Skip to content

Dear customers! Please send orders and inquiries to [email protected] or to Telegram @servix_uz instead of calling by phone: due to the high workload, the operator cannot answer all incoming calls.

Servix

Network adapters and transceivers: SFP+, SFP28, QSFP28 — what connects to what

Servix technical team · Updated

In brief. Speed is determined by the whole chain: the card, the module or cable, the switch port and the fiber type. SFP+ is 10G, SFP28 is 25G and backward compatible, QSFP+ is 40G with breakout into 4 × 10G. Within a rack, a 1–⁠5 meter DAC cable is cheaper and more reliable; between floors, optics: 10G over OM3 reaches up to 300 meters, 25G only up to 70. Half of the “link won't come up” problems are vendor coding in the transceiver and mismatched FEC at 25 gigabits.

The network adapter and transceivers are an inexpensive part of the project, yet it is easy to lose a week troubleshooting them: the link won't come up, the speed is wrong, the module is not detected. We look at what connects to what and in what order to check it. Prices and specific models are collected separately in the selection network adapters, transceivers and DAC, and the network adapters themselves with SFP+ ports (often searched as “SFP plus”), SFP28 and QSFP28 for high-speed 100G networks are in the catalog network adapters for servers with prices and availability.

What speed is needed

Speed

Typical use

What to look at

1GbE

management, auxiliary servers, small offices

not enough for virtualization today

10GbE

the working minimum for virtualization hosts and storage system connections

a single backup stream takes up an entire gigabit

25GbE

the current choice for new projects

cost close to 10GbE, 2.5 times more headroom

40/100GbE

backbones between switches, hyperconverged nodes

usually excessive for server access

If the project is new, plan for 25GbE on servers and 100GbE on backbones. The price difference compared to 10/40GbE is small, and the operating horizon is noticeably longer. A practical criterion that works better than theoretical bandwidth: how long the backup window takes. If a full backup does not fit into the night, the network is the bottleneck, and the cure is moving to 10 or 25 gigabits.

Copper or fiber, and over what distance

  • RJ45 (twisted pair). Convenient and familiar, but for 10GbE it has higher power consumption, more latency and a strict length limit — 30 meters on Category 6 cable and up to 100 meters on 6a. Justified within a rack or a floor where there is no way to run fiber.
  • DAC cable. A copper cable with non-removable transceivers at the ends. The cheapest option for connections within a rack: passive DAC works over 1–⁠5 meters, active up to 10. It requires no connector cleaning and hardly heats up.
  • AOC cable. An active optical cable with attached modules, up to several tens of meters. Convenient between adjacent racks when you do not want to build a link from two modules and a patch cord.
  • Optics with SFP+/SFP28/QSFP transceivers. The universal option. Multimode SR: 10G — up to 300 meters over OM3 and up to 400 over OM4; 25G — only 70 and 100 meters respectively; 100G SR4 — up to 100 meters over OM4. Single-mode LR — up to 10 kilometers.

Note the gap in reach between 10 and 25 gigabits: a run that worked fine at 10G over OM3 may not come up at all after moving to 25G. This is one of the typical causes of “we just replaced the modules and everything broke”.

The main trap is transceiver compatibility

Many network equipment manufacturers check the vendor ID in the transceiver and refuse to bring up a port with a “foreign” module. Hence the practical conclusions:

  • A transceiver is selected for a specific model of switch or network adapter, not as “just an SFP+ 10G”.
  • Compatible modules from third-party manufacturers are usually coded for the required vendor and work, but their compatibility is checked before purchasing a batch, not after. On Cisco, third-party modules are enabled with a command, but the port is then no longer covered by warranty support.
  • Intel X520 and X710 series network adapters have historically been picky about modules: some firmware versions bring up the link only with Intel modules or with modules explicitly allowed in the driver.
  • Both ends of the link must use the same fiber type and the same wavelength. A multimode module will not connect to a single-mode one.

Form factors: don't mix them up

  • SFP — 1G
  • SFP+ — 10G
  • SFP28 — 25G (physically compatible with SFP+, the port usually supports both speeds)
  • QSFP+ — 40G
  • QSFP28 — 100G

A QSFP port can be split into four lower-speed lanes with a special cable — convenient when you need to connect several servers to one high-speed switch port. The reverse is not possible: four SFP+ ports cannot be combined into one 40-gigabit link other than by link aggregation, and that is a different logic and different load balancing.

QSFP28 and 100G: when it is needed and what connects to what

Requests about QSFP28 come in more often than real 100-gigabit projects — and almost always because of confusion with QSFP+. The connectors look the same, but these are different speeds: QSFP+ is 40G, QSFP28 is 100G. A QSFP28 port usually also accepts a 40-gigabit QSFP+ module; the reverse does not work: a QSFP28 module will physically fit into a QSFP+ port, but the link will either be 40G or will not come up at all.

A hundred gigabits in a corporate network are needed in three cases: uplinks between core switches, connecting all-flash storage nodes or vSAN and Ceph cluster nodes, and GPU nodes for inference. For a regular virtualization host with a couple dozen machines, this is excessive — 25G works there, and often 10G.

Connector

Speed

What to connect with

Typical reach

SFP+

10G

DAC, AOC, SR module over OM3/OM4

DAC up to 5 m, SR up to 300 m over OM3

SFP28

25G

DAC, AOC, SR module over OM4

DAC up to 5 m, SR up to 70–⁠100 m

QSFP+

40G

DAC, AOC, SR4 over MPO

SR4 up to 100–⁠150 m over OM4

QSFP28

100G

DAC up to 3–⁠5 m, AOC up to 30 m, SR4 over MPO, LR4 over single-mode

SR4 up to 100 m, LR4 up to 10 km

Three practical points that stall a 100G project after the purchase:

  • The fiber connector is different. 100G SR4 and 40G SR4 modules work not over an LC pair but over a multi-fiber connector MPO-12: eight fibers, four in each direction. Regular LC patch cords do not fit them, and the cabling must be designed for MPO in advance — redoing the cross-connects costs more than the modules themselves.
  • PCIe lanes. A 100G card needs a PCIe 3.0 x16 or PCIe 4.0 x8 slot. In a PCIe 3.0 x8 slot it will physically work but will be limited to about 63 Gbps — that is, you pay for a hundred and get sixty. Before purchasing, check which slots and what width the riser of the specific server provides.
  • Heat. QSFP28 modules run noticeably hotter than SFP+. In a densely packed 1U chassis with weak airflow, the module goes into thermal protection, and this looks like “the link flaps for no reason”. For 100G, plan for proper ventilation and do not install such cards in slots with no airflow.

A separate note on breakout: a QSFP28 → 4 × SFP28 cable turns one 100G switch port into four 25G ports — this is how four servers are connected to one core port. Similarly, QSFP+ breaks out into 4 × SFP+ at 10G. The reverse is not possible: four SFP+ ports will not become one forty-gigabit link other than through link aggregation, and that is about session balancing, not about the speed of a single stream.

DAC cables are sorted by speed: 10G, 25G, 40G and 100G.

Modules of all four form factors are in the catalog: SFP, SFP+, SFP28 and QSFP optical modules — from 170,600 UZS for a gigabit SFP, SFP28 25G from 619,000 UZS, QSFP28 100G SR4, LR4 and BiDi from 1,399,000 UZS, plus DAC cables. Network adapters in the catalog go up to 40G QSFP+ (price on request) and 25G SFP28; they are collected here; a 100-gigabit card is selected separately for the project, together with the cabling and connector calculation.

Why the link won't come up: the diagnostic sequence

Check along the chain, not at random — this way the problem is found in minutes.

  1. Physical layer. Is the port LED on, and is the module detected in the switch diagnostic output (show interface transceiver and equivalents)? A module that cannot be read is either a vendor coding issue or a dead module.
  2. Type and speed. Do the speeds match on both ends, and is the port not set to a fixed speed that differs from the module's capabilities?
  3. Fiber. Type (OM3, OM4, single-mode), run length, pair polarity and connector cleanliness. A dirty connector produces “came up and went down” under load rather than a clean failure.
  4. FEC at 25G. Forward error correction negotiation between the card and the switch sometimes has to be set manually: if the modes do not match, the link either does not come up or flaps. At 10G, there is no such class of problems.
  5. Driver and firmware. Outdated card firmware is a common reason a new module is “not visible”. Updating the adapter firmware is part of the standard checklist before commissioning.

What to consider when choosing a card for a server

  • OCP slot versus PCIe. Modern HPE and Dell servers have a dedicated OCP 3.0 slot for the network adapter. Using it frees full-size PCIe slots for a RAID controller or GPU. The proprietary formats of the previous generation — HPE FlexibleLOM (FLR) and Dell rNDC — fit only their own platforms.
  • CPU offload. Enterprise-class cards can take over part of the processing — checksums, segmentation, VXLAN tunnel processing. On loaded virtualization hosts, this noticeably reduces CPU load.
  • Hypervisor support. Check the card against the compatibility list of your virtualization platform before purchasing. Exotic adapters without a driver included in the hypervisor are a source of unpleasant surprises during updates.
  • Number of ports. Two ports are the minimum for a fault-tolerant connection. A single port means that a failure of the cable or switch port takes the server out of service.
  • Power and cooling. 25 and 100 gigabit cards run noticeably hotter than gigabit ones; in a short 1U chassis with several adapters, this should be taken into account in the layout.

Typical virtualization host connection scheme

Separate the traffic: management, production virtual machine networks, storage, VM migration. The separation can be physical (different ports) or logical (VLANs on shared ports), but each group must have two paths through different switches. Connecting both server ports to one switch leaves a single point of failure and defeats the redundancy.

The working minimum for a cluster node: two 10 or 25 gigabit ports for data and storage with aggregation, and two gigabit ports for management. If the storage system is connected via iSCSI, storage traffic is better moved to separate VLANs and not mixed with virtual machine migration: a migration spike can “eat” bandwidth from disk operations, and this looks like an array problem rather than a network one.

Related articles: choosing a switch for an office — in the guide 24- and 48-port switches; connecting storage systems via FC, iSCSI and SAS — in the selections HPE MSA 2060 and 2062 and Dell PowerVault ME5.

Questions and answers

Can a 40G QSFP+ module be installed in a QSFP28 port?

As a rule, yes: a QSFP28 port supports both 100G and 40G. It does not work the other way around — a QSFP28 module will fit mechanically into a QSFP+ port, but you will not get a hundred gigabits, and the link may not come up at all. Check the documentation of the specific switch: 40G support in QSFP28 ports is not guaranteed on all models.

What is needed for a 100-gigabit link?

A card with a QSFP28 port in a PCIe 3.0 x16 or PCIe 4.0 x8 slot, a QSFP28 port on the switch and a transmission medium: DAC up to 3–⁠5 meters within a rack, AOC up to 30 meters between racks, SR4 modules over an MPO-12 multi-fiber run up to 100 meters, or LR4 over single-mode up to 10 kilometers. The most expensive mistake is to lay a regular LC pair and discover that SR4 requires MPO.

Can an SFP28 module be installed in an SFP+ port?

The module will physically fit, but the link will come up at ten gigabits: the speed is limited by the port. The reverse combination works as standard — an SFP28 card handles an SFP+ module at 10 gigabits without issues, and this is a convenient migration path: first 25G cards, then switches.

How does DAC differ from AOC, and which to choose?

DAC is a copper cable with non-removable modules, cheap but short: 1–⁠5 meters passive. AOC is an optical cable with attached modules, reaches tens of meters and costs more. Within a rack, DAC is used; between racks, AOC or an assembly of two modules and a patch cord if the run goes through cable trays.

Is it mandatory to buy transceivers of the same brand as the switch?

Not mandatory, but compatibility must be checked before purchasing a batch. Compatible modules are coded for a specific vendor and work in most cases; the risk is that behavior may change after a switch firmware update. For a critical segment, branded modules are the sensible choice; for a test segment, compatible ones.

How many ports does a virtualization host need?

At least four: two for data and storage at 10 or 25 gigabits and two gigabit ports for management, with each pair going to different switches. A scheme with a single dual-port adapter also works, but a failure of the card itself takes down the entire node, so critical clusters use two cards.

We will select network adapters and compatible transceivers for your equipment — checked against your switch and server models: write to us, [email protected] or Telegram. Prices and availability are in the section network adapters and transceivers.

Illustration: Dmitry Nosachev, Wikimedia Commons (CC BY-SA 4.0).

Related reading