Skip to content

Dear customers! Please send orders and inquiries to [email protected] or to Telegram @servix_uz instead of calling by phone: due to the high workload, the operator cannot answer all incoming calls.

Servix

SFP28, QSFP28 and DAC for the server room and data center: what to choose for 25G and 100G

Servix technical team · Updated

In brief. Inside a rack, at 1–⁠3 m, servers are connected to the switch with a passive copper DAC cable: it is cheaper than a pair of modules with a patch cord and does not require connector cleaning. At 25G, the DAC length determines error correction: 1–⁠2 m cables usually work without FEC, 3 m cables usually need BASE-R FEC, 5 m cables need RS-FEC, and the mode must match at both ends. Between racks and rows, up to 100 m over OM4, SR optics are used: at 25G, this is a module with an LC pair; at 100G, SR4 with an MPO-12 connector for eight fibers. Between halls and buildings — LR and LR4 over single-mode with LC, up to 10 km, and with Fibreon modules up to 20 km as well.

This article is for those who design a server room or data hall for 25G and 100G: connecting servers to the access switch, uplinks to the core, links between halls. The basics — what SFP28 and QSFP28 are and how 25G differs from 10G — are in the article “Network adapters and transceivers”. Here is the next step: which cable or module to install on each segment, where FEC is needed, when to choose MPO and when LC, and how breakout cables work.

Three ways to connect two ports

An SFP28 or QSFP28 port can be connected with a copper DAC cable, an AOC active optical cable or a pair of optical modules with a patch cord between them.

Option

Length

When it fits

What to consider

Passive DAC

at Cisco — from 1 to 5 m for 25G and 100G

server and switch in the same rack

at 25G, FEC depends on the cable length and class; the length is fixed; a thick cable bends less easily

AOC

at Cisco — 1–⁠10 m for 25G, 1–⁠30 m for 100G

adjacent racks, when you do not want to build a link from modules

the modules are not removable; if it breaks, the whole cable is replaced

SR modules + OM4 multimode

25G and 100G SR4 — up to 100 m

between racks and rows, via patch panels

requires optical infrastructure; on 100G SR4, RS-FEC is mandatory; on 25G SR it is usually needed

LR and LR4 modules + single-mode

10 km, Fibreon has 20 km versions

between halls and buildings

the modules are more expensive than multimode ones

BiDi modules + single fiber

at 25G and 100G — up to 20 km for Fibreon models

when only one fiber strand is free

work only as a U and D pair

The Servix catalog has DAC cables and Fibreon modules for all these cases except AOC: 25G and 100G AOC cables are not currently in the catalog. If you need a 5–⁠30 m link, it is built from two SR modules and an OM4 patch cord — and the patch cord can then be replaced separately from the modules.

DAC at 25G: the length decides whether FEC is needed

At 10G, DAC cables were distinguished only by length. At 25G, this is not enough: IEEE 802.3 divides passive cables into three classes by electrical properties, and the class determines which error correction is needed. Intel, in its guide to 25G media, gives the standard's table:

Cable class per IEEE 802.3

Typical length

Error correction it works with

CA-25G-N

1–⁠2 m

without FEC, BASE-R FEC or RS-FEC

CA-25G-S

2–⁠3 m

BASE-R FEC or RS-FEC

CA-25G-L

3–⁠5 m

RS-FEC only

The length in the table is a guideline, not a rule. Intel itself notes that there are 3 m cables of class CA-25G-L and 3 m cables of class CA-25G-N: conductors are made with gauges from 22 to 32 AWG, and cables of the same length have different electrical properties. At Cisco, for example, the SFP-H25G-CU1M and CU2M cables work without FEC, the CU3M requires BASE-R FEC, and the CU5M requires RS-FEC.

Practical conclusions:

  • The FEC mode must match at both ends. If RS-FEC is enabled on the switch and the server network adapter works without FEC, the link will not come up or will “flap”. On Linux, the mode is viewed and set with ethtool --show-fec and ethtool --set-fec.
  • Do not disable auto-negotiation without need. Under IEEE 802.3, auto-negotiation is mandatory for copper 25GBASE-CR, and the sides also agree on the FEC mode through it.
  • Plan for RS-FEC with a 5 m cable. Fibreon has a 5 m SFP28 DAC — a compatible equivalent of Cisco SFP-H25G-CU5M, HPE JL489A and Dell 470-ACEY; enable RS-FEC on the ports.
  • Latency. According to Intel, BASE-R FEC adds about 80 ns, RS-FEC about 250 ns. For most tasks this is unnoticeable, but for latency-sensitive systems a short cable without FEC is preferable.

Fibreon DAC cables for 25G come in lengths of 1, 2, 3 and 5 m, with 28 AWG conductors. Passive DACs have no DDM diagnostics — the originals do not have it either.

DAC at 100G

A QSFP28 — QSFP28 cable is four 25G lanes in one jacket. In the same IEEE 802.3 table cited by Intel, such cables 1–⁠5 m long belong to class CA-25G-L and are designed to work with RS-FEC. Cisco 100G DAC cables are available in lengths from 1 to 5 m.

Fibreon 100G cables come in lengths of 1, 2 and 3 m: 1 m with 30 AWG conductors, 2 and 3 m with 26 AWG. The thicker the conductor, the stiffer the cable, so in a dense rack check in advance that there is enough room for bending at the ports and in the cable organizer. A 1 m cable is suitable for connecting switches in adjacent units.

Optics: SR or LR

If the neighboring port is more than a few meters away, optics are needed.

  • 25GBASE-SR. 850 nm, multimode, LC pair: 70 m over OM3 and 100 m over OM4. For its module, Cisco requires enabling RS-FEC on the port; power consumption is up to 1.2 W. Intel specifies that only 25G SR modules with an especially low error rate work without FEC, so enable RS-FEC on ports with SR modules. The FBN-S25-SR module is a compatible equivalent of Cisco SFP-25G-SR-S, HPE JL484A and Huawei SFP-25G-SR.
  • 25GBASE-LR. 1310 nm, single-mode, LC pair, 10 km; Fibreon also has a 20 km version. The modules are in the section SFP28 25G.
  • 100GBASE-SR4. Four 25G lanes at 850 nm, multimode, MPO-12 connector: 70 m over OM3 and 100 m over OM4; power consumption at Cisco is 3.5 W. RS-FEC is mandatory in this standard and cannot be disabled. The Fibreon compatible equivalent is FBN-Q100-SR4.
  • 100GBASE-LR4. Four wavelengths in the 1295–⁠1310 nm range in one fiber, single-mode, LC pair, 10 km; FEC is not used in this standard; power consumption at Cisco is 4 W. Fibreon has the FBN-Q100-LR4 for 10 km and a 20 km version.
  • 100G over a single fiber. If only one fiber strand is free between halls, a BiDi pair is used: at Fibreon, these are modules at 1291 and 1311 nm, up to 20 km, ordered as a pair.

QSFP28 modules consume several times more power than SFP28 and run hotter. Intel states directly in its driver documentation: if the airflow over the adapter and modules is insufficient, modules rated for higher temperatures are needed. Fibreon modules are rated for 0…70 °C, so install 100G adapters in slots with normal airflow. All Fibreon optical modules have DDM — the temperature is visible in the port diagnostics.

MPO or LC

This is the main question for 100G: the answer determines the cabling system of the hall.

SR4 and MPO-12. A 100GBASE-SR4 module transmits over four fibers and receives over four: eight of the twelve fibers of the MPO-12 connector are used. Regular LC patch cords do not fit it; MPO trunks and cassettes are needed. For a direct connection between two modules, a Type B polarity MPO cable, “key up — key up”, is usually used, but check the polarity and connector type (male or female) against the modules and panels before purchasing. The advantage of SR4 is price: in the Servix catalog, an SR4 module is noticeably cheaper than LR4 — compare the product pages.

LR4 and LC. An LR4 module works over a regular pair of single-mode fibers with LC connectors. If single-mode LC links are already installed in the hall or the backbone goes to another building, it is simpler and more reliable to use LR4 than to rebuild the patch panels for MPO.

The guideline is: a new hall with short distances and OM4 MPO trunks — SR4; existing single-mode LC infrastructure or distances over 100 m — LR4.

Breakout cables: one port into four

A QSFP port is four independent lanes. With a breakout cable, one port is turned into four lower-speed ports: QSFP+ 40G into four 10G SFP+, QSFP28 100G into four 25G SFP28. This way, four servers are connected to one switch port.

  • The port must support breakout. Breakout mode is enabled in the switch settings, and not all ports and models have it — check the switch documentation.
  • It does not combine back. Four SFP+ ports will not become one 40G link, only a link aggregation, and that has different load-balancing logic.
  • Where the cable splits. On a 3 m breakout cable, the ends diverge about 1 m from the QSFP connector — this is how the Cisco 40G cables are made. Take this into account when routing in the rack.
  • Optical breakout. An SR4 module can be split into four lanes through an MPO — 4 × LC cable, if the module and switch allow it. Cisco, for example, allows this scheme for QSFP-40G-SR4 and does not support it for QSFP-40G-SR4-S.
  • FEC. For a copper QSFP28 — 4 × SFP28 breakout cable, the same classes CA-25G-N, -S and -L apply as for regular SFP28 cables.

Of the Fibreon breakout cables, the catalog currently has a DAC 40G → 4 × 10G, 3 m, a compatible equivalent of Cisco QSFP-4SFP10G-CU3M and HPE JG330A. A 100G → 4 × 25G breakout cable is not yet among the Fibreon modules in the catalog.

Typical layout: what to install on each segment

Segment

Distance

What to install

Server — access switch in the same rack

1–⁠3 m

SFP28 25G DAC; at 1–⁠2 m usually without FEC

Server — switch in the adjacent rack

up to 5 m

5 m SFP28 DAC with RS-FEC or a pair of 25GBASE-SR modules

Access switch — core in the same hall

up to 100 m

100GBASE-SR4 over OM4 with MPO-12 or 100GBASE-LR4 over single-mode if the patch panels are LC

Switches in adjacent units

1–⁠3 m

DAC QSFP28 100G

Between halls and buildings

up to 10 km

100GBASE-LR4 or 25GBASE-LR

Long backbone

up to 20 km

20 km versions of LR4 and LR

One free fiber

up to 20 km

a BiDi pair at 100G or 25G

Four 10G servers to one 40G port

up to 3 m

DAC 40G → 4 × 10G breakout cable

100G modules are in the section QSFP28 100G; prices are on the product pages.

25G network adapters

On the server side, the SFP28 port is on the network adapter, and its controller sets the requirements for the module. Before ordering, check two things: which FEC modes the adapter supports and whether its manufacturer has restrictions on modules. 25G adapters with prices are in the section network adapters, for example HPE 640SFP28 with two 10/25G ports. We write the compatibility code for the module or cable for the specific adapter — tell us its model.

Frequently asked questions

What to choose inside a rack: DAC or optics?

DAC: at 1–⁠3 m it is cheaper than a pair of modules with a patch cord, barely heats up and does not require connector cleaning. Optics are needed where the cable length is not enough, or if flexible replacement of patch cords matters.

Why does the link not come up on a 5 m 25G DAC?

Most often because of FEC: a 5 m cable needs RS-FEC at both ends. Check the mode on the switch and on the adapter; on Linux, with the command ethtool --show-fec.

Can an SR4 module be connected with a regular LC patch cord?

No. SR4 has an MPO-12 connector; you need an MPO cable or trunk with the correct polarity. If the hall has only LC, choose LR4 over single-mode.

What is better for a 100G uplink in one hall: SR4 or LR4?

With OM4 MPO trunks and a distance of up to 100 m — SR4, it is cheaper. With single-mode LC patch panels or a distance over 100 m — LR4.

Is FEC needed for 100GBASE-LR4?

No, FEC is not used in the LR4 standard. For 100GBASE-SR4, RS-FEC is mandatory, and it is also enabled with 25G SR modules.

Send to [email protected] or on Telegram @servix_uz your layout: how many servers and switches, switch and network adapter models, distances between racks and halls, the type of fiber and connectors in the patch panels. We will select DAC cables and Fibreon modules for each segment and send a quote. The modules and cables are in stock in Tashkent, flashing for your equipment is free, warranty 1 year with replacement through Servix.

See also: network adapters and transceivers: SFP+, SFP28, QSFP28 — what connects to what, choosing a 10/25/40G network adapter, transceivers and DAC, compatible SFP modules: how they work and when to buy an original, how to choose an SFP for Cisco, HPE Aruba, Huawei and Juniper.

Related reading