100G InfiniBand EDR Solution: Empowering High-Performance Computing
Updated at Dec 21st 20231 min read
In the rapidly evolving landscape of high-performance computing (HPC), artificial intelligence, and hyperscale cloud infrastructures, the demand for ultra-high-speed data processing and low-latency communication is paramount. FS 100G EDR InfiniBand solution addresses these needs, providing cutting-edge technology to enhance network performance. This blog explores the capabilities of FS 100G EDR InfiniBand solution, focusing on the deployment of 100G QSFP28 EDR transceivers and cables, which are crucial for achieving higher data rates and improved network latency.

Introduction to InfiniBand HDR 100Gb/s Cables and Transceivers
InfiniBand EDR 100Gb/s Active AOC Cables
The NVIDIA InfiniBand MFA1A00-E001, a Class 1 FDA Laser-based active optical cable, is tailored for deployment in InfiniBand 100Gb/s EDR systems. Available in various lengths ranging from 1m to 100m, these active fiber cables are favored in high-speed InfiniBand HPC environments due to their predictable latency, Max. power consumption 3.5W, and airflow enhancement. Featuring hot-swappability, the NVIDIA InfiniBand MFA1A00-E001 facilitates easy installation and replacement. Each end of the cable incorporates four multimode fiber (MMF) transceivers with a data transmission rate of up to 26 Gb/s, surpassing the capabilities of Direct Attach Copper (DAC) cables in terms of airflow, bend radius, and transmission distances.

InfiniBand EDR 100Gb/s Passive Copper Cables
The NVIDIA InfiniBand MCP1600-E001E30 offers a cost-effective alternative for InfiniBand 100Gb/s EDR applications, with lengths available in 0.5m, 1m, 1.5m, 2m and 3m. This passive copper cable boasts four high-speed copper pairs, each supporting data rates of up to 25Gb/s. Featuring an EEPROM on each QSFP28 port for host system communication, this passive DAC solution ensures efficient connectivity for short-haul interconnections. It not only enables higher port bandwidth, density, and configurability, but also reduces power demand in the data center.

InfiniBand EDR 100Gb/s Optical Modules
The 100Gb EDR optical modules, packaged in QSFP28 form factor with LC duplex or MTP/MPO-12 connectors, are hot-swappable and suitable for both EDR InfiniBand and 100G Ethernet. These modules employ four high-speed differential signal channels, each operating at 25 Gbps. They can be categorized into QSFP28 SR4, QSEP28 PSM4, QSFP28 CWDM4, and QSFP28 LR4 based on transmission distance requirements.

QSFP28 Model | Interface Type | Rate | Fiber Type | Wavelength | Transmission Distance | Laser Type | DDM | Protocol |
MPO/MTP-12 | EDR | MMF | 850nm | 70/100m | VCSEL | Support | InfiniBand EDR | |
MPO/MTP-12 | EDR | MMF | 1310nm | 2km | DFB | Support | InfiniBand EDR | |
LC Duplex | EDR | SMF | 1271nm, 1291nm, 1311nm, 1331nm | 2km | DFB | Support | InfiniBand EDR | |
LC Duplex | EDR | SMF | 1310nm | 10km | EML | Support | InfiniBand EDR |
100Gb InfiniBand EDR System Scenario Applications
InfiniBand has gained widespread adoption in data centers, artificial intelligence, and other domains, primarily employing the spine-leaf architecture. In data centers, transceivers and cables play a pivotal role in two key scenarios:
Data Center to User: Involves end-user interactions such as web browsing, email exchange, and video streaming, connecting switches with servers or storage devices.
Data Center Interconnects: Facilitates data replication, software, and system upgrades, linking switches together.
Switch to Switch Interconnects
Spine and Leaf configurations of HDR100 InfiniBand built at 100 Gb/s, also connect Core Switch and Spine Switch at 100G EDR rate.

Product | Category | Distance | Application Area |
QSFP-100G-PCxxx | DAC | 0.5m to 3m | Spine switch to Leaf switch |
QSFP-100G-AOxxx | AOC | 1m to 100m | Spine switch to Leaf switch |
QSFP-SR4-100G | Optical Transceiver | ≤100m | Spine switch to Leaf switch |
QSFP-PSM4-100G | Optical Transceiver | ≤500m | Spine to Core switch |
QSFP-CWDM4-100G | Optical Transceiver | ≤2km | Spine to Core switch |
QSFP-LR4-100G | Optical Transceiver | ≤10km | Spine to Core switch |
Switches to Servers or GPU Computing Systems
Direct Attach Copper (DAC) is widely adopted in data centers due to its lack of electronics and minimal latency. Conversely, AOC active optical cables offer an integrated solution for short-distance multi-channel data communication, catering to communication distances from 1m to 100m. Ideal for supercomputers with InfiniBand standards like SDR, DDR, and QDR, AOC cables provide superior airflow heat dissipation and a smaller bend radius compared to copper cables. Additionally, they offer extended transmission distances and improved Bit Error Rate (BER) performance. However, for longer transmission distances, consideration of optical modules becomes necessary.

Product | Category | Distance | Application Area |
Q28-PC0xE | DAC | 0.5m to 3m | Leaf to NIC/Storage |
QSFP-100G-AOxxx | AOC | 1m to 100m | Leaf to NIC/Storage |
QSFP28-SR4-100G | Optical Transceiver | ≤100m | Leaf to NIC/Storage |
Summary
As the complexity of computational workloads continues to rise, FS's 100G EDR InfiniBand solution emerges as a powerful and reliable option for data centers, HPC environments, and AI applications. Offering a combination of high bandwidth, low latency, and reduced power consumption, this solution is designed to meet the demands of modern infrastructure. With FS's commitment to innovation and quality, customers can confidently choose 100G EDR InfiniBand to future-proof their networks and drive performance to new heights.