How to Connect 400G QSFP-DD Switch with ConnectX-7 OSFP NIC

When building a high-performance AI cluster, selecting the appropriate optical module is crucial for ensuring the speed and stability of network communications. With the continuous advancement of NVIDIA GPU cards, the computing power of single-card GPUs has seen significant improvements. As AI models grow in size, the demands for network bandwidth and transmission efficiency also increase. The 400G network has become the new standard. A 400G network needs to support higher data transfer rates while maintaining low latency and high reliability. In AI and big data applications, the speed and efficiency of data transfer directly affect the completion time of computing tasks. Therefore, a 400G network solution must be able to handle large data streams and minimize transmission delays while ensuring data integrity. RDMA over Converged Ethernet (RoCE) technology has emerged, providing an efficient solution for 400G networks. The QSFP-DD 400G SR4 optical module, as a new choice for high-performance networking, offers an efficient and economical solution for AI clusters.

400G Network Application Solutions

When choosing a high-performance network solution for AI clusters, there are mainly two options: InfiniBand networks and Ethernet-based RoCE networks. In a 400G network environment, we usually select the NVIDIA CX7 400G network card to pair with the H800 GPU server. Regarding switches, there are various 400G and 800G switches on the market based on different chips. It is worth noting that mainstream 400G switches on the market mostly use 56G Serdes, while the CX7 network card adopts 112G Serdes.

QSFP-DD 400G SR8, OSFP 400G DR4, and QSFP-DD 400G SR4 are three common choices, each with distinct features and application scenarios.

QSFP-DD 400G SR8

• SR8 indicates the use of multimode fiber to transmit over a distance of 100 meters, with “8” denoting 8 optical channels.
• Each channel operates at a rate of 50G PAM4, requiring 16 fibers (8 for sending and 8 for receiving).
• SR8 modules use MPO-16 or MPO-24 connectors to connect 8 pairs of fibers.
• Suitable for short-distance connections within data centers or between adjacent buildings.

OSFP 400G DR4

• DR4 indicates the use of single-mode fiber to transmit over a distance of 500 meters, with “4” denoting 4 optical channels.
• Each channel operates at a rate of 100G PAM4, requiring 8 fibers (4 for sending and 4 for receiving).
• DR4 modules use single-mode fiber connectors, such as LC or MPO-12.
• Suitable for connections between data centers or for long-distance internal connections.

QSFP-DD 400G SR4

• SR4 indicates the use of multimode fiber to transmit over a distance of 100 meters, with “4” denoting 4 optical channels.
• Each channel operates at a rate of 100G PAM4, requiring 8 fibers (4 for sending and 4 for receiving).
• SR4 modules use MPO-12 connectors to connect 4 pairs of fibers.
• Suitable for short-distance connections within data centers or between adjacent buildings.

When selecting these optical modules, considerations include transmission distance, fiber type (multimode or single-mode), connector type, and network bandwidth requirements. QSFP-DD SR8 and SR4 modules are typically used for short-distance multimode fiber connections, while OSFP DR4 modules are used for longer-distance single-mode fiber connections. Therefore, for switch-to-switch interconnects, the QSFP-DD 400G SR8 module is the preferred choice. For connections from switches to network cards, customers often choose QSFP-DD 400G DR4 optical modules and use OSFP 400G DR4 modules on the server side to achieve Serdes matching.

However, there is now a more cost-effective option available: the 400G QSFP-DD SR4 module.

The QSFP-DD 400G SR4 optical module is a high-density, high-speed optoelectronic component, designed to meet the demands of data centers and high-performance computing networks. It utilizes the QSFP-DD (Quad Small Form Factor Pluggable-Double Density) interface, offering higher port density and bandwidth capabilities. The 400G SR4 module supports a data transfer rate of 400Gbps, operates at a wavelength of 850nm, and uses MPO/MTP-12 multimode optical fibers, allowing for transmission distances of up to 100 meters. This makes it highly suitable for short-distance connections within data centers or between adjacent buildings.

Technical Advantages

The technical advantages of the QSFP-DD 400G SR4 module are primarily reflected in the following aspects:

• High-Speed Transmission: The 400Gbps transmission rate significantly enhances the network’s data processing capacity, meeting the high-speed network requirements of AI and big data applications.
• Low Power Consumption: Compared to other types of 400G modules, the SR4 module uses VCSEL lasers, which consume less power, helping to reduce overall operational costs.
• Cost-Effectiveness: The SR4 module is relatively low-priced, providing an economical choice for building high-performance networks.
• Ease of Deployment: The standard QSFP-DD interface and MPO/MTP-12 connectors make the SR4 module easy to install and maintain.

In this solution, the SR4 module is chosen on the switch side, and correspondingly, the OSFP 400G SR4 module is used on the server side (similar advantages can be observed when compared to the OSFP 400G DR4), achieving a match in Serdes.

As a new choice for high-performance networks, the QSFP-DD 400G SR4 optical module not only meets the current high-speed network requirements of data centers but also provides robust support for the future development of networks. Its introduction marks a significant step forward in network technology towards higher rates and greater efficiency, laying a solid foundation for network communications in the era of AI and big data. With continuous technological advancements and further cost reductions, the QSFP-DD 400G SR4 module is set to become the preferred solution for constructing modern high-performance networks.