NVIDIA and 800G Optical Transceiver Module

The launch of ChatGPT has driven the growth of AI. In mid-May, NVIDIA released the DGX H200 for AI, leading to a reshaping of industry demand with 800G optical transceiver orders pouring in. Companies such as Google, Meta, Microsoft, and others are also considering integrating AI into their operations, thereby driving the implementation of the AI industry chain both upstream and downstream. FiberMall, a producer of optical modules, is an integral part of this chain.

Currently, FiberMall is shipping 800G optical modules, with a larger quantity of multimode modules (800G SR8) and a smaller quantity of single-mode modules (800G DR8). As for the demand in data centers at the beginning of 2023, there are expectations of traffic growth, but limited investment funds pose a constraint. The industry’s estimation for the demand for 800G optical modules at the beginning of the year was around 3 million units. However, in June, there was a significant surge, almost doubling the demand, and it is projected to increase to tens of millions of units next year.

800G SR8 and 800G DR8

The fiber optic communication industry is in its early stages, and companies are gradually increasing their production capacity. As early as two to three years ago, many companies had already showcased 800G demonstration samples. During the OFC/ECOC exhibition in 2022, FiberMall introduced single-mode EML chips and multi-mode VCSEL chips for 800G transceiver modules, along with advancements in internal structures and principles.

This year’s OFC exhibition primarily focused on 200G EML technology, while the industrialization of 800G optical transceiver modules initially started with 8x100G modules. In the future, it will further develop into 4x200G and 8x200G 1.6T optical modules.

Multi-mode modules have a larger quantity compared to single-mode modules. FiberMall is capable of delivering several hundred thousand multi-mode 800G optical transceiver modules this year, indicating strong industrialization capabilities. From demand calculation to order placement and delivery, it is a process for companies. Therefore, some quantities are estimated for 2023, while others are projected for 2024, representing different time points in the industry’s perspective on the same demand.

The sudden increase in module demand is mainly driven by the interconnection of multiple chips, based on NVIDIA’s infrastructure.

With the increasing demand for GPUs, a significant number of basic transistors are required to achieve computation, storage, and control. Chip sizes of several hundred square millimeters are already considered large, and chip fabrication processes have been progressively shrinking, from 10nm to 7nm, 5nm, and so on. As the size of individual transistors decreases, hundreds of billions to trillions of “super” integrated transistors can be accommodated within an area of several hundred square millimeters.

To further enhance integration and computational power, relying solely on increasing integration density is no longer sufficient. There are limitations to the technological capabilities of semiconductor chips. One available approach is interconnecting multiple chips to achieve an equivalent larger scale of computing power.

Interconnection between chips serves the purpose of achieving connectivity, which can be accomplished using copper. For instance, copper foils on PCB boards can be employed as signal lines, or coaxial electrical connections can be established.

Optical fibers can also be utilized for interconnection. Optical fibers offer high capacity and low losses, enabling transmission over longer distances. However, the downside of optical fiber communication is its higher overall cost since it requires signal conversion between the “electrical” signals of integrated circuits and the “optical” signals of fiber optic communication.

An optical module is an interface for converting optical signals into electrical signals. A massive number of optical modules are required to support optical interconnection between integrated circuits.

This also explains why, in the early days, optical fibers were avoided whenever possible. It was primarily due to the high volume of interconnect traffic or when the interconnect distance was too long that optical fibers were used as a last resort.

Interconnection implies establishing connections between each pair of chips.

If we consider four chips, the number of chips increases in a linear relationship (2x). However, the interconnect traffic between chips does not increase linearly but rather exhibits exponential growth due to the complex topology.

As the topology becomes more complex, the interconnect traffic intensifies, and the distance between chips increases. This creates a dilemma regarding whether to choose copper wire or optical fiber for interconnection.

In this generation, where 256 chips are interconnected simultaneously, the diagram can only be annotated with ellipses, which represent large-scale chip-to-chip interconnection. From the perspective of optical modules, there is little difference between interconnecting copper wires, single-mode fibers, or multi-mode fibers in terms of physical interfaces.

The “Fiber in, Copper out” refers to the trend in the field of information transmission where optical fiber is increasingly being used as a medium for data transfer, gradually replacing copper cables. Optical fibers have the advantage of supporting high-capacity and long-distance information transmission. When optical fiber is chosen for data transmission, integrated circuits are used for functions such as storage, computation, and control, requiring the use of optical modules for optoelectronic conversion.

The current emergence of 800G optical transceiver orders is not due to a lack of demand for even larger capacities but rather because the industry’s capabilities have just reached this stage, with manufacturing capacity still being limited. Therefore, from a research and development perspective, there is continuous technological exploration and development of 1.6T, 3.2T, and 6.4T optical modules (some people also refer to them as optical engines), supported by the upstream and downstream industry chains. This includes advancements and innovations in areas such as silicon photonics, lithium niobate thin films, EML chips, and VCSEL chips.