The Era of AI Optical Communication: Three Major Industry Changes for CPO

Co-Packaged Optics (CPO) is a new type of optoelectronic integration technology. CPO is based on advanced packaging technology, integrating optical transceiver modules and ASIC (Application-Specific Integrated Circuit) chips into a single package to form a microsystem with specific functions.

CPO technology further shortens the electrical interconnect length between optical signal input and computing units. This not only increases the interconnect density between optical modules and ASIC chips but also achieves lower power consumption. It is an important technical solution to the problem of high-speed transmission of massive data in future big data processing.

According to Cisco data, from 2010 to 2022, the network switching bandwidth of global data centers increased by 80 times. The trade-off, however, was an 8-fold increase in switching chip power consumption, a 26-fold increase in optical module power consumption, and a 25-fold increase in the power consumption of SerDes (Serializer/Deserializer) in switching chips. Since optical interfaces rely on analog-digital hybrid SerDes technology, their energy efficiency evolves slower than the ASIC part. The cost per bit and power consumption reduction rate of optical interfaces lag far behind those of switch ASICs. To further reduce power consumption, it is necessary to shorten the distance of SerDes or reduce the number of SerDes. Hence, new technologies such as OBO (On-Board Optics), NPO (Near-Packaged Optics), and CPO have emerged in optical interconnect system architectures.

The Major Industry Changes for CPO in the AI Optical Communication Era

Accelerated Development of Silicon Photonics Technology; Continuous Maturation of CPO Silicon Photonics Engines

As the core technology of CPO, silicon photonics engines are maturing rapidly in the AI optical communication era. Silicon photonics technology is an ideal platform for integrating photonics and microelectronics. In today’s information society, characterized by “electronic computing, optical transmission,” the technical bottlenecks of microelectronics/optoelectronics are increasingly apparent. Silicon-based optoelectronics, compatible with the mature CMOS microelectronic process, is expected to become the best solution for integrating photonics and microelectronics. As the mainstream solution for current CPO optical engines, the maturity of silicon photonics technology is expected to drive the development of CPO further.

(1) From the perspective of silicon photonics technology applications, silicon photonics technology serves as the underlying technology for silicon photonics modules, CPO, and OIO optical engines. In the high-speed era, the expansion of silicon photonics optical communication is expected to further catalyze the maturation of silicon photonics engine technology.

Silicon photonics as an optical communication technology is expected to fully benefit from the development of AIGC (AI-Generated Content). Silicon photonics technology in data centers, whether on the chip side (OIO), device side (CPO), inter-device optical modules, or coherent optical communication between data centers, is expected to see further development.

• From the perspective of the development pace of silicon photonics, global companies are actively promoting the development of silicon photonics technology, further improving the silicon photonics industry chain.

At present, the silicon photonics technology industry is still developing, with the industry chain gradually taking shape. It now covers various aspects, including cutting-edge research institutions, design tool providers, device chip module vendors, foundries, IT companies, system equipment manufacturers, and users. Around 2010, the silicon photonics research system began transitioning from being led by academic institutions to being driven by manufacturers. There are several main development models for silicon photonics technology:

• National Project Support: For instance, the United States launched the “National Photonics Initiative” in 2014, funding the creation of an integrated photonics institute. Subsequently, in 2015, the AIM Photonics Institute was established with an investment of $610 million to standardize the integrated photonics platform, bringing together various industry chain segments. Other global related research projects and institutions include the EU’s Leti silicon photonics module mass production research program.
• Investment by IT Giants: Companies like Intel and IBM have been committed to silicon photonics technology research since around 2003, making long-term and substantial investments.
• Small Startups: Initially funded by venture capital, these companies are later acquired by larger enterprises for sustained investment, a key development model for silicon photonics. Examples include Acacia and SiFotonics.
• Emerging Startups: Some newly rising startups also contribute significantly to the development of silicon photonics.

(3) From the perspective of silicon photonics industry opportunities, the prosperity of silicon photonics solutions continues to increase, providing a favorable opportunity for companies to enter the CPO industry.

The 25th China International Optoelectronic Exposition (CIOE 2024) was held from September 11-13, 2024, at the Shenzhen International Convention and Exhibition Center. Driven by AI, the rapid upgrade from optoelectronic chips and optical devices/engines to optical modules towards higher speeds has accelerated. Simultaneously, new technologies represented by silicon photonics/CPO/thin-film lithium niobate/coherent optics are continuously maturing. Among them, the maturity and market attention of silicon photonics technology have significantly increased, with numerous companies investing in silicon photonics technology.

Leading Manufacturers Actively Deploy CPO, Further Catalyzing Industry Development

Leading chip manufacturers are actively deploying CPO technology, continuously launching silicon photonics CPO prototypes. Various CPO solutions have been introduced by major chip manufacturers, including Intel, Broadcom, Raonvus, AMD, Marvell, and Cisco, which have all showcased CPO prototypes at recent OFC exhibitions, achieving increased switching capacity and reduced power consumption. Companies like Nvidia and TSMC have also demonstrated their CPO plans. We believe that, on the one hand, CPO technology based on silicon photonics engines, as the mainstream solution, is likely to benefit from the development of silicon photonics technology. On the other hand, the entry of leading manufacturers is expected to further accelerate the improvement and development of the CPO industry chain.

Intel has been actively researching and developing pluggable optical modules and microring modulators. Since 2020, it has been leveraging its silicon photonics process platform to build CPO (Co-Packaged Optics) systems based on microring modulators. At the “OFC 2020” conference, Intel introduced its first CPO prototype, which integrates a 1.6Tbit/s silicon photonics engine with a 12.8Tbit/s programmable Ethernet switch, while also considering thermal management in its architectural design. At the 2024 IEEE ISSCC, Intel announced the latest advancements in its CPO technology, achieving a signal transmission rate of 4x64Gb/s with a low system power consumption of only 1.3pl/bit. Intel and Ayar Labs, having collaborated for many years, showcased the embedding of two 4Tb/s TeraPHY OIO chiplets into Intel Agilex FPGAs at the Supercomputing 2023 conference, supported by two SuperNova light sources that enable high-speed optical communication with 64 optical channels on each chiplet.

Broadcom unveiled its first CPO switch at the “OFC 2022” conference, combining the 25.6Tbps Tomahawk4 switch chip with an optical engine. In 2023, Broadcom introduced the Strata Tomahawk XGS5, which features a switching capacity of 51.2Tbps, consumes only 5.5W of power, and supports 800Gbps data rates. At the “OFC 2024” conference, Broadcom announced that it had delivered the industry’s first 51.2Tbps CPO Ethernet switch, the Bailly, to customers. This product integrates eight 6.4-Tbps silicon photonics optical engines with the StrataXGS Tomahawk5 switch chip, reducing optical interconnect power consumption by 70% and increasing silicon area efficiency by eight times.

Ranovus released the Odin brand analog-driven CPO 2.0 architecture at the “OFC 2021” conference. Developed in collaboration with IBM, TE, and Senko, this architecture achieves a 40% reduction in power consumption and cost savings by eliminating retiming functions and implementing an IC-efficient single-chip solution. At “OFC 2023,” Ranovus demonstrated the combination of an 800G direct-drive silicon photonics engine with AMD’s FPGA chips.

Marvell showcased its first CPO prototype with a bandwidth of 1.6Tbit/s at the “OFC 2022” conference. In “OFC 2023,” Marvell introduced a 51.2Tbit/s switching chip.

Cisco demonstrated a 25.6T switch prototype based on CPO technology at “OFC 2023,” featuring eight 3.2T silicon photonics engines, each equipped with eight 400G-FR4 silicon photonics chips, with each optical engine having a single-channel rate of 100Gbps.

Nvidia has been developing silicon photonics CPO. At the “2020 GTC” conference, Nvidia showcased a system architecture diagram that interconnects GPUs and switch chips through CPO and has been actively collaborating with companies like TSMC and Ayar Labs to develop CPO technology.

TSMC began collaborating with Luxtera in 2017 to develop a 12-inch silicon photonics process platform at the 65nm node. Subsequently, TSMC introduced advanced packaging and launched the COUPE1.0/2.0 platform, with a development roadmap aiming to achieve a 6.4Tbps optical engine by 2025.

TSMC has announced the launch of the COUPE platform. At the 2024 North American Technology Symposium, TSMC unveiled its 3D optical engine roadmap, planning to provide optical connections up to 12.8 Tbps for processors manufactured by TSMC. Given that copper cabling cannot meet the growing bandwidth demands, silicon photonics is poised to become a critical technology for future data centers. The Compact Universal Photonics Engine (COUPE) is one of the significant achievements in the field of silicon photonics. This technology utilizes TSMC’s SoIC-X packaging technology, stacking electronic integrated circuits (EIC) on top of photonic integrated circuits (PIC) to form an EIC-on-PIC structure. This structure achieves the lowest impedance at the chip-to-chip interface, resulting in maximum energy efficiency. Additionally, COUPE features a compact integrated design, wide wavelength compatibility, efficient optoelectronic conversion, scalability, and flexibility, making it suitable for various optical interconnect applications and meeting diverse needs.

Currently, TSMC’s 3D optical engines are in the development stage, with plans to gradually enhance transmission speeds and bring optical connections closer to the processors. The COUPE development plan includes three stages, each focused on increasing transmission rates and reducing power consumption:

1. 2025: TSMC’s first-generation 3D optical engine will be integrated into OSFP pluggable devices operating at speeds of 1.6 Tbps, doubling the maximum rate of current copper-based Ethernet solutions. The first-generation COUPE aims to achieve high bandwidth and improve power efficiency, both critical issues in modern data centers.
2. 2026: TSMC’s second-generation silicon photonics products plan to integrate COUPE into CoWoS packaging, achieving co-packaging of switch chips and optical devices. This will enable motherboard-level optical interconnects at speeds up to 6.4 Tbps. The second-generation power consumption is expected to be less than 50% of the first generation, with latency projected to be less than 10% of the first generation.
3. Third Generation: The goal is to integrate COUPE into processor packaging, with COUPE operating on CoWoS Interposer, targeting a transmission rate of 12.8 Tbps while bringing optical connections closer to the processor. This stage is still in the exploratory phase with no clear release date, but TSMC aims to further reduce power consumption and latency.

AI Era and High-Speed Switch Demand Growth

The AI era is driving the demand for high-speed switches, highlighting the advantages of CPO solutions. With optical interconnects increasingly penetrating racks and computing systems, switches—central network devices in optical communication networks—are evolving towards high speed, multiple ports, and low power consumption to meet the growing requirements of AI clusters. The AI era presents a significant market opportunity for CPO solution switches.

The bandwidth development of AI accelerates the evolution of interconnect speeds. Since 2019, the global data center industry has entered the computing power center stage. According to Cisco data, global data center network switching bandwidth increased 80-fold from 2010 to 2022. The rapid development of AIGC (Artificial Intelligence Generated Content) has further driven network architecture upgrades and GPU acceleration iterations, increasing the demand for higher bandwidth between devices. As 2023 marked the inaugural year of AI, interconnect speeds doubled in half the time. The evolution of data center switch chips is now in a phase of doubling capacity every two years, with expectations to reach 102.4T capacity by 2025, corresponding to 1.6T optical ports.