The Critical Role of Ethernet in AI Networks

The rapid advancement of artificial intelligence (AI) technology is revolutionizing the cloud computing and IT industries. Since the launch of Chat GPT in November 2022, the AI field has experienced an investment boom, attracting significant attention. Major cloud service providers have introduced new products and services to meet the growing demand for AI, while many large enterprises are actively exploring AI use cases such as generative AI (GenAI) to enhance operational efficiency and return on investment.

However, the rapid development of AI presents higher demands on the infrastructure of cloud service providers and enterprise data centers. Data, as the critical “fuel” for AI development, must be collected, protected, and transmitted efficiently. Organizations exploring new AI applications must address these challenges. To support the massive data and computational resources required by AI, we need to build more efficient and reliable network infrastructures.

In this context, Ethernet technology, with its mature and widespread ecosystem, is becoming a crucial support for AI network infrastructure. Ethernet shows strong potential to meet the high demands of AI and provide a unified platform, which significantly impacts the economic viability of AI. It can achieve consistent operational models across various networks and clouds, avoiding the high costs associated with maintaining multiple infrastructures.

Key Requirements for AI Network Development

• Speed: The rapid growth of AI services drives the need for higher speeds in data centers and edge networks, pushing networks towards new generations like 400 Gbit/s and even 800 Gbit/s.
• Privacy and Security: Networks must efficiently handle data while ensuring high-end encryption and security in multi-tenant environments to protect data privacy.
• Edge Inference: As enterprises deploy large language models (LLMs) or small language models (SLMs) and hybrid private AI clouds, the front-end deployment of inference capabilities will become a focal point.
• Short Job Completion Time (JCT) and Low Latency: Optimizing networks to provide lossless transmission, ensuring efficient bandwidth utilization through congestion management and load balancing, is key to achieving rapid JCT.
• Flexible Clusters: In AI data centers, processor clusters can be configured into various topologies. Optimizing performance requires avoiding oversubscription between layers or regions to reduce JCT.
• Multi-Tenant Support: For security reasons, AI networks need to separate data flows.
• Standardized Architecture: AI networks typically consist of back-end infrastructure (training) and front-end (inference). The generality of Ethernet allows for technical reuse between back-end and front-end clusters.

Continuous Innovation in Ethernet Technology

Ethernet technology is continuously innovating and developing to meet the higher demands on network scale by AI. Some key technological advancements include:

• Packet Spraying: This technology allows each network flow to access all paths to the destination simultaneously. The flexible ordering of packets fully utilizes all Ethernet links with optimal load balancing, enforcing ordering only when bandwidth-intensive operations require it in AI workloads.
• Congestion Management: Ethernet-based congestion control algorithms are crucial for AI workloads. They prevent hotspots and evenly distribute load across multiple paths, ensuring reliable transmission of AI traffic.

Unified and Optimized Enterprise Infrastructure

Enterprises need to deploy unified AI network infrastructure and operational models to reduce the cost of AI services and applications. Adopting standard-based Ethernet as the supporting technology is a core element. It ensures compatibility between front-end and back-end systems, avoiding the standardization process obstacles and economic impacts brought by different architectures. For example, Arista advocates building an “AI Center,” where GPUs are efficiently trained through lossless networks. The trained AI models are connected to AI inference clusters, allowing end users to query these models conveniently.

Market Advantages of Ethernet

Ethernet exhibits strong competitiveness in AI deployment due to its openness, flexibility, and adaptability. Its performance surpasses InfiniBand, and with the enhancements from the Ultra Ethernet Consortium (UEC), its advantages will further expand. Moreover, Ethernet is more cost-effective, has a broader and more open ecosystem, providing generality, unified operations, and skill sets for both back-end and front-end clusters, as well as platform reuse opportunities between clusters. As AI use cases and services continue to expand, the opportunities for Ethernet infrastructure will significantly increase, whether at the core of hyperscale LLMs or at the enterprise edge. AI-ready Ethernet can meet the demand and provide AI inference based on industry-specific private data.

In summary, Ethernet technology plays a critical role in AI network infrastructure. It can meet the multifaceted needs of AI in terms of speed, security, edge inference, and more. Through continuous technological innovation and extensive ecosystem support, Ethernet provides more efficient and cost-effective solutions for enterprises, promoting the widespread application and development of AI.