LightTrends Newsletter

LightCounting releases December 2022 report on High-Speed Cables, AOCs, EOM and CPO.

Increasing sales of AOCs and copper cables, illustrated in the figure below, are driven by the use of these cables as server connections and as interconnects in disaggregated switches and routers. Many lower speed (25G and below) cables are now also used in industrial and even consumer applications, but these market segments are not included in this report.

Market growth will be interrupted in 2023 because of an economic slowdown, but it will resume in 2024 with an estimated 14% and 25% CAGR from 2023 through 2027 for AOCs and Copper Cables, respectively. The emergence of Active Electronic Cables (AECs) will boost the growth rate for sales of copper cables, at the expense of AOCs. Adoption of Co-Packaged Optics (CPO) will also contribute to lower growth in sales of AOCs by 2027.

The majority of the high-speed cables are used in High Performance Computers (HPCs) and AI Clusters. The HPC market has seen optical speeds double with each successive generation, with ever faster adoption of each successive cycle. Mellanox’s InfiniBand HDR for 200G got off to a late start but since mid-2019 has been adopted more quickly than any previous rate.  Nvidia announced 400G InfiniBand in November 2021 and the next generation may not be far off. Nvidia’s latest GPU systems are designed for 800G optical transceivers, AOCs and copper cables to extend the reach on NVLinks – a proprietary interconnect technology, developed for scaling up GPU arrays. Nvidia expects that NVLink connections will lead the adoption of higher data rate connectivity, including 200G SerDes.

At the latest OCP summit, Craig Thompson of Nvidia presented a compelling argument for a 32x increase in the bandwidth of network connectivity needed in AI clusters. He also pointed out that achieving this goal with the current designs of pluggable optical transceivers is not realistic: it would double the cost of the whole system and add another 20-25% to the power consumption. Thompson emphasized that new designs of lasers and modulators are needed to enable reductions in cost and power of optical connectivity in AI clusters. CPO can potentially reduce the power consumption by 50%, but an additional 10x improvement is necessary to bring more optical connectivity into AI systems.

Access to Memory is another bottleneck for AI Clusters and HPCs. Disaggregating memory promises to solve this issue but the challenge is the connectivity needed - 100x the capacity compared to what is used today - and it needs to be power efficient. This is where CPO looks so promising. Indeed, it is seen as the only way to provide such vast, power-efficient connectivity.

We expect that shipments of CPO ports will increase from 50,000 in 2023 to 4.5 million by 2027. For comparison, annual shipments on AOCs and copper cables are projected to reach 10 million and 20 million units by 2027, respectively, excluding cables for industrial and consumer applications.

The Cloud companies are prioritizing investments into data centers and AI Clusters even during the current economic slowdown. For example, Amazon reduced their spending on the order fulfilment infrastructure by $10 billion and reallocated these funds to construction of datacenters in 2022.

There is a lot of speculation on how deep and long the current economic slowdown will be, but no one really knows. All the companies, including the largest Cloud vendors with huge financial resources, have to plan carefully for 2023. Once this uncertainly clears out, many markets will return to double-digit growth rates, including the ones covered in this report. We currently expect this to happen in 2024, but is the slowdown extends into another year, the industry will have to catch up even faster once the slowdown is over.