The new optoelectronic co-packaging technology may replace electrical interconnect devices in datacenters, significantly increasing the speed and energy efficiency of AI and Other computing applications.

Peking, December 12, 2024 /PR Newswire/ -- Recently, IBM (NYSE: IBM) announced its groundbreaking research results in optical technology, which are expected to significantly improve the efficiency of training and running generative AI models in datacenters. The new generation of optoelectronic co-packaged (co-packaged optics, CPO) process developed by IBM researchers achieves light-speed connectivity inside datacenters through optical technology, providing a strong complement to existing short-distance optical cables. By designing and assembling the first successfully announced polymer waveguide (PWG), IBM researchers demonstrated how optoelectronic co-packaging technology will redefine high-bandwidth data transmission between chips, circuit boards, and servers in the computing industry.

Today, fiber optic technology has been widely used for high-speed long-distance data transmission, achieving "light instead of electricity" to manage almost all Global commercial and communication transmissions. While the external communication networks of datacenters have adopted fiber optics, their internal racks still primarily use Copper wires for communication. GPU accelerators connected by wires may be idle more than half the time, needing to wait for data from other devices during large distributed training processes, leading to high costs and energy waste.

IBM researchers have discovered a new method to introduce optical speed and capacity into Datacenters. In a recently published paper, IBM showcased its first-ever prototype of optoelectronic co-packaging that enables high-speed optical connections globally. This technology can significantly enhance the communication bandwidth of Datacenters, minimize GPU downtime, and greatly accelerate the speed of AI operations. This innovation will achieve the following new breakthroughs:

• Reducing the cost of scaling generative AI applications: Compared to medium-distance electrical interconnects, energy consumption is reduced by over 5 times, while extending the length of Datacenter interconnect cables from 1 meter to several hundred meters.
• Increasing the training speed of AI models: Using optoelectronic co-packaging technology, the training speed of large language models is nearly five times faster than traditional wiring, reducing the training time of a standard large language model from three months to three weeks; for larger models and more GPUs, performance will be further enhanced.
• Significantly improving the energy efficiency of Datacenters: With the latest optoelectronic co-packaging technology, the electricity saved for training an AI model is equivalent to the total annual electricity consumption of 5,000 American households.

Dario Gil, Senior Vice President of IBM and Director of IBM Research, stated: "Generative AI requires increasing amounts of Energy and processing power, and Datacenters must upgrade accordingly. Optical Co-Packaging technology can help Datacenters confidently face the future. With breakthroughs in optical co-packaging technology, fiber optic cables will greatly enhance the data transmission efficiency of Datacenters, enabling more efficient communication between chips and handling of AI workloads, ushering in a new era of faster and more sustainable communication."

80 times faster than the current communication bandwidth between chips.
Thanks to recent advancements in chip technology, more and denser transistors can be accommodated on chips; for example, IBM's 2-nanometer chip technology can implant over 50 billion transistors on a single chip. The optical co-packaging technology aims to increase the interconnect density between accelerators, helping chip manufacturers to add optical pathways for Connection Chips on electronic modules, thus surpassing the limitations of existing electronic pathways. The new high bandwidth density optical structure described in IBM's paper, along with other innovative achievements, such as transmitting multiple wavelengths through each optical channel, is expected to increase the communication bandwidth between chips to 80 times that of wire connections.

Compared to the current state-of-the-art optical co-packaging technology, IBM's innovative achievements allow chip manufacturers to add six times the number of optical fibers at the edges of silicon photonic chips, a phenomenon referred to as "beachfront density." Each optical fiber is about three times the width of a human hair, with lengths ranging from a few centimeters to several hundred meters, capable of transmitting data at a trillion bits per second. The IBM team made use of standard packaging processes, packaging high-density Polymer Waveguides (PWG) on optical pathways with a 50-micron pitch and optically coupling them with silicon photonic waveguides.

The paper also points out that the aforementioned optoelectronic co-packaged modules use polymer optical waveguides with a 50-micron pitch, which have successfully passed all the pressure tests required for manufacturing for the first time. These modules need to withstand high humidity environments, temperatures ranging from -40°C to 125°C, and mechanical durability tests to ensure that the optical interconnect devices do not break or lose data even when bent. Additionally, researchers demonstrated a polymer optical waveguide technology with an 18-micron pitch: by stacking four polymer optical waveguide devices together, it can achieve connections of up to 128 channels.

IBM continues to lead semiconductor technology research and development.
Faced with the growing demand for AI performance, optoelectronic co-packaging technology has opened a new communication pathway and may replace external communication from electronic to optical modules. This technological breakthrough continues IBM's leadership in semiconductor innovation, including the world's first 2-nanometer chip technology, the first 7-nanometer and 5-nanometer process technologies, nanosheet transistors, vertical transistors (VTFET), single-chip DRAM, and chemical amplified photoresists.

The design, modeling, and simulation work for this project were completed in Albany, New York, USA, while the prototype assembly and module testing were undertaken by the IBM laboratory located in Bromont, Quebec, Canada, which is one of the largest chip assembly and testing bases in North America.

About IBM
IBM is a Global leader in Hybrid Cloud, AI and Enterprise Services, helping customers in over 175 countries and regions gain business insights from their data, simplify business processes, reduce costs, and gain competitive advantages in their Industry. More than 4,000 government and enterprise entities in critical infrastructure areas such as Financial Services, Telecommunications, and Healthcare rely on the IBM Hybrid Cloud platform and Red Hat OpenShift for rapid, efficient, and secure digital transformation. IBM's groundbreaking innovations in AI, Quantum Computing, Industry Cloud Solutions, and Enterprise Services provide our customers with open and flexible choices. A long-term commitment to corporate integrity, transparent governance, social responsibility, an inclusive culture, and a service spirit is the foundation of IBM's Business development. For more information, please visit:

Media Contact
Cui Shoufeng, shou.feng.cui@ibm.com