Scientists say they've eliminated a major AI bottleneck — now they can process calculations 'at the speed of light'

Scientists have developed a foundational architecture for next-generation optical computing — using light rather than electricity to power chips — that could revolutionize how artificial intelligence (AI) models are trained and executed.

At the heart of large language models (LLMs) and those based on deep learning lies a weighted organization structure called a “tensor” that works like a filing cabinet with sticky notes indicating which drawers are the most used.

When an AI model is trained to perform a task or function, such as recognizing an image or predicting a text string, it sorts the data into these tensors. In modern AI systems, the speed at which models can process tensor data — or sort through the filing cabinets — is a fundamental performance bottleneck that represents a hard limit on how large a model can become.

In typical light-based computing, models parse tensors by firing laser arrays multiple times. They function like a machine that scans a barcode on a package to determine its contents, except that in this case, each container references a math problem. The amount of processing power it takes to crunch these numbers scales with the models' inherent capabilities.

Although light-based computing is faster and more energy efficient at smaller scales, most optical systems can’t be run in parallel. Unlike graphical processing units (GPUs), which can be chained together to exponentially increase the amount and availability of processing power, light-based systems are typically run linearly. Because of this, most developers snub optical computing in favor of the parallel processing advantages of increased power at scale.

This scaling bottleneck is why the most powerful models made by the likes of OpenAI, Anthropic, Google, and xAI require thousands of GPUs running in tandem to train and operate.

But the new architecture, called Parallel Optical Matrix-Matrix Multiplication (POMMM), could negate the problem that's been holding optical computing back. Unlike previous optical methods, it conducts multiple tensor operations simultaneously using a single laser burst.

The result is a foundational AI hardware design with the potential to scale the tensor processing speed of a given AI system beyond state-of-the-art electronic hardware capabilities while reducing its energy footprint.

Next-generation optical computing and AI hardware

The study, published Nov. 14, in the journal Nature Photonics, details the results of an experimental optical computing prototype along with a series of comparative tests against standard optical and GPU processing schemes.

The scientists used a specific arrangement of conventional optical hardware components alongside a novel encoding and processing method to capture and parse tensor packages in a single laser shot.

They managed to encode digital data into the amplitude and phase of light waves, turning data into physical properties in the optical field — with these light waves combining to carry out mathematical operations such as matrix or tensor multiplications.

These optical operations don’t require additional power to process in this paradigm because they occur passively as the light propagates. This eliminates the need for control or switching during processing, as well as the power required to perform those functions.

"This approach can be implemented on almost any optical platform," lead author of the study, Zhipei Sun, leader of Aalto University’s Photonics Group, said in a statement. "In the future, we plan to integrate this computational framework directly onto photonic chips, enabling light-based processors to perform complex AI tasks with extremely low power consumption."

Zhang estimates the approach could be integrated into major AI platforms within three to five years.

An artificial general intelligence accelerator

Representatives described this as a step towards next-generation Artificial General Intelligence (AGI) — a hypothetical future AI system that's smarter than humans and can learn generally across multiple disciplines, independent of its training data.

Zhang added in the statement: "This will create a new generation of optical computing systems, significantly accelerating complex AI tasks across a myriad of fields."

While the paper itself doesn’t specifically mention AGI, it does refer to general-purpose computing several times.

The notion that scaling current AI development techniques is a viable path toward achieving AGI is so pervasive among certain sectors of the computer science community that you can buy t-shirts proclaiming that "scaling is all you need."

Other scientists, such as Meta's outgoing chief AI scientist Yann LeCun, disagree, saying that LLMs — the current gold standard AI architecture — will never reach AGI status regardless of how far and deeply they scale.

With POMMM, the scientists say they may have a critical piece of the hardware puzzle needed to remove one of the field's largest bottlenecks, allowing developers to scale well beyond the current paradigm’s foundational limits.