MIT's chip stacking breakthrough could cut energy use in power-hungry AI processes

Engineers from MIT say that stacking circuit components on top of each other could be the answer to creating more energy-efficient artificial intelligence (AI) chips. The logic and memory components, which respectively perform computations and store data, can transfer data more easily when in direct contact as opposed to when apart.

The team created a so-called "memory transistor" comprising both a logic element that can perform computations (the transistor) and a memory element. This nanoscale device has relatively few electrical defects, meaning it can operate more quickly while using less electricity, the scientists said in two studies presented Dec. 9 and Dec. 10 at the International Electron Devices Meeting in San Francisco.

The breakthrough is particularly relevant for energy-intensive applications like AI, deep learning and computer vision. According to the International Energy Agency (IEA), global electricity consumption by data centers is projected to rise by about 130% to reach around 945 Terawatt-hours by 2030, largely due to a growing dependence on AI.

Just a single interaction with ChatGPT can generate enough heat that you need the equivalent of a bottle of water for cooling. But most of the energy associated with AI is used for shuttling data between components rather than performing computations. Even just a small saving on-chip could have a huge impact, the scientists believe.

"We have to minimize the amount of energy we use for AI and other data-centric computation in the future because it is simply not sustainable,” lead author of the study Yanjie Shao, a postdoctoral researcher at MIT, said in a statement. "We will need new technology like this integration platform to continue that progress."

Stacking saves energy — but it's not easy

Modern chips contain logic circuits made of transistors; these are on/off switches that control the flow of current. These transistors combine to represent binary 1s and 0s, which is how chips process information. They also have memory circuits, containing transistors alongside other materials that can store the data.

Logic and memory circuits are traditionally kept separate, and data must travel between them through wires and interconnects, wasting energy in the process. While stacking the active components may seem an obvious solution, the challenge lies in doing so without causing damage. Deposition, the controlled formation of ultrathin layers that form these components, needs to be done at low temperatures, for example, because some transistors cannot withstand heat.

To overcome this issue, the scientists built their logic transistor with an active channel layer (the region where electricity flows) made from indium oxide. Crucially, the material can be deposited in a two-nanometer layer at around 302 degrees Fahrenheit (150 degrees Celsius). This is a temperature low enough not to affect other transistors.

Beyond the indium oxide transistor, the scientists vertically stacked a memory component — a 10-nanometer layer of ferroelectric hafnium-zirconium-oxide — that allows the device to store data as well as process it. The resulting memory transistor can switch on or off in just 10 nanoseconds and operates at less than 1.8 volts. The switching speeds of typical ferroelectric memory transistors tend to be orders of magnitude lower, and require voltages between 3 and 4V.

The memory transistor is made even more efficient by being built on the chip’s "back-end," where the wires and metal bonds that connect the front-end’s active components are found. Shao said that doing this makes the integration density of the chip much higher.

For the two studies, the memory transistor was only installed on a chip-like structure rather than in a functional circuit. The team hopes to improve the transistor’s performance such that it can be integrated first into a single circuit, and then into larger electronic systems.

"Now, we can build a platform of versatile electronics on the back end of a chip that enable us to achieve high energy efficiency and many different functionalities in very small devices," Shao said. "We have a good device architecture and material to work with, but we need to keep innovating to uncover the ultimate performance limits."