Making the shift from GPUs to ‘brainier’ computing in edge AI

Graphics processing units (GPUs) came up initially to make video gaming look more realistic. Now, they are the drivers of artificial intelligence (AI) as they process massive amounts of data that go into training AI algorithms.

GPUs are great for tasks that can be broken up into multiple parts and processed in parallel. If you think of the central processing (CPU) of your laptop as its ‘brain’, the GPU is like a swarm of tiny, specialized ‘brains’.

Chipmakers are cranking up their GPUs to keep up with the exploding demand for AI in everything from chatbots to the computer vision of guided missiles. Industry leader Nvidia reported $5 billion revenue in the last quarter.

Amid the heady commercial success of GPU makers, it is hard to make a business case for a new approach. But researchers believe traditional binary computing will hit physical limits, especially in the energy efficiency required for edge AI use cases such as the computer vision of autonomous vehicles or the industrial IoT (internet of things) of predictive maintenance. The energy guzzling of GPUs is also a worry as the world grapples with carbon emissions.

Hence the search for an alternative to the traditional von Neumann architecture where processing and storage are separate and involve frequent shuttling of data.

One approach is neuromorphic computing that draws inspiration from the human brain where memory and logic are fused. That’s how our brains can perform with a tiny fraction of the energy required in computers.

Breaking barriers

“What we are doing is to break down those boundaries between computing and storage. If the two are merged and every storage element begins to compute, what you get is much more capability in the whole system,” says Manan Suri, associate professor at IIT Delhi, who has been working on this for over a decade, right from his PhD thesis at the Institut National Polytechnique de Grenoble, France.

Massachusetts Institute of Technology (MIT) also named Suri in its annual list of the world’s top 35 innovators under the age of 35 as well as among the top 10 innovators in India.

The recognition from MIT came for Suri’s team turning a challenge in nano-electronics into an advantage. The challenge lies in memory devices behaving in unexpected ways as they are scaled down from macroscopic to nanoscale objects to keep up with Moore’s law, which postulated that the number of transistors on a chip would double every two years. So, a lot of effort, money and quality control go into ensuring that even a billion transistors on a chip will behave in a deterministic way, and not randomly.

But a human brain functions differently. There it would be inefficient to keep such tight control over the trillions of neurons and synapses, many of which die and get replaced each day. “It turns out that there are big margins for error in the behaviour of the neurons and synapses. So, the computation that is happening in nature is fundamentally tolerant,” says Suri.

On the other hand, a von Neumann architecture is absolutely deterministic, where two plus two in a processor always produces four. “Logic gates and operants have to precisely behave in the way they are supposed to. But this is not how natural systems work on real-life stimuli. So, if we are able to show that similar probabilistic or approximate computation can be done with nanoscale devices, then we would be able to make the case for future neuromorphic computing,” says Suri. The constant battle to control the behaviour of the building blocks of chips would then no longer be a disadvantage; it would instead become a part of the solution.

“We modified the learning rules or neural network algorithms to compute even with probabilities. So, the variability of ultra-scaled devices and circuits that people were struggling to control became an advantage here,” adds Suri.

New possibilities

It’s too early yet for nanoscale neuromorphic computing to come out of labs into commercial products, but AI startups have an eye out for new possibilities. One of them is Bengaluru industrial IoT startup Flutura. “In edge intelligence, what we are trying to do is see patterns in sensor data and respond to them without going to a central brain. For the processing workloads we have, the current GPUs and even the ones that are coming may not be good enough,” says co-founder Derick Jose.

The catch is that for any new computing architecture to be benchmarked against GPUs, chipmakers have to buy into futuristic nanoscale products. That’s a tough proposition to sell, given the current boom in GPUs.

“While the neuromorphic community is showing the functionality and accuracy of this form of computing, the conventional guys have solutions out there in the market. And in the last few years, their progress has been skyrocketing. You have a new benchmark to beat every day. The disadvantage is that an R&D space is competing with qualified products,” says Suri.

Sumit Chakraberty is a consulting editor with Mint.