The next generation of neural networks could live in hardware

However, once the network is up and running, things become much, much cheaper. Petersen compared his logic gate networks to a cohort of other ultra-efficient networks, such as binary neural networks, which use simplified perceptrons that can only process binary values. Logic gate networks performed as well as these other effective methods in classifying images in the CIFAR-10 dataset, which includes 10 different categories of low-resolution images, from “frog” to “truck.” He achieved this with less than one-tenth the logic gates required by those other methods and in less than a thousandth of the time. Petersen tested his networks using programmable computer chips called FPGAs, which can be used to emulate many different potential logic gate patterns; implementing networks in non-programmable ASIC chips would further reduce costs, as programmable chips must use more components to achieve their flexibility.

Farinaz Koushanfar, a professor of electrical and computer engineering at the University of California, San Diego, says she is not convinced that logic gate networks will be able to work when faced with more realistic problems. “It’s a cute idea, but I’m not sure how well it scales,” she says. She notes that logic gate networks can only be trained approximately, via a relaxation strategy, and approximations can fail. This hasn’t caused any problems yet, but Koushanfar says it could prove increasingly problematic as networks grow.

Even so, Petersen is ambitious. It plans to continue developing the capabilities of its logic gate networks and hopes to eventually create what it calls a “hardware foundation model.” A powerful network of general-purpose logic gates for vision could be mass-produced directly on computer chips, and these chips could be integrated into devices such as personal phones and computers. This could bring huge energy benefits, says Petersen. If these networks could effectively reconstruct photos and videos from low-resolution information, for example, then much less data would need to be sent between servers and personal devices.

Petersen admits that logic gate networks will never compete with traditional neural networks in terms of performance, but that is not his goal. It should be enough to make something that works and is as efficient as possible. “It won’t be the best model,” he says. “But it should be the cheapest.”