REDWOOD CITY, Calif.--(BUSINESS WIRE)--Using algorithms derived from its neuroscience research, Numenta announced today it has achieved dramatic performance improvements on inference tasks in deep learning networks without any loss in accuracy.
Today’s deep learning networks have accomplished a great deal but are running into fundamental limitations, including their need for enormous compute power. A large, complex model can cost millions of dollars to train and to run, and the power required is growing at an exponential rate. New algorithms are essential to break through this performance bottleneck.
The advances demonstrated by Numenta are achieved by applying a principle of the brain called sparsity. Numenta compared sparse and dense networks by running its algorithms on Xilinx FPGAs (Field Programmable Gate Array) for a speech recognition task using the Google Speech Commands (GSC) dataset. Using the metric of number of words processed per second, the results show that sparse networks yield more than 50x acceleration over dense networks on a Xilinx Alveo board. In addition, Numenta demonstrated the GSC network running on a Xilinx Zynq chip, a smaller chip where dense networks are too large to run, enabling a new set of applications that rely on low-cost, low-power solutions. Using the metric of number of words per second per watt, we show that the sparse networks use significantly less power than the most efficient dense network.
This proof-of-concept demonstration validates that sparsity can achieve significant acceleration and power efficiencies for a variety of deep learning platforms and network configurations, while maintaining competitive accuracy. This approach could enable machine learning technologists to:
- Implement larger and more complex networks using the same resources.
- Implement more network copies on the same resources.
- Implement deep learning networks on edge platforms where resource constraints prevent dense networks from running.
- Achieve large energy savings and lower costs due to scaling efficiencies.
Technical details can be found in a white paper that Numenta issued here today. Numenta is working with strategic partners to commercialize this technology. For information, contact the company at sparse@numenta.com.
The Future of AI is Based on Brain Principles
“New algorithmic-hardware approaches are required to advance machine intelligence,” said Priyadarshini Panda, Assistant Professor at Yale University in Electrical Engineering. “The brain offers the best guide for achieving these advances in the future. The results announced by Numenta demonstrate great promise by applying its cortical theory to achieve significant performance improvements.”
“Sparsity is foundational to how the brain works and offers the key to unlocking tremendous performance improvements in machine learning today,” said Subutai Ahmad, Numenta’s VP of Research and Engineering. “Going forward, Numenta’s neuroscience research has generated a roadmap for building machine intelligence which will yield equally exciting improvements in robustness, continual learning, unsupervised learning and sensorimotor integration.”
About Numenta
Numenta scientists and engineers are working on one of humanity’s grand challenges: understanding how the brain works. The company has a dual mission: to reverse-engineer the neocortex and to enable machine intelligence technology based on cortical theory. Co-founded in 2005 by technologist and scientist Jeff Hawkins, Numenta has developed a theory of cortical function, called the Thousand Brains Theory of Intelligence. Hawkins’ upcoming book, A Thousand Brains, details the discoveries that led to the theory and its potential impact on Artificial Intelligence. The team tests their theories via simulation, mathematical analysis, and academic and industry collaborations. Numenta publishes its progress in open access scientific journals, maintains an open source project, and licenses its intellectual property and technology for commercial purposes.
Numenta, Google, Xilinx, Alveo and Zynq are trademarks of their respective owners.
