Kaushik Roy - 2020 Arden L. Bement Jr. Award

Kaushik Roy – 2020 Arden L. Bement Jr. Award

Arden L. Bement Jr. Distinguished Lecture

Neuromorphic computing and the design of underlying hardware can lead to quantum improvements in energy efficiency with good accuracy.

Biography

Kaushik Roy is the Edward G. Tiedemann, Jr., Distinguished Professor of Electrical and Computer Engineering at Purdue University and Director of the Center for Brain-Inspired Computing (C-BRIC). He received his PhD from University of Illinois at Urbana-Champaign in 1990 and joined the Semiconductor Process and Design Center of Texas Instruments, Dallas, where he worked for three years on FPGA architecture development and low-power circuit design. His current research focuses on algorithms, circuits and architecture for energy-efficient cognitive computing, computing models and neuromorphic devices. Roy has supervised more than 85 PhD dissertations, and his students are well-placed in universities and industry. He is the co-author of “Low Power CMOS VLSI Design,” both the first and second editions, published by John Wiley & McGraw Hill.

Roy has received a National Science Foundation Career Development Award, IBM Faculty Partnership Award, ATT/Lucent Foundation Award, Semiconductor Research Corporation Technical Excellence Award, SRC Inventors Award, Purdue College of Engineering Research Excellence Award, Humboldt Research Award, IEEE Circuits and Systems Society Technical Achievement Award (Charles Desoer Award), Distinguished Alumnus Award from the Indian Institute of Technology, and the Semiconductor Research Corporation Aristotle Award in 2015. He also has served as a Department of Defense Vannevar Bush Faculty Fellow; Global Foundries Visiting Chair at National University of Singapore and Fulbright-Nehru Distinguished Chair.

Reengineering Computing with Neuro-Inspired Learning

Abstract

Advances in machine learning have led to computers matching or surpassing human performance in several cognitive tasks including vision, speech and natural language processing. However, implementation of such neural algorithms in conventional von-Neumann architectures are orders of magnitude more inefficient in power than the biological brain. Hence, we need fundamentally new approaches to sustain the exponential growth in performance beyond the end of the CMOS technology roadmap. Exploring the new paradigm of computing necessitates a multidisciplinary approach: exploration of robust learning algorithms inspired from neuroscientific principles, development of network and hardware architectures best suited for such algorithms and the creation of nanoscale devices that can closely mimic the neuronal and synaptic operations of the brain leading to a better match between the hardware substrate and the model of computation. In this talk, Roy will focus on his recent works on neuromorphic computing and the design of underlying hardware that can lead to quantum improvements in energy efficiency with good accuracy.

Research Accomplishments

Roy's research is founded on the key insight that today's computing fabrics – based on the Silicon CMOS transistors and von Neumann computer architecture – are ill-matched to the building blocks (neurons and synapses) as well as the computing architecture of the brain. This mismatch is in large part responsible for the orders of magnitude energy gap observed between artificial and natural intelligence; machines such as Google's AlphaGo and IBM's Watson that have surpassed human competitors consume hundreds of thousands of watts while the human brain consumes only around 20 watts.

His research has pioneered a holistic algorithms-to-devices approach to bridging the efficiency gap between current AI systems and the brain by proposing devices that directly emulate the basic neuronal and synaptic operations, by designing new circuits and architectures that embody the key information processing principles of the brain and by creating algorithms that bridge these hardware fabrics to AI/ML applications.

• Neuromimetic devices and circuits: Roy's group realized that the intrinsic physics of some of the emerging devices, including spintronics, could naturally emulate neurons and synapses of different bio-fidelity, leading to highly compact and energy-efficient hardware implementations, well beyond the capabilities of standard CMOS circuits. This is due to two factors – the inherent match between the characteristics of these devices and the functionality of a neuron/synapse/biological functions (leading to a drastic decrease in the number of devices required) and the possibility of ultra-low voltage operation.

• Neuro-mimetic hardware architectures: Even though the GPUs were instrumental in the rapid development of AI algorithms, their limitations in training and inference are apparent from the orders of magnitude energy efficiency gap that exists between the computing architecture of the brain and the von Neumann machines. This is mainly due to the memory bottleneck – the computation and storage are separate, leading energy-consuming traffic to fetch data from memory, computing in the processing unit and storing the results back to memory. To address such bottlenecks, and taking cues from the brain, Roy and his students developed new memory circuits that can effectively do processing of data in the memory itself.

• Learning and inference algorithms: Roy's group has been instrumental in pioneering a new generation of bio-plausible spiking neural network (SNN) architectures and training algorithms to achieve state-of-the-art accuracy with superior energy efficiency compared to today's artificial neural networks. His group developed new ways of training deep spiking networks with different types of spike-based input coding. They were the first to demonstrate deep SNNs (VGG-16, ResNet-34) capable of achieving state-of-the-art accuracy on industry strength datasets: CIFAR10 and ImageNet.