Implementation & analysys of neural networks on microcontrollers

Abstract

In this work, various neural network architectures such as convolutional neural network (CNN), deep neural networks (DNN), depth wise separable neural network (DS-CNN), have been deployed on different microcontrollers with Cortex-M cores to benchmark these microcontroller devices for different tasks such as vision sensing, keyword detection and gesture recognition applications. The parameters of interest here are the neural network architecture, latency, memory footprint, and the power consumption. It is observed that neural network architectures which use depth wise separable convolution operations (DS-CNN and MobileNet) perform better in terms of latency (4.5 in certain cases) and consume less power but have a higher memory footprint (1.1 to 7.3 ). It is also seen that an architecture with smaller memory footprint such as the CNN, does not guarantee faster response times. The results indicate that latency of an architecture is highly dependent on the number of operations each layer performs and not on the total number of operations. Using these observations, an analogy to the method of logical effort is proposed for optimizing the neural network architectures to achieve better latency on microcontroller devices. It is shown that energy savings of 85 Joules can be achieved on the edge by deploying microcontrollers as always-on devices for a vision sensing task.