Efficient Execution of Convolutional Neural Networks on Low Powered Heterogeneous Systems

Access to files

•   FULL-TEXT.PDF (pdf)

Abstract

Energy-efficient machine learning has been gaining interest due to the increase use of of machine learning, in particular deep learning, in applications that run on mobile and embedded devices. These devices are constrained in terms of resources in computation, memory and power, which limit the adoption of deep learning-based solutions, which are known to be power hungry. Research efforts that focus on reducing the energy consumption of machine learning is often referred to as Machine Learning on the Edge, the area of research to which the work in this thesis contributes. In this thesis, we identity and address three main issues related to enabling machine learning on the edge: the lack of software support to procure energy measurements, the lack of experimental evaluations based on energy use, and finally, the need for tools to rapidly explore neural network implementations in the context of emerging hardware. To address the first two issues, we first present, SyNERGY, a framework integrated in current deep learning software frameworks, that allows researchers to evaluate deep learning models and their optimizations on the metrics of both execution time and energy use on existing mobile platforms at different levels of granularity. To address the last issue, to explore efficient deep neural network implementations and hardware designs, a second tool, NNTaskSim, is proposed which supports the expression of neural network computations in a task-parallel framework and can be used to explore the execution of the resulting task graphs in the context of emerging hardware designs. The result of using SyNERGY is empirically gathered energy use and execution time data of existing deep learning models on current mobile devices. Based on the experimental data gathered, new, relatively low cost and accurate predictive models are explored and provided to estimate the energy use of new deep learning models. The experimental evaluation based on NNTaskSim shows that rapid exploration of neural network task graph executions can be done for prototype hardware, evaluated on the metrics of time and memory use, to reveal insights into the software and hardware design choices that lead to efficient solutions for deep learning systems.

Additional content not available electronically

Github Repository : https://github.com/Crefeda/SyNERGY

Keyword(s)

CNN; ConvNet; Convolutional Neural Networks; Dataflow mapping; Deep Learning; Energy; Energy efficiency; Energy prediction; Jetson TX1; Low power; Mobile Systems; Neural Architecture Search; Neural Network Accelerators; Performance; Power; Power estimation; Power measurement; Snapdragon 820; Task graph

Bibliographic metadata

Institutional metadata

Record metadata