Online Reconfiguration of Convolutional Neural Networks for Onboard Vision Applications
Creators: name: MAZOUZ, ALAA EDDINE
Publisher: name: University of Surrey
Year: 2022

description: The future of space exploration relies on developing novel systems for autonomous operations and
description: onboard data handling. The use of deep learning can provide techniques and models capable of solving
description: both existing and foreseen challenges identified in autonomous space applications. The existing
description: research on deep learning has seen a strong focus on supervised learning performance in terms of
description: accuracy, but current trends have shifted the attention towards speed and efficiency to meet the
description: requirements of embedded systems. This meant a need for conceptual changes to the existing techniques
description: and frameworks. In this context of computational efficiency, two recent branches of work can be
description: distinguished. One tries to eliminate redundancy in data and computation through compression and
description: optimization techniques. The other line exploits the fact that not all tasks require the same amount of
description: computation and explores selective and task-dependent execution of deep neural networks—however,
description: these two lines of work focus on offline reconfiguration and optimization.<br>
description: <br>
description: This research falls within the second category but aims to provide hardware engineers with the tools
description: to design, model and deploy onboard convolutional neural networks. Secondly, create reconfigurable
description: models and reconfiguration policies for runtime applications on embedded systems informed by the
description: requirements of the space domain. The aim is to develop tools capable of generating Field
description: Programmable Gate Array (FPGA) models of deep neural networks and explore introducing real-time
description: dynamic changes to the network during runtime to satisfy the requirements of autonomy and overcome
description: the risks of deploying static systems within harsh uncooperative environments.<br>
description: <br>
description: A new automated tool for generating FPGA models from high level convolutional neural networks
description: is proposed. The tool is accessible, easy to use with a model-based, modular design approach and thus
description: requires no in-depth knowledge of FPGA design. The generated hardware models are accompanied by
description: performance estimation and design exploration steps deemed necessary from an exhaustive review of
description: the literature on CNN to FPGA compilers. The Offline Design Exploration (ODE) is automatically
description: carried out using an analytical model, this only requires running a few scripts dedicated to performance
description: estimation and a Multi-Objective Optimization algorithm, this process allows for finding the most
description: efficient and optimized configurations of a model based on hardware and user constraints. <br>
description: <br>
description: Beyond the ability to quickly generate multiple hardware models, the need for runtime adaptivity is
description: tackled with a novel runtime reconfiguration methodology. Online Design Reconfiguration (ODR)
description: allows for attaining performance trade-offs during runtime with minimal resource overhead. Latency
description: and power trade-offs can be attained quickly and easily at runtime. To achieve this, a novel expandable
description: training technique for runtime reconfigurable and adaptive CNN models is proposed and tested. Training results in a unified model comprised of deployable “sub-network” IP cores that perform the
description: same task and share parameters but trade-off accuracy for speed and power consumption, this can be
description: done according to changing mission or environmental requirements. <br>
description: <br>
description: Finally, to further consolidate the runtime adaptivity aspect, the automated compiler is significantly
description: expanded to include backpropagation and the ability to train convolutional neural networks online. A
description: novel pipeline architecture can perform backpropagation directly on FPGA while reusing most of the
description: forward pass pipeline to minimize resource overhead, enabling online learning on FPGAs. The
description: streaming and pipelined architectures facilitate online and autonomous deployment, areas that are
description: overlooked in the literature. This is finally tested using a custom synthetic dataset, allowing us to
description: provide new insights on the feasibility of implementing Online Learning on FPGAs for autonomous
description: vision applications, especially for close proximity satellite applications.<br>
description: <br>
description: The generated designs achieved a 95x, 71x, and 18x throughput trade-off with resources for MNIST,
description: CIFAR-10, and SVHN architectures, respectively. In resource utilization, in terms of DSP Slices, the
description: proposed workflow achieved trade-offs of 44x for MNIST, 52x for SVHN, and 24x for CIFAR-10.
description: These trade-offs will allow designers to tailor implementations to their specific constraints and
description: objectives. The proposed Online Design Reconfiguration (ODR) policies achieved reductions in power
description: up to 25%, 28%, and 32% with 13x, 14x, and 50x gains in latency for a 0.7%, 2%, and 4% accuracy
description: loss in the MNIST, SVHN, and CIFAR-10 implementations respectively. When online learning
description: scenarios were tested, the FPGA pipeline’s performance was comparable to that of the GPU, with the
description: distinct advantage when it comes to power consumption, and x2.8, x5.8, and x3 speed up over GPU
description: was achieved on three architectures trained on MNIST, SVHN, and CIFAR-10 respectively.<br>