Network-level design space exploration of resource-constrained Networks-of-Systems
Creators: name: Zhao, Zhuoran
name: 0000-0003-1603-2712
Publisher: name: The University of Texas at Austin
Year: 2019
Subjects: Host-compiled and source-level simulation
Virtual platform prototyping
Deep learning
IoT
Edge computing

description: Driven by recent advances in networking and computing technologies, distributed application scenarios are increasingly deployed on resource-constrained processing platforms. This includes networked embedded and cyber-physical systems as well as edge computing in mobile applications and the Internet of Things (IoT). In such resource-constrained Networks-of-Systems (NoS), computation and communication workloads need to be carefully co-optimized yet are tightly coupled. How to optimally partition and schedule application tasks among an appropriately designed NoS architecture requires a simultaneous consideration of design parameters from applications and processing platforms all the way to network configurations. Traditionally, however, systems and networks are designed in isolation and combined in an ad-hoc manner, which ignores joint effects and optimization opportunities. To systematically explore and optimize NoS design spaces, a higher level of design abstraction on top of traditional system and network design is required. In this thesis, we propose a novel network-level design methodology for resource-constrained NoS optimization and design space exploration. A key component in such a design flow is fast yet accurate network/system co-simulation to rapidly evaluate NoS parameters with high fidelity. We first introduce a novel NoS simulator (NoSSim) that integrates source-level simulation models of applications supporting a wide range of estimation metrics with a host-compiled system simulation platform and a reconfigurable network simulation backplane to accurately capture system and network interactions. In order to consider dynamic NoS application scenarios, we further introduce a novel lightweight middleware aimed at making task mapping and load balancing decisions dynamically at runtime. Our middleware employs a distributed, peer-to-peer work stealing approach and a novel work scheduling method to minimize data synchronization overhead and better exploit available parallelism and computation/communication overlap in the presence of dynamic data sources and task dependencies. Finally, to explore design-time decisions, our co-simulation platform is combined with model generation tools and a multi-objective genetic search algorithm to provide a comprehensive and fully automated NoS design space exploration framework. We evaluate our proposed network-level design flow on a case study of distributed deep learning inference on IoT edge clusters. We developed DeepThings, a framework for adaptively distributed execution of Convolutional Neural Network inference applications on tightly resource-constrained IoT edge clusters. DeepThings employs a novel scalable Fused Tile Partitioning (FTP) for convolutional layers to minimize memory footprint while producing independent distributable tasks. Comprehensive evaluations of our network-level design flow are performed. Results show that our source-level application models can achieve more than 90% accuracy with simulation speeds ranging from 180 MIPS to 5740 MIPS at different abstraction levels. When integrated with system and network models, our NoSSim co-simulator can achieve more than 86% simulation accuracy on average as compared to a real world edge device cluster, where sensitivities to various design parameters are faithfully captured with high fidelity. Compared to existing work sharing methods, our distributed work stealing and work scheduling middleware can improve throughput by 1.7-2.2x under dynamic application scenarios with multiple dynamic data sources. Finally, when applying our network-level design space exploration methodology to the DeepThings application, design-time and runtime decisions are automatically optimized, where non-obvious NoS configurations are discovered outperforming manually-designed solutions by more than 45%. Comparing with a single device execution, more than 90% per-device energy consumption reduction and a 3.6x inference speedup can be achieved on a resource-constrained edge device cluster.