Controlling AI Engines in Dynamic Environments

A recent paper lists deploying AI in dynamic environments with unpredictable changes as a major research opportunity. To address this challenge we advocate the integration of AI and control systems. AI and machine learning are well-suited to deal with complexity, while control theory is specically designed to manage dynamics. e integration of these approaches has several benets: 1) fast reaction to dynamic changes, 2) error tolerance based on runtime feedback, and 3) robust design that guarantees that operating requirements will be respected when possible or the ability to report when those requirements cannot be met.1 1 e Challenge of System Dynamics e last several years have seen a massive transition of Articial Intelligence and Machine Learning technologies from academia and research labs into commodity products. AI systems are being deployed in all types of computing systems from image analysis in embedded systems to selecting and serving digital advertisements in the cloud. While this transition has been wildly successful, a recent survey notes several open research challenges in AI deployment, including operating “in dynamic environments, i.e., , environments that may change, oen rapidly and unexpectedly, and oen in non-reproducible ways [27].” Fortunately, control theory is an engineering discipline devoted to operating in dynamic environments. Furthermore, there is a rich history of deploying control theoretic solutions to computer management problems; e.g., meeting quality-of-service requirements in web servers [8, 9, 23]. Control theory’s appeal is that it provides a rigorous framework for designing systems that provably meet requirements despite unpredictable system dynamics. e drawback is that control deployment requires experts who can formulate dierence models of the computer system, a discipline in which most programmers are not trained [8]. While it seems natural to combine AI engines with control systems to build AI that is robust in dynamic environments, this combination requires developers who are experts in AI, control systems, and the actual application area in which the system is to be deployed. is combination of expertise is an unrealistic burden to place on a developer, as even AI experts are in short supply at the moment. We therefore advocate a general framework for composing AI engines with control that provides many of control’s guarantees without requiring control expertise from the user. e proposed combination of AI and control has three benets: • Formally analyzable dynamic response. • Fast adaptation to unpredictable dynamic events. • Increased robustness to errors in the learned model. 1is abstract is a summary of a full length paper to appear in ASPLOS 2018 [18]. Conference’17, Washington, DC, USA 2016. 978-x-xxxx-xxxx-x/YY/MM. . .$15.00 DOI: 10.1145/nnnnnnn.nnnnnnn 2 Allocating Heterogeneous Resource As a motivational problem we consider allocating resources in a heterogeneous multicore processor, specically Samsung’s Exynos Octa processor used in mobile devices and some internet-of-things devices [24]. e processor is based on an ARM big.LITTLE architecture with four big, high-performance cores and four LITTLE, energy ecient cores. e big cores support 19 clock speeds and the LITTLE cores support 14. e mobile and embedded systems running on these heterogeneous processors typically have latency requirements; i.e., they must deliver reliable performance to eectively process (e.g., video analysis) or produce (e.g., gaming) a data stream. Additionally, any latency requirement should be met while minimizing energy to prolong baery life or reduce costs. Finding the right combination of resources to meet a particular application’s latency requirement while minimizing energy is a complex optimization process, and a number of AI and ML approaches have been proposed for this problem [1, 4–7, 13, 17, 19, 20, 25, 31, 33]. All these approaches estimate the performance and energy of some resource conguration, but they have limited ability to deal with dynamic changes. In contrast, control theoretic approaches dynamically adjust resource usage based on the dierence between measured and expected behavior [2, 9, 11, 12, 14, 22, 26, 29, 30, 32]. Control formalisms, however, rely on “ground truth” models of application response to resource usage. Intuitively, combining learned models of complex hardware resources with control-theoretic resource management should produce predictable latency and two recent research projects explore such a combination. Recht et al. have proposed several approaches for combining statistical learning models with optimal control theory [3, 28]. Simultaneously, Homann et al. have developed OS[10] and hardware-level resource management systems [21] that combine learning and control to provide both energy and latency guarantees in dynamic environments. is prior work, however, still requires expertise in both learning and control methods to eectively deploy the proposed solution. Recent work, however, denes abstractions that allow a number of AI and learning techniques to be combined with an adaptive controller, maintaining control-theoretic formal guarantees [18]. is paper reviews those abstractions and demonstrates their benets. 3 Combining AI Engines and Control Systems Figure 1 shows the proposed approach of spliing resource management into learning and control sub-tasks and then composing their solutions. When a new application enters the system, an adaptive controller allocates resources using a generic model and records latency and power. e recorded values are sent to a learner, which estimates the application’s latency and power in all other resource congurations. e learner extracts those that are predicted to be Pareto-optimal and packages them in a data structure called the performance hash table (PHT).e PHT and the learner’s estimated error are sent to the controller, which tunes its internal parameters