Practical Coding-Theoretic Tools for Machine Learning Systems and by Machine Learning Systems
Creators: name: Kosaian, Jack
Publisher: name: Carnegie Mellon University
Year: 2023
Subjects: subject: 80401 Coding and Information Theory
subject: FOS: Computer and information sciences

description: Machine learning (ML) is now used in many domains, such as web services and<br> safety-critical systems. This has led to the development of ML systems for deploying<br> and training ML models. Beyond achieving high accuracy, ML systems must also<br> use computing infrastructure efficiently and tolerate unreliable infrastructure. <br> Coding-theoretic tools enable many systems to operate reliably without the sig-<br> nificant resource overhead that accompanies replication-based approaches. These<br> tools are used in production storage and communication systems, and there is grow-<br> ing interest in their use for distributed computing. <br> This thesis explores the interplay between ML systems and practical applications<br> of coding theory. Specifically, we show how ML systems can be made more reliable<br> and efficient via novel uses of coding-theoretic tools, and how coding-theoretic tools<br> can be expanded in reach and be made more efficient through techniques from ML<br> and ML systems. We illustrate this interaction via multiple thrusts: <br> 1) We show how properties unique to ML systems can be exploited to efficiently integrate coding-theoretic fault tolerance techniques into ML systems. First, we reduce the execution-time overhead of fault-tolerant inference on GPUs by up to 5.3× by exploiting trends in neural network design and GPU hardware. Second, we show how coding-theoretic tools can be coupled with the unique properties of recommendation models to enable low overhead fault tolerance in training.<br> 2) We demonstrate that co-designing coding-theoretic tools with ML systems<br> offers new opportunities to extend these tools beyond prior limitations. Specifically,<br> we enable resource-efficient fault tolerance in distributed prediction serving systems<br> by using ML to overcome a key barrier in prior coding-theoretic tools.<br> 3) We identify opportunities for ideas inspired by coding theory to be used to<br> improve the performance of ML systems, even when reliability is not a concern. We<br> show that the throughput and GPU utilization of specialized convolutional neural<br> network inference can be improved by up to 2.5× by combining images in a coding-<br> theory-inspired manner and making small modifications to the model architecture.<br> 4) Finally, we show that the encoding and decoding functions of one popular class of coding-theoretic tools, linear codes, can operate at higher throughput and with little developer effort via advancements in ML systems. We show how similarities between operations in linear codes and those in ML libraries enable linear codes to be represented via ML libraries, and thus allow libraries for computing linear codes to adopt the many optimizations that have gone into ML libraries. This approach outperforms custom libraries for computing linear codes by up to 2.2×.<br> Through these thrusts, this thesis demonstrates the promise of using coding-<br> theoretic in ML systems and ideas from ML systems in coding-theoretic tools to<br> bring about the next generation of efficient and reliable systems.