# Evaluating DL Model Scaling Trade-Offs During Inference via an Empirical Benchmark Analysis
## 7 Related Work
### Empirical Evaluations of DL Model Scaling  
The following presents both recent and notable scientific works introducing empirical evaluations for DL model scaling.  
[PERSON] et al. [20] investigated the pitfalls of one-dimensional model scaling for convolutional neural networks (CNNs) and proposed a scaling method for CNNs that utilizes dimensional relationship and runtime proxy constraints to improve accuracy and inference latency. In turn, [PERSON] et al. [35] introduced a study where they empirically projected the computational requirements for future DL applications based on the representational growth achieved (and targeted) by the AI community.  
[PERSON] et al. [14] presented an evaluation study to demonstrate how different compound scaling strategies affect the model parameters; activations; and, consequently, training time for different CNN model structures, focusing primarily on EfficientNet and RegNet models. With a focus on language models that adopt a transformers-based architecture, [PERSON] et al. [21] empirically showed that performance (measured by test loss) improves and follows a power-law relationship as the model size, dataset size, and availability of computational power are scaled during model training. When only scaling one factor to achieve the referenced performance, the other two must not become bottlenecks. For optimal performance, the authors showed that all three factors must be scaled-up in tandem.  
In a recent study, [PERSON] et al [22] capitalized on the previously mentioned work to show how scaling the model and dataset size can be explained under variance-limited and/or resolution-limited scaling regimes. In the end, the authors provided a theoretical framework that can be used to estimate upper and lower bounds for model loss depending on the aforementioned scaling regimes. Furthermore, [PERSON] et al. [36] presented an empirical analysis of three open and popular distributed deep learning frameworks to assess their performance and scalability during model training with experiments focusing on ResNet and ImageNet models as the examined workload. Finally, [PERSON] et al. [37] introduced an empirical study that examined a large corpus of DL model variants investigating unit test adoption and if/how metrics beyond accuracy are used to improve model robustness and reliability.  
Based on the provided list of existing works empirically evaluating model scaling, it is obvious that the majority of the aforementioned studies either focus on a single model structure (e.g., ResNet, RegNet, or CNN models) and/or introduce findings that are tailored to solely evaluating accuracy and overheads during the model training stage. We deviate from the norm and focus on the impacts different model scaling strategies have during runtime inference. By offering a detailed examination of trade-offs during model inference, our work can contribute to guiding AI practitioners in selecting and tuning DL models for efficient and effective deployment of intelligent IoT services in resource-constrained environments. Moreover, we note that although a direct comparison against other empirical studies cannot be performed, as the body of related work focuses on performance and accuracy during the model training phase, several of our findings can be confirmed by the aforementioned studies. For example, [PERSON] et al. [20] showcased that compound model scaling positively impacts classification accuracy for DNNs, while [PERSON] et al. [22] confirmed this but also noted (as we do) that the gains in terms of accuracy diminish and can plateau as a model attempts to scale using large compound coefficients. In turn, [PERSON] et al. [14] showed that although two model variants can present the same computational complexity, the output classification accuracy can (significantly) differ.