# A Comprehensive Survey on Real-Time Image Super-Resolution for IoT and Delay-Sensitive Applications
## 3 Real-Time Systems and Real-Time SR
### Real-Time Single Image Super-Resolution Methods
#### 3.3.1 Distillation-Based Real-Time Super-Resolution Methods  
Real-time SR based on distillation is a strategy that utilizes knowledge distillation to construct HR SR outcomes in real-time circumstances. These types of models follow the approach of student and teacher models. The intuition behind these models, training a faster and tiny model that behaves like a student and mimics the attributes and performance of larger models, is to create more accurate SR tasks. Figure 6 displays how knowledge distillation works in a tiny student model to replicate a big teacher model and harness the teacher's knowledge to achieve comparable or superior accuracy. Here, larger models act  
Figure 5: Taxonomy of real-time super-resolution methods. as teachers. Distillation-based SR technologies frequently use lightweight network designs and optimization approaches to attain real-time performance. Some solutions, for example, make use of network topologies with a limited number of parameters, such as MobileNet or ESPCN. The authors of [63] presented the first model based on information distillation, known as the Information Distillation Network (IDN). The IDN architecture is made up of three major components: feature extraction blocks, stacked information distillation blocks (DBs), and reconstruction blocks. Within the DB, an enhancement unit and a compression unit collaborate to efficiently extract both local long- and short-path characteristics. Specifically, the enhancement unit combines two types of characteristics, whereas the compression unit distills more usable information through consecutive blocks. The IDN model has a noteworthy benefit in that it uses very few layer filters and group convolution. However, the IDN still has a significant weight, emphasizing the need for more efficient models while deploying in real-time systems. In [64], the authors proposed a feature-domain adaptive method utilizing contrastive distillation for lightweight SISR. They addressed the limits of previous feature distillation approaches that use Euclidean distance-based loss and developed a feature-domain contrastive loss. This novel technique allows student networks to learn more detailed information from the teacher's representation in the feature domain and be efficient while deploying on real-time systems. As a solution for IDN's obstacles, the authors of [65] presented the Information Multi-Distillation Network (IMDN) for SISR, which uses cascaded Information Multi-Distillation Blocks (IMDBs). This architecture allows for the sequential extraction of hierarchical features using a distillation module. These traits are subsequently integrated by a fusion module, which prioritizes them according to their importance. A Contrast-Aware Channel Attention (CCA) layer is used for task assessment. Furthermore, an Adaptive Cropping Strategy (ACS) is created to handle actual photos of various sizes using the same trained model. While the IMDN overcame some of the difficulties associated with the IDN's high weights, there is still significant room for improvement in the IMDN paradigm. In [15], an innovative approach named the Residual Feature Distillation Network (RFDN) is described, which uses a channel-splitting operation to build a Feature Distillation Connection Block (FDCB). This block is an upgraded version of the previous IMDN [65]. The RFDN focuses on learning discriminative feature representations by combining numerous feature distillations with a shallow residual block (SRB). The SRB, which consists of a convolutional layer, a skip connection, and an activation unit, uses residual learning to improve performance. This addition marginally enhances computation without adding any additional parameters compared to the original IMDN block. Additionally, the model incorporates an Enhanced Spatial Attention (ESA) mechanism to further improve  
Figure 6: The framework for knowledge distillation work through the teacher-student model. performance. As a result, the RFDN network is lighter in weight and runs faster than the IMDN.