## 3 Preliminary and Model
### Traffic Degradation Image Restoration Model
#### 3.4.1 Overall Architecture  
The overall architecture of TDIRM is shown in Figure 5. Its core methodology employs triple conditional control to constrain the restoration process, thereby enhancing both restoration quality and fidelity.  
For the original degraded image requiring restoration, a lightweight Swin transformer is first employed to perform preliminary enhancement, improving the quality of the de  
Figure 4: Restoration results by SDM under different text prompts.  
Figure 5: Network structure of traffic degradation image restoration model (TDIRM).  
graded input to some extent. The enhanced image, denoted as DI, serves as the initial input to the model. The image semantics encoder (ISE) processes DI to extract semantics information, encoding it into latent features as the first condition embedding (SE). This embedding contains feature representations that align with the latent description of the real background while excluding degradation-related information.  
Simultaneously, DI is encoded by a pre-trained VAE encoder to produce latent features as the second condition embedding DIE. DIE contains the global content understanding and structural layout of the degraded image, thereby enhancing the fidelity of the restored image.  
Finally, an instance segmentation model is applied to DI to obtain a segmentation map of key traffic elements such as vehicles, pedestrians, and traffic signs. Segmentation maps are further encoded by a pre-trained VAE encoder into latent features, forming the third condition embedding, SgeE. SgeE contains critical information on traffic elements such as category, location, and size, ensuring that the restoration process does not arbitrarily alter essential traffic elements.  
The three condition controls are integrated into the reverse sampling process of the stable diffusion model (SDM) by using a ControlNet-based approach to guide the reconstruction of degraded images. Similar to ControlNet architecture, the denoising module in TDIRM consists of two components: the U-Net denoising module from SDM and a trainable ControlNet copy. The difference is that in the U-Net denoising module of SDM, original U-Net encoder blocks are retained, while original attention modules in the middle block and decoder blocks are replaced with the triple control embedding attention module (TCE) in order to enhance restoration fidelity. The parameters in the U-Net encoder blocks remain frozen to preserve pre-trained feature extraction capability. The middle and decoder blocks are partially trainable.  
The image semantics embedding SE is concurrently input into both the U-Net denoising module and the ControlNet module. In contrast, DIE and SgeE are first processed by ControlNet before being input to TCE attention in the middle and decoder blocks of the U-Net denoising module. This is used to guide the restoration process of SDM reverse sampling.  
TDIRM adopts the same loss function as the original SDM, as specified in Equation (2):  
\[\mathcal{L}_{\text{TDIRM}}=\mathbb{E}_{t,\epsilon,\mathbf{z}_{t},c}\Big{[}\| \ \epsilon-\epsilon_{\Theta}(\mathbf{z}_{t},\mathbf{t}|c)\ \|_{2}^{2}\Big{]} \tag{2}\]  
\(\mathbf{t}\in\{1...T\}\) represents time step of the diffusion process. \(\epsilon\sim\text{N}(0,1)\) is the standard Gaussian noise. \(\mathbf{z}_{t}\) is the potential feature after adding noise. \(\epsilon_{\Theta}\) is the predicted noise by network. \(c\) is control condition.  
Ultimately, using triple condition control to constrain the sampling process of SDM can effectively enhance the quality and fidelity of the restored image while utilizing SDM's powerful prior knowledge about the real world.