# Fast Fault Line Selection Technology of Distribution Network Based on MCECA-CloFormer
## 4 Fault Line Selection Model Based on MCECA-CloFormer
### CloFormer  
CloFormer is a lightweight vision transformer designed with a dual-branch architecture, incorporating both local and global pathways. In the global branch, standard attention mechanisms are employed to capture the global low-frequency information of the input, whereas the local branch combines context-aware weights with shared weights to extract high-frequency local information from the input. This dual-branch configuration facilitates the model's enhanced perception of both high and low-frequency components in the input feature maps. Experimental evidence demonstrates its superior performance across a range of visual tasks, including image classification, object detection, and semantic segmentation [24], as illustrated in Figure 5. The overarching architecture consists of four main components: the Conv stem, Clo blocks, ConvFFN, and the Output module.  
Figure 4: ECA module structure.  
Conv stem. This component primarily comprises five convolutional layers, with the detailed structure illustrated in Figure 5a.
* Clo block. This part is constituted by a local branch and a global branch. As shown in Figure 5b, within the global branch, \(K\) and \(V\) are initially downsampled, followed by the execution of a standard attention process on \(Q\), \(K\), and \(V\) to extract global low-frequency information, as outlined in Equation (15).  
\[X_{global}=\text{Attntion}(Q_{g},\text{Pool}(K_{g}),\text{Pool}(V_{g})) \tag{15}\]  
In the local branch, linear transformations are first applied to obtain \(Q\), \(K\), and \(V\). Subsequently, depthwise convolution (DWConv) with shared weights is employed on V for local feature extraction, as illustrated in Equations (16) and (17).  
\[Q,K,V=\text{FC}(X_{in}) \tag{16}\]  
\[V_{s}=\text{DWconv}(V) \tag{17}\]  
where FC denotes a fully connected layer.  
Meanwhile, local information aggregation for \(Q\) and \(K\) is performed using deep convolutions, following which the Hadamard product of the aggregated \(Q_{l}\) and \(K_{l}\) is computed. This product undergoes a series of non-linear transformations to yield context-aware weights within the range of \(-1\) to \(1\). These generated context weights are then employed to enhance the local features \(V_{s}\). The entire process is outlined in Equation (18).  
\[\left\{\begin{array}{l}Q_{l}=\text{DWconv}(Q)\\ K_{l}=\text{DWconv}(K)\\ Attn_{t}=\text{FC}(\text{Swish}(\text{FC}(Q_{l}\odot K_{l})))\\ Attn=\text{Tanh}(\frac{Attn_{l}}{\sqrt{d}})\\ X_{local}=Attn\odot V_{s}\end{array}\right. \tag{18}\]  
Figure 5: CloFormer structure diagram.  
where \(d\) is the number of token's channels, \(\odot\) is the Hadamard product.  
Finally, the outputs from the global and local branches are concatenated along the channel dimension, followed by passing through a fully connected layer to yield the final output of the Clo block, as illustrated in Equation (19).  
\[\left\{\begin{array}{l}X_{t}=\text{Concat}(X_{local},X_{global})\\ X_{out}=\text{FC}(X_{t})\end{array}\right. \tag{19}\]  
where \(\text{Concat}(\cdot,\cdot)\) denotes the concatenation of two tensors along the channel dimension.  
* ConvFFN. This component primarily bolsters its local information aggregation capacity by introducing depthwise convolution following the GELU activation function. Two modes are adopted: one without downsampling (depicted on the left in Figure 5c), which is utilized in all layers except the last, and another incorporating a downsampling module (shown on the right in Figure 5c), which is employed in the final layer of the CloFormer architecture.
* Output module. The output module consists of a global average pooling layer and a linear classifier. The overall structure of CloFormer is depicted in Figure 5d, wherein a CloLayer is composed of a combination of two Clo blocks and a ConvFFN, with downsampling incorporated in the ConvFFN of the final layer.