# Multi-Scale Flame Situation Detection Based on Pixel-Level Segmentation of Visual Images
## 3 Method
### The Framework of Flame-SeaFormer
#### 3.2.1 Flame Segmentation Model: SeaFormer
The symbol \(\oplus\) indicates an element-wise addition operation. Mul means multiplication. _Squeeze axial attention:_ For the feature maps \(x\in\mathbb{R}^{H\times W\times C}\) obtained in the previous step for each SeaFormer layer, matrix transformation is applied to derive the query, key, and value, which is expressed as  
\[(Q^{s},K^{s},V^{s})=(W^{s}_{q},W^{s}_{k},W^{s}_{v})x \tag{2}\]  
where \(Q^{s}\in\mathbb{R}^{H\times W\times C_{\#}}\), \(K^{s}\in\mathbb{R}^{H\times W\times C_{\#}}\), and \(V^{s}\in\mathbb{R}^{H\times W\times C_{\#}}\) represent the query, key, and value, respectively. \(W^{s}_{q}\in\mathbb{R}^{C_{\#}\times C}\), \(W^{s}_{k}\in\mathbb{R}^{C_{\#}\times C}\), and \(W^{s}_{v}\in\mathbb{R}^{C_{v}\times C}\) are learnable weight matrices. Firstly, the query matrices are horizontally and vertically compressed by averaging:  
\[Q^{s}_{h}=\frac{1}{W}\big{(}Q^{s\rightarrow\big{(}C_{\#},H,W\big{)}\mathds{1 }_{W}}\big{)}^{\rightarrow\big{(}H,C_{\#}\big{)}} \tag{3}\]  
\[Q^{s}_{v}=\frac{1}{H}\big{(}Q^{s\rightarrow\big{(}C_{\#},W,H\big{)}\mathds{1 }_{H}}\big{)}^{\rightarrow\big{(}W,C_{\#}\big{)}} \tag{4}\]  
where \(z\rightarrow(\cdot)\) denotes element-wise multiplication, and \(\mathds{1}_{m}\in\mathbb{R}^{m}\) is a vector with all elements equal to \(1\). The compression operation on \(Q\) is also performed on \(K\) and \(V\), obtaining \(Q^{s}_{h}\in\mathbb{R}^{H\times C_{\#}}\), \(K^{s}_{h}\in\mathbb{R}^{H\times C_{\#}}\), \(V^{s}_{h}\in\mathbb{R}^{H\times C_{\#}}\), \(Q^{s}_{v}\in\mathbb{R}^{W\times C_{\#}}\), \(K^{s}_{v}\in\mathbb{R}^{W\times C_{\#}}\), and \(V^{s}_{v}\in\mathbb{R}^{W\times C_{\#}}\). Secondly, a multi-head attention mechanism is employed to self-adaptively obtain the dependency relationships between horizontal and vertical flame pixels. Additionally, pixel-level addition is utilized to fuse the horizontal and vertical fire feature information to obtain a global fire feature. The feature value \(f^{s}(i,j)\) at position \((i,j)\) within this feature map is expressed as follows:  
\[ \begin{split} f^{s}(i,j)&=\sum_{p=1}^{H}\text{ softmax}_{p}\Big{(}(Q^{s}_{h(i)}+r^{q}_{h(i)})^{T}(K^{s}_{h(p)}+r^{k}_{h(p)})\Big{)}V^{s}_{h( p)}\\ &+\sum_{p=1}^{W}\text{softmax}_{p}\Big{(}(Q^{s}_{v(j)}+r^{q}_{v( j)})^{T}(K^{s}_{v(j)}+r^{k}_{v(j)})\Big{)}V^{s}_{v(j)}\end{split}  \tag{5}\]  
where position embedding \(r^{q}_{h}\), \(r^{k}_{h}\in\mathbb{R}^{H\times C_{\#}}\) are introduced to make \(Q^{s}_{h}\) and \(K^{s}_{h}\) perceive their positions in the compressed axial feature. These position embeddings are obtained through linear interpolation from the learnable parameters \(B^{q}_{h}\), \(B^{k}_{h}\in\mathbb{R}^{L\times C_{\#}}\), where \(L\) is a constant. In the same way, \(r^{q}_{v}\), \(r^{k}_{v}\in\mathbb{R}^{W\times C_{\#}}\) is applied to \(Q^{s}_{v}\), \(K^{s}_{v}\). Finally, a \(1\times 1\) convolution operation is employed to linearly transform and adjust the input feature map in the channel dimension. \[F^{s}=\text{Conv}(f^{s}) \tag{6}\]  
where Conv means the convolution operation. _Detail enhancement kernel:_ To compensate for the loss of local details caused by the compression operation, an auxiliary kernel based on convolution is introduced to enhance the spatial details. As shown in Figure 5, another query, key, and value set is obtained from the feature map \(x\). \[(Q^{e},K^{e},V^{e})=(W^{e}_{q},W^{e}_{k},W^{e}_{v})x \tag{7}\]  
where \(W^{e}_{q}\in\mathbb{R}^{C_{\#}\times C}\), \(W^{e}_{k}\in\mathbb{R}^{C_{\#}\times C}\), and \(W^{e}_{v}\in\mathbb{R}^{C_{v}\times C}\) are learnable weight matrices. Firstly, the three matrices are concatenated in the channel dimension. \[f^{e}=\text{Concat}(Q^{e},K^{e},V^{e}) \tag{8}\]  
where Concat represents the concatenation operation. Then, the concatenated feature matrix is processed through a block composed of a \(3\times 3\) deep convolutional layer and a batch normalization (BN) layer. This step enables the aggregation of local details from \(Q^{e},K^{e},V^{e}\).