## 3 Methods
### Encoder
#### 3.1.2 Interleaving Branch Stage  
This interleaving branch stage alternates between using swin transformer blocks and Window Grouping Attention (WGA) for inter-view communication, as shown in Figure 2b. This stage ensures the efficient integration of spatial and relational information across multiple views.  
This section introduces the Window Grouping Attention (WGA) mechanism that effectively captures the relationships between views. For a given input feature view \(X_{i}\ \in\ \mathbb{R}^{P\times D}\), where the number of tokens is \(P\), the feature dimension is \(D\), and tokens are grouped into \(G\) groups, each group \(G_{j}\) is defined as follows:  
\[G_{j}=\big{\{}x_{i,k}\ \big{|}\ i=1,\ 2,\...,\ N;\ k\ \in\ W_{j}\big{\}} \tag{3}\]  
where \(x_{i,k}\) denotes the k-th token from the i-th view and \(W_{j}\) specifies the index of tokens in the j-th spatial window. This ensures that grouping tokens in \(G_{j}\) comes from the same spatial window in all views, promoting local attention by maintaining inter-view consistency, which is key for capturing significant spatial relationships and ensuring that features from each view contribute to the final aggregated representation.  
Within each group \(G_{j}\), tokens engaged independent self-attention, and they are computed as follows:  
\[\text{Attention}_{\text{j(window)}}\big{(}Q_{j},\ K_{j},\ V_{j}\big{)}= \text{Softmax}\ \bigg{(}\frac{Q_{j}K_{j}{}^{T}}{\sqrt{d_{k}}}\bigg{)}V_{j} \tag{4}\]  
where \(Q_{j},\ K_{j},\ V_{j}\ \in\ \mathbb{R}^{M_{g}\times D}\) represent the queries, keys, and value matrices obtained from the grouped tokens and the number of tokens from each group \(M_{g}\). By reducing self-attention, the output of this stage is a spatial feature map for each view.  
Figure 2: Illustration architecture of the encoder, (**a**) swin transformer block, and (**b**) Window Grouping Attention (WGA) block.  
attention between grouped tokens, WGA establishes efficient computation while preserving long-range relationships between different views.  
After applying the self-attention operation, all groups are sent for separate processing before their outputs are combined into an inter-view feature representation. This divide-and-conquer approach allows WGA to focus on token correlations between spatial windows across views, effectively understanding intra-view details alongside inter-view dependencies.  
Furthermore, to improve inter-view associations, Inter-View Feature Signatures (IFSs) [11] have been implemented. The IFS assigns view-specific encodings to the tokens, which ensures that the model can distinguish tokens originating from different views. Specifically, for each token \(x_{i,j}\) in the group \(G_{j}\), the feature signature is defined as \(f_{i,j}=\text{IFS}\left(\phi\big{(}x_{i,j}\big{)}\right)\), where \(\phi\big{(}x_{i,j}\big{)}\) denotes the trainable projection. The feature signatures for all views are expressed as \(f_{j}=\{f_{1,j},\ f_{2,j},\...,\ f_{N,j}\}\), where \(N\) denotes the number of views. When the IFS is added to the output of WGA, the following occurs:  
\[\text{Attention}_{\text{i}(\text{window})}\big{(}Q_{j},\ K_{j},\ V_{j}\big{)}= \text{Softmax}\ \left(\frac{Q_{j}K_{j}{}^{T}}{\sqrt{d_{k}}}\right)V_{j}+f_{j} \tag{5}\]  
where the addition of the IFS in WGA allows the model to distinguish between tokens from different views, enriching feature representation with view-specific information. This enhances the model's ability to maintain consistency across multiple views and improves the accuracy of the feature aggregation process.  
The interleaving branch stage combines WGA with the view-specific differentiation of the IFS to produce a characterized, diverse, and consistent representation of features across multiple views. This allows the encoder to efficiently handle multi-view inputs and provide rich feature maps for critical tasks, such as 3D reconstruction.