# Automatic Medical Image Segmentation with Vision Transformer
## 3 Method
### Infection Segmentation Network
#### 3.2.4 Reverse Attention Module  
Clinicians typically segment regions of infection in a two-step process--coarsely localizing regions of infection, and then accurately annotating local tissue structures. We mirrored this workflow with two components. First, the PPD acts as a coarse locator, generating a global localization map \(S_{g}\) indicating the rough locations of an infection region without structural details. Next, a progressive reverse attention (RA) framework acts as the refined labeler to discriminatively delineate infections by removing estimated regions from high-level feature representations [53; 54]. Rather than simply aggregating all levels [54], we dynamically acquired reverse attention across three parallel high-level feature maps. This progressively exploited complementary regions and details from the deeper layers. This was achieved by eliminating elements of the current estimations from the up-sampled high-level features. The resulting RA features \(R_{i}\) are obtained through fusing high-level feature maps \(\{f_{i},i=3,4,5\}\) and edge-aware features \(e_{att}=f_{2}\) from the edge attention module using the reverse attention weights \(A_{i}\):  
\[R_{i}=C(f_{i},Dow(e_{att}))\odot A_{i} \tag{5}\]  
where \(dow(\cdot)\) is the down-sampling operation; \(C(\cdot)\) is the concatenation operation. Our network mimics the clinician workflow with coarse localization followed by fine labeling of structural details. The global guidance provided by the localization map and the progress  
Figure 2: Paralleled partial decoder to obtain global map. sive reverse attention mechanism allow the module to selectively zoom in on informative regions. The RA weight \(A_{i}\) is crucial for salient object detection [46], with the definition as:  
\[A_{i}=\epsilon(\odot(\sigma(P(S_{i+1})))) \tag{6}\]  
where \(P(\cdot)\) is the up-sampling operation; \(\sigma(\cdot)\) represents the sigmoid activation function; and \(\odot(\cdot)\) denotes the reversed subtraction operation between the input and a matrix with each element as 1. The reversed subtraction \(\epsilon\) subtracts the localization map from a matrix of ones to reverse the activation in Equation (5). Figure 3 shows the details of this procedure. This reverse attention mechanism essentially reverses the background to foreground patterns learned by the CNN, helping to disentangle infection regions from normal tissue. The \(A_{i}\) weights then filter out irrelevant regions while passing through discriminative cues that are progressively refined across stages. Together with the edge guidance from the edge attention module, this multi-level erasing and fusion process elegantly captures subtle characteristics that are difficult for standalone networks to model, moving towards an expert-level understanding. Importantly, the erasing strategy, which is guided by the RA module, progressively refines the initial coarse prediction map into a more accurate final segmentation prediction. The reverse attention weights are learned to erase the prior coarse estimations and focus on complementary infection details. By iteratively refining the localization in this manner, the initial coarse prediction is transformed into a precise final segmentation map. The reverse attention module progressively refines coarse segmentation predictions in a hierarchical manner to enhance accuracy and capture finer details. It disentangles infection regions from normal tissues by using attention mechanisms to selectively focus on amplifying features of infection regions while suppressing irrelevant features of normal tissues. This highlights infection boundaries and contrasts between infected and healthy areas. The module employs an iterative erasing strategy that gradually removes less relevant features from the coarse predictions to emphasize salient infection-related features.