# Noise Suppressed Image Reconstruction for Quanta Image Sensors Based on Transformer Neural Networks
## 3 Method Description
### Detail of the QIS-SPFT Denoiser  
The SM is utilized to extract structural information from quanta images that contain complex noise components. Its superior denoising performance is attributed to its serial architecture, which is primarily composed of three components: convolutional layers, activation functions, and Transformers. The first and third convolutional layers employ _Conv_. Given that this study focuses on grayscale noisy images directly generated by light intensity, the input and output channels of the first convolutional layer are set to 1 and 64, respectively. The second convolutional layer, \(Conv+R\), integrates linear and nonlinear components to extract richer features. In this context, \(Conv+R\) is defined as the combination of a convolutional layer, denoted as \(Conv\), and an activation function, specifically \(Relu\). The convolutional layer \(Conv\) extracts linear features, whereas the \(Relu\) function acts as a piecewise function, transforming linear information into nonlinear features. The Transformer structure is employed to explore relationships between different patches and to dynamically learn weights for various inputs, enabling adaptive noise reduction across diverse areas in the image. The architecture of SM is shown in Figure 4 and the output of SM is mathematically represented by Equation (10). \[ \begin{split} O_{SM}&=SM(I_{N})\\ &=T(C(C(R(C(I_{N})))+C(I_{N})),\end{split}  \tag{10}\]  
In this network, the Transformer architecture is implemented through an encoder that incorporates Multi-Head Self-Attention (MHSA) and Channel Feature Enhancement (CFE) mechanisms. These components are mathematically represented by Equation (11). \[ \begin{split} T(I_{T})&=CFE(O_{MHSA}+I_{T})+I_{CFE} \\ &=FCL(FCL(R(LN(O_{MHSA}+I_{T}))))+I_{CFE},\end{split}  \tag{11}\]  
Figure 4: Detail of the serial module. where, \(FCL\) denotes the fully connected layer, \(LN\) represents layer normalization, \(I_{T}\) is the Transformer input, \(O_{MHSA}\) is the MHSA output, and \(I_{CFE}\) is the CFE input. The MHSA mechanism extracts global contextual information to enhance significant features for image denoising. It employs normalization layers to standardize the feature distribution, which are then fed into three parallel branches, each comprising a fully connected layer. The outputs of these branches are designated as \(Q\), \(K\), and \(V\). The features are integrated via self-attention, further refined through an \(FCL\), and then used as the \(MHSA\) output. Residual operations merge features from Transformer inputs and \(MHSA\) outputs into the \(CFE\). This process is described by Equation (12):  
\[ \begin{split} O_{SM}&=T(O_{IN\_T})\\ &=CFE(MHSA(O_{IN\_T})+O_{IN\_T})+O_{IN\_CFE}\\ &=CFE(FCL(\text{softmax}\\ &\big{(}\frac{FCL(LN(O_{IN\_T}))\times FCL(LN(O_{IN\_T}))^{T}}{d} \big{)}\\ &\times FCL(LN(O_{IN\_T})))+O_{IN\_T})+O_{IN\_CFE}\\ &=CFE(FCL(\text{softmax}(\frac{Q\times K^{T}}{d})\times V)+O_{IN\_T})+O_{IN\_CFE}\\ &=CFE(O_{MHSA}+O_{IN\_T})+O_{IN\_CFE},\end{split}  \tag{12}\]  
The MHSA mechanism is illustrated in Equation (12), which efficiently captures global structural information. Concurrently, normalization layers and fully connected layers are employed to standardize features, thereby improving denoising performance. The output of the MHSA is then further processed through a series of fully connected layers and normalization steps within the CFE mechanism, where it is combined with the original inputs to achieve a more effective representation of clean image information. As shown in Figure 5, the parallel module consists of two interacting parallel subnetworks that capture complementary features at different levels. To enhance the effectiveness of deep learning, a residual learning mechanism is employed in the subnetworks, where the residual between the input to the convolutional layer and the output after the convolutional layer is computed and used as the input to the Transformer. This process can be expressed by Equation (13). \[