# Noise Suppressed Image Reconstruction for Quanta Image Sensors Based on Transformer Neural Networks
## 1 Introduction  
With the continuous advancement of Moore's Law, the resolution of conventional complementary metal-oxide-semiconductor image sensors (CIS) has steadily increased, leading to a reduction in pixel size. However, at the deep sub-diffraction-limit (deep-SDL) scale, smaller pixels result in decreased full well capacity (FWC), which have a negative impact on signal-to-noise ratio (SNR) and limits the dynamic range of the sensors [1]. To address these limitations, the concept of the digital film sensor (DFS) was proposed in 2005 [2], and was formally termed the quanta image sensor (QIS) in 2011 [3]. Recent advances [4] reveal that the latest QIS prototype has achieved a read noise below 0.2 e - r.m.s at room temperature, with a pixel resolution of 16.7 million and a pixel pitch of 1.1 \(\upmu\)m. This state-of-the-art sensor is expected to incorporate smaller pixel pitches, higher frame rates, lower read noise, and reduced dark current while maintaining high performance at room temperature and production costs comparable to traditional CIS [5]. These features make QIS highly suitable for low-light imaging applications. Numerous studies in computer vision have demonstrated the effectiveness of the sensor in addressing low-light scenarios [6, 7, 8, 9]. The QIS is built on an array of ultra-small photodetectors referred to as 'jots' [10], each of which is quantized using an analog-to-digital converter (ADC). A distinguishing feature of QIS is its ability to perform spatio-temporal oversampling through high-frequency sampling. In this context, the oversampling unit, known as a 'cubicle', is composed of 3D arrays of 'jots'. The main purpose of the spatio-temporal oversampling process is to suppress Poisson noise induced by photon incident events, thereby improving image quality [11]. After oversampling, a reconstruction process is carried out to convert the data in the 'cubicle' into reconstructed pixel values. The concept of spatio-temporal oversampling is illustrated in Figure 1.  
The analysis of noise and bit error rates in QIS was first introduced in [12; 13], followed by detailed mathematical modeling and SNR formulations [14]. One of QIS's key advantages is its ability to programmatically adjust spatial and temporal oversampling rates for high SNR image reconstruction [10]. Therefore, robust reconstruction algorithms are essential for achieving high-quality imaging. Early research framed image reconstruction as a photon number estimation challenge. [PERSON] et al. [15] proposed a Maximum Likelihood Estimation (MLE)-based algorithm for this purpose and [PERSON] et al. [16] proposed a Maximum A Posteriori (MAP) estimation based on the Alternating Direction Method of Multipliers (ADMM). In addition to photon number estimation methods, a non-iterative reconstruction method based on Block-Matching and 3D Filtering (BM3D) denoiser [17] and a deep learning reconstruction method based on convolutional neural networks (CNN) [18] have also been proposed. The photon detection capability makes QIS an optimal choice for imaging in low-light conditions. However, the low SNR bit-planes pose challenges to image reconstruction, thus the appropriate noise suppression method can improve the quality of reconstructed images. The photon number estimation methods [15; 16] suppress noise based on imaging principles. However, the estimated values of these methods significantly deviate from the true values and the mathematical expressions become very complex and difficult to solve as the bit depth increases. Additionally, the non-iterative BM3D method [17] utilizes complex frequency domain filtering algorithms to achieve better denoising results compared to photon number estimation methods. However, the time complexity of this method is high and the performance of the BM3D denoiser has been proven to be lower compared to the denoisers of the neural network [19]. The QISNet reconstruction method [18], based on CNN, has been proposed for noise suppression in QIS image reconstruction.