# The Quanta Image Sensor: Every Photon Counts
## 1 Introduction  
The Quanta Image Sensor (QIS) was conceived in 2004 and published in 2005 [1; 2; 3; 4] as a forward look at where image sensors may go in the 10 to 15-year future as progress in semiconductor device technology would allow sub-diffraction limit (SDL) pixels to be readily implemented, and advancement in circuit design and scaling would permit greater pixel throughput at reasonable power dissipation levels. Active research began in 2008 at Samsung (Yongin, Korea) [5] but was short lived due to economic pressure in that period. Research began anew at Dartmouth in 2011 and was supported from 2012 to the present by Rambus Inc. (Sunnyvale, CA, USA). Since 2011, progress has been made in pixels, readout circuits, and image formation [6; 7; 8; 9; 10; 11; 12; 13; 14; 15; 16; 17; 18; 19; 20; 21; 22; 23; 24; 25; 26]. In this paper, the QIS concept and progress to date is reviewed. In the QIS, SDL pixels (e.g., 200 nm-1000 nm pitch) are sensitive to single photoelectrons, so that the presence or absence of one electron will result in a logical binary output of 0 or 1 upon readout. The specialized pixel, called a \"jot\", (Greek for \"smallest thing\") needs only the smallest full-well capacity (FWC). It is envisioned that a QIS will consist of hundreds of millions or perhaps billions of jets read out at perhaps 1000 fields per second, resulting in a series of bit planes, each corresponding to one field. The bit data can be thought of as a jot data cube, with two spatial dimensions (x and y) with the third dimension being time. Output image pixels are created by locally processing the jot data cube to create a representation of local light intensity received by the QIS. Since this processing occurs post-capture, great flexibility isafforded in choosing the effective spatial dimensions of a pixel as well as its temporal dimension (e.g., digital integration time). Conceptually, if the bit data is an accurate representation of the collection and counting of photoelectrons, the combining of jet data is noiseless, allowing functionality such as time-delay-and-integration (TDI) to be performed post-capture on the data in an arbitrary track direction. In fact, different tracks can be used in different portions of the image. Indeed, even relative motion of objects within the field of view can be determined and refined iteratively to optimize the image generation process. Spatial and temporal resolution can also be adjusted for different portions of the image. After the QIS concept was introduced in 2005, the concept was applied for use with single-photon avalanche detectors (SPADs) by the group at the University of Edinburgh [27; 28; 29; 30] as published starting in 2014 as part of their SPAD research program. Other work on SPADs published in 2005 shows concurrent conception of some of the same ideas [31], and in 2009 a \"gigavision camera\" using binary pixels was proposed by researchers from Ecole Polytechnique Federale de Lausanne (EPFL) [32; 33; 34; 35; 36]. Some characteristics of the QIS can be traced to photographic film as reported in 1890 [37] and other characteristics were observed in photon-counting devices implemented using vacuum tubes and solid-state devices [38; 39; 40; 41; 42; 43]. Progress in other devices achieving sub-electron and deep sub-electron read noise has mainly been made in the past few years [44; 45; 46; 47; 48; 49; 50; 51]. Possible applications of the QIS include scientific low-light imaging such as in life sciences (e.g., microscopy), defense and aerospace, professional and consumer photography and cinematography, multi-aperture imaging, cryptography (quantum random number generation), direct detection of low-energy charged particles, and others. To realize the QIS in a convenient form, several theoretical and technological issues require exploration.