Calibrating photon counts from a single image

Current methods for detector gain calibration require acquisition of tens of special calibration images. Here we propose a method that obtains the gain from the actual image for which the photon count is desired by quantifying out-of-band information. We show on simulation and experimental data that our much simpler procedure, which can be retroactively applied to any image, is comparable in precision to traditional gain calibration procedures. Optical recordings consist of detected photons, which typically arrive in an uncorrelated manner at the detector. Therefore the recorded intensity follows a Poisson distribution, where the variance of the photon count is equal to its mean. In many applications images must be further processed based on these statistics and it is therefore of great importance to be able to relate measured values S in analogue-to-digital-units (ADU) to the detected (effective) photon numbers N. The relation between the measured signal S in ADU and the photon count N is given by the linear gain g as S = gN. Only after conversion to photons is it possible to establish the expected precision of intensities in the image, which is essential for single particle localization, maximum-likelihood image deconvolution or denoising [Ober2004, Smith2010, Afanasyev2015, Strohl2015]. The photon count must be established via gain calibration, as most image capturing devices do not directly report the number of detected photons, but a value proportional to the photoelectron charge produced in a photomultiplier tube or collected in a camera pixel. For this calibration typically tens of calibration images are recorded and the linear relationship between mean intensity and its variance is exploited [vanVliet1998]. In current microscopy practise a detector calibration to photon counts is often not done but cannot be performed in retrospect. It thus would be extremely helpful, if that can be determined from analysing the acquisition itself – a single image. A number of algorithms have been published for Gaussian type noise [Donoho1995, Immerkaer1996] and Poissonian type noise [Foi2008, Colom2014, Azzari2014, Pyatykh2014]. However, all these routines use assumed image properties to extract the information rather than just the properties of the acquisition process as in our presented algorithm. This has major implications for their performance on microscopy images governed by photon statistics (see Supplementary Information for a comparison with implementations from Pyatykh et al. [Pyatykh2014] and Azzari et al. [Azzari2014] which performed more than an order of magnitude worse than our method). Some devices, such as avalanche photodiodes, photomultiplier tubes (PMTs) or emCCD cameras can be operated in a single photon counting mode [Chao2013] where the gain is known to be one. In many cases, however, the gain is unknown and/or a device setting. For example, the gain of PMTs can be continuously controlled by changing the voltage between the dynodes and the gain of cameras may deviate from the value stated in the manual. To complicate matters, devices not running in photon counting mode, use an offset Ozero to avoid negative readout values, i.e. the device will yield a non-zero mean value even if no light reaches the detector, S = gN + Ozero.  Ozero is sometimes changing over time (“offset drift”). Traditionally, a series of about 20 dark images and 20 images of a sample with smoothly changing intensity are recorded [vanVliet1998]. From these images the gain is calculated as the linear slope of the variance over these images versus the mean intensity g = var(S)/mean(S) (for details see Supplementary information). In Figure 1 we show a typical calibration curve by fitting (blue line) the experimentally obtained data (blue crosses). The obtained gain does not necessarily correspond to the real gain per detected photon, since it includes multiplicative noise sources such as multiplicative amplification noise, gain fluctuations or the excess noise of emCCDs and PMTs. In addition there is also readout noise, which includes thermal noise build-up and clock induced charge. The unknown readout noise and offset may seem at first glance disadvantageous regarding an automatic quantification. However, as shown below, these details do not matter for the purpose of predicting the correct noise from a measured signal. Let us first assume that we know the offset Ozero and any potential readout noise variance Vread. The region in Fourier space above the cut-off frequency of the support of the optical transfer function only contains noise in an image [Liu2017], where both Poisson and Gaussian noise are evenly distributed over all frequencies [Chanran1990, Liu2017]. By measuring the spectral power density of the noise VHF in this high-frequency out-of-band region and accounting for the area fraction f of this region in Fourier space, we can estimate the total variance Vall=VHF/f of all detected photons. Then the gain g is then obtained as (1) g = !!""!!!" where we relate the photon-noise-only variance Vall-Vread to the sum offset-corrected signal over all pixels in the image (see Online Methods). The device manufacturers usually provide the readout noise leaving only the offset and gain to be determined from the image itself in practise. To also estimate both, the offset together with the gain, we need more information from the linear meanvariance dependence than given by equation (1). We achieve this by tiling the input image, e.g. into 3×3 sub-images, and process each of these sub-images to generate one data point in a meanvariance plot. From these nine data points we obtain the axis offset (Ono-noise). We then perform the gain estimation (1) on the whole image after offset correction . As seen from Figure 1 the linear regression of the mean-variance curve determines the axis offset ADU value Ono-noise at which zero noise would be expected. Yet we cannot simultaneously determine both offset Ozero and readout noise Vread . If either of them is known a priori, the other can be calculated: Vread = g(Ozero Ono-noise), which is, however, not needed to predict the correct noise level for each brightness level based on the automatically determined value Ono-noise. To test the single-image gain calibration, simulated data was generated for a range of gains (0.1, 1, 10) with a constant offset (100 ADU), a range of readout noise (1, 2, 10 photon RMS) and maximum expected photon counts per pixel (10, 100, ..., 10). Offset and gain were both determined from band-limited single images of two different objects (resolution target and Einstein) without significant relative errors in the offset or gain (less than 2% at more than 10 expected maximum photon counts) using the proposed method (see Supplementary Figures S1-S3).  1 quantitatively compares the intensity dependent variance predicted by applying our method individually to many single experimental in-focus images (shaded green area) with the classical method evaluating a whole series of calibration images (blue line). Note that our single-image based noise determination does not require any prior knowledge about offset or readout noise.  2 shows a few typical example images acquired with various detectors together with the gain and offset determined from each of them and the calibration values obtained from the standard procedure for comparison. We evaluated the general applicability of our method on datasets from different detectors and modes of acquisition (CCD, emCCD, sCMOS, PMT, GAsP and Hybrid Detector).  3 quantitatively compares experimental single image calibration with classical calibration. 20 individual images were each submitted to our algorithm and the determined offset and gain was compared to the classical method. The variance of a separately acquired dark image was submitted to the algorithm as a readout noise estimate, but alternatively the readout noise specification from the handbook could be used or a measured offset at zero intensity. As seen from Figure 3 the singleimage-based gain calibration as proposed performs nearly as well as the standard gain calibration using 20 images. The relative gain error stays generally well below 10% and for cameras below 2%. The 8.5% bias for the HyD photon counting system is unusually high, and we were unable to find a clear explanation for this deviation from the classical calibration. Using only lower frequencies to estimate VHF (kt =0.4) resulted in a much smaller error of 2.5% in the single-photon counting case suggesting that dead-time effects of the detector might have affected the high spatial frequencies. Simulations as well as experiments show a good agreement of the determined gain with the ground truth or gold standard calibration respectively. The bias of the gain determined by the single-image routine stayed below 4% (except for HyD). For intensity quantification any potential offset must be subtracted before conversion to photon counts. Our method estimates the photon count very precisely over a large range of parameters (relative error below 2% in simulations). Our method could be applied to many different microscopy modes (widefield transmission, fluorescence, and confocal) and detector types (CCD, emCCD, sCMOS, PMT, GAsP and HyD photon counting), because we only require the existence of an out-of-band region, which purely contains frequency independent noise. This is usually true, if the image is sampled correctly. As discussed in the Supplementary Information the cut-off limit of our algorithm can in practise be set below the transfer limit and single-image calibration can even outperform the standard calibration if molecular blinking perturbs the measurement In summary we showed that single image calibration is a simple and versatile tool. We expect our work to lead to a better ability to quantify intensities in general.