# Optimal transfer functions for bandwidth-limited imaging

Sjoerd Stallinga, Niels Radmacher, Antoine Delon, Jörg Enderlein

► To cite this version:

Sjoerd Stallinga, Niels Radmacher, Antoine Delon, Jörg Enderlein. Optimal transfer functions for bandwidth-limited imaging. Physical Review Research, 2022, 4 (2), pp.023003. 10.1103/PhysRevResearch.4.023003. hal-04929190

HAL Id: hal-04929190

https://hal.science/hal-04929190v1

Submitted on 4 Feb 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
Optimal transfer functions for bandwidth-limited imaging

Sjoerd Stallinga*

Department of Imaging Physics, Delft University of Technology, 2628 CJ Delft, The Netherlands

Niels Radmacher†

III. Institute of Physics, Georg-August University, 37077 Göttingen, Germany

Antoine Delon ◦,‡

Université Grenoble Alpes, CNRS, LIPhy, 38000 Grenoble, France

Jörg Enderlein ◦,§

III. Institute of Physics, Georg-August University, 37077 Göttingen, Germany
and Cluster of Excellence “Multiscale Bioimaging: from Molecular Machines to Networks of Excitable Cells” (MBExC),
Georg August University, 37077 Göttingen, Germany

<figure><img src="image_2.png" /></figure>

(Received 14 October 2021; revised 17 February 2022; accepted 1 March 2022; published 1 April 2022)

One of the fundamental limits of classical optical microscopy is the diffraction limit of optical resolution. It results from the finite bandwidth of the optical transfer function (or OTF) of an optical microscope, which restricts the maximum spatial frequencies that are transmitted by a microscope. However, given the frequency support of the OTF, which is fully determined by the used optical hardware, an open and unsolved question is what is the optimal amplitude and phase distribution of spatial frequencies across this support that delivers the “sharpest” possible image. In this paper, we will answer this question and present a general rule how to find the optimal OTF for any given imaging system. We discuss our result in the context of optical microscopy, by considering in particular the cases of wide-field microscopy, confocal image scanning microscopy (ISM), 4pi microscopy, and structured illumination microscopy (SIM). Our results are important for finding optimal deconvolution algorithms for microscopy images, and we demonstrate this experimentally on the example of ISM. They can also serve as a guideline for designing optical systems that deliver best possible images, and can be easily generalized to nonoptical imaging such as telescopic imaging, ultrasound imaging, or magnetic resonance imaging.

DOI: 10.1103/PhysRevResearch.4.023003

I. INTRODUCTION

Conventional optical microscopes have a resolution limit
due to the wave nature and resulting diffraction of light. This
is embodied by the fact that a microscope transmits only spa-
tial frequencies of a sample up to some maximum values. The
total set of all transmitted frequencies comprises the support
of the so-called optical transfer function (OTF). The OTF
is the Fourier transform of the point spread function (PSF),
the image of an ideal infinitesimally small point emitter.
Recorded data are often deconvolved using a model of image
formation for the given imaging system [1–6]. The goals of
such a deconvolution can be several: (i) correcting optical

aberrations of the imaging system, (ii) maximizing spatial
resolution and contrast, or (iii) minimizing out-of-focal plane
contributions. An appropriate model of the imaging system
is also required for nondirect imaging methods, that need an
image reconstruction procedure to arrive at the final image.
For example, image reconstruction in structured illumination
microscopy (SIM) [7–11] requires finding an optimal apodiza-
tion function. In any case, an optimal deconvolution or image
reconstruction needs a valid model of the “ideal” OTF (or,
equivalently PSF) for a given imaging system, which would
yield the best possible image of a sample.

The goals of this paper are threefold. First, we want to
answer the fundamental question in optical imaging what the
ideal OTF is given the support in spatial frequency space, and
how such an ideal OTF can be found from first principles.
Second, we want to illustrate this concept to find the ideal
OTF for a range of microscopic imaging modalities. Third,
we wish to demonstrate a practical application of the concept
of an ideal OTF in linear deconvolution.

II. THEORETICAL FOUNDATION

In what follows, we will focus on image formation in flu-
orescence microscopy. Let us start by considering the image

Published by the American Physical Society under the terms of the
Creative Commons Attribution 4.0 International license. Further
distribution of this work must maintain attribution to the author(s)
and the published article's title, journal citation, and DOI.

<footnote>*s.stallinga@tudelft.nl</footnote>

<footnote>†niels.radmacher@uni-goettingen.de</footnote>

<footnote>‡antoine.delon@univ-grenoble-alpes.fr</footnote>

<footnote>§jenderl@gwdg.de</footnote>
formation of a general optical microscope. To be as general as possible, we consider three-dimensional image formation, either by taking consecutive images at different focal planes along the optical axis with a wide-field microscope (with or without structured illumination), or by three-dimensional point scanning with a laser scanning microscope (e.g., conventional confocal microscope, image scanning microscope, STED microscope). For an optical system that is isoplanatic along all directions [12], the final 3D image $I(\mathbf{r})$ is related to the sample function $S(\mathbf{r})$ (distribution density of fluorescent emitters) by a linear convolution of the form

$$I(\mathbf{r}) = \int d\mathbf{r}' U(\mathbf{r}-\mathbf{r}') S(\mathbf{r}') \Leftrightarrow \tilde{I}(\mathbf{k}) = \tilde{U}(\mathbf{k}) \tilde{S}(\mathbf{k}), \quad (1)$$

where $\mathbf{r}$ and $\mathbf{k}$ are the coordinates in real and Fourier space, respectively, and a tilde above a symbol denotes its Fourier transform. Here, $U(\mathbf{r})$ is the PSF, and $\tilde{U}(\mathbf{k})$ is its Fourier transform, the OTF. Along all directions $\hat{\mathbf{n}}$, there exists a maximum value $k_{\max}(\hat{\mathbf{n}})$ beyond which the OTF is zero, which means that no spatial frequencies of the sample distribution $\tilde{S}(\mathbf{k})$ beyond values $k_{\max}(\hat{\mathbf{n}})\hat{\mathbf{n}}$ are transmitted by the microscope (final support of the OTF). This fundamentally limits the achievable spatial resolution along direction $\hat{\mathbf{n}}$ to $2\pi/k_{\max}$. We will denote by $\mathcal{M}$ the set of all points of the support of the OTF in Fourier space, i.e.,

$$\mathcal{M} = \{\mathbf{k} \mid |\tilde{U}(\mathbf{k})| > 0\}. \quad (2)$$

This support $\mathcal{M}$ is solely determined by the microscope's hardware properties, such as the numerical aperture of its objective, the size of the confocal aperture and magnification at the aperture's position (for a confocal microscope), the spatial modulation period of the excitation intensity (for a structured illumination microscope), or the intensity of the stimulated emission laser (for a STED microscope). But given this support $\mathcal{M}$, the question is what is the distribution of complex-valued amplitudes over $\mathcal{M}$ that yields the best possible PSF.

At this point it makes sense to make more explicit what we understand to be the ideal OTF and PSF. Our position is that an ideal PSF of an incoherently imaging fluorescence microscope should be (i) positive everywhere ($U(\mathbf{r}) \ge 0$), as this avoids edge ringing and negative pixel artifacts in linear deconvolution and image filtering, as well as being consistent with the physical requirement of having nonnegative image signals, (ii) as sharp as possible, which can be quantified by a maximum OTF summed over the OTF support, or equivalently a maximum PSF peak value

$$U(0) = \int_{\mathcal{M}} \frac{d\mathbf{k}}{(2\pi)^p} \tilde{U}(\mathbf{k}), \quad (3)$$

where $D$ is the dimensionality of $\mathcal{M}$. The advantage of a sharpness metric that is based on the OTF is that it is independent of any specific object features, in contrast to, e.g., the steepness of imaged edges.

There is a large interdependency between the above two requirements. For example, a gain in contrast and overall sharpness may be achieved by having a relatively high OTF for large spatial frequencies, as this will emphasize the representation of small scale features in the image. It appears, however, that an unbalanced OTF, with comparable or even

higher values for larger spatial frequencies than for lower spatial frequencies, gives rise to violations of the positivity constraint for the PSF. A metric to quantify sharpness should therefore take into account the OTF at all spatial frequencies, a straightforward way to do this is to simply take the average over all spatial frequencies in the OTF support. The quest for a best possible OTF giving rise to a positive PSF has already been addressed by Lukosz, who has derived upper bounds for such an OTF [13,14]. The concept of the Lukosz bound has also been applied in image reconstruction for SIM [15,16]. Here, we improve on these results by proposing an exact recipe for obtaining the ideal OTF given the support $\mathcal{M}$.

Let us start with the simplest case of an infinitely extended support in one dimension. This is a trivial case with the well-known answer that a constant OTF, $\tilde{U}(\mathbf{k}) = \text{const}$, yields the most narrow PSF, namely a delta function. In that case, the image $I(\mathbf{r})$ and the sample function $S(\mathbf{r})$ become identical. For the one-dimensional case with a limited support $|k| < k_{\max}$, one may still assume that the Fourier transform of uniform amplitude, $\tilde{U}(k) = \text{const}$ for $|k| < k_{\max}$, yields the most compact PSF. A quick calculation shows that this leads to a function proportional to $\sin(k_{\max}x)/x$, which is not strictly nonnegative, while strict nonnegativity is a fundamentally important requirement for a physically valid PSF. Expanding on this idea, it is proposed that the most compact PSF is given by the square of this function, $[\sin(k_{\max}x/2)/x]^2$, which corresponds to the Fourier transform of the autoconvolution of the rescaled original uniform amplitude distribution, $\tilde{U}(2\mathbf{k})$, over half the support $|k| < k_{\max}/2$, i.e., $\tilde{U}(2\mathbf{k}) \otimes \tilde{U}(2\mathbf{k})$. This autoconvolution does indeed have the original support $|k| < k_{\max}$, and its Fourier transform is by definition strictly nonnegative. Generalizing this idea to three dimensions, we arrive at the core hypothesis of our paper: For a given support $\mathcal{M}$, the most compact positive PSF is found as the Fourier transform of an OTF, which is the autoconvolution of a uniform amplitude distribution.

This problem is closely connected to Minkowski sums. A Minkowski sum of two sets of points $A$ and $B$ is defined by

$$A \oplus B = \{\mathbf{a} + \mathbf{b} \mid \mathbf{a} \in A \wedge \mathbf{b} \in B\}. \quad (4)$$

If one has two functions $\tilde{f}(\mathbf{k})$ and $\tilde{g}(\mathbf{k})$ defined over finite supports $A$ and $B$, respectively, then their convolution $\tilde{f}(\mathbf{k}) \otimes \tilde{g}(\mathbf{k})$ has the support $A \oplus B$. Thus, given the set of points $\mathcal{M}$ of an OTF's support, what we need to find is another set of points $\mathcal{M}^{1/2}$ so that

$$\mathcal{M} = \mathcal{M}^{1/2} \oplus \mathcal{M}^{1/2}. \quad (5)$$

Knowing this set $\mathcal{M}^{1/2}$, the ideal OTF is found by an autoconvolution of a uniform amplitude distribution over $\mathcal{M}^{1/2}$, and the ideal PSF is the Fourier transform of this OTF. It is trivial to find $\mathcal{M}^{1/2}$ for a convex set $\mathcal{M}$, which is $\mathcal{M}$ scaled down in extent by a factor of 2. For general sets $\mathcal{M}$, in particular nonconvex or disjoint sets, there is no general algorithm known for finding $\mathcal{M}^{1/2}$. However, for many cases of practical relevance in microscopic imaging, one can find $\mathcal{M}^{1/2}$ by considering how the OTF is physically calculated.

We will now show that a uniform phase and amplitude distribution across $\mathcal{M}^{1/2}$ gives rise to a maximum sharpness metric $U(0)$ as defined in Eq. (3). That is, given a Minkowski
decomposition of the OTF support $\mathcal{M}$ in Eq. (5) we can construct the ideal OTF. A positive (and real) PSF can always be written as $U(\mathbf{r}) = |E(\mathbf{r})|^2$, where the field distribution $E(\mathbf{r})$ can be written as a Fourier transform:

$$E(\mathbf{r}) = \int \frac{d\mathbf{k}}{(2\pi)^D} \tilde{E}(\mathbf{k}) \exp(i\mathbf{k} \cdot \mathbf{r}), \quad (6)$$

where the support of $\tilde{E}(\mathbf{k})$ is $\mathcal{M}^{1/2}$. Without loss of generality we may write [for $\mathbf{k} \in \mathcal{M}^{1/2}$]:

$$\tilde{E}(\mathbf{k}) = E_0 \exp(iW_c(\mathbf{k})), \quad (7)$$

where $W_c(\mathbf{k}) = W(\mathbf{k}) + iV(\mathbf{k})$ is the complex aberration function, with real and imaginary parts $W(\mathbf{k})$ and $V(\mathbf{k})$ that represent the phase and amplitude aberration functions, respectively. The constant $E_0$ is fixed by the normalization condition:

$$\begin{aligned} 1 &= \int d\mathbf{r} |E(\mathbf{r})|^2 = \int \frac{d\mathbf{k}}{(2\pi)^D} |\tilde{E}(\mathbf{k})|^2 \\ &= E_0^2 \int_{\mathcal{M}^{1/2}} \frac{d\mathbf{k}}{(2\pi)^D} |\exp(iW_c(\mathbf{k}))|^2. \end{aligned} \quad (8)$$

With the aid of this equation, $E_0$ can be solved and we find for the PSF peak value:

$$U(0) = \frac{1}{(2\pi)^D} \frac{\left| \int_{\mathcal{M}^{1/2}} d\mathbf{k} \exp(iW_c(\mathbf{k})) \right|^2}{\int_{\mathcal{M}^{1/2}} d\mathbf{k} |\exp(iW_c(\mathbf{k}))|^2}. \quad (9)$$

The PSF peak value is usually expressed in the dimensionless Strehl ratio:

$$S = \frac{U(0)}{A_p} = \frac{|\langle \exp(iW_c) \rangle|^2}{\langle |\exp(iW_c)|^2 \rangle}, \quad (10)$$

where the *pupil area* is defined as

$$A_p = \int_{\mathcal{M}^{1/2}} \frac{d\mathbf{k}}{(2\pi)^D}, \quad (11)$$

and where the *pupil average* of any function $F(\mathbf{k})$ is defined as

$$\langle F \rangle = \frac{1}{A_p} \int_{\mathcal{M}^{1/2}} \frac{d\mathbf{k}}{(2\pi)^D} F(\mathbf{k}). \quad (12)$$

Using $|\langle a|b \rangle|^2 \le \langle a|a \rangle \langle b|b \rangle$ for $a = \exp(iW_c)$ and $b = 1$ we find that $S \le 1$ and that the maximum is obtained in case $W_c(\mathbf{k}) = 0$, i.e., for zero phase and amplitude aberrations, leading to a uniform complex amplitude distribution over the pupil support $\mathcal{M}^{1/2}$. This finalizes the proof, and enables a direct computation of the ideal OTF once the Minkowski decomposition of the OTF support is made.

An interesting corollary of the current formalism is that to lowest order in the phase and amplitude aberration functions:

$$S = 1 - W_{\text{rms}}^2 - V_{\text{rms}}^2, \quad (13)$$

where the root mean square phase aberration is given by $W_{\text{rms}}^2 = \langle W^2 \rangle - \langle W \rangle^2$ (and likewise for the root mean square amplitude aberration $V_{\text{rms}}$). This suggests a generalization of Maréchal's criterion for aberration tolerances to include amplitude aberrations as well. In fact, in the current form phase and amplitude aberrations can be treated on an equal footing.

It appears that the decomposition of the OTF support $\mathcal{M}$ into the pupil $\mathcal{M}^{1/2}$ via the Minkowski sum is not unique.

Consider for example the case of a circular support of a 2D-OTF. Then our procedure leads to a circular pupil scaled down twofold in size compared to the OTF support. An annular pupil, however, is an equally valid solution $\mathcal{M}^{1/2}$ to the Minkowski sum decomposition. Such pupil shapes can be incorporated into the current formalism because these sets $\mathcal{M}^{1/2}$ are subsets of the full set $\mathcal{M}^{1/2}$, i.e., $\mathcal{M}^{1/2} \subset \mathcal{M}^{1/2}$ for any solution $\mathcal{M}^{1/2}$ to the Minkowski sum decomposition. This can be taken into account by an amplitude aberration function $V(\mathbf{k}) = 0$ for $\mathbf{k} \in \mathcal{M}^{1/2}$ and $V(\mathbf{k}) = \infty$ for $\mathbf{k} \notin \mathcal{M}^{1/2}$. Then the Strehl-ratio can be computed as

$$S = \frac{1}{A_p} \int_{\mathcal{M}^{1/2}} \frac{d\mathbf{k}}{(2\pi)^D}, \quad (14)$$

which satisfies $S \le 1$, i.e., the overall performance, quantified by the summed OTF over the support $\mathcal{M}$, is always worse. This fits with the well-known behavior that annular pupils can give rise to a narrower central peak of the PSF, but always at the expense of enhanced side lobes [17–19]. We mention that for a small range of spatial frequencies the use of annular pupils possibly gives rise to a higher OTF than out ideal OTF, but for the average OTF over all spatial frequencies this does not appear to be the case.

In what follows, we will demonstrate the application of our concept of an optimal PSF/OTF on four examples: a wide-field microscope, an image scanning microscope, a 4p microscope, and a structured illumination microscope.

### III. WIDE-FIELD MICROSCOPE

As a first example, we consider a perfectly imaging, aberration-free wide-field microscope equipped with an objective of numerical aperture NA. Here and it the next sections, we will work in the scalar approximation of optical imaging, i.e., neglecting the vector character of the electric-magnetic field—discussion and justification of our results in the light of a full vector theory of electromagnetic radiation will be given in Sec. VII. Thus, following the scalar theory of imaging through a high-aperture optics, the electric field distribution of the image of a point emitter is given by Eq. (6) where the finite two-dimensional integration domain is an axisymmetric cap of the sphere $|\mathbf{k}| = nk_0$ defined by arctan($k_\perp/k$) $\le \theta_{\text{max}}$, with $k_0$ being the length of the emission light’s wave vector in vacuum, n the refractive index of the objective’s immersion medium (sample space), $k_\perp$ the modulus of the transverse component of $\mathbf{k}$ perpendicular to the optical axis, and $\theta_{\text{max}}$ the maximum half angle of light collection of the microscope’s objective. This angle is connected with the objective’s numerical aperture by NA = $n \sin \theta_{\text{max}}$. A cross section of the integration domain is shown in Fig. 1(a). It is important to note that the above electric field distribution is given as a function of the coordinate $\mathbf{r}$ in—the corresponding transverse position in image space is the transverse component of $\mathbf{r}$ multiplied by magnification $M$.

For an isotropically emitting point source, energy conservation implies that the amplitude function $\tilde{E}(\mathbf{k})$ is equal to $\sqrt{\cos\theta/\cos\theta'}$, where $\theta = \arctan(k_\perp/k_\parallel)$ with $k_\parallel$ being the component of the wave vector along the optical axis, and the angle $\theta'$ in image space is connected to $\theta$ via Abbe’s sine condition, $Mn' \sin\theta' = n \sin\theta$, where $M$ is the image
<figure><img src="image_3.png" /></figure>

FIG. 1. OTF of a wide-field microscope. [(a),(b)] $(k_x, k_z)$-cross section of the axisymmetric Fourier transform $\tilde{E}(\mathbf{k})$ of the electric field distribution, Eq. (6). Optical axis is along $k_z$. Extent along the $k_x$ direction is $\pm kNA/n$, and along the $k_z$ direction $k[1 - \sqrt{1 - (NA/n)^2}]$. Numerical aperture NA of the microscope's objective was set to 1.2, refractive index $n$ of objective's immersion medium (sample space) to 1.33 (water immersion), and image magnification $M$ to 60. (c) OTF of wide-field microscope as convolution of (a) with (b), i.e., the autoconvolution of $\tilde{E}(\mathbf{k})$ with an extent double as large in all directions as that of the Fourier transform in (a) and (b). All three distributions in (a)–(c) are normalized by their maximum value.

magnification, and $n'$ the refractive index of image space (typically $n' = 1$) [20,21].

The PSF is the absolute square of the electric field distribution, $U(\mathbf{r}) = |\mathbf{E}(\mathbf{r})|^2$, which means that the OTF is the three-dimensional autoconvolution of $\tilde{E}(\mathbf{k})$, as shown in Fig. 1. For axisymmetric functions such as $\tilde{E}(\mathbf{k})$ considered here, such autoconvolutions can be computed using the theory of Ref. [22]. We follow the numerically efficient and more generally applicable method using Hankel transforms, see SI of Ref. [23]. The resulting PSF, i.e., Fourier transform of the OTF shown in Fig. 1(c), is presented in Fig. 2(a).

Following our hypothesis, the most optimal OTF is obtained by an autoconvolution of a uniform amplitude distribution yielding the same support as the original OTF. As already mentioned, this reduces to finding a set of points $M^{1/2}$ so that its Minkowski sum with itself yields the support of the OTF in Fig. 1(c). For such nonconvex sets, no general

<figure><img src="image_4.png" /></figure>

FIG. 3. OTF of ISM. (a) Autoconvolution of the Fourier transform of the focused excitation electric field, Eq. (6), with $\tilde{E}(\mathbf{k}) = \sqrt{\cos\theta}$, and for a modulus of the wave vector, which is assumed to be 6/5 that of the fluorescence emission light. (b) OTF of light detection efficiency distribution for light detection with an infinitely small pinhole. All other microscope parameters were again set to the same values as for the wide-field microscopy example, i.e., NA = 1.2, $n = 1.33$, and $M = 60$. (c) OTF of ISM obtained by the convolution of (a) with (b).

solution for finding $M^{1/2}$ is known, but in the present case, it is obviously known from the very physics of how the OTF is calculated, and is the support of the Fourier transform of the electric field as shown in Figs. 1(a) and 1(b). Thus, the ideal OTF is found by replacing in (6) the $k$-dependent amplitude by a constant one, $\tilde{E}(\mathbf{k}) = \text{const}$, and then autoconvolving the result. We do not show the resulting OTF, which looks very similar to the original one, but present the result for the corresponding PSF in Fig. 2(b). One finds indeed a slight improvement in spatial resolution (quantified by the square root of the variance values of three-dimensional Gaussian distributions fitted to the PSFs). However, the improvement in lateral resolution is only ca. 10%, and in axial resolution it is negligible. Thus, a perfectly adjusted wide-field microscope operates close to the absolute optimum in the sense that it transmits the spatial frequencies of the sample in a close-to-optimum manner.

## IV. IMAGE SCANNING MICROSCOPY

Next, we consider an image scanning microscope (ISM), that scans the sample with a diffraction-limited focus, and records at each scan position an image of the excited fluorescence. From the resulting 3 + 2-dimensional data set (three dimensions for scan positions, two dimensions for the images taken at each scan position), a final image is calculated that is equivalent to an image recorded with a conventional laser-scanning confocal microscope having an infinitely small confocal aperture. Thus, the PSF of the final ISM image is given by the product of the excitation intensity distribution and the light collection efficiency of detecting light through an infinitely small confocal aperture. The Fourier transform of the former is given by the autoconvolution of the Fourier representation of the electric field in sample space when focusing a perfectly plane wave through the objective. This Fourier representation is given again by (6), but now with amplitude function $\tilde{E}(\mathbf{k}) = \sqrt{\cos\theta}$, and at the wavelength of the excitation light (which is smaller than that of the fluorescence emission, resulting in a larger modulus $k$ of the wave vector). This autoconvolution of this Fourier transform is shown in Fig. 3(a). The Fourier transform of the light collection

<figure><img src="image_5.png" /></figure>

FIG. 2. Comparison of actual and ideal PSFs of a wide-field microscope. (a) PSF obtained via a Fourier transform of the OTF shown in Fig. 1(c). Length of shown yellow coordinate vectors is one wavelength. (b) Ideal PSF obtained from the OTF calculated as the autoconvolution of two *uniform* amplitude distributions with the same frequency support as those shown in Figs. 1(a) and 1(b). At the top of both panels are indicated the square root variances (in units of wavelength) of the distributions when fitting their lateral and axial projections with Gaussian functions. The curves on the left show cross sections along the x and z directions for both panel (a) (solid lines) and panel (b) (dotted lines).
<figure><img src="image_6.png" /><figcaption>FIG. 4. Ideal OTF of ISM. [(a),(b)] Uniform amplitude distributions over the frequency support of the convolution of the Fourier transforms of excitation and detection electric fields. Please note the hole of missing frequencies in the middle, which is due to the different wavelengths (and thus lengths of the wave vectors) of excitation and emission. (c) Ideal OTF of ISM as obtained by the autoconvolution of (a) and (b).</figcaption></figure>

<figure><img src="image_7.png" /><figcaption>FIG. 6. OTF of a type-C 4pi microscope. (a) Fourier transform of the excitation intensity distribution. Again, excitation wavelength is set to 5/6 of fluorescence emission wavelength. (b) Fourier transform of the light detection efficiency distribution. All microscope parameters were again set to the same values as for the wide-field microscopy example, i.e., NA = 1.2, n = 1.33, and M = 60. (c) OTF of a type-C 4pi microscope obtained by the convolution of (a) with (b).</figcaption></figure>

efficiency for an infinitely small pinhole is the same as the OTF of a wide-field microscope as shown in Fig. 1(c), and is reproduced in Fig. 3(b). Finally, the result for the OTF of ISM is shown in Fig. 3(c), and the corresponding PSF in Fig. 5(a).

Again, following our core hypothesis, the ideal OTF is obtained as an autoconvolution of a uniform amplitude distribution yielding the same support as that of the OTF shown in Fig. 3(c). For finding the corresponding $M^{1/2}$, we can use the associative property of Minkowski addition: For example, for two sets $A$ and $B$ we have $(A \oplus A) \oplus (B \oplus B) = (A \oplus B) \oplus (A \oplus B)$. Thus, in our case of ISM, the sought-after $M^{1/2}$ is the support of the convolution of the Fourier transform of the excitation electric field with the Fourier transform of the detection electric field. This is shown in Figs. 4(a) and 4(b). Thus, the ideal OTF of ISM is found as the autoconvolution of a uniform amplitude distribution over this support and is shown in Fig. 4(c). The resulting PSF is presented in Fig. 5(b). As can be seen, for ISM, the optimal amplitude distribution yields a significantly smaller PSF, and thus an appropriate deconvolution of ISM images using this PSF can indeed significantly improve image resolution and contrast (brightness per image area) [24].

V. 4PI MICROSCOPY

As a third example, we consider the case of a so-called type-C 4pi microscope [25,26]. In such a microscope, the sample is illuminated from both sides through two identical

objectives that constructively focus the light of two mutually coherent laser beams, and the excited fluorescence emission is detected through the same two objectives by constructive interference, as is done also in 3D interferometric photoactivated localization fluorescence microscopy (iPALM) [27] or isotropic stimulated emission depletion (isoSTED) microscopy [28]. The Fourier representation of the excitation electric field is again given by Eq. (6), but now with an integration domain $\Omega$ consisting of two segments of the sphere $|\mathbf{k}| = nk_0$ defined by $|\arccos(k_\parallel/k_\perp)| \le \theta_{max}$. The resulting distribution looks very similar to the one shown in Fig. 1(a), but with a second spherical segment that is a mirror image of the one shown reflected on the horizontal axis, and both having a larger radius (due to the excitation wavelength being shorter than the emission wavelength). The autoconvolution of such a distribution and thus the Fourier transform of the excitation intensity distribution is presented in Fig. 6(a). The Fourier transform of the light collection efficiency distribution is obtained analogously and is shown in Fig. 6(b). Thus, the OTF of a type-C 4pi microscope is finally obtained by the convolution of both Fourier transforms and shown in Fig. 6(c). The corresponding PSF is seen in Fig. 8(a).

Using the same considerations as in the previous example, the set $M^{1/2}$ is obtained as the support of the convolution of the Fourier electric field representations for excitation and detection, and is shown in Figs. 7(a) and 7(b). The autoconvolution of a uniform amplitude distribution over this support

<figure><img src="image_8.png" /><figcaption>FIG. 5. (a) PSF of ISM as obtained via the Fourier transform of the OTF shown in Fig. 3(c). (b) Ideal PSF of ISM as obtained by a Fourier transform of the ideal OTF shown in Fig. 4(c). The curves on the left show cross sections along the x and z directions for both panel (a) (solid lines) and panel (b) (dotted lines).</figcaption></figure>

<figure><img src="image_9.png" /><figcaption>FIG. 7. Ideal OTF of a type-C 4pi microscope. [(a),(b)] Uniform amplitude distributions over the frequency support of the convolution of the Fourier transforms of excitation and detection electric fields. (c) Ideal OTF of a type-C 4pi microscope as obtained by the autoconvolution of (a) and (b).</figcaption></figure>
<figure><img src="image_10.png" /><figcaption>FIG. 8. (a) PSF of a type-C 4pi microscope, being the Fourier transform of the OTF seen in Fig. 6(c). (B) Ideal PSF of a type-C 4pi microscope calculated as the Fourier transform of the OTF seen in Fig. 7(c). The given values of square root variances, $\sigma_{x,z}$, refer to fits of the lateral and axial projections of the central peaks of the PSFs with Gaussian functions. The curves on the left show cross sections along the x and z directions for both panel (a) (solid lines) and panel (b) (dotted lines).</figcaption></figure>

then yields the ideal OTF of a type-4 4pi microscope and is shown in Fig. 7(c). Its Fourier transform then gives the ideal PSF as presented in Fig. 8(b). As can be seen, this PSF is in size not much different from that of the central peak of the original one, but now the axial side peaks are significantly suppressed. Using this ideal PSF for image deconvolution promises an efficient suppression of ghost images due to the axial side peaks in the original PSF, which is a serious issue in 4pi microscopy.

## VI. STRUCTURED ILLUMINATION MICROSCOPY

As a last example we consider a three-dimensional structured illumination microscope (3D SIM) [9–11]. This example is different from the previous ones because the OTF and PSF now, as will be seen, are no longer axially symmetric. In a 3D SIM microscope, illumination is done by the superposition of three plane waves, one traveling along the optical axis and two with propagation angle $\theta = \pm \arcsin(\beta NA/n)$ with respect to the optical axis while sharing a common incidence plane. Here $\beta$ is a numerical coefficient that is chosen to be close to 1. For the sake of simplicity we will set it to the best possible value $\beta = 1$ in the following. This generates a 3D periodic excitation intensity pattern in sample space. The Fourier transform of this intensity distributions consists of seven delta-function peaks in the plane of incidence of the inclined excitation plane waves and is shown in Fig. 9(a). The OTF of detection of the 3D SIM is the same as that of a wide-field microscope, Fig. 1(c), and is shown, for the sake of completeness, again in Fig. 9(b). By taking several images for different relative positions and orientations (usually by rotating around the optical axis in angular steps of 60°) of this excitation intensity pattern with respect to a sample, and applying a sophisticated image reconstruction algorithm, one finally obtains an image with doubled lateral resolution (as compared to a wide-field microscope) and an axial resolution and sectioning comparable to a confocal microscope.

It is difficult to define a generic OTF for 3D SIM, because the OTF (and PSF) will depend on the specific details of the image reconstruction algorithm used, such as the different spatial frequency filtering steps. Whatever the image reconstruction algorithm is, the resulting OTF will have the same

<figure><img src="image_11.png" /><figcaption>FIG. 9. Comparison of spectral support of the OTFs for 3D SIM and for ISM. For the sake of simplicity, the excitation and detection wavelengths are set equal. (a) OTF of excitation intensity distribution of 3D SIM. Shown are the 7 peaks at the center {0, 0, 0} and positions $\mathbf{k} = nk_0\{\pm 2 \sin \theta_{\text{max}}, 0, 0\}$ and $\mathbf{k} = nk_0\{\pm \sin \theta_{\text{max}}, 0, \pm 2(1 - \cos \theta_{\text{max}})\}$. The green-dotted line shows the frequency support of the corresponding OTF for ISM excitation, which is identical to the OTF shown in the middle panel. (b) OTF of wide-field detection. (c) Superposition of (b) shifted to all peak positions shown in (a) plus all additional peak positions obtained by rotating the peaks around the vertical (optical) axis in steps of 60°. The result visualizes the coverage of the frequency space by 3D SIM. Green line shows the limit of the frequency support for the ISM OTF, obtained by autoconvolution of (b).</figcaption></figure>

support as the linear superposition of copies of the distribution shown in Fig. 9(b), shifted to the peak positions of Fig. 9(a). This is visualized in Fig. 9(c) showing a cross section of such a superposition obtained by shifting the OTF of Fig. 9(c) to all 19 peak positions in 3D, 7 as shown in Fig. 9(a), and 12 additional from rotations of the nonaxial peaks around the vertical axis by 60° and 120°. This cross section visualizes the frequency range covered by SIM. A direct application of our procedure, i.e., finding the effective pupil support $M^{1/2}$ given the OTF support $M$, appears to be too involved in view of the complex rotationally asymmetric shape of the OTF support for 3D SIM. A good approximation, however, can be found by comparison to the case of ISM. The green line in Fig. 9(c) delimits the frequency support of the OTF of an ideal ISM (for the situation of no Stokes shift) as obtained through an autoconvolution of the distribution of Fig. 9(b). As can be seen, SIM covers nearly the same frequency space as ISM, and thus a fairly good approximation of the optimal OTF of 3D SIM is given by that of the corresponding ISM (same excitation and emission wavelengths). Thus, the ideal OTF (and PSF) of 3D SIM is expected to be very similar and well approximated by that of ISM as derived in the section about ISM above.

## VII. HIGH NUMERICAL APERTURE AND POLARIZATION EFFECTS

Up to now, all above considerations were based on a scalar approximation of image formation, thus neglecting the vector character of the electromagnetic field and corresponding polarization effects in fluorescence excitation and imaging, which become especially important for objectives with high numerical aperture. However, as will be explained in the following, our considerations and definition of the ideal PSF and OTF will be fully valid also for the general vectorial case. As an example, let us consider image formation in a conventional wide-field microscope. Following the wave-vector theory of
Richardson and Wolf [20,21], a point dipole emitter with dipole amplitude vector **d** positioned in the focal plane and on the optical axis generates an electric field distribution in the image plane of the microscope that is proportional to the following plane-wave superposition:

$$ 
\mathbf{E}(\mathbf{r}) = \int_{0}^{2\pi} d\psi \int_{0}^{\eta_{\text{max}}} d\eta \sin \eta \sqrt{\frac{\cos \eta}{\cos \eta'}} \cdot [\hat{\mathbf{e}}_{p}(\hat{\mathbf{e}}'_{p} \cdot \mathbf{d}) + \hat{\mathbf{e}}_{s}(\hat{\mathbf{e}}_{s} \cdot \mathbf{d})]e^{i\mathbf{k} \cdot \mathbf{r}}, \quad (15)
$$

where $\eta$ and $\psi$ are azimuthal and polar integration angles, the relation between angles $\eta$ and $\eta'$ is given by Abbe's sine condition, $M \sin \eta = n \sin \eta'$, with M being the image magnification and n the refractive index of the objective's immersion medium (ideally equal to that of the sample), $\mathbf{k} = -k_0 \{\sin \eta \cos \psi, \sin \eta \sin \psi, \cos \eta\}$ is the wave vector along the propagation direction of the plane waves, with $k_0 = 2\pi/\lambda$ its length at wavelength $\lambda$, and $\hat{\mathbf{e}}_p = \{\cos \eta \cos \psi, \cos \eta \sin \psi, -\sin \eta\}$, $\hat{\mathbf{e}}'_p = \{\cos \eta' \cos \psi, \cos \eta' \sin \psi, \sin \eta'\}$ and $\hat{\mathbf{e}}_s = \{-\sin \psi, \cos \psi, 0\}$ are azimuthal and polar polarization unit vectors. The maximum integration value $\eta_{\text{max}}$ is defined by the numerical aperture NA of the objective via $\eta_{\text{max}} = \arcsin(NA/M)$. The factor $\sqrt{\cos \eta' / \cos \eta}$ assures energy conservation during imaging. The corresponding magnetic field distribution is given by a similar plane wave superposition

$$ 
\mathbf{B}(\mathbf{r}) = \frac{1}{c} \int_{0}^{2\pi} d\psi \int_{0}^{\eta_{\text{max}}} d\eta \sin \eta \sqrt{\frac{\cos \eta}{\cos \eta'}} \cdot [-\hat{\mathbf{e}}_{s}(\hat{\mathbf{e}}'_{p} \cdot \mathbf{d}) + \hat{\mathbf{e}}_{p}(\hat{\mathbf{e}}_{s} \cdot \mathbf{d})]e^{i\mathbf{k} \cdot \mathbf{r}} \quad (16)
$$

and the final intensity distribution in the image plane is given by the time-averaged Poynting vector component along the optical axis, which is

$$ U_d(\mathbf{r}) = -\frac{c}{8\pi} \Re[\hat{\mathbf{e}}_z \cdot (\mathbf{E} \times \mathbf{B})] \quad (17) $$

with $\hat{\mathbf{e}}_z$ being a unit vector along the optical axis (z axis). Please note that all equations were written in such a coordinate system where the light from the dipole to the image plane propagates along the negative z direction. For an isotropic distribution of emitters, the last expression has still to be averaged over all dipole orientations, which then yields the final PSF of the system as

$$ U(\mathbf{r}) = \langle U_d(\mathbf{r}) \rangle_d = -\frac{c}{8\pi} \langle \Re[\mathbf{e}_z \cdot (\mathbf{E} \times \mathbf{B})] \rangle_d, \quad (18) $$

where $\langle \cdot \rangle_d$ denotes averaging over all orientations of **d**.

Although this expression for the PSF looks now much more complicated than what we have seen for all PSFs in the scalar approximation of the previous sections, the core difference is the presence of the vectorial prefactors in front of the exponents $\exp(i\mathbf{k} \cdot \mathbf{r})$ in the above integrals, so that the integrals return vector functions instead of scalar functions. However, when comparing these integrals with the scalar approximation of Eq. (6), one can see that the frequency support of the Fourier transforms of all these functions is identical! Furthermore, the PSF calculated with the vectorial representations of the

electric and magnetic field is given by the real part of a cross product of both fields, which corresponds in Fourier space to a sum of several scalar convolutions. As a result, also the frequency support of this Fourier-transformed PSF is identical to that of the OTF calculated in the scalar approximation. The vector character of the electric and magnetic field makes the calculations of the PSF and OTF much more complicated than in the scalar approximation, but it does not at all change the frequency support of the final OTF. This support remains the same, irrespective of any alterations to the amplitude, phase, or polarization in the pupil or the light path towards the detector.

Thus, all our previous derivations of ideal PSFs and OTFs remain fully intact, and one can use them with same validity for optimally deconvolving images that are obtained with high-NA objectives where vector effects can no longer be neglected. Even more, in the case of a wide-field microscope, the complex vectorial structure of the electric and magnetic fields will lead to a PSF/OTF that is suboptimal (because it will inadvertently lead to a nonuniform amplitude distribution of the resulting OTF in Fourier space), but which can now be improved by deconvolving an image using the ideal OTF as derived in the scalar approximation. It remains an open question how close the ideal OTF can be approximated in a high-NA wide-field microscope by engineering the phase, amplitude, and polarization in the pupil plane. This would amount to maximizing the z component of the Poynting vector in focus, given the total power flowing through the microscope's pupil towards the detector. An approach similar to Ref. [29], where the electric field magnitude in focus is optimized, could be a good starting point.

## VIII. APPLICATION IN LINEAR DECONVOLUTION

The concept of an ideal OTF is important for often applied linear deconvolution methods. Let us assume, the actual experimental PSF of a given microscope is $U_{\text{exp}}(\mathbf{r})$, which can take into account all kinds of imperfections such as optical aberrations, angle-dependent transmission efficiency of the objective etc. These imperfections will affect the measured image $I_{\text{exp}}(\mathbf{r})$ (blurring, reducing contrast, etc.). If one knows the ideal OTF $\tilde{U}(\mathbf{k})$ over the same frequency support as that of $\tilde{U}_{\text{exp}}(\mathbf{k})$, one can calculate a rectified Fourier-transformed image as

$$ \tilde{I}(\mathbf{k}) = \frac{\tilde{U}(\mathbf{k})}{\tilde{U}_{\text{exp}}(\mathbf{k})} \tilde{I}_{\text{exp}}(\mathbf{k}). \quad (19) $$

Just dividing by $\tilde{U}_{\text{exp}}(\mathbf{k})$ seemingly results in a better, more sharp, outcome of the deconvolution. In that case, however, the effective OTF will be essentially flat across the OTF support $M$, which will violate the PSF positivity constraint. The key thesis of this paper, namely that the best possible bandwidth limited linear representation of any object is the convolution of the ground truth object with the ideal OTF, implies that the multiplication with the ideal OTF in Eq. (19) is a necessary ingredient of the linear deconvolution.

Of course, a naive scheme such as Eq. (19) is numerically dangerous because the denominator $\tilde{U}_{\text{exp}}(\mathbf{k})$ tends to zero when the frequency vector $\mathbf{k}$ approaches the boundary of the
frequency support $\mathcal{M}$, so one has to apply some regularization to prevent numerical boosting of high-frequency image noise, see, e.g., Ref. [24] for the case of image scanning microscopy (ISM), Ref. [30] for the case of super-resolution optical fluctuation imaging (SOFI), or Ref. [31] for the case of structured illumination microscopy (SIM). In the simplest case, one adds a small number to the denominator to prevent divergence effects, i.e.,

$$ \tilde{I}(\mathbf{k}) = \frac{\tilde{U}(\mathbf{k})}{\epsilon + \tilde{U}_{\text{exp}}(\mathbf{k})} \tilde{I}_{\text{exp}}(\mathbf{k}), \quad (20) $$

where $\epsilon$ is typically chosen to be much smaller than the maximum of $|\tilde{U}_{\text{exp}}(\mathbf{k})|$. As an example we have recorded movies of single luminescent quantum dots (QDs) with a wide-field microscope, so that we could apply a super-resolution fluorescence optical imaging (SOFI) analysis for obtaining a super-resolved second-order SOFI image. In second-order SOFI, one calculates from the movie of recorded images $I(\mathbf{r}, t)$ a second-order cumulant image via

$$ I_{\text{SOFI}}(\mathbf{r}) = \sum_{\tau>0} \langle \delta I(\mathbf{r}, t) \delta I(\mathbf{r}, t + \tau) \rangle_t, \quad (21) $$

where $\delta I(\mathbf{r}, t) = I(\mathbf{r}, t) - \tilde{I}(\mathbf{r})$ with $\tilde{I}(\mathbf{r})$ being the average (over the whole movie) intensity recorded at position $\mathbf{r}$, the angular brackets denote averaging over time $t$, and cumulants are summed for all possible correlation delay times $\tau$ (for more details, see Ref. [30]). The PSF of a second-order SOFI image is equal to the square of the PSF $U_{\text{exp}}$ of the original wide-field microscope, or equivalently, its OTF is the autoconvolution of the wide-field OTF. Thus, the “ideal” OTF for second-order SOFI is the autoconvolution of a uniform amplitude distribution having the same frequency support as that of the wide-field OTF $U_{\text{exp}}$. In Fig. 10 panel (a), the sum image of the movie recorded for a single QD is shown (emission wavelength 670 nm, oil-immersion objective with 1.35 NA, pixel size 55.6 nm), and panel (b) shows the resulting second-order SOFI image. We fitted the wide-field image of the QD (i.e., the wide-field PSF $U_{\text{exp}}$) with a scalar approximation of an ideal wide-field 2D-PSF, $U_{\text{exp}}(\rho) \propto (J_1(\kappa\rho)/\rho)^2$, where $J_1$ is the first-order Bessel function of the first kind, and $\kappa$ is a fit parameter. This PSF corresponds to an OTF that is the autoconvolution of a uniform amplitude distribution over a disk of spatial frequencies with radius $\kappa$. Thus, this wide-field PSF is “ideal” in the sense of our paper, and for the second-order SOFI image the ideal PSF is thus given by this fitted PSF scaled down by a factor of two. This corresponds to an upscaling of the corresponding OTF by the same factor. In panel (c) of Fig. 10, we show the resulting image when applying the deconvolution of Eq. (20) with this upscaled OTF, an the curves in panel (D) are cross sections for the images shown in panels (a)–(c). When fitting the images with 2D-Gaussian distributions, we indeed find that the three widths of the resulting Gaussians scale ca. as $1:1/\sqrt{2}:1/2$, as expected for second-order SOFI and its optimal deconvolution Eq. (20).

Algorithms of more sophistication take into account the frequency-dependence of image noise [4,32,33]. A recent proposal of one of us for Wiener filtering for SIM [31] would translate in the current context to finding the Wiener filter

<figure><img src="image_12.png" /><figcaption>FIG. 10. SOFI imaging of a single blinking quantum dot on immobilized on a glass cover slide. Panel (a) shows a sum image of a movie of 1000 images, taken with a frame rate 31 Hz on an oil immersion objective with 1.35 NA. Yellow bar is 0.5 µm. Panel (b) shows the second-order SOFI image as calculated following Eq. (21). Panel (c) shows the deconvolved image using Eq. (20), with $\epsilon = 0.1 \max |\tilde{U}_{\text{exp}}(\mathbf{k})|$.</figcaption></figure>

$\tilde{W}(\mathbf{k})$ that minimizes

$$ E = \left\langle \int d\mathbf{k} |\tilde{W}(\mathbf{k}) \tilde{I}_{\text{exp}}(\mathbf{k}) - \tilde{U}(\mathbf{k}) \tilde{S}(\mathbf{k})|^2 \right\rangle, \quad (22) $$

where the angular brackets $\langle \dots \rangle$ indicate averaging over all noise realizations. That is, we seek a linear deconvolution $\tilde{I}(\mathbf{k}) = \tilde{W}(\mathbf{k}) \tilde{I}_{\text{exp}}(\mathbf{k})$ that best matches the ideal OTF representation of the ground truth object spectrum $\tilde{S}(\mathbf{k})$. Using that

$$ \langle \tilde{I}_{\text{exp}}(\mathbf{k}) \rangle = \tilde{U}_{\text{exp}}(\mathbf{k}) \tilde{S}(\mathbf{k}) \quad (23) $$

$$ \langle |\tilde{I}_{\text{exp}}(\mathbf{k})|^2 \rangle = |\tilde{U}_{\text{exp}}(\mathbf{k})|^2 |\tilde{S}(\mathbf{k})|^2 + \tilde{N}(\mathbf{k}), \quad (24) $$

with $\tilde{N}(\mathbf{k})$ the noise variance in Fourier space, it may be found that the optimum Wiener filtered deconvolution is given by

$$ \tilde{I}(\mathbf{k}) = \frac{\tilde{U}(\mathbf{k})}{\tilde{U}_{\text{exp}}(\mathbf{k})} \left[ \frac{\text{SSNR}(\mathbf{k})}{1 + \text{SSNR}(\mathbf{k})} \right] \tilde{I}_{\text{exp}}(\mathbf{k}), \quad (25) $$

where $\text{SSNR}(\mathbf{k}) = |\tilde{U}_{\text{exp}}(\mathbf{k})|^2 |\tilde{S}(\mathbf{k})|^2 / \tilde{N}(\mathbf{k})$ is the spectral signal to noise ratio. Such an algorithm prevents unwanted boosting of noise, as the $\text{SSNR}(\mathbf{k})$ goes to zero when $\mathbf{k}$ approaches the boundary of the frequency support $\mathcal{M}$, just as the factor $\tilde{U}_{\text{exp}}(\mathbf{k})$ in the denominator.

We have applied this linear deconvolution principle in combination with the proposed ideal OTF concept to a sample of the so-called synaptonemal complex that was imaged both with a Zeiss Airyscan setup, based on the ISM super-resolution technique [24], and with a rescan confocal microscopy (RCM, confocal.nl) setup [34] operating on a
Nikon microscope. The underlying image formation of the ISM and RCM techniques are identical, even though the experimental implementations are quite different. In particular, the experimental OTF is found by convolution of the excitation and detection OTF, as explained earlier in this paper. This results in a PSF that effectively is a factor $\sqrt{2}$ more narrow than the PSF of conventional wide-field fluorescence microscopy, while the OTF support size is increased by a factor 2. Wiener filtered linear deconvolution as described by Eq. (25) is therefore expected to lead to an additional $\sqrt{2}$ improvement in apparent sharpness.

There were slight differences between the microscopes, in particular the magnification ($M = 60$ for Zeiss and $M = 150$ for Nikon, using a $1.5 \times$ adapter), numerical aperture ($NA = 1.40$ for Zeiss and $NA = 1.45$ for Nikon) and the back-projected pixel size ($a = 42.6$ nm for Zeiss and $a = 43.3$ nm for Nikon). The slightly higher NA of the Nikon machine implies that more light is collected for the same exposure time and that the depth of focus is somewhat less. For a sample refractive index $n = 1.515$ there is a 13% smaller depth of focus (this scales proportional to $\lambda/(1 - \cos \alpha)$ with $\sin \alpha = NA/n$).

The raw data of all 32 detector segments of the Airyscan detector, which are arranged on a hexagonal grid [35], were available. The relative shift of the 32 segments was estimated using the findshift function of DIPLib [36], revealing a spiral arrangement that enables identification of the correct grid position of the segment. Then, the images of the 32 segments were added to emulate a confocal image. For the ISM modality we used the found relative shifts to align the 32 images before adding them. The ISM method prescribes an image shift that is $1/2$ times the true distance between detector segments. The shift estimation, however, already takes this correctly into account. The estimated distance between detector segments on the hexagonal grid appears to be $2 \times (2\sqrt{3}) \times a = 98.4$ nm in sample space, which gives a distance $5.9 \mu m$ in the detector plane. The diameter of the entire set of 32 segments is then approximately $6 \times 98.4$ nm $= 590$ nm, corresponding to $1.59 \lambda/NA \approx 1.30$ Airy units, which fits reasonably well with practical pinhole diameter settings in confocal microscopy.

The dominant noise source is shot noise governed by Poisson statistics. This implies a noise variance in Fourier space $\tilde{N}(k)$ that is constant and equal to the total (expected) number of collected photons in the image. For the assessment of signal and noise levels it is therefore needed to convert the raw image signals into photon counts. The raw ISM detector segment images and the RCM images were fed into a routine for estimating the gain and offset [37], giving a gain equal to 4.45 and an offset equal to 0 for the Zeiss system, and a gain of 2.51 and offset of 97.4 for an assumed readout noise of 1.0 e (standard deviation) for the RCM setup, that uses an sCMOS camera (Hamamatsu Orca Flash 4.0). With these parameter estimates the raw data is converted to measured photon counts. It appears that the total number of collected photons is about $4 \times$ higher for the RCM data set compared to the confocal/ISM data set, implying SNR levels that are about 2x higher.

The SSNR is subsequently estimated by averaging over rings in Fourier space [31]. It appears that this estimate of the

<figure><img src="image_13.png" /><figcaption>FIG. 11. Application of the ideal OTF concept in linear deconvolution. Image of a synamptonemal complex sample with confocal (a), ISM (b), and RCM (c) modality. Inset of red box in (c) for the confocal (d), ISM (e), and RCM (f) modality. Linearly deconvolved inset images for the confocal (g), ISM (h), and RCM (i) modality. SSNR estimated by Fourier space ring averaging and normalized SSNR according to Eq. (26) for the confocal (j), ISM (k), and RCM (l) modality.</figcaption></figure>

SSNR becomes unreliable for the highest spatial frequencies (where essentially the SSNR goes to zero). To fix this, an ad hoc normalization

$$SSNR(k) \rightarrow \frac{|\tilde{U}_{\text{exp}}(k)|^2}{\gamma^2 + |\tilde{U}_{\text{exp}}(k)|^2} SSNR(k) \quad (26)$$

for $\gamma = 0.05$ is applied, enforcing the physical asymptotic behavior close and beyond the spatial frequency cutoff, without affecting the estimated SSNR for the spatial frequencies within the OTF support.

Figure 11 shows the results of this analysis. The ISM and RCM images before linear deconvolution already show a comparable resolution advantage over the emulated confocal image. Linear deconvolution has a clear impact on all three modalities: Noise is suppressed and features appear more clear, in particular the two-line substructure of the filaments can be more clearly recognized in the linearly deconvolved ISM and RCM images.

## IX. CONCLUSIONS

We have presented a method for obtaining optimum OTFs for a given support in Fourier space and computed the
optimum OTF for a range of microscopy modalities. We applied the concept to linear deconvolution of spot scanning microscopy, taking into account the impact of shot noise. There could be alternative or even more general ways to incorporate other noise sources in the treatment, such as detector thermal noise, read-out noise etc. For example, a possible modification of our algorithm to incorporate noise effects could be to adequately reduce the support of the uniform amplitude distribution, which is used for calculating, via autoconvolution, the optimal OTF.

Another possible way to extend the current idea is to explore a numerical, Gerchberg-Saxton-like approach. In each iterative step, the ideal OTF is updated by projecting the Fourier transform of the ideal PSF on the given support $M$, and the ideal PSF is subsequently updated by taking the absolute value of the Fourier transform of the ideal OTF. Such an approach could be of use if the shape of the OTF support is too complex for an exact direct computation of the ideal OTF, as, e.g., for 3D-SIM.

However, the main emphasis of the current paper was to introduce the general idea of an optimal OTF, and to demonstrate it on four idealized examples. Finally, it should be mentioned that our considerations may be valid far beyond

the scope of optical microscopy, for example in magnetic resonance imaging, ultrasound imaging, or for deconvolution of telescopic images.

## ACKNOWLEDGMENTS

J.E. and A.D. acknowledge financial support through project MICROSCATTAB, which is jointly financed by a Grant of the German Research Foundation (DFG, Project No. EN297/15-1) and the Agence National de la Recherche (ANR, Project No. ANR-16-CE92-0003-01). J.E. thanks also for financial support by the DFG under Germany's Excellence Strategy - EXC 2067/1-390729940, and by the European Research Council (ERC) via project smMIET (Grant Agreement No. 884488) under the European Union's Horizon 2020 research and innovation programme. N.R. acknowledges financial by the German Federal Ministry of Education and Research via its funding program Photonics Research Germany (Grant Contract No. 13N15327).

We are grateful to Anne Sentenac for proofreading and advice. We thank Jeroen Kole for providing ISM and RCM image data.

The authors declare no conflicts of interest.

[1] W. H. Richardson, Bayesian-based iterative method of image restoration, J. Opt. Soc. Am. **62**, 55 (1972).

[2] L. B. Lucy, An iterative technique for the rectification of observed distributions, Astron. J. **79**, 745 (1974).

[3] G. M. P. van Kempen, L. J. van Vliet, P. J. Verveer, and H. T. M. van der Voort, A quantitative comparison of image restoration methods for confocal microscopy, J. Microsc. **185**, 354 (1997).

[4] P. J. Verveer, M. J. Gemkow, and T. M. Jovin, A comparison of image restoration approaches applied to three-dimensional confocal and wide-field fluorescence microscopy, J. Microsc. **193**, 50 (1999).

[5] W. Wallace, L. H. Schaefer, and J. R. Swedlow, A working person's guide to deconvolution in light microscopy, BioTechniques **31**, 1076 (2001).

[6] J.-B. Sibarita, Deconvolution microscopy, *Advances in Biochemical Engineering*, edited by J. Rietdorf, Microscopy Techniques (Springer, Berlin, Heidelberg, 2005), Vol 95.

[7] Heintzmann R. and C. Cremer, Laterally modulated excitation microscopy: Improvement of resolution by using a diffraction grating, Proc. SPIE **3568**, 185 (1999).

[8] M. G. L. Gustafsson, Surpassing the lateral resolution limit by a factor of two using structured illumination microscopy, J. Microsc. **198**, 82 (2000).

[9] M. G. L. Gustafsson, L. Shao, P. M. Carlton, C. J. R. Wang, I. N. Golubovskaya, W. Z. Cande, D. A. Agard, and J. W. Sedat, Three-dimensional resolution doubling in wide-field fluorescence microscopy by structured illumination, Biophys. J. **94**, 4957 (2008).

[10] F. Kraus, E. Miron, J. Demmerle, T. Chitiashvili, A. Budco, Q. Alle, A. Matsuda, H. Leonhardt, L. Schermelleh, and Y. Markaki, Quantitative 3D structured illumination microscopy of nuclear structures, Nat. Protocols **12**, 1011 (2017).

[11] R. Heintzmann and T. Huser, Super-resolution structured illumination microscopy, Chem. Rev. **117**, 13890 (2017).

[12] W. T. Welford, Aplanatism and isoplanatism, in *Progress in Optics*, Vol. 13 (Elsevier, Amsterdam, 1976), pp. 267–292.

[13] W. Lukosz, Properties of linear low-pass filters for nonnegative signals, J. Opt. Soc. Am. **52**, 827 (1962).

[14] W. Lukosz, Übertragung nicht-negativer Signale durch lineare Filter, J. Mod. Opt. **9**, 335 (1962).

[15] C. H. Righolt, J. A. Slotman, I. T. Young, S. Mai, L. J. van Vliet, and S. Stallinga, Image filtering in structured illumination microscopy using the Lukosz bound, Opt. Express **21**, 24431 (2013).

[16] C. H. Righolt, S. Mai, L. J. van Vliet, and S. Stallinga, Three-dimensional structured illumination microscopy using Lukosz bound apodization reduces pixel negativity at no resolution cost, Opt. Express **22**, 11215 (2014).

[17] G. Toraldo Di Francia, Super-gain antennas and optical resolving power, Nuovo Cimento **9**, 426 (1952).

[18] I. J. Cox, C. J. R. Sheppard, and T. Wilson, Reappraisal of arrays of concentric annuli as superresolving filters, J. Opt. Soc. Am. **72**, 1287 (1982).

[19] E. T. F. Rogers, J. Lindberg, T. Roy, S. Savo, J. E. Chad, M. R. Dennis, and N. I. Zhelude, A super-oscillatory lens optical microscope for subwavelength imaging, Nat. Mater. **11**, 432 (2012).

[20] E. Wolf, Electromagnetic diffraction in optical systems-i. an integral representation of the image field, Proc. R. Soc. London A **253**, 349 (1959).

[21] B. Richards and E. Wolf, Electromagnetic diffraction in optical systems, ii. Structure of the image field in an aplanatic system, Proc. R. Soc. London A **253**, 358 (1959).
[22] C. J. R. Sheppard, M. Gu, Y. Kawata, and S. Kawata, Three-dimensional transfer functions for high-aperture systems, *J. Opt. Soc. Am. A* **11**, 593 (1994).

[23] O. Schulz, C. Pieper, M. Clever, J. Pfaff, A. Ruhlandt, R. H. Kehlenbach, F. S. Wouters, J. Großhans, G. Bunt, and J. Enderlein, Resolution doubling in fluorescence microscopy with confocal spinning-disk image scanning microscopy, *Proc. Natl. Acad. Sci. USA* **110**, 21000 (2013).

[24] C. B. Müller and J. Enderlein, Image Scanning Microscopy, *Phys. Rev. Lett.* **104**, 198101 (2010).

[25] S. Hell and E. H. K. Stelzer, Properties of a 4pi confocal fluorescence microscope, *J. Opt. Soc. Am. A* **9**, 2159 (1992).

[26] M. Schrader, M. Kozubek, S. W. Hell, and T. Wilson, Optical transfer functions of 4pi confocal microscopes: Theory and experiment, *Opt. Lett.* **22**, 436 (1997).

[27] G. Shtengel, J. A. Galbraith, C. G. Galbraith, J. Lippincott-Schwartz, J. M. Gillette, S. Manley, R. Sougrat, C. M. Waterman, P. Kanchanawong, M. W. Davidson *et al.*, Interferometric fluorescent super-resolution microscopy resolves 3D cellular ultrastructure, *Proc. Natl. Acad. Sci. U.S.A.* **106**, 3125 (2009).

[28] R. Schmidt, C. A. Wurm, S. Jakobs, J. Engelhardt, A. Egner, and S. W. Hell, Spherical nanosized focal spot unravels the interior of cells, *Nat. Methods* **5**, 539 (2008).

[29] H. P. Urbach and S. F. Pereira, Field in Focus with a Maximum Longitudinal Electric Component, *Phys. Rev. Lett.* **100**, 123904 (2008).

[30] T. Dertinger, R. Colyer, R. Vogel, J. Enderlein, and S. Weiss, Achieving increased resolution and more pixels with superresolution optical fluctuation imaging (sofi), *Opt. Express* **18**, 18875 (2010).

[31] C. S. Smith, J. A. Slotman, L. Schermelleh, N. Chakrova, S. Hari, Y. Vos, C. W. Hagen, M. Müller, W. van Cappellen, A. B. Houtsmuller, J. P. Hoogenboom, and S. Stallinga, Structured illumination microscopy with noise-controlled image reconstructions, *Nat. Methods* **18**, 821 (2021).

[32] E. H. K. Stelzer, Contrast, resolution, pixelation, dynamic range and signal-to-noise ratio: Fundamental limits to resolution in fluorescence light microscopy, *J. Microscopy* **189**, 15 (1998).

[33] E. A. Ingerman, R. A. London, R. Heintzmann, and M. G. L. Gustafsson, Signal, noise and resolution in linear and nonlinear structured-illumination microscopy, *J. Microscopy* **273**, 3 (2019).

[34] G. M. R. De Luca, R. M. P. Breedijk, R. A. J. Brandt, C. H. C. Zeelenberg, B. E. de Jong, W. Timmermans, L. N. Azar, R. A. Hoebe, S. Stallinga, and E. M. M. Manders, Re-scan confocal microscopy: Scanning twice for better resolution, *Biomed. Opt. Express* **4**, 2644 (2013).

[35] J. Huff, The Airyscan detector from Zeiss: Confocal imaging with improved signal-to-noise ratio and super-resolution, *Nat. Methods* **12**, i (2015).

[36] www.diplib.org.

[37] R. Heintzman, P. K. Relich, R. P.J. Nieuwenhuizen, K. A. Lidke, and B. Rieger, Calibrating photon counts from a single image, arXiv:1611.05654.