# 12. Super-Resolved Off-Axis Holographic Multiplexing

*(Natan T. Shaked*)

## 12.1. Status

Super-resolved quantitative imaging of rapid dynamic biological processes with fine spatial details and without labeling is of interest for many medical, biological, and biophysical studies. Holographic imaging uses interference to acquire the complex wave front of a sample, containing both its amplitude and quantitative phase profiles, where the latter represents the structure of the sample. In transmission imaging mode, the quantitative phase is proportional to the optical path delay of the sample, which is equal to the product of the sample thickness and its refractive index, enabling visualization of mostly transparent biological samples, which have only small absorption, such as biological cells in vitro, in a meaningful contrast without using cell labeling. Holography is diffraction limited, as many other optical imaging techniques, and various techniques of super-resolution can be integrated with holographic imaging, thus gaining not only amplitude super-resolution, but also label-free quantitative-phase super-resolution. Generally, there are two main approaches to holographic wave-front acquisition. In on-axis holography,[383] the two beams interfere with no angle between them, which causes mixing of the sample and reference beam intensities with the complex wave front of the sample. Therefore, this approach requires acquisition of three or four phase-shifted holograms of the same sample instance in order to isolate the sample complex wave from the unwanted waves. If these holograms are acquired sequentially, at different times, fast dynamic processes cannot be recorded. If these holograms are acquired together, three or four camera planes are needed, requiring extended sensor sizes with slower frame rates, and causing registration problems between the different camera planes. Alternatively, parallel on-axis holography can be implemented by more complicated camera setups such as using pixel division.[384]

Off-axis holography[385] captures the complex wave front of the sample within a single camera exposure, which is highly relevant for quantitatively acquiring dynamic samples. This is done by inducing a small angle between the sample and reference beams creating the interference pattern of the hologram. This single-exposure mode is possible since in the spatial-frequency domain, there is a full separation between the auto-correlation (DC) terms, or the sample and reference beam intensities, and each of the cross-correlation (CC) terms, each of which contains the complex wave front of the sample. This spatial-frequency separation typically occurs across a single axis, which allows compressing additional information across the other axes as well. This can be done by optical multiplexing of several holograms with different interference fringe orientations into a single multiplexed hologram, acquired in a single camera exposure, and full reconstruction of the complex wave fronts encoded. Each of these complex wave fronts can contain additional information from the imaged sample, meaning that multiplexing allows recoding more information with the same number of camera pixels. This is highly beneficial for highly

dynamic samples that need to be recorded without multiple camera exposures. The multiple wave-front information multiplexed into the hologram can be used to obtain super-resolution by illuminating the sample from different angles and building a synthetic aperture that is larger than the one provided by the optical system. Thus, although the microscope objective can pass only limited spatial-frequency content of the sample due to a limited numerical aperture, passing simultaneously different angular perspectives of the sample though the microscope objective, each of them encodes different spatial-frequency content of the sample, can be used to reconstruct a larger synthetic aperture, eventually achieving a super-resolved reconstructed wave front. Holographic multiplexing is the mechanism that enables separation of the different spatial-frequency contents from the simultaneous angular channels, allowing the correct reconstruction of the super-resolved wave front of the sample.

Early implementations of simultaneous super-resolution by synthetic aperture by multiplexed holography used ultrasonic light diffraction to generate small diffraction angles, hence obtained a low synthetic aperture.[386] Later on, Leith et al.[387] used a spatially incoherent source and diffraction gratings to obtain simultaneous super-resolution holographic imaging across a single dimension. Mico et al.[388] used a vertical-cavity surface-emitting laser (VCSEL) array to constitute individually coherent sources that are mutually incoherent, so that each tilted beam provided a separate spatial-frequency content of the sample and interfered with the coinciding reference beam in one-dimensional holographic multiplexing geometry.

**Figure 21** presents the off-axis holography process and how it can be used to obtain super-resolution by simultaneously capturing an increased synthetic aperture, which is larger than the one provided by the optical system. This is done by optical multiplexing of several complex wave-fronts together, while sharing the camera grayscale dynamic range, which brings to negligible errors for mostly transparent phase objects.[389] As shown in Figure 21a from left to right: in regular off-axis holography, the sample is illuminated by a single beam S1 and is magnified by a microscope objective MO; a mutually coherent clear reference beam R is superimposed with the sample magnified image on a digital camera at a small off-axis angle; where an off-axis hologram with straight parallel fringes is created; this hologram is digitally Fourier transformed, resulting in central DC terms and a pair of shifted complex-conjugate CC terms; one of the CC terms is cropped and centered; and inverse Fourier transformed, resulting in the complex wave front (amplitude and quantitative phase) of the sample, which is limited by the numerical aperture of the system. The quantitative phase enables visualizing biological samples in a label-free manner. In Figure 21b, to increase the resolution by simultaneous synthetic aperture, multiplexing two complex wave fronts into a single off-axis hologram is performed. From left to right: the sample is illuminated from two angles by S1 and S2, each collecting a different spatial-frequency content from the sample; the magnified sample images are superimposed with a reference wave R on the camera, and captured by the camera in a single exposure; where a multiplexed off-axis hologram is created; after a digital Fourier transform, we obtain the central DC terms, two pairs of shifted CC terms, and unwanted cross-terms CCx, none of which overlaps with the other terms; two cropped CC terms, CC1 and CC2, are positioned
<figure><img src="image_24.png" /><figcaption>**Figure 21.** Principles of label-free quantitative imaging with super-resolution by off-axis holography. a) Regular off-axis holography. b) Multiplexing two wave fronts into a single hologram. c) Multiplexing six wave fronts into a single hologram.</figcaption></figure>

in the spatial-frequency domain, so that the numerical aperture is synthetically increased.; after an inversed Fourier transform, we obtain the complex wave front of the sample with increased resolution. From this wave front, we can extract the quantitative phase profile of the sample, which requires two-dimensional phase unwrapping digital processing to avoid $2\pi$ ambiguities, or experimentally acquiring the sample with several wavelengths and generating a larger synthetic wavelength.[390]

## 12.2. Current and Future Challenges

The size of the synthetic aperture directly affects the resolution of the reconstructed image. Constructing large synthetic apertures requires collecting many angular projections. Highly dynamic samples require simultaneous collection of all angular projections. Recent implementations provided off-axis holographic multiplexing of up to six simultaneous wave-front channels, each carries different spatial-frequency content from the sample, and all are acquired in a single camera exposure, which can then be reconstructed to an extended synthetic aperture and the super-resolved image.[90] As shown in Figure 21c from left to right, six beams S1–S6 illuminate the sample from different angles, each capturing a different spatial-frequency content of the sample; these beams are superimposed on the camera with a reference beam R, resulting in a six-pack multiplexed off-axis hologram; digitally Fourier transforming this hologram results in the central DC terms and six pairs of shifted CC terms, assuming that unwanted cross-terms are avoided, all of them efficiently occupying the entire spatial-frequency domain without

overlaps; CC1–CC6 are cropped and positioned in the spatial-frequency domain, isotopically for optimal resolution increase, so that an extended synthetic aperture is constructed; after an inverse Fourier transform, we obtain the super-resolved complex wave front of the sample. Six is the maximum number of wave fronts that can be multiplexed for optimal spatial-bandwidth consumption, without overlap of CC terms in the spatial frequency domain. Alternatively, when imaging sparse samples, more than six complex wave fronts can be multiplexed.[90] The resolution enhancement analysis and its limit in synthetic aperture setting have been discussed elsewhere.[95]

For multiplexing multiple wave fronts into a single hologram, we need to avoid unwanted cross-terms between nonmatching pairs of beams, which might overlap with the wanted terms and diminish the capability for full multiplexing. An example of a pair of unwanted cross-terms between the two sample beams themselves, created when multiplexing two wave fronts, is shown in Figure 21b, where the unwanted cross-terms appear across the diagonal direction on the spatial-frequency domain. This problem might become critical when more than two wave front channels are multiplexed, due to possible overlaps between the various terms.

Another issue is that even without overlaps of terms in the spatial-frequency domain, there is a risk for loss of information since multiple off-axis fringe patterns share the same dynamic range of the camera grayscale levels.

Finally, the current setups of off-axis holographic multiplexing are bulky, expensive, require optical expertise to align and use, and are sensitive to mechanical vibrations, making them less attractive for working in biological labs and medical clinics.
## 12.3. Advances in Science and Technology to Meet Challenges

Regarding the first problem, there are three possible solutions in order to avoid unwanted terms in holographic multiplexing: using different polarizations, using different light sources or wavelengths, and using coherence gating.

Beams of orthogonal polarizations do not interfere with each other. Therefore, it is possible to encode the wave fronts to different polarization states, and use polarizers after the sample to separate the channels to different interference fringe orientations on the camera. This solution, however, allows multiplexing up to four wave front channels, due to the lack of more than two orthogonal polarization states.

Using different illumination wavelengths for each sample and reference beam pair is a more valid solution for avoiding unwanted cross-terms. Color filters can be used after the sample to split each beam pair of a different wavelength to create another interference fringe orientation on the camera.

A different solution to the problem of unwanted cross-terms is based on utilizing the low-coherence properties of the light source. Here, we can avoid the unwanted cross-terms by making sure that the path difference between nonmatching pairs of beams is longer than the coherence length of the source, and therefore they do not interfere on the camera. This can be obtained by newly develop two-dimensional VCSEL arrays or other sets of coherent sources which are mutually incoherent. Another solution is using a super-continuum source with a tunable filter as a low-coherence source with delay lines in the optical paths (such as glass windows of various thicknesses), which create a different beam path for each pair of sample and reference beams. Here again, nonmatching sample and reference beams will not create interference, thus avoiding unwanted cross-terms between the imaging channels. More-elegant solutions might be developed with the introduction of new, controllable low-coherence sources, as well as specialized two-dimensional diffraction grating to split the beams in the optical system with inherited optical path delays.

Regarding the second problem of several imaging channels that share the same dynamic range of the camera, it is expected to be less critical for phase objects, such as biological cells in vitro and thin elements tested in optical metrology, which are mostly transparent, thus inducing minimal reconstruction-quality loss. When the amplitude of the sample becomes non-negligible, solving this problem requires cameras with larger bit depth. The signal-to-noise ratio (SNR) of the reconstruction when using 8-bit cameras versus 16-bit cameras for off-axis holographic multiplexing was quantified in Ref. [389] The introduction of rapid digital cameras with large bit depths is expected to significantly improve the ability to multiplex the maximum number of possible wave-front channels with minimal SNR decrease.

The third problem of bulkiness, meticulous alignment, and sensitivity to vibrations is expected to be solved by the introduction of nearly common-path modules, which are portable to existing imaging systems. Rather than splitting the reference and sample beams right after the light source,[385, 391] dictating bulky, complex and vibration-sensitive imaging setups, these modules create the reference beam from the sample beam itself, yielding more portable setups that are less sensitive to vibrations, inex-

pressive, and easier to align and use, even in clinical setting. Off-axis multiplexing of two and three wave-front fields of view of the samples has been implemented recently with these compact external interferometric modules,[392–395] but their application for super-resolution holographic imaging with synthetic aperture, as well as multiplexing of more than three wave-front channels inside these compact modules, are yet to be achieved.

One of the high-impact medical applications for off-axis holographic multiplexing is label-free super-resolved imaging flow cytometry that requires imaging several thousand of cells per second during rapid flow, due to its ability for high-resolution image analysis of a large number of cells during rapid flow, for liquid samples obtained from body fluids.

## 12.4. Conclusions

Simultaneous wave-front acquisition is a powerful tool for label-free super-resolution imaging of dynamic samples, creating quantitative contrast even for mostly-transparent samples, such as biological cells in vitro. Super-resolution in this setup can be obtained by multiplexing several wave fronts, each of which samples a different spatial-frequency content of the sample, into the same camera image. Each of these wavefronts is optically encoded into an off-axis interference pattern, with fringes oriented differently. Following the multiplexed off-axis hologram acquisition, a synthetic aperture of larger spatial-frequency content can be reconstructed, thus obtaining super-resolved imaging. By avoiding unwanted cross-terms of non-machining interference patterns and using cameras with large bit depths, up to six simultaneous wave-front channels can be reconstructed. By using external portable modules, these systems are expected to become more available and affordable, providing new rapid super-resolved label-free quantitative imaging capabilities for clinical diagnosis and biological assays.

# 13. Tomographic Diffractive Microscopy

*(Bertrand Simon*, Nicolas Verrier, Matthieu Debailleul, Olivier Haeberlé)*

## 13.1. Introduction

Conventional optical microscopy is mostly used as a simple imaging tool, for detection of features, or morphological measurements. This is because the complex interaction of light with the observed sample usually translates into intensity-only images. Also, resolution is still limited in far-field optical microscopy (except for fluorescence microscopy). We here present tomographic microscopy, an extension of Digital Holographic Microscopy, emphasizing its true 3D imaging capabilities, its ability to retrieve the complex index of refraction distribution within the observed sample, also achieving this with an improved resolution compared to conventional intensity-only microscopies. Common configurations are described and future evolutions are presented, as well as challenges this technique still faces.
## 13.2. Early Developments

For unlabelled samples, things changed with the invention of Ga-bor holography.[250] Allowing for precise measurement of amplitude and phase of a diffracted field, holography paved way for quantitative measurements of optical constants at microscopic level. Seminal paper by Wolf[248] has established the relation linking the diffracted field to the index of refraction distribution creating this field. The classical treatment starts from the Helmholtz equation in an inhomogeneous medium, introducing n as the complex refractive index of refraction and λ as the wavelength in vacuum. For weakly diffractive specimens, considering that the first Born approximation is valid, the diffracted field amplitude e is small compared to the incident field of amplitude A<sub>inc</sub>, which finally leads to write in Fourier space[91]:

$$e(k, k_{ill}) = 8\pi^3 A_{inc} k_0^2 \tilde{N}(k - k_{ill}) \quad (18)$$

Here $k_0 = 2\pi/\lambda$, $k_{ill}$ is the wavevector of the illumination, and k represents the diffracted wavevector. This equation provides direct correspondence between the diffracted far-field amplitude $e(k, k_{ill})$ in direction k, from $k_{ill}$ illumination, and the Fourier coefficient of the relative permittivity of the object $\tilde{N}(k-k_{ill})$: when Born approximation holds, the diffracted field depicts a subset of the 3-D Fourier transform of the sample permittivity distribution. In linear scattering and with momentum conservation, this subset of $k_{obj}$ is simply constructed using the fundamental equation: $k_{obj} = k_{diff} - k_{ill}$, and implying energy conservation $||k_{diff}|| = ||k_{ill}||$.

However, despite some promising early results,[396, 397] the technique did not attract the interest of the community. This can be explained by the cost or non-availability of lasers, CCDs and computers at that time to acquire and process data.

## 13.3. Status

Situation has changed at the beginning of the new millenium, reviving the domain,[256, 398, 399, 95] which has experienced growing development in recent years. Implementations of the technique have been very diverse, emphasizing the advantages and drawbacks of its various configurations. Figure 22 describes the construction of the Optical Transfer Function (OTF) support for transmission configurations in Digital Holographic Microscopy and several variants of Tomographic Diffractive Microscopy (TDM),[400] a technique also known as synthetic aperture microscopy, tomographic phase microscopy, holotomography, optical diffraction tomography, scanning holographic microscopy, optical microtomography, phase nanoscopy, holographic tomography...

TDM with illumination rotation (TDM-IR) is the most common. It has for main advantage that not having to rotate the sample makes it a more general approach, applicable to a large variety of samples. Conventional sample preparation between glass and cover glass, between cover glasses or in Petri dishes can be used. Furthermore, when high-resolution is targeted, use of high numerical objective-condenser configuration is mandatory. As a consequence, working distances are small, and inducing sample rotation in that case can be difficult. Rotating the illumination is on the contrary simple and can be performed in a variety

<figure><img src="image_25.png" /><figcaption>Figure 22. Optical Transfer Function (OTF) for various configurations in transmission TDM. a) Digital holographic microscopy: the OTF depicts a cap of sphere with large lateral, but limited longitudinal extension. b) With inclined illumination, same positions of $k_{diff}$ vectors provide new $k_{obj}$ vectors. c,d) With many illuminations, a filled, extended OTF is obtained. e) With sample-rotation, an almost completely filled sphere is obtained (but of lesser extension than in previous case). f,g) OTFs combining illumination and sample rotation with 2, and 4 evenly spaced specimen positions. Bottom row: *Betula pendula* pollen grain images in DHM and three TDM variants. Left composite image depicts x-y slice in the focal plane, showing improved resolution in TDM, but only twice that of DHM. Contrast is however better thanks to better optical sectioning. Right composite image highlights the large differences in imaging capabilities along the optical axis. Only TDM-IRSR can deliver true improved and isotropic resolution, while holography is not, strictly speaking, a true 3-D imaging method when considering transparent samples: only *optical thickness* can be measured.</figcaption></figure>

of ways. Rotating mirrors,[398] rotating Pechan prisms,[396] galvanometric mirrors and fast tip-tilt mirrors[400] have been used. Alternately, pure optical means have also been successfully implemented, involving Spatial Light Modulators[401] or Digital Micromirror Devices.[402] A specific advantage of illumination rotation is that exact knowledge of the illumination direction is not
mandatory, as it can indeed be directly retrieved from the processing of the recorded holograms.[91][403] The main drawback of this approach is the peculiar form of its OTF, Figure 22c, which translates into strongly anisotropic resolution. While this approach indeed delivers the best lateral-resolution[403, 95] low sectioning capabilities result from the so-called missing-cone of unrecorded frequencies along the optical axis.

TDM with sample rotation (TDM-SR) on the contrary can deliver (almost) isotropic resolution as deduced from Figure 22e. The main difficulty in that case is to ensure accurate rotation of the sample. A simple and efficient approach can be to embed cells within a rotating capillary, but one has then to deal with curved optical surfaces, which may induce aberrations. In that case also, larger working distance are mandatory, leading to use of lower aperture objectives. Optical and acoustical tweezers have also been successfully used.[404] While more challenging, they allow using conventional sample preparation and use of high Numerical Aperture (NA), high magnification objectives. Cells being often polarized, electric oscillating field can also be used to induce rotation. In some cases, the sample itself can be directly rotated,[397] such as when characterizing optical fibres, to which samples can also been attached, or textile yarns, pulled-glass micropipettes, SNOM (Scanning Near Field Optical Microscopy) tips[400]... In some cases such as blood cells, one can even use their natural tendency to tumble when moving through microfluidic channels to perform microtomography.[262]

Combing illumination with sample rotation (TDM-IRSR) has also been proposed: an extended and fully filled OTF is obtained[400] (Figure 22f,g). This adds advantages of both above-mentioned TDM variants, however also combines their difficulties, which probably explains why it is not more popular.

The lateral and axial extensions of the OTFs for these configurations have been detailed in,[400] with $\lambda$ the wavelength of observation, $n$ the index of refraction of the immersion medium, and $\theta$ the aperture collecting angle of the microscope (so that $NA = n \sin\theta$):

$$ \Delta v_{x,y}^{Holo} = \frac{2n \sin \theta}{\lambda} \qquad \Delta v_{z}^{Holo} = \frac{n(1 - \cos \theta)}{\lambda} \qquad (19) $$

For TDM with illumination rotation (IR) one obtains:

$$ \Delta v_{x,y}^{TDM-IR} = \frac{4n \sin \theta}{\lambda} \qquad \Delta v_{z}^{TDM-IR} = \frac{2n(1 - \cos \theta)}{\lambda} \qquad (20) $$

With sample rotation (SR), one has:

$$ \Delta v_{x,z}^{TDM-SR} = \frac{4n \sin(\theta/2)}{\lambda} \qquad \Delta v_{y}^{TDM-SR} = \frac{2n \sin \theta}{\lambda} \qquad (21) $$

and when combining both approaches, one gets:

$$ \Delta v_{x,y,z}^{TDM-IRSR} = \frac{4n \sin \theta}{\lambda} \qquad (22) $$

So, TDM provides two main advantages: firstly, it delivers quantitative images of the index and absorption within samples. Then, it allows for improved resolution compared to DHM, thanks to its extended support, and improved resolution compared to conventional microscopy, thanks to its (theoretically)

constant transmission OTF, while high frequencies are otherwise rapidly attenuated in intensity-only, conventional transmission microscopy.

TDM with white light[256] is another approach for tomography, consisting in using white light illumination and a SLIM (Spatial light interference microscopy) module.[405] When changing the k-vector norm in Figure 22a, the position of the detected cap of sphere is fixed, but its radius changes, so that the final OTF takes the shape of a bowl.[91][256] Similar results are obtained when varying the wavelength in a holographic microscope as in.[406] While the lateral resolution and axial sectioning capabilities are lower, the main advantage of this method is that it can be more easily implemented onto a conventional wide-field transmission microscope.

TDM in reflexion mode is also possible. While TDM in transmission is mainly used for studying transparent biological samples, TDM in reflexion has been developed for investigating surfaces and interfaces, e.g. for microelectronics device inspection. In that case, high index of refraction contrast and noticeable polarization effects have to be taken into account, but when appropriate inversion methods are implemented, those lead to the highest demonstrated resolution in label-free, far-field microscopy, with $\lambda/10$ features successfully identified,[407] albeit requiring prior knowledge about the specimen's index of refraction to achieve such performances.

Advanced image reconstructions: in parallel with hardware development, very substantial efforts have been devoted to improving image reconstruction. The fastest approaches are based on direct Fourier transforms, under Born and Rytov approximations, but their application is limited to small samples exhibiting low index of refraction contrasts (see for example Refs[408, 409]). Iterative methods (re)computing both the reconstructed image and its spectrum, comparing it to the acquired data, such as the Gerchberg-Papoulis method allow for better reconstructions, but are more time consuming.

For thicker samples, samples with high index contrast, or sample inducing multiple scattering, one has to use more advanced methods, based on the propagation of the electromagnetic field through the sample.[410] This can be performed assuming samples properties to simplify computations. But in the most extreme cases, when considering thick samples with high index contrast, birefringence, or semi-conductor, or metallic samples, complete electromagnetic problem must be involved, using Finite Difference-Time Difference (FDTD) solver.[411] While these methods have proven being much more precise, and sometimes essential to get realistic results, it is at a strong cost in terms of computational power, memory and time. But with the continuous progress of computers in terms of processing power and memory, one can expect these advanced methods to be more widely used.

Then, in recent years, artificial intelligence and machine learning has also gained strong interest in the domain. A main advantage of these approaches is their efficiency at reducing missing-cone induced reconstruction artefacts.[412, 413] Convolutional Neural Networks (CNN) have been successfully used, as well as methods taking benefit from dictionary of structures. Skipping prior training is advantageous, but requires more computational effort. Conversely, large sets of properly labelled training images are difficult to constitute, and prior training is time consuming,
but then rapid reconstructions are possible when the network has been trained.

Finally, an interesting hardware-software joined development is the possibility to avoid interferometric measurements, by numerical reconstruction of the phase. This can be for example done from sets of intensity images, solving Transport of Intensity Equation,[⁴¹⁴] or using wavefront analysers.[⁴¹⁵] Doing so simplifies the microscope optical design, but transfers complexity towards numerical reconstruction of the amplitude/phase data from intensity only images.

## 13.4. Current and Future Challenges

In the last decade, TDM has turn from a promising technique into a well-established one to study microscopic specimens with a 3-D, label-free, and quantitative approach.

It has found applications in many fields, and is actively developed by several groups worldwide. Companies like TomoCube and NanoLive are now commercializing systems, for end-users not interested in instrumentation development.

But there are challenges not yet addressed to further improve the technique, and widen applications.

The importance of working with isotropic-resolution images is vastly overlooked. Even simple quantities as volumes are difficult to accurately measure at sub-micrometric scale in transmission-only TDM, because of the missing-cone problem.[⁴⁰⁰] Index of refraction or species concentrations are even more problematic. Demonstrated approaches to tackle this limitation require combining sample rotation with illumination rotation, and as a consequence are limited to freestanding specimens, while 4PI[³⁹⁸] and mirror-assisted TDM[⁴¹⁶, ⁴¹⁷] are promising for all kind of samples, but have not yet been fully demonstrated.

The index of refraction in itself provides some selectivity, but the technique could be much more specific, if a full multimodal implementation of TDM would be available. Hyperspectral,[⁴¹⁸, ⁴¹⁹] polarized[³⁹⁸, ⁴²⁰] TDM could in principle add some chemical selectivity to imaging.

To study dynamic samples, or for high-throughput screening, possible in holographic, fluorescence and conventional staining microscopy, a real-time acquisition, reconstruction, and 3D display of the observed sample image would be very helpful (see Refs.[421, 422, 423, 424, 425, 247] and references therein). This will require important development to fasten acquisitions and numerical image computation.

Another direction of evolution of the technique will be the observation of large samples, in a similar way the development of SPIM (Selective Plane Illumination Microscopy) fluorescence microscopy has allowed going from cellular level to small organisms, organoids, embryos, or even small animals organs.

Finally, TDM is in fact still diffraction limited. The thriving development of fluorescence nanoscopy relies on nonlinearities such as saturation, or molecular switching, which are the keys to break the Abbe limit. But switching molecules have no equivalent in terms of index of refraction. Resolution under 100 nm in unlabelled optical microscopy relies on capturing the evanescent field by SNOM or with microsphere-assisted microscopy. But such techniques are by definition limited to surface imaging.

Obtaining true superresolution in TDM now represents the grand challenge in the domain.

## 13.5. Conclusions

Tomographic microscopy is gaining attention in biology, thanks to its capacity of imaging unlabelled samples at high resolution, with some specificity provided by quantification of the sample index of refraction (see recent reviews[²⁴⁷, ⁴²²–⁴²⁵] and references therein). Future developments could increase its domains of application towards material sciences, or metrology of micro-nanofabrication.

# 14. Scanning Near-Field Optical Microscope for Quantitative Nano-Imaging and Nano-Spectroscopy

(Sheng Wang*, Mengkun Liu*)

## 14.1. History and Status

Optical microscope and spectroscopy have been versatile and powerful characterization tools to study light-matter interactions in condensed matter and biological systems. Conventional optical techniques are constrained by the well-known Abbe diffraction limit, which restricts the spatial resolution to be approximately half the wavelength of the illumination light. Driven by vibrant research into quantum materials and molecular interactions at the nanoscale, researchers have been actively developing label-free and super-resolution nanoscopy with wide spectral energies spanning from below 0.1 meV to above 1000 meV. The idea of utilizing high-momentum components of evanescent waves to achieve subdiffraction imaging dated back to as early as late 1920s.[⁴²⁶, ⁴²⁷] Edward H. Synge, starting in 1928, proposed a series of experimental schemes to collect the surface-bound electromagnetic field from a nanoscale orifice in a planar metallic film (Figure 23a) or from metallic particles (Figure 23b) placed within nanometers from the sample. The former concept formed the basis for the aperture-type scanning near-field optical microscope (a-SNOM, Figure 23c) and the latter one brought about the scattering-type scanning near-field optical microscope (s-SNOM, Figure 23d). However, the implementation of these ingenious proposals were hampered by various technical difficulties, notably the controllable placement of the nanoscale aperture in close vicinity of the sample and extraction of the weak nanoscale light from the overwhelming background. These technical hurdles had been later overcome with the invention of various scanning probe microscopes (SPM) including scanning tunneling microscope (STM) and atomic force microscope (AFM).

Prototypical aperature/waveguide based scanning probes achieving super-resolution imaging at visible and microwave wavelengths was demonstrated in the 1980s.[⁴²⁶, ⁴²⁷] Since then, steady improvement and perfection in SNOM techniques have been made. a-SNOM features great operation flexibilities with aperture/fiber/waveguide coupled near-field tips that can yield various types of experimental schemes and clean near-field signal extraction.[⁴²⁸] Notably, aperture waveguide method in terms
<figure><img src="image_26.png" /><figcaption>**Figure 23.** Schematic illustrations of scanning near-field optical microscope (SNOM). a) The original proposal of aperture-type SNOM (a-SNOM) by Edward H. Synge. The nanoscale orifice can convert localized near field into propagating far-field wave. b) The original proposal of scattering-type SNOM (s-SNOM) by Edward H. Synge. The metallic nanoparticle can scatter localized near field into propagating far-field wave. c) Modern a-SNOM based on a scanning probe microscope with an aperture/fiber/waveguide coupled near-field tip. d) Modern s-SNOM based on a tapping-mode atomic force microscope. The integration of Fourier-transform infrared spectroscopy (FTIR) with s-SNOM gives rise to Nano-FTIR. The integration of pump-probe spectroscopy with s-SNOM ends up as ultrafast nanoscopy, which combines nanometer spatial resolution with femtosecond temporal resolution.</figcaption></figure>

of impedance-matched coaxial cables has been leveraged to ef-
ficiently guide microwave radiation to the tip apex for near-
field interaction with the sample. This configuration at mi-
crowave regime is commonly referred to as scanning microwave
impedance microscope (sMIM) and can routinely image sam-
ple conductivity and permittivity at a spatial resolution of ≈20–
100 nm.[429] s-SNOM at infrared frequencies started to gain
momentum in the 1990s with the development of broadband
light sources and the need for their integration.[38, 426, 427] The
active modulation of tip-sample distance and subsequent high-
harmonics demodulation of the scattering optical signal substan-
tially suppress the background scattering and can achieve a spa-
tial resolution of ≈10–20 nm in routine operation. Due to the scat-
tering nature, s-SNOM working in visible, infrared (IR), and ter-
ahertz (THz) wavelength regimes have been developed and em-
ployed to characterize a wealth of optical phenomena.[38, 430–433]
Importantly, the pseudo-heterodyne detection scheme has been
developed for background-free near-field spectroscopy. This
scheme simultaneously retrieves near-field optical signal ampli-
tude and phase by interferometric detection of scattered light
utilizing a phase-modulated reference wave.[434] When the tip-
sample convolution effects are considered with sufficient mod-
eling, complex dielectric functions of the sample can be obtained
in a quantitative fashion.[38, 435]

Diffraction-limited optical spectroscopy methods can be nat-
urally extended to deep subwavelength resolution when inte-
grated with SNOM.[38, 431] The integration of Fourier-transform
infrared spectroscopy (FTIR) with s-SNOM gives rise to Nano-
FTIR (Figure 23d), which allows broadband infrared nano-

spectroscopy with nanometer spatial resolution (<20 nm).[431, 436]
Nano-FTIR has proven to be a powerful analytical tool to probe
low-energy excitations and correlations in complex heteroge-
neous solid matter and molecular vibrational fingerprints in bi-
ological specimens. The integration of s-SNOM with ultrafast
time-resolved techniques such as pump-probe spectroscopy is
of utmost interest because it enables the interrogation of non-
equilibrium dynamics of quantum materials at the nanoscale
(Figure 23d).[431, 432] This ultrafast nanoscopy has been suc-
cessfully developed for some spectral ranges and has provided
unique insight into the dynamics of polaritonic modes of van
der Waals semiconductors.[430–432, 437] While SNOM is facile to
operate in ambient conditions, a cryogenic environment has
also been realized to study temperature-sensitive effects such
as phase transitions and electron–phonon coupling in quantum
materials.[38, 438, 439]

**14.2. Current and Future Challenges**

There have been multiple ongoing forefronts of extending the
capabilities of SNOM. One is the realization of multimodal
SNOMs, which interface optical nano-imaging with nanome-
chanical and nanoelectrical mapping to better unravel the
structure-property relationship of nanomaterials. The correla-
tive nano-imaging serves to examine the intriguing interplay
between photo-excitation, electronic properties, and mechani-
cal responses in heterogeneous materials, such as photoinduced
electrical and mechanical effects in organic-inorganic perovskites
and two-dimensional materials.[440, 441] The differing SPM operation modes for different types of characterizations make it challenging to achieve a multifunctional SNOM. s-SNOM is based on a tapping-mode AFM with moderate modulation of the tip-sample distance. On the contrary, nanomechanical and nanoelectrical methods usually require intimate contact between the tip and the sample. Consequently, complex tip manipulation or multipath scanning methods are usually required.

Another forefront is the integration of various conventional far-field optical spectroscopy techniques with SNOM. Broadband spectroscopy spanning from visible to THz requires optimal tip geometries and sizes for different wavelengths. The commercially available metallic AFM tips (e.g., PtIr-coated tips) usually have the best performances in the infrared range because the tip configuration offers optimal field enhancement with mid-IRilluminations (e.g., CO₂ laser at 10.6 µm).[433, 436, 439] Specially designed tips are therefore largely mandatory at visible and THz frequency ranges for achieving higher detection sensitivity and signal-to-noise ratio (SNR). In ultrafast pump-probe SNOM, low SNR is also a major concern due to the nominally weak pump-induced change in the near-field probe signal.

In-liquid and in-situ infrared vibrational nano-spectroscopy can provide intrinsic, label-free nanoscale chemical information on biological, chemical and electrochemical processes in aqueous environment. However, strong broadband IR absorption from vibrational and rotational modes of liquid water and aqueous solutions presents a large undesirable background.

Magneto-optical spectroscopy is instrumental in investigating topological photonics and magnetoplasmonics. SNOM operating in high magnetic fields for magneto-optical nano-spectroscopy has so far remained elusive. A significant barrier to the practical realization of a magnetic SNOM is the efficient optical open access to a cryogenic sample-scan AFM in ultrahigh vacuum (UHV) chambers.

Lastly, the quantitative analysis and interpretation of SNOM data have been nontrivial because detailed fitting and stringent modeling are usually required to include the intricate convoluted tip-sample effects.[38, 428, 429, 435]

### 14.3. Advances in Science and Technology to Meet Challenges

The development and perfection of diverse SPM configurations pave the way for a multimodal SNOM. A frequency-modulation Kelvin force probe microscope based on a tapping-mode AFM can be seamlessly combined with s-SNOM, providing simultaneous surface potential and optical responses at nanoscale resolution.[440] AFM operating in peak force tapping mode offers delicate control of the interaction force between tip and sample, enabling robust tip-sample intimate contact and large modulation in one cycle. Careful interfacing SNOM with peak force mode promises a multimodal SPM capable of correlative nanoscale optical, electrical, and mechanical mapping for cross-domain research.[441] In the quest for a magnetic SNOM, tremendous efforts are underway to customize a sample-scan cryogenic AFM in a high magnetic field in order for easy access to optical excitation and detection. An alternative approach is guiding light via a tapered conical fiber tip in a tip-scan a-SNOM or s-SNOM in a cryogenic UHV chamber with a high magnetic

field, hence obviating the need for external optical open accesses. To circumvent the unavoidable water IR absorption background for in-liquid SNOM, graphene-encapsulated liquid cells have been developed as a thin IR-transparent membrane to separate AFM operating in air and the liquid phase. IR s-SNOM with in-liquid AFM operation and optimized total internal reflection excitation has been demonstrated as a universal approach to infrared nano-spectroscopy in native aqueous environment due to enhanced signal emission and minimized water IR absorption and far-field background.[442]

Advances in electromagnetic modeling and nanofabrication have enabled on-demand tip engineering for specific purposes. Quantum and nonlocal effects have been included in nanophotonics modeling to provide more accurate predictions on local field enhancement and tip-sample near-field interaction.[443] The improved simulation results offer crucial guidance for controllable and cost-effective nanofabrication of desired near-field tips. For visible illuminations, tip apex nanostructure can be properly designed to leverage plasmonic resonance effect to enhance near field and elevate detection sensitivity. The tip length at THz range needs to be critically engineered for the tip to function as an efficient optical antenna for s-SNOM operation.[430] For a-SNOM, an advanced campanile probe composed of a metal-dielectric-metal waveguide to efficiently guide light to the apex has been shown in both simulation and experiment to be reasonably broadband and highly efficient for spectroscopic nano-imaging.[444] Tip engineering is also conducive to ultrafast pump-probe SNOM. A smart tip design can offer optimal near-field enhancement for both the pump and probe lights. In addition, proper tip engineering can boost lateral sensitivity for measurements of the in-plane photoresponse. The on-demand tip design and fabrication are markedly powerful yet far from easy. Further improvement and perfection are needed to achieve reliable and reproducible tip engineering for versatile applications.

Beyond rigorous analytical models,[38, 435] recent progress on data-driven machine learning (ML) methods can partially tackle the complex tip-sample convolution problems in s-SNOM.[445] An artificial neural network can be trained to learn the correlation between the sample's dielectric function and the experimentally measured near-field signal using a limited set of nano-spectroscopy data. The ML method enables fast and robust extraction of quantitative material properties from tip-specific raw data without sophisticated finite-element numerical simulations.

Part of the challenges and prospects are summarized in **Figure 24**. Note that due to the limited scope of this section and the limitations of the authors, Figure 24 is by no means an exhaustive roadmap and serves merely as a guiding resource.

### 14.4. Conclusions

SNOM technique has been progressively perfected as a regular and robust nano-imaging and nano-spectroscopy instrument, which serves to reveal quantitative structure-property relationship at the nanoscale. While challenges remain in the interfacing with other SPM modalities and in the integration with various far-field optical techniques, researchers have paved plausible pathways en route to a broadband and multimodal nanoscale
<figure><img src="image_27.png" /><figcaption>**Figure 24.** Pathways en route to broadband and multimodal SNOM for quantitative nano-imaging and nano-spectroscopy.</figcaption></figure>

characterization platform. With the exciting new technological
and theoretical advances, SNOM technique is poised to become
an essential label-free and quantitative nanoscopy method with
exceptional spatial, temporal, and spectral resolution.

Theme III: Nonlinear Optics Approaches

15. Absorption-Based Super-Resolution Imaging

(Yeran Bai, Ji-Xin Cheng*)

**15.1. Status**

Super-resolution microscopy enables the visualization of pre-
viously unseen details in conventional imaging where the
illumination light cannot be focused smaller than the diffraction
limit. Due to the widespread application of fluorescence imaging
in biomedical research, the majority of efforts in super-resolution
imaging have been pursued along the direction of fluorescence
nanoscopy.[⁹] However, fluorescence imaging suffers from issues
including photobleaching, phototoxicity, and perturbation to in-
tracellular functions. On the contrary, the absorption happens
universally in light-matter interactions, hence allowing label-free
detection. Based on electronic transition and vibrational tran-
sition, UV-Vis and infrared (IR) absorption spectroscopy have
found broad applications in chemistry, biomedical and material
science.[⁴⁴⁶, ⁴⁴⁷] Despite its rising popularity, the spatial resolution
in absorption-based imaging remains to be diffraction-limited.
To address the unmet need to improve the spatial resolution of
absorption-based imaging, many methods have been proposed.
The first category is near-field IR absorption spectroscopy. By uti-
lizing a nanometer-sized aperture or an atomic force microscopy
(AFM) tip, near-field IR spectroscopy has been demonstrated
with nanometer resolution and high sensitivity down to single-
molecule level.[⁴⁴⁸, ⁴⁴⁹] In AFM-IR nanoscopy, a AFM tip is used to

sense the absorption-induced thermal expansion. The IR spectra
with nanometer spatial resolution are obtained by sweeping the
IR light to different wavelengths and record the tip oscillations.
The second category is termed optical photothermal detection
(Figure 25), where the pump beam is selected at the absorption
peaks of the analyte, and the probe beam is selected outside the
absorption range.[450–452, 65] For molecules with high absorption
coefficient but low quantum yield, the absorbed energy is con-
verted into heat, which causes a localized temperature increase.
Since the physical property including volume and refractive index
are dependent on temperature, the local environment experi-
ences a transient change due to the photothermal process. The
probe beam passing through the sample carries the modulation
by the environment, which can be detected through lock-in or op-
tical phase measurement. For the electronic state photothermal
detection, a nearly threefold increase of spatial resolution com-
pared to diffraction limit has been demonstrated for nanoparti-
cles with non-linear absorption when the pump beam exceeds
the linear absorption fluence threshold.[450] For vibrational state
photothermal detection, various efforts have been demonstrated
to break the diffraction limit of the IR pump beams. Since
the spatial resolution is determined by the probe beam in the
photothermal imaging, one order of magnitude increase over
the diffraction-limited resolution has been demonstrated with a
visible or near-IR probe beam.[451, 452, 65] The development of IR
photothermal imaging also enables unprecedented performance
in label-free chemical imaging for live cells in the aqueous
environment at the sub-micrometer resolution.[453, 454, 65] The
third category is based on ground-state depletion in transient
absorption processes. The pump beam is split into two arms
with one arm temporally modulated but spatially unmodified;
while the other arm is sent to a spatial light modulator to create
a doughnut-shape beam shape. When these two pump beams
combine with the modulated probe beam and sent to a sample,
super-resolution imaging is achieved at the very center of the
focal spot.[66]
<figure><img src="image_28.png" /><figcaption>**Figure 25.** Concept and spatial resolution characterization of IR photothermal imaging. a) Visible beam propagation geometry when the IR beam is off. b) When the IR light is on, the photothermal effect leads to deflection of the visible beam. c) IR photothermal imaging of a 500-nm polymer bead. The horizontal and vertical line profile cross the center of the bead indicating a sub-micrometer spatial resolution. Adapted with permission.[65] Copyright 2016, The Authors, some rights reserved; exclusive licensee AAAS. Distributed under a Creative Commons Attribution NonCommercial License 4.0 (CC BY-NC).</figcaption></figure>

**15.2. Current and Future Challenges**

Although the near-field methods have provided the highest spatial resolution so far, the low imaging throughout restrains broader applications of these methods. On the contrary, far-field methods greatly improve the speed and the field of view. A typical acquisition time get a single-cell image is about tens of seconds with the pixel dwell time of 1 to 30 ms for a scanning geometry.[451, 65] However, to observe highly dynamic samples, higher imaging speed is expected. Additionally, the pump-probe signal is only generated at the overlapped area of the two beams. For IR photothermal imaging, the focal spot between the IR pump and visible probe has one order of magnitude difference and thus only about 1% IR photons contribute to the signal generation.

In addition to the need for further increasing photon budget and push imaging speed, depth-resolved imaging is also required for most biological-related research. The near-field methods are restricted to characterize the surface of the samples. For nonlinear absorption, since it requires overlap of two tightly focused beams to produce a signal, depth-resolved imaging can be realized in scanning the foci in 3D.[450, 66] The depth-resolved IR photothermal imaging has also been demonstrated in the same manner.[65] On the other hand, it is harder to achieve depth-resolved imaging in widefield modalities since the beams are not tightly focused.

To facilitate the adoption of super-resolution methods to biologists, multimodal imaging capacities are needed. Although IR spectroscopy provides fingerprints of the analyte, the specificity for certain intracellular organelles or certain proteins still needs further exploration. Combining the abovementioned methods with other analytical tools could maximize the content acquired. Adding fluorescence capability would be a very appealing option since it is the most widely used tool for biologists. However, conventional IR imaging does not compatible with fluorescence imaging. The spatial resolution between fluorescence imaging and IR imaging makes it hard to register the point of interest.

The detection sensitivity in current far-field super-resolution absorption-based methods is limited to sub-micrometer particles. Single bacterium sensitivity has been achieved both with the scanning system and the widefield system.[451, 455, 456] Using confocal configuration, a single virus has been imaged with the scanning far-field IR photothermal microscope.[457] Since the photothermal process is transient, with a typical thermal decay time at hundreds of nanoseconds to several microseconds, the system needs to be optimized for improving detection efficiency. Simulation of the photothermal processes suggests that a high peak intensity heating beam should be selected to maximize the temperature increase for nanosized objects.[452] In the meantime, heating power should be considered to avoid thermal damage when observing live cell samples.
<figure><img src="image_29.png" /><figcaption>**Figure 26.** Widefield IR photothermal imaging setup and live-cell chemical maps. a) Mid-IR pump and visible probe are loosely focused to the sample and thus enables the widefield detection of the photothermal responses. b,c) Label-free live ovarian cancer cell imaging targeted at the lipid C=O bond vibration and the protein amide I band. Scale bars: 10 µm. Adapted with permission.[454] Copyright 2019, The Authors, some rights reserved; exclusive licensee AAAS. Distributed under a Creative Commons Attribution NonCommercial License 4.0 (CC BY-NC).</figcaption></figure>

**15.3. Advances in Science and Technology to Meet Challenges**

To boost the imaging speed, widefield photothermal imaging has been proposed.[454, 458] In the widefield imaging modality (Figure 26), the illumination area of the IR and visible beams are matched. Therefore, all pump photons can be fully utilized to generate signal. Synchronization between lasers, cameras, and modulation electronics enabled ultrafast chemical imaging speed over a large field of view.

To improve the depth imaging resolvability in widefield-detected photothermal imaging, methods such as optical diffraction tomography[459] have been proposed. Multiple angle illumination of the visible beams expanded the bandwidth in the frequency spectrum, and thus improved the resolution. Other 3D imaging modalities such as optical coherence tomography can also be coupled into current photothermal systems to achieve volumetric imaging capability.

The multimodal imaging is easy to implement in the far-field super-resolution techniques due to the two-beam configuration in pump-probe experiments. The probe beam is spectrally in the visible or near-IR range, which falls in most fluorescence excitation spectra. Work has been done using the fluorescence dye as a thermal sensor to probe the photothermal processes, where the fluorescence emission efficiency is a function of temperature.[460] In this work, fluorescence is used as a detector as well as an indicator of certain organelles. Integrating the specificity of fluorescence labeling and IR photothermal imaging would make fingerprinting different organelles under certain microenvironments possible. In addition to fluorescence imaging, other analytical methods such as Raman spectral measurements are also compatible with IR photothermal imaging. Li et al. demonstrated the combination of IR and Raman on the same setup, where a 532-nm probe laser in IR photothermal detection also functions as the Raman excitation laser.[455, 456]

The detection sensitivity in IR photothermal imaging can be further improved. For the current scanning system, most of the demodulation methods are based on lock-in amplifiers. However, the lock-in amplifier might not be the ideal selection to demodulate the signal. Since the pulsed heating beam is typically with a low duty cycle (<10%), the corresponding photothermal signal produces many harmonics. While the lock-in amplifier only picks up fundamental frequencies, the higher harmonics signal is neglected which leads to lower detection sensitivity. A recently developed photothermal dynamic imaging system successfully addresses this limitation by digitizing the whole photothermal process.[461] After performing the Fourier transform, all the harmonics is added to enhance the signal-to-noise ratio. In the meantime, all recorded data adds another benefit of time-resolved imaging. By performing the digitization, a four-fold signal-to-noise ratio is demonstrated. In addition to the data streaming optimization, the careful design of sample substrate would also improve the detection sensitivity. Substrate engineering such as interferometric scattering or plasmonic scattering is an effective way to boost imaging sensitivity.[221, 462]

**15.4. Conclusions**

Absorption-based super-resolution imaging is evolving exponentially over the past few years. Providing label-free, nanometer to sub-micrometer resolution capability, chemical selectivity, absorption-based super-resolution imaging offers unprecedented performance in investigating nanomaterials and biomedical samples. Looking into the future, it is anticipated that this rising field will continue to expand and will deepen the understanding about molecular machinery inside a cell and a functional material.

**16. Pump-Probe Super-Resolution Microscopy**

(Paolo Bianchini*, Behjat S. Kariman, Alberto Diaspro)

**16.1. Introduction**

Optical microscopy uniquely provides non-invasive imaging of biological specimens and is an essential tool in life sciences. In particular, fluorescence microscopy is probably the most widely used technique to image living specimens, and much effort has been dedicated to improving it over one of its intrinsic limits, i.e., resolution. However, it requires the introduction of exotic fluorescence stains, which could become the major obstacle to studying a living specimen in its physiological conditions. Label-free methods result in the solution to such a limit without missing consolidated characteristics, i.e., specificity and resolution. Therefore, label-free chemical imaging is becoming one of the best choices to answer this challenge.

In this context, non-linear optical processes—i.e., when observable signal non-linearly depends on the illumination intensity—have captured the attention of life scientists for the development of label-free and super-resolved
<figure><img src="image_30.png" /><figcaption>**Figure 27.** Energy level diagrams of pump-probe microscopy based on contrast mechanisms of a) ground state depletion, b) stimulated emission, c) excited state absorption, and d) two-photon absorption.</figcaption></figure>

approaches.[14, 15, 463–465] In particular, pump-probe microscopy can image a wide range of specimens at high speed and resolution using different modalities: transient absorption, stimulated Raman scattering (SRS), coherent anti-stokes Raman scattering (CARS), second harmonic generation (SHG), two-photon excitation fluorescence (ZPEF), etc. The saturation of some light-driven processes can be exploited in fluorescence microscopy to shrink the point spread function beyond the diffraction limit. Among those methods, it is worth noting stimulated emission depletion (STED) nanoscopy[15] and saturated excitation (SAX)[466] microscopy, which are based on far-field focused laser scanning systems. Point scanning remains one of the most flexible approaches since it enables a vast assortment of spectroscopical experiments while imaging. It can also explore the advantages of using proper wavelengths in a broad range of the spectrum from ultraviolet (UV) through visible (vis) up to the near-infrared (NIR) energies. For instance, many innovations exploiting NIR allow imaging at increased depth, improving resolution, contrast, molecular specificity, and, most importantly, label-free capabilities.[467] Illumination with NIR laser light very often deals with non-linear frequency conversion and non-linear absorption processes. Such light-matter interactions provide molecular contrast without needing exogenous labels and preserve the local biochemical environment of the target molecules.[468] In general, they involve the second $\chi^2$ and third $\chi^3$ order of susceptibilities.[469] Transient absorption is related to third-order non-linearity susceptibility but rarely implicates fluorescence emission. Its recent application in microscopy has broadened the range of suitable targets while providing high contrast in weak or non-fluorescent samples.[66, 67] The contrast mechanism comes from the observation of processes such as ground-state depletion (Figure 27a), stimulated emission (Figure 27b), excited-state absorption (Figure 27c) and two-photon absorption (Figure 27d).

When such processes are saturable with the irradiation intensity, they can be exploited for super-resolution microscopy.[30]

Transient absorption can be exploited by a pump-probe approach, which is a well-known spectroscopic method used to study the time-dependent ultrafast phenomena in matter. When applied for microscopy imaging, it brings the following advantages: while keeping specificity, it is label free[470]; it is not invasive nor destructive, thus suitable for living cell and tissue imaging[471]; it has three-dimensional (3D) sectioning capability and a high penetration depth.[472] Dong et al., in the 1990s, realized the first pump-probe microscope to measure the fluorescence lifetime through SE detection.[473] But only in 2007, thanks to the introduction of a high-frequency modulation scheme to detect melanins in cells,[474] transient absorption microscopy has started to be strongly developed and applied in many different research fields, from material to biological sciences. Pump-probe microscopy possesses non-fluorescent-based contrast, single-particle sensitivity, and high temporal resolution. Therefore, not only it allows characterizing the carrier dynamics of metallic and semiconducting nanostructures for studying their optoelectronic properties, but it also became relevant in mapping weakly or non-fluorescent nanostructures (like single-walled carbon nanotubes, nanodiamonds, gold nanoparticles...) in biomedical applications inside living cells and tissues.[475] For instance, the Cheng group investigated the properties of single-walled carbon nanotubes (SWNTs), distinguishing semiconducting from metallic SWNTs by means of the phase of the pump-probe signal.[476] Among carbon-based materials, graphene is one of the most intensively studied.[477] Due to its linear electronic band structure, graphene shows a wavelength-independent, broadband optical absorption in the NIR part of the spectrum,[478] and a large third-order susceptibility $\chi^{(3)}$.[479] These properties make graphene very well suited to be studied by pump-probe super-resolution microscopy using NIR wavelengths. The pump-probe signal of graphene is linear with respect to the applied pump and probe powers and the number of graphene layers.[480] While at low powers, the linear behavior is satisfied, at higher powers, the signal saturates, as predicted by the state-filling effect of the higher electronic states.[481]
<figure><img src="image_31.png" /><figcaption>**Figure 28.** a) Example of a two-beam laser-scanning pump-probe nanoscope setup. BS: beam splitter, EOM: an electro-optical modulator for intensity modulation, vpp: vortex phase plate, DM: dichroic mirror, PBS: polarizing beam splitter, SH: scanning head unit, O: objective, C: condenser, det: detector. The pump beam is shown in green while the probe beam in red. b) Temporal modulation behavior of input and output pulse trains before and after interacting with the sample. Depending on the light-molecule interaction, the probe beam can undergo either a relative gain (SE, SRG, GSD) or a loss (TPA, ESA) in its output intensity (ΔT), exhibiting in-phase and anti-phase modulation, respectively. The pump temporal modulation is shown as a black square wave. The input probe intensity is marked by a dashed grey line. The tunable temporal delay (Δt) of input pulses is shown in the inset graph. c) Normalized pump-probe and saturated pump-probe images of SLG folding and defects. Scale bar 2 µm. Zoomed regions I and II are also presented, and line profiles across the arrows are shown as black dots (non-saturated case, PP) and red dots (saturated case, SPP). Gaussian fits of the non-saturated data are shown as solid black lines, while Lorentzian single-peak (I) and double-peak (II) fits of the saturated data are shown as solid red lines. The dashed red lines in graph II highlight the single Lorentzian peaks retrieved from the analysis. The obtained resolution is marked in the graphs as full-width-half-maximum of the fitted curves. Adapted with permission.[67] Copyright 2019, American Chemical Society.</figcaption></figure>

Such a particular property enables far-field nanoscopy as defined by the Nobel laureate Stefan Hell in 2003.[482, 483] Therefore pump-probe microscopy become nanoscopy, a valuable, fast, sensitive, and non-destructive tool to map the local excited state dynamics and characterize the nano-defects.[484, 67]

## 16.2. The Pump-Probe Setup Enabling Super-Resolution

In a typical pump-probe microscope (Figure 28a), a pump pulse excites the molecule, and a probe pulse interrogates the transient excited states. Optical pump and probe can provide images by detecting changes in probe light intensity induced by the pump on the sample carrier population in either transmission or reflection modality. The pump-probe process at the molecular level can be defined by the absorption coefficient, an electronic transition between different levels.

Depending on the kind of transition that the material undergoes, the probe intensity can exhibit either a loss or a gain. The reason includes the following possible effects: excited-state absorption (loss), and two-photon absorption (2PE) (loss), stimulated emission (SE) (gain), and ground-state depletion (GSD) (gain) shown in (Figure 28b). The latter was used in microscopy to overcome diffraction barrier light in 2007,[485] following the

most general concept of RESOLFT (reversible saturable optical fluorescence transition).[486] The number of molecules in the ground state is decreased upon photoexcitation due to an intense pump beam, leading to increased transmission of probe pulse and absorption reduction at probe frequency. When GSD can be saturated, e.g., carbon materials like graphite and graphene, a STED-like scheme is applicable, and super-resolution is realized by detecting the remaining probe absorption in the neighborhood of the zero-intensity feature of the depletion beam.[67]

Such light-matter interaction is recorded as an intensity variation of the probe beam, which is detected by a non-descanned photodiode, a notch filter removes the pump signal, and the modulation is isolated from the background by electronic filtering with a lock-in amplifier locked at the frequency imposed to the electro-optical modulator, which controls the pump intensity. Subdiffraction spatial resolution is then realized by applying an unmodulated doughnut-shaped pump (Figure 28a). The pulses of such a beam need to be temporally aligned with the pump and probe ones, and this is generally realized with optical delay lines. To avoid molecular selection rules and to realize a good doughnut beam featuring a real zero-intensity level at the center, circular polarization at the sample plane is required.[487] This condition can be easily realized by means of two retarder plates, i.e., a half- and a quarter-wave plate.
## 16.3. Far-Field Nanoscopy Imaging of Single Layer Graphene

Pump-probe nanoscopy has been successfully demonstrated to image graphite nanoplatelets,[66] and single layer graphene (SLG).[67] The depletion of their ground state exponentially decays with the increase of the pump power. It has the same behavior as the excited fluorescence state in the case of STED. So that the final resolution will be limited only by the maximum saturation-pump power can be applied to the sample.

From the published results setting the saturation power to 2.0 MW cm⁻² it is already enough to achieve a resolution of about 225 nm (≈ λ/3.0) on graphite,[66] while on graphene with about 7.0 MW cm⁻² is possible to achieve a resolution of the order of ≈ λ/10 (Figure 28c).[67]

In Figure 28c, we show an example of comparison between conventional pump-probe (PP) and saturated pump-probe (SPP) images of SLG with its multilayer islets. The spatial resolution improvement thanks to saturation beam is remarkable.

## 16.4. Conclusions and Perspectives

The pump-probe approach allows resolving and monitoring chemical and atomic ultrafast processes without the need for fluorescence emission. The latter is a standard in microscopy, allowing specificity and super resolution requesting a dedicated probe.[358] Therefore, a label-free method is of particular interest. It can avoid invasive procedures and allow long imaging sessions without loss in signal, i.e., photobleaching. However, since pump-probe microscopy is realized by employing ultrafast and ultrashort-pulse laser sources, such a method allows for multimodal imaging in which fluorescence, harmonic generation, and coherent Raman scattering, when needed, can be easily combined, and correlated. In some cases, like for carbon nanomaterials, the transient absorption process is saturable at the time scale of the laser pulses, thus, super-resolution can also be achieved following the general RESOLFT concept. In particular, graphene, which recently becomes one of the most studied materials because of its implication in many sciences and society fields, shows a linear electronic band structure, and a wavelength-independent, broadband optical saturable absorption in the NIR, together with a large third-order susceptibility χ(3). This characteristic makes pump-probe nanoscopy ideal for mapping the carrier dynamics in all its different graphene forms. Nevertheless, since transient saturable absorption is not a unique property of graphene, we may envision novel pump-probe nanoscopy implementations on many other molecules, able to probe the molecular organization and dynamics of a cell with a spatiotemporal resolution that can assist in deciphering the most puzzling mechanisms of life.

# 17. Super-Resolution Raman Microscopy

*(Katsumasa Fujita)*

## 17.1. Status

Raman microscopy has emerged as one of the key modalities of imaging methods for biological and medical research. Vibrational spectroscopy of endogenous molecules in a sample can

provide information about biological samples that is not available using conventional imaging techniques. While the small cross-sectional area of Raman scattering has hindered powerful analytical capabilities in the life sciences, recent developments in optics and laser technology have resulted in techniques that improve image acquisition speed for both spontaneous and coherent Raman scattering microscopy. Surface-enhanced Raman scattering (SERS) is also used to reduce image acquisition time and increase the sensitivity of Raman detection. As with other optical imaging techniques, the spatial resolution of Raman microscopy is limited by the wave nature of light. Therefore, many attempts have been made to overcome this limitation using super-resolution fluorescence imaging strategies such as stimulated emission depletion (STED), single molecule localization microscopy (SLM), and structured illumination microscopy (SIM).[488]

## 17.2. Current and Future Challenges

Although Raman scattering and fluorescence emission are different physical phenomena, they can share the same techniques to overcome the resolution limit. In particular, spontaneous Raman scattering has the same imaging properties as fluorescence microscopy, which uses incoherent light to generate sample images. The techniques used for super-resolution fluorescence imaging are essentially valid for spontaneous Raman imaging. Coherent Raman microscopy (e.g., coherent anti-Stokes Raman scattering microscopy (CARS), stimulated Raman scattering microscopy (SRS)) uses coherent light to generate images and has different imaging properties from fluorescence microscopy. However, since they are based on laser scanning, the methods used in STED microscopy[15] and saturation excitation (SAX) microscopy[466] are effective in overcoming the resolution limitation.

The STED-based technique is used for both spontaneous and coherent Raman microscopy using laser scanning. By superimposing a donut-shaped beam on the Gaussian spot for Raman excitation, the signal at the peripheral region of the Gaussian spot is suppressed, limiting the volume of signal generation within the excitation spot. Several methods have been proposed to suppress signal generation such as prepopulation of vibrational state, ground state depletion, pump photon depletion, and phonon suppression. In any case, in order to obtain spatial resolution beyond the limit, it is important to introduce a nonlinear relationship between signal suppression/generation and excitation intensity, otherwise the enhancement will be limited to the contrast enhancement of high spatial frequencies.

The use of STED microscopy in Raman imaging is more difficult than in fluorescence imaging because of the short lifetime of the vibrational excitation state. The typical fluorescence lifetime of dyes with high quantum yields is a few nanoseconds, whereas the lifetime of vibrationally excited states is as short as ≈ps. Therefore, higher laser intensities are needed to suppress the Raman signal, and it is difficult to suppress the signal deeply enough to obtain high spatial resolution as in STED microscopy of fluorescence. Fluorescence microscopy can even offer more time to manipulate the signal through transition to triplet and dark state, photoisomerization, which are used in ground state
<figure><img src="image_32.png" /><figcaption>**Figure 29.** Examples of label-free super-resolution Raman imaging. A mouse brain tissue observed by a) conventional line illumination and b) structured line illumination Raman microscopy using 532 nm excitation. Raman peaks at 1682 cm⁻¹ (amide-1, red) and 2848 cm⁻¹ (CH₂ stretching, green) were used for image reconstruction. c) the intensity profiles of Raman band observed between the arrows in a) and b). d)-f) SRS images of an expanded HeLa cell. A vibrational mode of CH₃ at 2940 cm⁻¹ was detected by using 1031.1 nm Stokes and 791.2 nm pump beams. e) and f) are zoomed images of the yellow box region in d) and e), respectively. g) the line profile on the yellow line in f). Scale bars: 30 µm in d) and 2 µm in e) and f). a–c) Reproduced under the terms of the CC BY 4.0 License.[493] Copyright 2015, The Authors, published by Springer Nature. d–g) Reproduced under the terms of the CC BY 4.0 License.[496] Copyright 2021, The Authors, published by Springer Nature.</figcaption></figure>

depletion (GSD) microscopy and RESOLFT microscopy. It is difficult to realize these techniques in Raman microscopy as effectively as in fluorescence microscopy.

Another approach to achieve super-resolution imaging in laser scanning Raman microscopy is to introduce a higher-order nonlinear response in the Raman effect. By saturating the Raman effect, a higher-order nonlinear relationship is induced between the signal and the incident light, and the signal generation is localized within the focal volume. So far, the use of saturation in CARS and SRS has been demonstrated[489, 68] for high-resolution coherent Raman imaging based on SAX microscopy. It has also been demonstrated to induce higher-order CARS in a sample and to achieve super-resolution vibrational imaging of biological samples.[490] The use of photothermal nonlinearity in Raman scattering has also been exploited to realize super-resolution imaging in confocal Raman microscopy.[491]

SIM can be used for spontaneous Raman and SERS imaging because its imaging properties are the same as those of fluorescence microscopy. Since a typical SIM involves wide-field detection, applying SIM to filter-based Raman microscopy is straightforward.[492] However, it sacrifices the spectral resolution and require a long measurement time for hyperspectral imaging. The SIM-based technique has also been applied to line illumi-

nation Raman microscopy, where the sample is irradiated with a line-shaped focus and the image is formed by conventional imaging for the direction of the irradiated line (Figure 29a-c).[493] Similarly, image scanning microscopy (ISM) is also effective to improve the spatial resolution of Raman microscopy.[494] The use of 4π optics was also demonstrated, which realized the detection of the detection volume for the axial direction effectively.[495]

As mentioned above, manipulation of Raman signals is not as easy as fluorescence signals, so there are few applications of SLM in Raman imaging. It is well known that SERS signal show blinking by reflecting the condition of molecules on a metal surface. This blinking effect was exploited to perform SLM in SERS imaging.[497] However, there are still challenges in applying this technique because it requires manipulation of SERS hotspots on the sample.

### 17.3. Advances in Science and Technology to Meet Challenges

Although the super-resolution techniques used for fluorescence imaging have been explored, the spatial resolution achieved for Raman imaging is still limited. This is because of the difficulty of manipulating the Raman effect in target molecules. To push
the current limit, innovative ways to control the Raman effect, such as the capability of switching Raman effect and induction of highly nonlinear relations between incident light and Raman effect, need to emerge. Another difficulty in the high-resolution Raman imaging is its weak signal. The reduced detection volume in high-resolution imaging can provide only small detection signals, which requires a much longer exposure time. The reduction of signal-to-noise ratio (SNR), making the practical resolution lower. Therefore, increasing the Raman signal or reducing noise is inevitable to improve the spatial resolution further. Recently, a technique to improve the SNR by using the squeezed light has been demonstrated in SRS microscopy,[498] which is expected to contribute to the improvement of spatial resolution in SRS imaging.

The use of non-optical methods is another direction. Recently, expansion microscopy has been applied to super-resolution SRS imaging. In expansion microscopy, the volume of the sample is expanded by swellable polyelectrolyte hydrogels, of which volume can be highly increased by being immersed in a solvent, before observation.[499] The size of the sample can be increased by a factor of 4.5 to 20 in the linear dimensions, and a more detailed spatial distribution of the target can be observed than with conventional techniques, even using diffraction-limited optical microscopy. The expansion microscopy technique has been applied to observe tissue samples by SERS and SRS microscopy (Figure 29d-g).[496, 500] Although this technique cannot be used on live samples, it extends the usefulness of vibration microscopy for investigating microstructures in cells.

Switching the capabilities of Raman scattering has recently been demonstrated using chemical compounds.[501] It has been shown that isomerization of the molecular structure can change the Raman spectrum. This switching capability can be used for both STED and SLM-type super-resolution techniques. At this time, it may be difficult to apply this strategy to molecules intrinsic to the samples, but as our understanding of the photophysical properties of molecules improves, it will eventually show us how to manipulate the Raman scattering cross-section of molecules.

## 17.4. Conclusions

The development of super-resolution Raman imaging has proceeded mainly by using methods developed for fluorescence microscopy. Although these methods are effective in principle, it is practically difficult to achieve a spatial resolution of 100 nm or higher. Furthermore, it would be necessary to develop a method suitable for Raman imaging, where pre-preparation of samples is not expected, unlike fluorescence imaging. In other words, we need to develop a fully optical super-resolution technique that does not rely on the manipulation of sample properties.

Currently, the main applications of Raman spectroscopy are in material science and industry, such as nanomaterials, semiconductors, batteries, polymers, and pharmaceuticals. Although super-resolution techniques have been intensively developed for the observation of biological samples so far, exploring application in materials science is important for super-resolution Raman imaging. In fact, these applications typically involve samples that are more resistant to photodamage and produce more signal than biological samples, which can be advantages in realizing super-resolution imaging. Since such material analysis is

performed without labeling, label-free super-resolution Raman imaging is inevitable. Also, while SEM and AFM are the primary choices for high-resolution imaging, the ability to provide high-resolution 3D imaging, a major advantage of optical microscopy, is attractive to this field.

# 18. Silicon Non-Linearity Based Super-Resolved Pump-Probe Imaging

(Moshe Sinvani, Zeev Zalevsky)

## 18.1. Introduction

In optical microscopy the property of diffraction limits the resolution and yields that one cannot image spatial details being smaller than half of the wavelength of the illuminating light because such small spatial features cause light to propagate as evanescent rather than a harmonic wave. Various concepts were proposed in the attempt of bypassing this physical limitation.[502–505]

One important breakthrough that initiated the field of nanoscopy and which was applied for fluorescence imaging proposed a new type of scanning fluorescence microscope.[14, 287] This imaging concept could reach a resolution of 35nm and it was named stimulated emission depletion (STED).[506] Further improvement of that concept was generated in years to follow[507–509] leading eventually to imaging capabilities of ≈5nm for fluorescent color centers in diamond crystal.[510] STED concept exploited the typical characteristics of the fluorescent molecules. Two beams were used to obtain the super-resolved imaging. The first is a pulsed laser beam acting as a pump beam having optimal Gaussian point spread function (PSF) and being focused on the examined sample while acting as the fluorescence excitation source. A second pulsed laser beam which has a donut shape, is stimulating depleting fluorescence laterally around the center of the excitation beam spot while leaving the center region of the focal spot active to continue and emit fluorescence. Due to this process the imaging PSF is laterally sharpened since the lateral regions of the PSF coinciding with the donut shape beam are significantly attenuated and only the central part of the PSF of the excitation beam produce emission from the sample, which leads to enhanced imaging resolution.

Besides STED additional approaches were also introduced aiming to produce a nanometric imaging resolution while all using various physical aspects of the fluorescence effect such as the stochastically sparse subsets of fluorophores, namely the photoactivated localization microscopy (PALM) approach[244, 511–514] or as stochastic optical reconstruction microscopy (STORM) approach.[515] PALM exploits the spontaneously occurring phenomenon of photobleaching while STORM exploits the reversible switching between a fluorescent on-state and a dark off-state of a fluorescence dye.

Example of other methods developed in the field of nanoscopy and which also involve interaction of probe-pump beams include for instance adding to the surface that is to be imaged a photochromic absorption modulated layer[516] or using the concept of saturated transient absorption microscopy where usage of materials with very small absorption saturation level[66] is applied to
<figure><img src="image_33.png" /><figcaption>**Figure 30.** Illustration of the operation principle for the two possible directions: a) A narrow pump Gaussian beam at 532 nm, creates a hole in the middle of a wider IR probe beam. b) Donut shape 532 nm pump beam blocks the periphery of the Gaussian IR probe beam and transmits only a narrow beam in its center. Reproduced with permission.[69] Copyright 2018, Optical Society of America under the terms of the OSA Open Access Publishing Agreement.</figcaption></figure>

obtain sub-diffraction resolution in far field. Additional methods use various types of silicon-coated gold nanoparticles that allow to generate a time multiplexing super-resolved imaging by modifying the PSF into a donut-like shape while utilizing the plasma dispersion effect (PDE)[517] or by using coherent anti-Stokes Raman scattering (CARS) approach.[518–520]

In our research, we use the plasma dispersion nonlinearity occurring in silicon in order to realize a super-resolved nanoscope capable of providing significant advantage to the micro-electronic industry in the field of failure analysis. Our concept involved a pump-probe scanning architecture that maps the silicon wafer with nano-metric resolution that could be extracted not only from the surface of the integrated circuit but also from its deeper layers. The photons of the pump are absorbed in the silicon chip and generate free electrons that shape the PSF of the probe that is near infra-red (IR) wavelength of 1550nm. The reshaping of the probe PSF is done through the plasma dispersion non-linearity causing either narrowing of the PSF or to the generation of a narrow dip in its center i.e. in both cases the PSF of the probe has high spatial frequencies being higher than the limits sets by the laws of diffraction. When such sharpened PSF scans the inspected sample (i.e. the PSF is multiplied by the spatial content of the target), the high spatial frequencies of the sample are down-converted and become low spatial frequencies that can propagate from the sample to the imaging sensor as harmonic rather than evanescent waves and thus be sensed and decoded.

**18.2. The Concept**

As mentioned above our method is based upon PDE in silicon
in which near infra-red (IR) laser probe beam (at wavelength that
regularly is not absorbed in silicon) can be spatially shaped by
the silicon's locally changed refractive index (its real as well as
its imaginary part) that is modified via another pump laser beam

that its photons are being absorbed in the silicon and by that cre-
ate change in the free electrons density. The spatial shaping of the
probe beam generates in its PSF higher spatial frequencies and
after scanning the inspected object may produce a super-resolved
image. Note that the proposed super resolving mechanism is a
non-linear effect because the absorption coefficient for the probe
is proportional to the power of the pump and therefore following
the Beer-Lambert law, the total absorption of the probe is expo-
nentially related to the absorption coefficient.

Unlike in the STED method, the proposed approach has flex-
ibility in the manner of how the PSF of IR probe beam is done.
One direction is to induce a hole or a dip in the center of the
probe's PSF by applying a narrower Gaussian pump beam coinc-
ding with the center of the probe's PSF[518] (Figure 30a). How-
ever, in this case, the PSF has a shape requiring post-processing
decoding to reconstruct the super-resolved image. Another direc-
tion is to apply a donut shape pump beam. In this case, the central
part of the IR beam passes through while its lateral surrounding
is significantly attenuated (Figure 30b). Then the scanning of the
narrowed probe PSF produces the super-resolved result without
the need for any further post-processing. In both cases, high spa-
tial frequencies are induced and reflected in the nano scales via
the same free charge carrier-induced absorption mechanism.

To further demonstrate the proposed operation principle we
show some experimental results presenting the breaking of the
diffraction limit by generating a narrow dip in the center of the
probe's beam PSF that can lead also to sub-wavelength super-
resolved imaging capabilities when performing scanning with
the experimentally shaped PSF beam.[69]

In Figure 31, we presented how that shaping of the probe PSF
is obtained. In order to obtain a super-resolved image the pump
and the probe beam need to synchronously scan the inspected
sample. They need to do the scanning together because in order
to obtain superresolution, the probe PSF needs to be shaped. This
shaping is obtained only due to the pump beam. The process of
<figure><img src="image_34.png" /><figcaption>**Figure 31.** The dip generated in the Gaussian IR probe beam was induced by the green pump beam. One diffraction limit unit is 600 µm. a). The image of the transmitted Gaussian probe beam and its profile. b). The shaped probe beam and its profile. The probe beam is having a dip that is superimposed on its center by the pump beam. Reproduced with permission.[69] Copyright 2018, Optical Society of America under the terms of the OSA Open Access Publishing Agreement.</figcaption></figure>

<figure><img src="image_35.png" /><figcaption>**Figure 32.** Gaussian probe IR laser beam at the diffraction limit scan across a resolution target containing 3-bars having a period of 500 µm: a) Without applying the pump beam. No resolution improvement is obtained. b) A scan of the target with the pump beam creating a dip in the probe's beam PSF. Super-resolution is obtained and the target is reconstructed. The red lines are the direct scan results. The blue lines are the direct scan results deconvoluted with the shape of the probe beam having the central dip. Reproduced with permission.[69] Copyright 2018, Optical Society of America under the terms of the OSA Open Access Publishing Agreement.</figcaption></figure>

super-resolved reconstruction obtained following the scanning of the inspected sample is presented in **Figure 32**. The super-resolving experiment of Figure 32 includes imaging a resolution target that is composed of 3-bars having spatial period of 500 µm.

The IR beam was focused on the silicon surface with a very low NA lens (one diffraction limit unit is 600 µm) and the camera was without a lens. When scanning the resolution target only with the probe beam and without illuminating with the pump beam, the probe's PSF is not narrowed, and no superresolution is obtained (Figure 32a). However, when the scanning is performed both with the probe as well as with the pump beam, the probe's PSF is narrowed and the superresolving result is extracted (Figure 32b). In both cases, the raw scan is colored in red. Knowing the beam PSF we used MATLAB software to perform a standard Wiener filter algorithm to de-convolute the raw scan, and the results are colored in blue. Superresolved results are clearly seen in the images of Figure 32b.

Further development of our approach while using tunable pulses for the pump and the probe allowed us to play with the position of the focus of the probe beam and control its focusing and defocusing processes in deeper layers of the silicon chip[521–526] and not only on the wafer’s surface. We used probe near IR (at $\lambda = 1.55$ µm) beam and a pump beam at $\lambda = 775$nm with 30 ps pulse width, with a donut shape (generated via beam

shaping vortex phase plate). Both the probe and the pump beams were lunched collinearly and simultaneously (with some time delay between the pump and the probe pulses). Thus, the proposed PSF shaping method is not only suitable for overcoming the diffraction resolution limit in silicon microscopy, but it may also allow doing this deep under the silicon surface dependent on the pump wavelength and the pump/probe lasers pulse widths.

## 18.3. Conclusions

The proposed super resolution concept that is based upon silicon non-linearity called plasma dispersion, to generate failure analysis super-resolved image with nano metric resolution after scanning the integrated circuit with a pump-probe collinearly aligned beams. Our super-resolving approach allows improving the imaging resolution beyond the diffraction limit and towards the nanometric scale but obtaining it not only on the surface of the silicon wafer but also deeper inside, which is being a significant advantage in the field of failure analysis of integrated circuits.

# 19. The Rise of Mie-Tronics-Based Nonlinear Scattering Imaging

*(Xiangping Li*, *Guan-Jie Huang*, *Shi-Wei Chu*)*

## 19.1. Status

Traditional super-resolution imaging mostly relies on manipulating nonlinear behaviors of fluorescence emission, including on/off switching of the molecules deterministically or stochastically, saturation of excitation/emission, and high-order multiphoton excitation.[488] Despite the great success of super-resolution fluorescence techniques in cell biology and biomedical sciences, one major challenge is the limited photon yield due to photobleaching and low photostability. Therefore, developing a fluorescence-free contrast agent is highly desirable for super-resolution techniques.

Light scattering by nanoparticles is ubiquitous in many subjects of optics. The associated Mie theory and optical properties may lay the foundation for a new type of imaging probes. The excitation and interaction between radiating multipoles of different orders expand the emerging field of “Mie-tronics” to further
<figure><img src="image_36.png" /><figcaption>**Figure 33.** Optical nonlinearity and super-resolution imaging in plasmonic NS. Panels (a), (d), and (g) are various types of nonlinearities, including sub-linearity (a), super-linearity (d), and all-optical switch (g), from a single gold nanosphere (80- or 100-nm diameter). b,e,h) Super-resolution scattering images are obtained by SAX microscopy (b), where images are reconstructed by signal demodulated at $f_m$ and $3f_m$, where $f_m$ is the temporal modulation frequency; by super-linear microscopy (e), where images are formed under low and high illumination; and by SUSI microscopy (h), where a $\lambda = 592$-nm donut suppression beam is applied. Panels (c), (f), and (i) are corresponding line profiles, quantifying the resolution enhancement. a–c) Reproduced with permission.[532] Copyright 2014, American Physical Society. d–f) Reproduced with permission.[534] Copyright 2014, American Chemical Society. g–i) Reproduced under the terms of the CC-BY license.[529] Copyright 2016, Springer Nature.</figcaption></figure>

manipulate Mie resonances from imaging to nanophotonic applications. Interested readers may refer to this recent review to explore advances in Mie-tronics.[527] We have proposed to adopt scattering from Mie-resonators, especially in plasmonic and dielectric nanostructures (NS) as a novel super-resolution contrast. The reasons include 1) scattering intensity from Mie-resonators is particularly strong due to resonance enhancement; 2) scattering does not suffer from photobleaching; 3) NS scattering offers high photostability. However, typically intrinsic nonlinearities of metal and dielectric materials are weak.

Recently, our groups unraveled giant nonlinearities in both plasmonic and dielectric Mie-resonators,[528, 529, 70] whose behaviors can be manipulated by excitation wavelength and NS morphology. The underlying nonlinear mechanism is based on photothermal and thermo-optical effects, which interact with Mie-resonance and modulate NS scattering intensity. In brief, as laser excitation wavelength is located near the Mie resonance peak of NS, absorption and the consequent photothermal heating becomes more efficient.[70] Subsequently, temperature rise modifies the NS refractive index via thermo-optical effect, and

the index alteration is amplified by Mie resonance, leading to significant scattering variation, that corresponds to nonlinear behaviors.

In the theoretical aspect, a self-consistent model based on Maxwell and heat equations has been developed to quantitatively deduce the Mie enhanced photo-thermo-optic nonlinearity.[530] In experiment, we develop an x-scan method based on a laser scanning microscope to characterize the optical nonlinearities of a single NS.[531] The idea of x-scan is to scan a focused Gaussian beam laterally across a NS, whose diameter is much smaller than the Gaussian beam width, and collect its emission. If the NS exhibits a nonlinear response, the resulting images deviate from the Gaussian profile (linearity) and the nonlinear coefficient is derived directly from the deviation. Through the x-scan method, various nonlinearities are observed, including sub-linearity, super-linearity, and all-optical switch (Figures 33 and 34).

These nonlinearities are all applicable to super-resolution imaging in both plasmonic and dielectric NSs. As a NS is illuminated by a focused Gaussian beam, whose intensity gradually increases, the nonlinear behaviors should first appear in the
<figure><img src="image_37.png" /><figcaption>**Figure 34.** Optical nonlinearity and super-resolution imaging in dielectric NSs. Panels (a), (d), and (g) are various types of nonlinearities, including sub-linearity (a), super-linearity (d), and all-optical switch (g), from a single silicon NS or silicon NS array. b,e,h) Super-resolution scattering images are obtained by SAX microscopy (b); by differential signal processing in super-linear microscopy denoted as differential reverse saturation scattering (DRSS) (e); and by STED-like microscopy (h), where a $\lambda$ = 693-nm donut suppression beam is applied under low and high illumination to present different scattering saturated levels. Panels (c), (f), and (i) are corresponding line profiles, quantifying the resolution enhancement. a–c) Reproduced under the terms of the CC-BY license.[70] Copyright 2020, The Authors, published by Springer Nature. d–f) Reproduced under the terms of the CC-BY license.[528] Copyright 2020, The Authors, published by Springer Nature. g–i) Reproduced with permission.[536] Copyright 2021, OSA publishing.</figcaption></figure>

centermost region of the focus. Therefore, the extraction of nonlinear signals results in effective reduction of the point spread function, and thereby super-resolution imaging.

In the case of sub-linearity, saturation excitation (SAX) microscopy achieves super-resolution via extracting nonlinear scattering by temporal modulation of excitation plus harmonic demodulation of emission (Figure 33a–c for plasmonic and Figure 34a–c for dielectric NSs).[532, 70] A more straightforward super-resolution approach is through super-linearity,[533] which enables direct observation of the resolution enhancement (Figure 33d–f).[534] Combining sub-linear and super-linear effects, along with differential signal processing, the single-NS image width can be further reduced to below 50 nm (Figure 34d–f).[528]

It is well known that all-optical switch of emission enables resolution enhancement. The most famous example is stimulated emission depletion (STED),[488] which relies on switch on/off fluorescence. We pioneered in demonstrating all-optical switch with scattering from both plasmonic and dielectric NSs.[529, 70] That is, scattering intensity decreases as increasing the illumination intensity. Borrowing the setup from STED microscopy, where peripheral scattering is suppressed by a donut beam, we succeed

in resolving two adjacent plasmonic scatterers by suppression of scattering images (SUSI) (Figure 33g–i).[529] Ouyang et al. extended this concept, introducing saturation scattering competition (SCC), where excitation and the depletion beams are at the same wavelength to prevent the chromatic aberration.[535] Very recently, the concept of SUSI is demonstrated in dielectric NSs via manipulating the polarization state of illumination, as shown in Figure 34g–i.[536]

## 19.2. Current and Future Challenges

We have shown that scattering, from plasmonic or dielectric nanoparticles, exhibits a significant nonlinear response due to the interplay of Mie resonance and photo-thermo-optic effects. The nonlinear Mie scattering provides a novel contrast for super-resolution imaging, via the technical approaches of SAX, super-linear microscopy, SUSI, etc. However, the general purpose of super-resolution microscopy should be the investigation of “unknown” structures, but in current status, most experiments adopt plasmonic and dielectric NSs with well-defined morphologies (size, shape, etc.). Typically, only one or two exci-
tation wavelengths are selected to meet the NS resonance peaks, and to optimize the nonlinear efficiency. Therefore, one major challenge is to develop a new scattering-based nonlinear super-resolution technique that is able to examine unidentified NSs.

A common goal of optical super-resolution techniques is to push toward the resolution of electron microscopy. The state-of-the-art fluorescence nanoscopy has now enabled ≈1-nm spatial resolution in 3D[537]! It is critical to borrow experiences from fluorescence imaging to boost the spatial resolution of scattering-based microscopy. In principle, the resolution of nonlinear imaging techniques such as SAX and SUSI is unlimited, depending on how strong is the nonlinear response. Thus, one major challenge is to substantially improve the nonlinear light-matter interactions in Mie resonators.

From Figures 33 and 34, the best resolution of nonlinear scattering imaging is about 40-nm, which is much smaller than the NS diameter itself. An interesting question is to understand the underlying mechanism of the coherent scattering super-resolution image formation. For instance, the ultrafast temporal dynamics of the nonlinear Mie-scattering has not yet been thoroughly scrutinized.[538]

## 19.3. Advances in Science and Technology to Meet Challenges

In order to generate nonlinear scattering in NSs with various morphologies and resonance wavelengths, one plausible strategy is to adopt a broadband light source such as a super-continuum laser. Please note that most of the results in Figures 33 and 34 are carried out with continuous-wave lasers, while commercially available fiber-based super-continuum laser in general has a nanosecond pulse width, and custom-made super-continuum may have pulse width as short as a few femtoseconds. Therefore, on the one hand, combining super-continuum with a wavelength-selection tool and x-scan confocal microscopy may provide the desirable capability of generating nonlinear scattering in many different NSs, thus enabling the observation of “unknown” structures. On the other hand, the nanosecond/femtosecond pulse nature of the super-continuum laser offers a playground to explore the nonlinear scattering effect in diverse time scales. In particular, the nanosecond pulse may be the most efficient in transient heating of the NS, while the femtosecond pulse shall be preferred to study the carrier dynamics by pump-probe spectroscopy.

The second challenge is to enhance spatial resolution of scattering images, via the approach of enhancing optical nonlinearity. It is well known that nonlinear light-matter interactions are significantly amplified in a high-Q cavity. In most results of Figures 33 and 34, the Q-factor of Mie resonance is merely on the order of 10. Manipulating the interaction between multipoles of different orders can lead to exotic optical phenomena such as nonradiating anapole,[539] supercavity mode, and quasi-bound states in the continuum,[540] pushing Q-factor of single-particle Mie resonance toward 10³, and that of a metasurfaces as high as 10⁶! The unprecedented high-Q resonance may lead to significant enhancement in the image precision. Under a tightly focused scenario of microscopy, an emerging trend is to explore spatially dependent mode excitation,[541] and multipole interference to enhance local field as well as nonlinearity.

## 19.4. Concluding Remarks

Through the investigations over the past decade, giant photothermal nonlinearities on scattering has been unraveled, in both plasmonic and dielectric nanoparticles. Various nonlinear responses are discovered, and applied to resolution enhancement on the nanoparticles. Looking back to the title of this roadmap article “Label-free Super-resolution Imaging”, the readers may feel confused that these nanoparticles shall serve as extrinsic “labels” when applying to biological specimens. However, optical microscopy is useful not only for biology, but also for material sciences. We would like to advocate the development of nonlinear Rayleigh scattering microscopy as a novel contrast mechanism to inspect metal and semiconductor nanostructures. In addition, the nonlinear light-matter interaction may readily extend from Rayleigh scattering, into absorption or other forms of emission, to form an array of super-resolution imaging contrast tools.

# 20. Label-Free Super-Resolution Microscopy by Non-Linear Photomodulated Reflectivity

*(Omer Tzang, Dror Hershkovitz, Ori Cheshnovsky*)

## 20.1. Status

The urge to explore the properties, structure, dynamics, and functionality at the nanoscale has inspired the onset of ultrahigh-resolution imaging techniques. In particular, super resolution (SR) fluorescence-based microscopy, with resolution down to few nanometers is very useful in biological research where functional groups can be selectively labeled. However, in many cases, in material inspection or imaging of unprocessed biological tissues, far-field, label-free, imaging modality is desirable. Near field Scanning Optical microscopy does provide label-free SR by coupling light to proximate nanostructures, however it is confined to a range of few tens of nanometers off the surface. Recently, far-field label-free SR methods, based on ground state depletion[66] and Photo-acoustics[542] were introduced. Our methodology, Non-linear Photo-Modulated Reflectivity (NPMR) is based on the nonlinear changes in the reflectance of materials, ΔR, caused by an ultra-short pump pulse.[543]

In NPMR, a modulated train of ultra-fast pump pulses focused on the sample, photo-induces temperature and/or charge-carriers density changes, within the diffraction limited spot. The material undergoes fast exchange dynamics of between various degrees of freedom. Electronic excitation (10–500 fs) is followed by carrier-carrier and carrier-phonon scattering/thermalization processes (500 fs–10 ps),[544] leading to changes in the dielectric function. A spatially overlapping, delayed, constant-intensity train of probe pulses monitors the induced nonlinear ΔR by the detection of the high harmonics changes in the probe reflectance. The idea is illustrated in **Figure 35**. A comprehensive account on NPMR is found in a review chapter.[545]

The nonlinear components of reflection due to photomodulation (at $\omega_m$) are demodulated at the corresponding harmonic frequencies ($\omega_m, 2\omega_m, 3\omega_m, ...$) in a lock-in amplifier. The nth harmonics components of the reflectivity scale with the
<figure><img src="image_38.png" /><figcaption>**Figure 35.** (Left) The essentials of NPMR. A train of intensity modulated pump pulses (red) is focused on a sample. Focused and position overlapping train of probe pulses (green), with constant intensity, are delayed by a few ps. The increase of the sample temperature, induced by the pump pulses give rise to changes in the probe reflectivity (blue line). The black dots represent the changes in the probe-reflectivity. The harmonics analysis of changes in reflectivity with a lock-in amplifier, provide the nonlinearity of the response. (Right) the improvement in resolution of at test sample increasing with the harmonics order. Reproduced with permission.[466] Copyright 2015 American Chemical Society.</figcaption></figure>

nth power of the excitation. As a result, the effective PSF<sub>pp</sub> comprises of the product of PSF<sub>probe</sub> and that of the pump laser to the power of the nonlinearity order n, PSF<sub>pump</sub><sup>n</sup>. In all dimensions, the PSF<sub>pp</sub> scales down by √n for Gaussian focused beams. By recording the NPMR while scanning the sample, super-spatial resolution is achieved. A pure sinusoidal modulation of the pump beam is needed to warrant that the weak nonlinear response originates solely from the physical response of the system. This purity is achieved by driving an Acousto-Optic Modulator (or other modulators) by an Arbitrary Waveform Generator, tuned to generate a pure sinusoidal light intensity modulation.[546] The short (≈100s) optimization process, reduces the harmonic distortion down to 10<sup>-4</sup>.

How about the optimal time scales? A short ≈1 ps pump pulse, is desirable to prevent the photo-excited spatial distribution of induced properties from diffusing and blurring the initial excitation profile. The time delay between pulses of the pump and the probe (P&P) should be optimized to match the relevant physical property (0–2 ps). The repetition rate of the P&P should allow

for a thorough electronic and thermal relaxation between subsequent pairs of laser pulses. A schematic view of the experimental system is presented in **Figure 36**.

SR using NPMR was first demonstrated on Si nanostructures,[543] and showed the reduction the PSF down to 85nm (Figure 35 left). It was further applied to plasmonic structures of gold and V<sub>2</sub>O<sub>3</sub>. By its nature, NPMR excels in characterizing semiconductors and metals in vacuum, ambient and liquid, semi-transparent, and opaque systems. An equivalent, fluorescence-dependent SR technique, is SAX.[466] There, harmonics of the fluorescence, caused by saturating the modulated excitation, are detected.

## 20.2. Current and Future Challenges

One significant difficulty in a routine use of P&P technique is the need to spatially co-focus and fine-tune the delays of the P&P pulses, and eliminates the need for these adjustments.[547] In Single laser NPMR, the train of laser pulses is intensity modulated,

<figure><img src="image_39.png" /><figcaption>**Figure 36.** A schematic view of the experimental system. The role of the galvo mirror is to enable the combination of SPOM and NMPR.</figcaption></figure>
<figure><img src="image_40.png" /><figcaption>**Figure 37.** (Left) Super resolution Photo-modulated reflectivity using single color or two colors. The sample consists of Au double lines, 125 nm wide, with gaps of 370, 270, and 180 nm, respectively. a) Line imaging of probe reflection (red) and pump reflection (purple). b) Two-color photo-modulated reflectivity line imaging using first (blue), and second (black) harmonics. c) Single color photo-modulated reflectivity using first (orange), and second (black) harmonics. Note the resolution enhancement (95 ±5 nm) at $2\omega_1$ in both two and single-color modalities. Reproduced with permission.[548] Copyright 2012, OSA Publishing. (Right) The effect of apodization: Line scan of SR target using clear beam (blue) and apodized beam (brown).</figcaption></figure>

and its reflectance is measured at higher harmonics. The pump pulse excites the sample and the changes in its reflectivity are measured simultaneously. The measured $\Delta R$ is a weighted average, of changed in reflectivity induced in early stages of the same pulse. This approach is lacking optimized P&P delay or optimized probe wavelength, and thus reduces the signal substantially. It is also highly dependent on the timescales of photo-excited dynamics in the examined samples. However, due to its simplicity, this modality may be beneficial to many applications, as depicted in **Figure 37** (left). A set of 100 nm thick, 125 nm wide pairs of gold stripes fabricated on Sapphire substrate, shows an equal resolution of 95 ± 5 nm for the second harmonics of both single laser and P&P. We inferred that the reflectivity changes due to electron excitation in Au are fast enough (approximately 100 fs) to be detected by a single 2 ps pulse. In Silicon sample, $\Delta R$ evolves slower (≈700 fs) and is not probed efficiently by NMPR with a single 2 ps pulse.

In NMPR, the signal of high orders, n, degrades exponentially, while SR evolves as the $\sqrt{n}$. Therefore, we combined NMPR with other, efficient resolution enhancement modalities:

### 20.2.1. Spatial overlap modulation Nonlinear optical Microscopy (SPOM-NOM)

The enhancement of resolution in SPOM nonlinear optical microscopy (SPOM-NOM), is achieved by modulating the spatial overlap between two color pulses, and monitoring the nonlinear optical processes excited by a combination of two-color pulses.

SPOM-NOM was used for resolution enhancement in sum frequency imaging[548] or stimulated Raman scattering.[549] However, the point spread function of SPOM, $PSF_{SPOM}$, representing the second derivative of the diffraction limited PSF, suffers from artifacts resulting from its negative lobes.

The combination of NPMR with SPOM, while benefiting from improved resolution, dramatically reduces the negative lobes.[547] The combined PSF consists of the product of $PSF_{pump}^n$ and $PSF_{SPOM}$. The negative lobes of $PSF_{SPOM}$ overlap with low intensities in $PSF_{pump}^n$. In our studies, the probe was spatially modulated at $\omega_{SPAT}$ using a mirrored galvanometer, while the intensity of the pump was modulated at $\omega_m$ using the AOM. The co-demodulated signal at $\omega_{ref} = n\omega_m + 2\omega_{SPAT}$ combines the integrated response of SPOM (second order) and NPMR. The incorporation of SPOM improved the resolution by ≈15%, down to 85 ±5 nm, in accordance with our simulation. It replaced additional 1–2 higher harmonics in NPMR, yet provided about tenfold higher signal. As simulated, the intrinsic negative lobes of the PSF of SPOM are significantly reduced due to the slimmer PSF of NPMR.

### 20.2.2. Resolution Improvement via Beam Shaping

So far, we based resolution enhancement on Gaussian diffraction-limited spots. However, by employing a phase and/or amplitude filter in the beam path, we can engineer a smaller central spot at the focal plane. The annular tails in the PSF, generated by such filters, are less pronounced in high orders of the $PSF_{pump}^n$, and are practically nulled in NPMR due
to lack of spatial overlap with a probe Gaussian beam. Spatial filters have demonstrated substantial improvement in resolution in confocal setups.[550] To prove the principle in NPMR, we used an annular gold amplitude mask.[551] to obtain a symmetric donut-shaped beam. Figure 37 (right) shows the effect of the beam apodization. In two scans we tested SR by measuring the second order NPMR, using a single color, linearly polarized, 400nm laser and a 0.95NA air objective. The resulting resolution was 155nm and 130 nm for the clear and the apodised beams respectively. This ≈16% reduction in the PSF width, matches the theoretical prediction. We emphasize that sophisticated beam shaping such as Bessel beams or super oscillation beams should further improve the resolution.

## 20.3. Advances in Science and Technology to Meet Challenges

Raman microscopy, with its material-specific spectral features, is a label-free counterpart to fluorescence microscopy. While Raman tip enhancement methods,[552] interleaved with scanning probe microscopy, provide rich information with nanometric resolution, they are confined to surface studies. Fujita and coworkers introduced a far-field combination of structured illumination and micro-Raman microscopy to achieve SR. We used photomodulated nonlinear Raman scattering to achieve SR.[491] However, this approach proved impractical, due to its very low signals. The progress in resonance-enhanced time-gated Raman spectroscopy, with cross sections orders of magnitude higher, may resolve the problem and enable label-free SR with chemical recognition. In this modality only the interfering fluorescence is suppressed by time-gating the Raman signal to overlap with a sub-nanosecond excitation pulse.[553]

## 20.4. Conclusions

We have presented the essentials of NPMR for far-field, label-free, SR microscopy. Unlike some other modalities, in NPMR sample treatment or preparation is not required. NPMR is suitable to characterize variety of materials: at any wavelength that materials absorb, semiconductors, and metals, in vacuum or ambient, in liquids or solids.

While the signal of high orders, n, degrades exponentially, the incremental improvement in SR evolves as the $\sqrt{n}$. In order to enhance resolution it is practical to combine NPMR with other resolution enhancement modalities, as discussed. In any SR technique the sensitivity strongly declines with the size of the examined objects. One way to partially cope with this problem is to exploit resonance enhancement in NPMR. E.G. tuning the Probe wavelengths near to a plasmonic resonant wavelength, can enhance the sensitivity of measurements for specific sizes and materials.

# 21. Structured Illumination-Based Nonlinear Label-Free Super-Resolution Microscopy

*(Mikko J. Huttunen*, Stefan G. Stanciu)*

<figure><img src="image_41.png" /><figcaption>**Figure 38.** Schematic for nonlinear structured illumination-based super-resolution microscopy. By spatially modulating the incident/excitation fields, one can detect higher spatial frequencies associated with the sample. After reconstruction, an image with improved resolution can be formed.</figcaption></figure>

## 21.1. Status

When objects are imaged using ultra-fast light sources with high peak-power, the occurring light-matter interaction may become nonlinear giving rise to novel contrast mechanisms that enable the imaging of various samples based on their intrinsic properties (Figure 38).[554] The main advantages of nonlinear optical microscopy, also known as multiphoton microscopy (MPM) are that I) the nonlinear processes provide novel, label-free contrast mechanisms, and II) it becomes easier to improve penetration depth by using longer wavelength excitation. Over the past years, an important focus of attention has been placed on the development of nonlinear super-resolution techniques, which are likely to lead to many interesting bioimaging applications for label-free nonlinear optics (NLO).[472]

Among the most widely used nonlinear microscopy techniques are those based on second-harmonic generation (SHG), third-harmonic generation (THG),[555] and coherent anti-Stokes Raman scattering (CARS).[556] The main advantage of these nonlinear optical processes is that they occur intrinsically in samples providing means to perform label-free imaging. The SHG signals originate from non-centrosymmetric molecules, with collagen, the most abundant protein in the human body, and the main constituent of the extracellular matrix, being one of them.[557] THG signals mainly originate from interfaces where sharp spatial variations of the refractive indices occur, enabling the imaging of lipid droplets and cellular membranes, among others.[558] The most flexible processes in terms of achievable contrast in NLO microscopy are coherent Raman scattering processes, such as CARS or stimulated Raman scattering (SRS), that provide true chemical contrast by allowing specific imaging of molecules whose chemical bond vibrational frequencies match with the difference of a pair of pump and probe beams.[559]

In addition to above, two-photon excitation fluorescence (2PEF) and three-photon excitation fluorescence (3PEF) processes, also largely popular in the context of NLO microscopy, can intrinsically take place in samples, provided that the
<figure><img src="image_42.png" /><figcaption>**Figure 39.** Label-free tissue imaging with non-linear optical microscopy modalities. a) TPEF+SHG and b) H&E images of murine colon. The TPEF+SHG image (green:TPEF, red:SHG) was collected ex vivo through an endomicroscope objective. Adapted under the terms of the CC-BY license.[575] Copyright 2019, The Authors, published by Wiley-VCH GmbH. c) THG+SHG (green:THG, red:SHG) and d) H&E images of human brain affected by glioblastoma, showing hypercellularity, vascular proliferative changes, and aggregation of macrophages (arrows). Adapted under the terms of the CC-BY license.[555] Copyright 2019, The Authors, published by Wiley-VCH GmbH. e) SRS+SHG (green: lipids SRS@2845cm⁻¹, blue: proteins SRS@2930cm⁻¹, red: collagen SHG) and f) H&E images of laryngeal squamous cell carcinoma tissues. Adapted with permission.[559] Copyright 2019, Ivyspring International Publisher. g) CARS+TPEF+SHG (green:TPEF, red:CARS, blue:SHG), h) H&E and i) virtual H&E images of human colon. The virtual H&E image was computed from the NLO image using Deep Learning. Adapted with permission.[570] Copyright 2021, The Authors, published by Optica Publishing Group.</figcaption></figure>

respective multiple of the pump wavelength coincides with some electronic transition present in the sample.[472] Most commonly, nonlinear autofluorescence is utilized to image reduced nicotinamide adenine dinucleotide phosphates[NAD(P)H] and flavin adenine dinucleotides (FAD).[560] Particularly the ratio of these signals, known as the redox ratio, is commonly utilized to investigate metabolic cell activity, that can be linked to severe diseases/disorders such as to carcinogenesis.

The above-discussed nonlinear processes have already enabled many applications (Figure 39).[472] SHG microscopy is now commonly utilized to characterize collagen morphology of tissues, e.g. providing valuable information of fibrosis, wound healing and carcinogenesis.[561, 562] Coherent multiphoton processes (2/3PEF) excel in calcium imaging, due to which combined 2/3PEF and THG microscopy are often used for in-vivo imaging of the brain.[563] In addition to the above applications, coherent Raman scattering techniques, particularly CARS microscopy, have been found to be useful e.g. in time-resolved analysis of lipid droplet growth, assembly and composition.[556] As shown in several landmark works, coherent Raman modalities are highly complementary to NLO techniques based on autofluorescence and harmonic generation (Figure 39).

## 21.2. Current and Future Challenges

The most obvious challenges to address in label-free nonlinear imaging are *imaging speed, penetration depth*, and achievable *resolution*. The latter challenge can be alleviated to a degree by utilizing structured-illumination-based super-resolution techniques,[72, 564-566] such as image scanning microscopy (ISM) or rescan microscopy.[73, 567, 568] The achievable resolution improvement depends on the order of the nonlinear process and of the technique used, but in essence ≈100 nm resolution is already achievable for some NLO modalities.[569] A major challenge is to develop techniques that could further improve the achievable lateral resolution to the range of values (<20 nm) now already available in super-resolved fluorescence microscopies based on contrast-agents. However, in terms of current and future challenges, lateral resolution is only a single figure of merit that limits the emergence of novel applications. Many of the nonlinear structured-illumination microscopy techniques are based on widefield configuration resulting in somewhat poor axial resolution.

In addition to axial resolution, in some imaging applications, e.g. intravital imaging, the penetration depth is as well an
important parameter. Nonlinear microscopy modalities in general are advantageous in this respect because absorption and scattering of incident light can be considerably decreased when moving to longer pump wavelengths.[571] However, it has turned out to be very challenging to combine high (≈100 nm) spatial resolution with long (>100 µm) penetration depths.[569] Furthermore, even when compromises in resolution can be accommodated, imaging deep tissues (e.g., <1 mm) is still a major problem. Improvements in deep tissue imaging with high resolution can be expected to take place by advances in machine learning and artificial intelligence-based methods.

The demands of imaging are also shifting towards large-area and volumetric imaging.[572, 573] For example in histopathologic applications, whole slide imaging using more conventional imaging modalities, that have been combined with automated image processing approaches based on artificial intelligence are quickly becoming commonplace.[574] It can be expected that similarly also MPM modalities will be developed to stages, where large-area imaging becomes a default option. In addition to whole slide imaging, an upcoming trend in histopathology seems to be the shift to volumetric imaging, which has many advantages over traditional approaches that rely on imaging microns thick sample sections. In short, never-ending need for speed exists.

A final challenge that many of the nonlinear super-resolution microscopy techniques face is that the achievable contrast mechanisms in label-free imaging are somewhat restricted subsequently limiting the potential imaging applications. Fortunately, MPM modalities have already found many niche applications. However, the limited contrast admittedly restricts to some extent the applicability of developed techniques.

## 21.3. Advances in Science and Technology to Meet Challenges

With current knowledge, it seems likely that the lateral resolution cannot be improved much below the ≈100 nm limit, unless new ideas to circumvent the diffraction limit emerge. However, many of the current imaging applications might not benefit dramatically from improved *lateral* resolution. Instead, improvements in the *axial* resolution are highly desirable. This challenge could be remedied for example by combining structured-illumination microscopy with the temporal focusing scheme,[569] or by using laser-scanning structured illumination microscopy (SIM) or nonlinear image scanning microscopy (ISM).[568]

The limitations in the achievable penetration depth are another major existing challenge in bioimaging applications. A straightforward, although technologically non-trivial solution is to continue nudging the excitation and emission wavelengths towards longer wavelengths inside the second near-infrared region (NIR-II, 1000–1700 nm),[571] where water absorption is however an important challenge to address, together with losses in resolution. Here, further development of commercially available and affordable ultrafast laser systems seems necessary, facilitating new research groups and imaging facilities to acquire/build NIR-II capable nonlinear imaging systems.

Enhancing the penetration depth is also extremely relevant to current NLO endomicroscopy efforts, which suggest that we

are not far from a next generation of NLO tomographs capable of imaging not only easily accessible body parts, such as skin, but also other parts of the human body that can benefit of NLO contrast and resolving power, such as the gastrointestinal tract (Figure 39a).[575] However, additional progress in miniaturization is obviously required in order to probe internal organs, such as the heart, kidney, lungs, liver, etc., that have been thoroughly discussed in the context of ex-vivo NLO imaging. Furthermore, in-vivo imaging is particularly dependent on imaging speed, to be discussed below. This is because internal organs have their inherent movement due to blood circulation, which yields motion artifacts. Avoiding these issues requires high-speed scanning, which poses additional complications compared to enhancing imaging speeds of bench-top systems.

Steady improvements in the imaging speeds will likely take place by further development of imaging systems. The most straightforward approach is to move away from single-point scanning modalities towards multi-point, line-scanning or even widefield modalities.[572, 573] Furthermore, widefield techniques could be made considerably faster by more widespread use of low/medium repetition rate amplified laser systems, that would allow reaching ≈100 GW cm⁻² illumination peak intensities while still keeping mean powers compatible with live imaging.[573]

The final challenge mentioned above concerns the limited contrast mechanisms that arise in label-free imaging applications. Most of nonlinear imaging modalities (SHG, THG, 2PEF, and 3PEF) provide contrast only towards quite specific cell and tissue types. However, smart control of the polarization state of the excitation and/or emission beams might provide new possibilities, particularly when coherent SHG and THG modalities are used.[576] In the longer run, further development of coherent Raman scattering techniques, such as CARS and SRS, seems however a more probable remedy for the limited contrast.[556] In addition, the challenges associated with limited contrast and subsequent interpretation of the NLO images are already being alleviated by the emergence of artificial intelligence methods for digital staining, e.g., based on generative adversarial networks.[570] These allow the translation of raw NLO images, which can be cumbersome to interpret even by experts, into versions that closely resemble traditional histopathology images, and whose content is therefore highly accessible (Figure 39g-i). Furthermore, artificial intelligence has also been demonstrated as having interesting potential for NLO based automated histopathology.[559, 562]

## 21.4. Conclusions

The field of label-free non-linear optical microscopies has witnessed important progress over the past years, and a wide palette of applications has been reported to date. Advances will take place with further technology development. Particularly promising are the recent activities where artificial intelligence and computational imaging approaches are combined with state-of-the-art imaging techniques. One can envisage that soon non-linear label-free large-scale imaging techniques become standard techniques that e.g., facilitate human patient diagnosis both ex-vivo and in-vivo.
# Theme IV: Advanced Superlens Designs

## 22. Surface Electromagnetic Wave Lenses and Transformation Optics

(Vera N. Smolyaninova, Igor I. Smolyaninov*)

### 22.1. Introduction

Starting from early 2000s several novel and important research directions have developed in label-free super-resolution optical microscopy,[577] which are quickly demolishing the resolution limit of conventional optical microscopy due to diffraction. The first one is making use of nonlinear optics.[287] In general, the nonlinear optical microscopy techniques implement photo-switching of and saturation of fluorescence down to single molecule level, so that spatial resolution of the order of 20 nm is achieved. However, these techniques rely on spatial scanning of either samples or light sources, which is a relatively slow mechanical process. During approximately the same time considerable progress in transformation optics, plasmonics,[578] and metamaterials[122] enabled drastic improvement in spatial resolution of conventional linear far-field optical microscopy. First, John Pendry realized that negative refractive index metamaterials may be made into a flat “perfect lens,” so that high-resolution (almost perfect) optical images could be obtained due to negative refraction. Such a perfect image is formed by the amplified evanescent waves (or surface plasmons) which exist at the boundaries separating the positive and negative index materials. Recent examples of the experimental realization of this approach may be found in.[579] However, the magnification of a planar “perfect lens” is limited to 1. A major step to bypass this limitation was achieved by the development of surface plasmon-assisted microscopy,[124] in which image magnification in two spatial dimensions has been demonstrated in the geometric optics limit. Soon it was realized that the improved spatial resolution of 2D plasmonic microscopy relies on the “hyperbolic” dispersion law of surface plasmons, which may be expressed as

$$k_{xy}^2 - |k_z|^2 = \frac{\epsilon_d \omega^2}{c^2} \qquad (23)$$

where $\epsilon_d$ is the electric permittivity of the metal film substrate, $k_{xy} = k_p$ is the component of the plasmon wave vector along the propagation direction, and $k_z$ is the wave vector component (exponential decay constant) in the perpendicular direction. This approach was further developed in the “optical hyperlens” designs by Narimanov et al.[126] and Engheta et al.,[127] which make use of a concentric arrangement of multiple thin metal and dielectric cylinders, which form a hyperbolic optical metamaterial. Such a metamaterial arrangement results in a strongly anisotropic tensor of dielectric permittivity, in which the radial $\epsilon_r$ and the tangential $\epsilon_\theta$ tensor components have opposite signs. The dispersion relation of such a metamaterial

$$\frac{k_r^2}{\epsilon_\theta} - \frac{k_\theta^2}{|\epsilon_r|} = \frac{\omega^2}{c^2} \qquad (24)$$

does not exhibit any diffraction limit. Light propagates through such a hyperbolic metamaterial in the form of well-defined radial

<figure><img src="image_43.png" /><figcaption>Figure 40. When a glycerin microdroplet is illuminated near its edge with a tapered fiber tip, an image of the tip may be seen at the opposite edge of the microdroplet. Adapted with permission.[131] Copyright 2010, The Optical Society.</figcaption></figure>

rays. If a radiating sample is placed near the inner rim of such a hyperlens structure, the lateral separation of rays radiated by the sample increases towards the outer rim, thus providing image magnification. Such magnifying hyperlenses were realized in both 2D and 3D configurations.[212, 580, 581] Far-field optical resolution of at least 70 nm has been achieved using a 2D plasmonic hyperlens. The goal of this paper is to outline further promising directions in 2D microscopy, which would potentially move the resolution scale of linear far-field optical microscopy toward the 10 nm scale and beyond. We anticipate that this may be accomplished by using the newly discovered class of surface electromagnetic waves which propagate along gradual interfaces of lossy optical media.[582, 583]

### 22.2. Two-Dimensional Transformation Optics

As mentioned above, a plasmonic microscope used in the 2D geometric optics mode may exhibit considerably increased spatial resolution compared to the conventional three-dimensional microscopes. This is an expected result, since operation of plasmonic microscopes relies on the hyperbolic dispersion law—see Equation (23). In an additional recent development, we also showed that such transformation optics (TO) devices as various invisibility cloaks, which require quite sophisticated spatial profiles of anisotropic electric permittivity and magnetic permeability, may be emulated by microfabricated tapered waveguides and micro-droplets.[131] This interesting fabrication shortcut results in broadband low-loss performance of the TO devices in the visible frequency range, which is very difficult to achieve by conventional means. Moreover, the same approach was successfully applied to fabrication of the inverted Eaton and Maxwell fisheye microlenses,[131] which were recently introduced as superb imaging devices, which do not require negative refraction.[130] For example, a refractive index distribution which needs to be emulated in a microdroplet or a microlens so that a resulting 2D device would act as an inverted Eaton lens is described by the following equation:

$$n = 1 \text{forr} < R, \text{ and } n = \sqrt{\frac{2R}{r} - 1} \qquad (25)$$

This refractive index distribution may be emulated by varying the microdroplet thickness, as described in.[131] An example of such a microdroplet-based Eaton lens is shown in Figure 40.

As illustrated in Figure 41, two segments of the Eaton lens which have different magnitudes of parameter R can be
<figure><img src="image_44.png" /><figcaption>**Figure 41.** Numerical simulations of image magnification ($M = R_1/R_2 = 2$) using a compound inverted Eaton lens. Adapted with permission.[131] Copyright 2010, The Optical Society.</figcaption></figure>

combined in order to achieve image magnification. In such a configuration the image magnification equals $M = R_1/R_2$, while the image resolution is defined by the wavelength of the optical mode (regular light or surface plasmon) which is used in the imaging device. We should also note that near-field image magnification has been proposed in recent experiments with high-index liquid-immersed microspheres.[584]

## 22.3. Towards Nanometer-Scale Resolution using Novel Surface Wave Modes

The familiar examples of surface electromagnetic waves include surface plasmons, which may exist at sharp interfaces between metals and dielectrics,[578] and Zenneck waves, which may exist at surfaces separating highly lossy conductive media and lossless dielectrics. However, consideration of a “more physical” situation in which the electric permittivity (or conductivity) of a material changes continuously across some gradual interface, leads to appearance of a new class of deeply subwavelength surface waves.[582, 583] It appears that low-loss surface electromagnetic waves may exist at planar gradual interfaces between lossy media, even if the dielectric losses are very high on both sides of the interface. This very recent and highly unexpected result seems to be applicable to most any electromagnetic frequency range from the very low frequencies (VLF) of radio waves all the way up to the UV light. The TM wave equation in such cases takes the form of a one-dimensional Schrodinger equation

$$ -\frac{\partial^2 \Psi}{\partial z^2} + \left( -\frac{\epsilon(z)\omega^2}{c^2} - \frac{1}{2}\frac{\partial^2 \epsilon}{\partial z^2} + \frac{3(\partial\epsilon/\partial z)^2}{\epsilon^2} \right) \Psi = -\frac{\partial^2 \Psi}{\partial z^2} + V\Psi = -k^2\Psi \quad (26) $$

where the wave function is introduced as $E_2 = \psi/\epsilon^{1/2}$. Note that for the TM wave the effective potential energy $V(z)$ near the interface between highly lossy media may become almost purely real even if “$\epsilon(z) \gg \epsilon'(z)$”. This is happening if the gradient terms dominate in Equation (26). It appears that such materials as graphite and silicon (which have almost purely imaginary $\epsilon$ in the UV range) appear to be highly suitable for UV nanophotonics applications. As illustrated in Figure 42, the typical wavelengths of these newly discovered surface waves appear to fall into a few nanometers range, while propagation length of these waves

<figure><img src="image_45.png" /><figcaption>**Figure 42.** Numerical modeling of excitation and scattering of surface electromagnetic waves in a gradient waveguide made of doped graphite at $\lambda_0 = 275$ nm. The guided UV field propagating through the waveguide is scattered by a 4 nm diameter metal nanowire which is located near the waveguide.</figcaption></figure>

may be much larger than the skin depth in a homogeneous lossy medium.[212] As a result, these novel surface waves and the 2D microscopy arrangements described above appear to be ideal to further push resolution of the far-field linear optical microscopy techniques down to nanometer scale.

## 22.4. Conclusions

We have described several examples of two-dimensional super-resolution microscopy devices, which may be nanofabricated and studied using currently available modern fabrication techniques. These devices achieve 70 nm spatial resolution with conventional surface plasmons, which far exceeds resolution of conventional optical microscopy. Utilization of the newly discovered class of surface electromagnetic waves which propagate along gradual interfaces of lossy optical media should further bring the spatial resolution of 2D microscopes down to 10 nm scale and beyond. While the material base of nanophotonic devices using surface plasmons is limited to a few low-loss materials such as good metals and some oxides, the novel class of surface waves is supported by a much larger number of highly lossy media. In particular, such lossy materials as graphite and silicon may be ideal materials to use in UV nanophotonics applications.

## 23. Perfect Imaging

(Ulf Leonhardt*, Sahar Sahebdivan*)

Richard Feynman[585] questioned the diffraction limit of imaging, for a simple physics reason: the propagation of light is reversible. The equations that govern the light propagation, Maxwell's equations, are time-reversible. Changing the sign of the magnetic-field components in an electromagnetic wave would send the wave back to where it came from. If it was emitted from a point source it would, upon time reversal, focus with point-like precision. One may object that sending light back to its
<figure><img src="image_46.png" /><figcaption>**Figure 43.** Wave propagation on the surface of a sphere. The wave is emitted from a point source at some point on the surface. While propagating, the wave initially expands but then focuses on the point antipodal to the source. If the wave gets absorbed at the image the focus has point-like precision as the peaks show.</figcaption></figure>

source does not qualify as imaging, as the source and the image are the same. But, manipulating both the electric and the magnetic field, we may deflect the light and let it reverse to a different point, forming a distinct, perfect image there.

The diffraction limit is not a fundamental law of nature, but rather a feature of conventional lenses – that is Feynman’s argument.[⁵⁸⁵] Yet there is a counter-argument: for time reversal, it is not enough to reverse the field, the source must be reversed, too. Suppose we only reverse the field and then let it focus in empty space. An incoming spherical wave $(4\pi r)^{-1}e^{-ikr}$ with wavenumber $k$ produces the outgoing wave $-(4\pi r)^{-1}e^{ikr}$ where the sign describes the reflection of light. The interference between the incoming and the outgoing wave gives a focal spot proportional to sinc($kr$), which agrees with the diffraction limit! However, if we also reverse the source, no outgoing wave is there and the light is perfectly focused, as has been experimentally demonstrated.[⁵⁸⁶] Reversing the source is easy: instead of an emitter, we just need an absorber. The absorber takes out the incident wave and records the image. Even if only a small part of the incident light is absorbed, that part is perfectly focused. Perfect imaging is possible in principle.

Whether perfect imaging is possible in practice depends on the practicalities of time-reversing the field. For microwaves[¹²⁸] one can use an array of receivers, electronics to conjugate the field, and emitters to send it back. However, there is a way to do all this passively, with gradually varying index media. It makes use of the connection between dielectric media and geometries.[⁵⁸⁷] Imagine that light is confined to the surface of a sphere (Figure 43). Suppose a light wave is emitted from one point on the sphere, say the South Pole. The outgoing wave expands until it contracts and focuses on the North Pole, as if it got time-reversed while crossing the Equator. Obviously, this is also true for all other source points: any point on the sphere is perfectly imaged at its antipodal point (Figure 43). The curved space of the sphere performs the time-reversal required for perfect imaging.

<figure><img src="image_47.png" /><figcaption>**Figure 44.** Stereographic projection. In the stereographic projection, a line is drawn from the North Pole of the sphere to the point to be projected. Where this line intersects the plane through the Equator lies the projected point. The top picture shows the stereographic projection in a cut through the sphere. The lower picture shows the wave on the sphere (Figure 1) projected accordingly. The stereographic projection is a conformal map that preserves angles and just distorts distances. An isotropic material does the same. In particular, Maxwell's fish eye[⁹] acts like a sphere in stereographic projection.[⁸] As light can be perfectly focused on a sphere (Figure 43) it can be perfectly focused in Maxwell's fish eye,[¹³⁰] too, provided the image gets absorbed and does not interact with the source.[⁶⁰¹, ⁶⁰³]</figcaption></figure>

Curved geometries like the surface of the sphere affect the diffraction limit in another way as well: they modify the uncertainty relation between position and momentum.[⁵⁸⁸, ⁵⁸⁹] In the absence of sources and drains – and without curvature – the diffraction limit follows from the uncertainty relation.[⁵⁹⁰] So curved spaces will modify the resolution of imaging. However, this effect goes away in the limit of a large radius of curvature, whereas the sphere, how large it may be, would, in the presence of source and drain, time-reverse and perfectly focus light.

Now, curved spaces like the sphere can be implemented with dielectric media.[⁵⁸⁷] In the stereographic projection[⁵⁸⁷, ⁵⁹¹] (Figure 44) the surface of the sphere is projected onto the plane. Here a variable measure of length plays the role of the curved two-dimensional surface in three-dimensional space. The measure of length is the refractive index. For implementing a sphere in the $(x, y)$ plane in appropriate units we need the index profile[⁵⁹¹] $n = 2n_0/(1 + x^2 + y^2)$. Maxwell himself wrote down[⁵⁹²] such a profile and noticed that light rays go in circles in it and are perfectly focused. Much later it was shown[¹³⁰, ⁵⁹³] that not only rays but waves are perfectly focused, too. Maxwell's profile[⁵⁹²] is known as Maxwell's fish eye, because such gradually varying index media reminded Maxwell on the eyes of fish. It is the prototype of
absolute optical instruments[594] where from any given point in the source region all rays focus at one or several points in the imaging region.

Absolute optical instruments come with restrictions: they can-
not really magnify or, to be more precise, their magnification is
given by the ratio of refractive indices around the image and the
source. Furthermore, both the object and the image lie inside the
device like the focal points in an elliptical mirror, except that for
an absolute optical instrument all source and image points are
foci.

Absolute optical instruments capable of perfect imaging re-
quire a considerable index range. In the case of Maxwell's fish
eye, the device is infinitely extended and the index goes to zero at
infinity. Luckily, one can modify the fish eye[130] by surrounding it
with a mirror at the unit circle (in device units) without compro-
mising its functionality,[130] which makes the device finite and
confines the index range to {n₀,2n₀}. Achieving a factor of two
in index modulation is still difficult to implement by doping a
material, but it can be done with planar waveguides in silicon
photonics.[595] Here the thickness of a silicon layer on silica plays
the role of the refractive index; with variable thickness one can
implement refractive-index profiles like Maxwell's fish eye.[595]

Does perfect imaging work? The community reacted with con-
siderable skepticism[596] after the idea was proposed.[596] The
lively discussion in the literature has been very helpful in refin-
ing and clarifying the ideas, but the ultimate test of any physics
idea is the experiment. Two types of experiments[597-599] were per-
formed, both with microwave radiation. In the first,[597] the spher-
ical geometry (Figure 43) was implemented for microwaves. The
waves were confined between two closely spaced spherical mir-
rors such that they effectively propagated on the surface of a
sphere. Radiation was injected at one point by a cable and ex-
tracted by another cable movable around the image position. At
well-defined resonances, radiation was only transmitted when
the source and the detector were aligned with 1/105 wavelength
precision. This proves that perfect imaging works, but apparently
only at resonance frequencies.[600, 601]

In the second set of experiments,[598, 599] modified fish eyes
were made using microwave metamaterials. Two coaxial cables
inserted at the image points served as sources, and an array of
cables as detectors in the imaging area. The detectors appeared
to have resolved the two sources, despite them being consider-
ably closer to each other than half a wavelength. However, the re-
sults were not reproduceable in computer simulations. Further-
more, the figure of the main experimental results showed a sub-
tle inconsistency. The plotted intensities of the image points were
perfectly symmetric, although the sources were not. As was later
confirmed,[602] only half of the image data were plotted and the
other half were "not measured". According to Ref. [602] the im-
age values were "assumed to be symmetrical" while, at the same
time, the two sources were given different phases, making the
field naturally asymmetrical.

Why did the experiments in Refs[598, 599] not work? The prob-
lem was that the detectors were acting back onto the sources.[601]
Both the sources and the detectors were cables connected to a
microwave synthesizer. The sources were not just transmitting a
fixed amount of radiation, but they were also receiving radiation
reflected at the sources and they were reacting to it. In Feynman's
objection to the diffraction limit, there is a clear causal distinc-

tion between source and image. The source emits first, then the
radiation is focused by effective time reversal, and the image is
taken last. In the experiments in Refs[598, 599] the image acts back
onto the source as if the future would act back onto the past,
messing up the causal order between source and image. Only
at well-defined sharp resonances, like in the experiment on the
sphere,[597] no radiation is reflected and all radiation goes through
from the sources to the detectors with perfect resolution.[601]

Despite a few caveats,[601] one may still agree with Feynman[585]
that here is no fundamental reason why perfect imaging should
not work. The experiments[597-599, 602] and their analysis[600, 601, 603]
have highlighted a crucial requirement for achieving subwave-
length resolution with absolute optical instruments: the field re-
flected at the detectors must not interact with the sources. If this
is guaranteed then perfect resolution is perfectly possible.

24. Microsphere Superlens and Metamaterial Solid Immersion Lens

(Zengbo Wang*, Boris Luk'yanchuk, Limin Wu)

**24.1. Status**

In 2011, super-resolution imaging by microsphere superlens emerged as a simple yet effective method to overcome the diffraction limit that limits the resolution of conventional lenses.¹³⁶¹ Significant progress has since been made. Here, key advances including the development of scanning superlens system, metamaterial solid immersion lens (mSIL), super-resolution physics, and bio-superlens are discussed along with the challenges in this field. For more detailed review on the technique and other superlens applications in interferometry, endoscopy, and others, please refer to refs. [604, 605].

**24.2. Microsphere Nanoscopy**

24.2.1. Overview

The field of microsphere superlens research dates back to 2000 when it was discovered that a microsphere could generate subwavelength focus.[606] This effect became known as ‘photonic nanojet (PNJ)’ since 2004 and was widely used in laser cleaning, laser direct nano-writing and signal enhancement,[605] among other applications. The achievement of 80 nm resolution in laser patterning by microsphere[607] motivated the research on microsphere nanoscopy, first published in 2011.[136] As shown in **Figure 45a**, the technique uses microsphere as superlens to image the contacting nanoscale objects. The superlens collects and transforms the near-field evanescent waves, which carry the high-spatial-frequency information about the object, into the propagating waves reaching the far-field and leading to a formation of a magnified virtual image. The evanescent-to-propagating-conversion (ETPC) efficiency determines the final imaging resolution.[608] Further improvement of the resolution can be accomplished by enhancing the ETPC efficiency. These ideas motivated the development of the mSIL superlens
<figure><img src="image_48.png" /><figcaption>**Figure 45.** Microsphere nanoscopy. a) Contact mode setup and imaging examples (50–100 nm samples). Reproduced with permission.[136] Copyright 2011, Springer Nature. b) Resolution analysis by the PSF convolution method ($\lambda$ = 405, confocal mode). Reproduced with permission.[138] Copyright 2015, Wiley-VCH. c) Non-contact scanning superlens imaging setup and examples (microsphere attached to AFM tip, sample: 80–90 nm nanodevices, and fluorescent-labeled actin filament). Reproduced with permission.[612] Copyright 2016, Springer Nature. d) SR mode demo at size parameter $q$ = 26.94164 with peak value $\approx$43.5k and theoretical resolution $\approx \lambda/3 - \lambda/6$ for $n$ = 1.5, $q$ = 0–90 microspheres.</figcaption></figure>

discussed below, which provides improved ETPC efficiency with enhanced optical super-resolution and imaging quality.[144]

In contact mode, the microsphere superlens can resolve 50–100 nm scale objects (e.g., nanostructures and devices (Figure 45), subcellar structures and adenovirus[609]) using a wide-field microscope. Smaller features, i.e., 15–25 nm nanogaps, can be resolved with superlens under a confocal microscope.[138, 610] Since resolution of an imaging system is often characterized by the point spread function (PSF) instead of by the minimal resolvable feature sizes, Allen et. al. developed a convolution-based resolution analysis method and derived the best resolution for microsphere nanoscopy is $\approx \lambda/6 - \lambda/7$ (Figure 45b).[138] A higher estimation for resolution of $\lambda/8$ could be obtained if the final image's contrast is adjusted for clarity before convolution, giving calibrated resolution of $\approx \lambda/6 - \lambda/8$ for the technique. Such method is now widely used to calculate the PSF resolution for the superlens, which avoids exaggerated resolution claim beyond $\lambda/10$ based on the minimal resolvable feature sizes.

In non-contact mode, resolution of the superlens drops rapidly when particle-sample distance ($\Delta z$) increases, from $\lambda/7$ at $\Delta z$ = 0 (contacting) to $\lambda/3.8$ at $\Delta z \approx \lambda/2$ (half wavelength). Super-resolution typically degrades if the distance exceeds one wavelength. Extending the working distance (WD) of a superlens is a major challenge for this technique which is discussed later.

A variety of microspheres have been used as superlenses for imaging, including BaTiO₃ (BTG), Polystyrene (PS) and SiO₂ microspheres with typical size between 3 and 80 µm. For an optimum imaging, the optical contrast (OC, i.e., refractive index ratio

between microsphere and surrounding media) is recommended within 1.4–1.75.[136, 611] Therefore, for high-index microspheres such as BTG ($n$ = 1.9–2.1) and others, an immersion media (e.g., water or transparent resin) is often used to optimize the OC to maximize the performance.

### 24.2.2. Scanning Superlens

The ability to position microsphere superlens at desired location and scanning over an area are essential for practical applications. Single microsphere has a narrow field-of-view (FOV), scanning is utilized to expand FOV for imaging larger-area and for dynamic imaging. Several scanning schemes have been demonstrated, such as integration with AFM system[612] and encapsulation of microsphere in solid film.[138] Figure 45c shows the AFM-based scanning superlens system built by attaching a microsphere to an AFM tip and use precision motion system of AFM to control particle-sample distance and scanning across the sample surface. The system can work in both contact and non-contact scanning modes. A 96 × 96 µm² sized sample image was obtained in 3 mins with super-resolution at $\lambda/6.3$ level for 80–90 nm objects, which is about 200 times faster compared to the ordinary AFM. Another scanning approach is to bond the microsphere directly with the objective lens to form a unibody design,[613] which was used in commercial microsphere nanoscope developments.[614] The resolution of commercial systems is limited to $\approx$137–150 nm
<figure><img src="image_49.png" /><figcaption>**Figure 46.** Metamaterial solid immersion lens (mSIL). a) Concept of mSIL and synthesis approach. b) Near-field coupling between nanoparticles in mSIL transforms incident propagating wave into large-area structured evanescent wave illumination of substrate at FWHM resolution ≈8 nm. c1) Super-resolution imaging of 60 nm feature on IC chip by mSIL. c2) Bottom surface of mSIL detached from c1 sample. d) 50 nm Polystyrene particle imaged by mSIL. e) 45 nm IC chip imaged by mSIL. Reproduced with permission.[144] Copyright 2016, The Authors, published by AAAS.</figcaption></figure>

(measured by PSF) due to difficulties of attaining contact scanning mode.

### 24.2.3. Super-Resolution Physics

A complete theory for microsphere nanoscopy is still under development and one of the latest results is the wave theory of virtual imaging by microsphere.[615] The mechanism behind microsphere nanoscopy has been under debate since its birth. The PNJ effect was first considered as the main mechanism. However, calculations show that super-resolution strength by PNJ is weak, typically ≈λ/2−λ/3 for n = 1.5–1.6 particles. To explain the strong super-resolution ≈λ/6−λ/8 observed in experiments, other mechanisms were studied. Excitation of whispering-gallery mode (WGM) in microsphere allows to explain resolution up to ≈λ/4. Very recently, new super-resonance (SR) modes in microsphere were discovered.[616] A typical SR mode field distribution is shown in Figure 45d. It has a pair of highly localized hotspots (|E|² > 10⁴−10⁵, three orders higher than PNJ of 10−10²) near the bottom and top apex of the microsphere. A strong resolution of ≈λ/3−λ/6 is observed in Figure 45d. Deeper resolution and stronger field enhancement (10⁹−10¹¹)[617] by SR effect is possible for other particles, which requires further investigation. Moreover, some other mechanisms also contribute to resolution enhancement, such as plasmonic substrate effect and non-traditional illumination method (e.g., partial and inclined illumination[278, 612] and near-field evanescent-wave illumination using fluorescent nanowire[618] and localized plasmonic structured illumination[619]).

### 24.2.4. Bio-Superlens

Another trend in the field is the development of biological superlens using biomaterials such as spider silks,[620] cyanobacteria, live yeast cells,[621] and lipid droplets,[622] where typically 100 nm features can be resolved (not PSF). These bio-superlenses may open the intriguing route for developing multifunctional biocompatible bioimaging tools for sensing and single-cell diagnosis, which will be further advanced in the future.

## 24.3. Metamaterial Solid Immersion Lens (mSIL)

A notable achievement in the field is the development of mSIL in 2016. mSIL is an artificially engineered three-dimensional all-dielectric superlens assembled by high-index nanoparticles (Figure 46) that supports enhanced ETPC efficiency. Exploiting 15 nm high-index (n = 2.55) TiO₂ nanoparticles as building blocks, we fabricated TiO₂ mSIL with widths of 10–20 µm (Figure 46a) and demonstrated excellent super-resolution performance. It generates a sharp image with a resolution of at least 45 nm (≈λ/8.5 PSF resolution, Figure 46c), which exceeds the resolution of all previous superlenses. A new super-resolution mechanism was discovered in mSIL. The near-field coupling between neighboring nanoparticles in closely stacked media can effectively guide and transform the propagating wave into a large-area array of structured evanescent wave illumination field (Figure 46b). Inversely, the composite media supports highly efficient ETPC that lead to enhanced super-resolution. Similar works have been reported using other materials like ZrO₂
to replace TiO₂.[⁶²³] Recently, Dhama et al. designed and fabricated full-sphere TiO₂ mSIL and compared the imaging performance with BTG microsphere. The results confirmed that mSIL superlens performs consistently better than BTG superlens in terms of imaging contrast, sharpness, clarity, field of view, and resolution.[⁶²⁴] Besides mSIL, gradient solid immersion lenses (Maxwell fisheye) may provide another route toward near-perfect super-resolution imaging.[⁶²⁵, ⁶²⁶]

## 24.4. Current and Future Challenges

The imaging contrast by microsphere nanoscopy is often low due to the relatively weak ETPC efficiency. The low-contrast problem can be partially solved by mSIL with enhanced ETPC. While mSIL has shown greater imaging resolution and quality over other superlenses, its structure-integrity, lifetime in air/liquid and suitability for scanning imaging remains unknown which demands more investigations.

Increasing WD in dielectric superlens nanoscopy while retaining super-resolution is a key challenge in the field. Using partial and inclined illumination have shown the possibility to extend the working distance from sub-wavelength scale to more than one wavelength scale.[⁶¹²] Other proposals are needed to extend WD to at least 5 µm scale to enable a truly 3D super-resolution imaging of biological details and processes. Developing a higher speed nanoscopy system remains another challenge for the technique. Deeper tissue imaging, and combination of superlens with other super-resolution techniques (e.g., fluorescent nanoscopy) to achieve multi-modal super-resolution imaging systems will also be the future challenge for the technique.

## 24.5. Advances in Science and Technology to Meet Challenges

Recent advances in photonics, nanomaterials, metamaterials, and artificial intelligence (AI) could be utilized to address the discussed challenges. Introducing superlenses like microsphere/mSIL into a conventional optical microscope system leads to unwanted aberrations that reduce imaging contrast and quality, despite the resolution is improved locally at a region under the microsphere. This contradiction could be solved by using adaptive optics technology to correct the aberrations so that high-contrast super-resolution image can be obtained.[⁶²⁷] Another possible solution is to use metasurface, which can be designed and placed in front of the dielectric superlenses to realize similar function to the adaptive optics. Highly tuneable Metamaterials and metasurfaces will also be the promising solution to enable the development of long-WD superlens due to its flexibility in phase, amplitude, polarization, and wavefront engineering.[⁶²⁸] Combining superlenses with multiphoton microscopy may offer another solution to develop a long-WD superlens imaging system, e.g., using nonlinear effect to enhance resolution at the far-field zones. Due to significant amount of data generated during scanning superlens over a large-area, AI and machine learning technologies are particularly useful to process the big data to generate desired output image or extract features from a large-image. The latter property is especially useful for developing systems capable of tracking dynamics in biological samples.

## 24.6. Conclusions

Dielectric superlenses made from microsphere and nanoparticles and other materials have proven to be the effective tools to overcome the diffraction limit. Optical super-resolution of ≈λ/6–λ/8 (measured by PSF) has been demonstrated in real-time, label-free imaging of a variety of samples and processes in fields such as biology, material, and medicine research. The superlens technology has the potential to revolutionize the field of optical microscopy when the discussed challenges are resolved.

# 25. Microspherical Superlens Imaging: Fundamental Origins of Super-Resolution

*(Alexey V. Maslov*, Vasily N. Astratov)*

## 25.1. Status

The use of dielectric microspheres in contact with label-free or fluorescent objects, the technique to which we will refer here as microspherical superlens imaging (MSI),[¹³⁶, ¹³⁸, ⁵⁸⁴, ⁶¹¹, ⁶²⁹] see Figure 47, revolutionized the biomedical and nanoplasmonic imaging a decade ago because of its significantly better resolution compared to the standard microscopy and its relative simplicity compared to the advanced fluorescence (FL) microscopy represented by localization[⁶³⁰] or stimulated emission depletion (STED)[⁶³¹] techniques. The experimentally observed MSI resolution exceeds the solid immersion lens (SIL) limit, $\delta = \lambda/(2n_s \sin\theta)$, where $\lambda$ is the vacuum wavelength, $\theta$ is the half angle of the light collection cone, and $n_s$ is the refractive index of the microsphere. An intriguing situation in this area is that after a decade of intense experimental and theoretical studies, the origin of this super-resolution remains largely obscure.

The main properties of MSI can be summarized as the following three experimental findings: a) the resolution is systematically higher for smaller microspheres ($2 \mu m < D < 10 \mu m$, where D is the diameter) and can reach $\lambda/6-\lambda/7$ exceeding the SIL limit[¹³⁸, ⁶¹¹]; b) higher index ($n_s \approx 2$) microspheres inside liquid or plastic coverslips provide higher resolution[⁵⁸⁴] that can be qualitatively explained by the SIL effect; c) best resolution results are usually obtained for nanoplasmonic objects[⁶²⁹] or for FL objects coupled to short-period nanoplasmonic arrays.[⁶³²] The most rigorous approach to the explanation of resolution so far was based on exact numerical solutions of the Maxwell equations. The computational results for the point spread function (PSF), which is the classical estimate of resolution, show significantly larger width than the experimental resolution, as illustrated in Figure 47c.

## 25.2. Current and Future Challenges

A few approaches to explain the MSI super-resolution have been considered.

### 25.2.1. Photonic Nanojets

It was suggested originally that the observed super-resolution is linked to the ability of microspheres to form nanojets.[¹³⁶] This
<figure><img src="image_50.png" /><figcaption>Figure 47. a) Schematic of MSI in the transmission geometry. b) Virtual image of a dipole marked by the dot near the microsphere. The image intensity is plotted as a function of the transverse coordinate y and position x of the focal plane of the objective. The refractive index of the microsphere is n_s = 1.4 and of the background is n_b = 1. c) Resolution δ for imaging in water n_b = 1.33 background for the index contrast n_s/n_b = 1.4. The horizontal lines define two diffraction limits (0.5λ/n_b for water and 0.5λ/n_s for SIL) and the colored area defines the super-resolution regime. The data points for frames (b) and (c) were taken from Ref. [140]; for frame (c) they were rescaled for the immersion case.</figcaption></figure>

stimulated several studies in which the modeling of nanojet formation was performed[633, 634] assuming its reciprocal relation to resolution. However, the reciprocity of focusing and imaging may work with propagating waves, but if evanescent waves are involved it is likely to be invalid.[140] Besides that, the width of photonic nanojets can be only slightly smaller than the diffraction limit of λ/2 in air and only for microspheres with diameters much smaller than used in the experiments.[604, 635, 636] Thus, the existence of a direct relation between the super-resolution and the nanojet formation was subjected to serious doubt.

### 25.2.2. Incoherent Imaging and PSF modeling

In the incoherent imaging regime, the image is obtained as a convolution of the PSF and the object shape (field intensity). One way to explain the super-resolution in MSI is to demonstrate the narrowness of the PSF. This direction was followed in several studies which applied various computational techniques to point-source (dipole) emission near microspheres,[139, 140, 637, 638] Due to the complexity of the problem most of the PSF simulations are two-dimensional (2D)[139, 140, 637] except some cases where the Mie

theory is used in 3D.[638] In 2D geometry, see Figure 47a, one can put a point current source at distance d from the microsphere

$$J(x, y, t) = \hat{j}_0 \delta(y) \delta(x + R + d) e^{-i \omega t} \quad (27)$$

and calculate its emission which is collected by the objective forming an image.[140] The solution of the Maxwell equations describes the outcoupling of not only the propagating but also the evanescent components of the point source. The prominence of WGMs in microspheres calls for separate analysis in resonant and nonresonant cases.

Figure 47b shows a typical image in the non-resonant case for continuously moving focal plane of the objective. The virtual image is formed below the microsphere. Figure 47c summarizes the resolution (the FWHM of the PSF corrected for magnification) in the case of liquid immersion. In the nonresonant case, the resolution is ≈λ/3, which is slightly better than the diffraction limit of λ/(2n_b) of imaging in water, and depends very weakly on the gap d. This means that the evanescent fields do not affect the PSF width. In the resonant case, the resolution can reach λ/4, which is still noticeably below the experimental observations, and the images typically have sidelobes. The resolution increases with decreasing gap which correlates with the excitation of WGMs. The role of polarization on the PSF was also studied by considering dipoles oriented tangentially and radially relative to the microsphere surface.[637, 639] The narrowest PSF, about 0.28λ in air, is achieved for radial polarization. While the PSF analysis provides some insight into MSI properties, it fails to explain the experimentally observed super-resolution.

### 25.3. Advances in Science and Technology to Meet Challenges

Many modalities of experimental MSI and variety of objects with different properties used in these studies suggest that it is unlikely that a single factor would reveal the origin of super-resolution.

#### 25.3.1. Coherent Imaging Regime

In the Abbe coherent imaging theory,[640] the imaged object is represented as a sum of periodic components defined by their spatial frequencies. The incident wave is diffracted by each component. The diffracted waves are collected by the objective and directed to an image plane, where interference between the diffracted waves produces an image. The incoherent imaging regime is realized only as a limiting case of coherent imaging under special arrangements, for example, if the object is illuminated by a very broad angular spectrum of plane waves that are mutually incoherent. This condition can, however, break down, for example, due to the presence of a microsphere.

In coherent imaging the resolution is defined as the smallest period L = λ/2 of the object's spatial component which can create a propagating diffracted wave. A microsphere near an object can scatter even evanescent diffracted components, similar to plane waves in the Mie theory. A key question is whether these waves can be used to form an image. To demonstrate the possibility of image formation from an evanescent wave with an in-plane
<figure><img src="image_51.png" /><figcaption>**Figure 48.** Imaging of subwavelength current distribution (Equation (28)). a) Scattered energy W (normalized to some $W_0$) toward the objective ($-\pi/2 < \varphi < \pi/2$) as a function of the phase index of the current for several values of the particle size. The nonresonant case is exemplified by $kR = 5$, 8, 20, the resonant case by $kR = 20.382$ (WGM with azimuthal number $m = 24$, see Figure 4 in Ref. [643]). The colored area defines the region $n_{ph} > n_s$ in which the resolution above the SIL limit can be achieved. b,c) Images produced by the current with $n_{ph} = 1.6$ for various focal plane locations for $kR = 20$ and $kR = 20.382$. The images are normalized to their maximum values, which in (c) is 46 times higher than in (b). The oscillating curves near the microspheres in (b,c) illustrate current (Equation (28)).</figcaption></figure>

wavenumber $h_0$, let us consider a current source placed parallel to the horizontal y-axis near the bottom of a microsphere with $n_s = 1.4$ in a background medium with $n_b = 1$, as schematically illustrated in Figure 48b,c:

$$J_y(x, y) = 2j_0\cos(h_0y) \delta(x + R) e^{-i\omega t} \quad (28)$$

A gap between the current and the microsphere will only change the intensities of the emission and of the resulting image. For the results in Figure 48, the scattering problem was solved rigorously using the cylindrical functions. The phase velocity of the current wave is $v_{ph} = \omega/h_0 = c/n_{ph}$, where $n_{ph}$ is the phase index. The evanescence of the field created by current (Equation (28)) in the absence of the microsphere is provided by $h_0 > n_b\omega/c$. This is equivalent to the condition that the spatial current period $L = 2\pi/h_0 = \lambda/n_{ph}$ is smaller than the wavelength in the background material $\lambda/n_b$, that is $L < \lambda/n_b$ or $n_{ph} > n_b$. The physical meaning of the standard SIL limit is that for $n_{ph} > n_s$ (or $L < \lambda/n_s$), the current would not emit in a uniform medium even with index $n_s$ and, therefore, no image should be formed.

### 25.3.2. Fundamental Role of the Curvature of the Microsphere

The situation is dramatically changed in the presence of a microsphere since it scatters the evanescent field creating waves which go towards the objective, see Figure 48a. The key property is that while the emission decreases monotonically with increasing $n_{ph}$, it persists even for $n_{ph} > n_s$. There is no any qualitative change near the value $n_{ph} = n_s$. It was pointed out in Refs. [641, 642] that an evanescent wave can be converted to propagating by the curvature of the microsphere through a modified Snell's law. The condition for such conversion, as defined by Eq. (13) in Ref. [642], implies that if the spatial period $L$ of an evanescent wave incident on the sphere satisfies $L > \lambda/n_s$, its conversion to a propagating wave is possible. However, this condition is equivalent to the SIL limit. In contrast, Figure 48a demonstrates the possibility of outcoupling of very high spatial frequency components, which exceed the SIL limit of the microsphere. The scattering of evanescent waves by spheres is often used to excite WGMs but is also related to imaging. [643, 644] In the non-resonant case, the higher extraction efficiency for higher spatial frequencies ($n_{ph} > n_s$) is evident for the intermediate value of $kR = 8$ compared to the smaller $kR = 5$ or larger $kR = 20$. This qualitatively supports the experimentally observed higher resolution for mesoscale microspheres with diameters of about several wavelengths. The emission efficiency affects the visibility of fine features on the images. While the microsphere can outcouple high spatial frequency components, can they form an image by the objective? The image from the evanescent field generated by the subwavelength current, see Figure 48b, supports the possibility of imaging with a resolution better than in the SIL method, that is for $n_{ph} > n_s$.

In the calculation of the PSF, the microsphere also outcouples the evanescent component of the point current so why would the coherent regime provide a better resolution than incoherent? It is likely that in the infinitely broad spectrum of the point source, the contribution of the evanescent components is simply too weak to be noticeable. Indeed, the emission efficiency decreases with increasing spatial frequency of the evanescent field, see Figure 48a. In contrast, in the coherent imaging the evanescent components can play a dominant role since the diffraction on a high spatial period component of the object shape will not produce any propagating waves, except perhaps the zeros order. This, therefore, greatly limits the spatial bandwidth allowing the evanescent field contributions to be revealed.

### 25.3.3. Resonant Imaging

Figure 48a shows that the WGM resonance provides a much stronger current emission for all $n_{ph}$ with a maximum when the phase velocities of the current and of the WGM are close. As a result, the image at the WGM resonance has much higher intensity than off-resonance, compare Figures 48b and 48c. In the resonant case, the image can be defined more by the properties of the WGM itself rather than by the current. Using this property, however, one can also envision a new imaging regime. Under a broadband frequency illumination, different spatial components of the diffracted evanescent fields can potentially excite different resonant modes. This may give a mapping between the spatial component of the evanescent field and the spatial distribution of the exited WGM, which then can potentially be transformed into
an image. Despite being more efficient, the resonant imaging can lead to many artifacts since the observed image can be dominated by the properties of the WGMs rather than by the object. Yet it offers exceptional opportunities for combining imaging and detection.

### 25.3.4. Coherent Imaging of Resonant Structures and Nanoplasmonics

Many objects used for MSI resolution benchmarking have resonances, for example, localized surface plasmon resonances (LSPR), and the so-called plasmonic hot spots. Examples of such objects are bow-ties, dimers, plasmonic molecules, nanowires, nanoclusters, and arrays. In this case, the field that creates the image is not the diffracted incident field, but rather the reradiated modal field, which can be greatly enhanced.[645] This resonantly enhanced coherent field, in which high spatial frequency components are dominant, can be outcoupled by the microsphere enabling the image formation. Some modes may allow seeing extraordinarily small features but usually with distortion. The microsphere can potentially play a role in the excitation of the resonant modes. Alternatively, they can also be excited by fluorescent molecules. By contrast, if mutually incoherent waves are incident from all angles on a small resonator, the excitation of resonances is unlikely since it requires some coherence of the driving field at the resonator location.

### 25.3.5. Integration with Other Super-Resolution Techniques

Due to widefield imaging and flexible illumination conditions MSI is suitable for integration with other super-resolution approaches and methods considered in this Roadmap. One example is interferometric techniques[145-147] discussed in Topic 28. Another example is the integration of micro-spheres or millimeter-scale ball lenses[646, 647] with cellphone imaging that permits building pocket-size microscopes as discussed in Topic 26.

### 25.4. Conclusions

The theoretical explanation of super-resolution values beyond the classical SIL limit represents an intriguing question laying at the intersection of fundamental physics of imaging and advanced numerical modeling methods. It is shown that the approach based on coherent imaging is likely to be relevant to many experimental situations and is promising for explaining the observation of extremely small features. The illumination (which includes multiple object-microsphere scattering) can create coherent fields in the region between the microsphere and the object. These fields can be enhanced by plasmonic resonances. The subwavelength scale of these fields in the object plane results in their evanescence in the normal direction in the absence of the microsphere. Due to the curvature of the microspherical surface these fields can be outcoupled and then create a magnified super-resolved image observable by the microscope objective. The narrow spatial bandwidth of the evanescent components may allow seeing even rather weak evanescent signals. However, coherent imaging

is prone to artifacts and image distortion effects. The WGM resonances of microspheres can significantly boost the outcoupling of the evanescent fields, but they can also lead to the presence of artifacts in the optical images. Our vision for future developments in this area is based on combination of several factors: a) coherent imaging regime, b) curvature of microsphere, c) WGM resonant enhancement as well as d) nanoplasmonic enhancement due to LSPRs and hot spots as main physical causes of super-resolution. The validity of this consideration for the explanation of super-resolution imaging and the specific roles played by these factors in different imaging scenarios require further confirmation, which can include specially designed experiments, theoretical and computational studies.

## 26. Smartphone Ball Lens Microscopy

(Vasily N. Astratov*, Boya Jin)

### 26.1. Status

Since introducing megapixel CCD and CMOS sensor arrays in digital cameras in the 1990s and 2000s, the motivation to develop portable and lightweight microscope systems never faded away. Such pocket-size microscopes can be taken to the world's most remote locations. They can also help to answer some fundamental questions in astrobiology, such as to provide evidence of bacterial life on different planets or meteorites. Most importantly, they would allow certain medical procedures such as biopsy-free diagnostics of the skin cancers that can be performed at the point-of-care without using complicated facilities such as reflectance confocal microscopy, optical coherence tomography, multiphoton microscopy (MPM), and Raman spectroscopy.[648]

All these applications require label-free imaging with resolution sufficiently close to the classical diffraction limit, but not necessarily require optical super-resolution imaging. As an example, such diffraction-limited resolution is desirable for studying the spatial distribution of the lymphocytes and tumor cells to achieve a prognostic significance for melanoma diagnostics.[649, 650] The resolution of ordinary smartphones, however, is limited at the 20–30 µm level due to the finite sizes of the pixels in the sensor array and insufficient magnification (M) of the smartphone optics. This resolution is well below the classical diffraction limit, $\lambda/(2NA)$, where $\lambda$ is the illumination wavelength (or emission wavelength in the case of fluorescence (FL)), $NA = \sin\theta$ is the numerical aperture, $n$ is the index of refraction of the media surrounding the radiating point, and $\theta$ is the half angle of the collection cone that enters the objective. Resolution of standard cell phones can be enhanced down to $\approx 1.5$ µm using microoptics solutions for FL samples,[651–655] however further increase of the resolution is made difficult by the limited magnification and significant spherical aberrations of such systems.

In this Topic, we briefly review main design approaches to development of smartphone microscopes and discuss the prospect of application of microspherical superlens imaging (MSI) for enhancing the resolution of cellphone microscopy. Previously, applications of MSI to nanoplasmonic structures
<figure><img src="image_52.png" /><figcaption>**Figure 49.** Designs of cellphone microscopy based on: a) Lensless shadow imaging on top of the sensor array, adapted with permission.[660] Copyright 2005, Elsevier. b) Digital in-line holographic imaging.[662–664] c) Combination with the microscope objective, adapted under terms of the CC-BY license.[651] Copyright 2009, The Authors, published by the Public Library of Science (PLOS). d) Microoptics solutions.[652–655]</figcaption></figure>

allowed to achieve resolution ≈λ/6−λ/7 which exceeds the solid immersion lens (SIL) limit (≈λ/(2n)).[136, 584, 632, 656, 657] However, the smartphone optics doesn't have the quality of microscope objectives used in the previous MSI studies. Recently, it was shown that significantly larger millimeter-scale ball lenses with refractive index close to two provide large image magnifications (M>30) in contact conditions with the samples that allows to overcome the resolution limitations coming from the pixilation of the cellphone sensor array and achieve truly diffraction-limited resolution ≈0.7 µm for visible light.[646, 647, 658, 659] It allows developing pocket-size handheld microscopes with the diffraction-limited resolution for a variety of applications.

## 26.2. Current and Future Challenges

Smartphone-based microscopy is attractive for developing diagnostics in low-resource settings, but its practical designs faced significant limitations.

### 26.2.1. Shadow Imaging

One of the first lensless designs includes a microfluidic channel on top of the CCD sensor array illuminated with a uniform light field, as illustrated in **Figure 49a**. The shadows of floating worms or other biomedical objects can be detected with the resolution determined by the sensor's pitch size, typically in 5–10 µm range.[660] The resolution can be increased by using additional nanoaperture arrays in combination with the microscope imaging.[661] The latter technique has been termed optofluidic microscopy (OFM). The improved resolution of OFM methods, however, comes at the expense of increasing the size and complexity of the imaging system, so that it cannot be viewed as a pocket-size imaging solution.

### 26.2.2. Digital In-line Holographic Microscopy (DIHM)

Another lensless imaging solution is offered by the DIHM method where a spherical wave emanating from an aperture illuminates an object, and the interference of the light waves that passed through the objects (such as cells or microparticles) with the unscattered light creates the hologram of each object, which is detected using cellphone detector array, as schematically illustrated in Figure 49b. The lensfree hologram of each object permits rapid reconstruction of its microscopic image through digital processing.[662–664] The lens-free holographic approach provides a large field-of-view (FoV), but with a limited, micron-scale resolution combined with a significant computational load. In addition, this approach cannot be used to image samples prepared on standard glass slides due to the requirement that the sample be positioned in close proximity to the sensor.

### 26.2.3. Smartphone Combined with the Microscope Objective

As an example, a combination with 0.85 NA 60× achromat objective and a 20× wide field microscope eyepiece resulted in a system with FoV ≈ 180 µm, an effective magnification onto the camera face of 28×, and a measured spatial resolution *d* = 1.2 µm.[651] This resolution value is three times larger than the classical diffraction limit. Such a system reminds classical microscope design and, as a result, it is rather bulky and heavy.

### 26.2.4. Microoptics Solutions

To reduce the size and weight of imaging system, the long attachment with microscope objective was replaced with microlenses, as illustrated in Figure 49d. The use of ball lenses leads to significant aberrations that reduce image quality towards the edges of FoV.[653] There is also a tradeoff between the magnification
<figure><img src="image_53.png" /></figure>

<figure><img src="image_54.png" /><figcaption>**Figure 50.** a) Comparison of images of various biomedical samples taken by a standard microscope with 10 × (NA = 0.25) objective (left column) and by proposed cellphone microscopy (right column) through LASFN35 ball lens with D = 2.0 mm. Pairs of cellphone images are obtained at different positions of the ball lens. b) Image of double-stripe nanoplasmonic object resolved through the ball lens by cellphone microscopy at λ = 480 nm. c) Image of “Siemens star” nanoplasmonic object resolved through the ball lens by cellphone microscopy at λ = 480 nm. d) Intensity cross-sections measured using Siemens star target at different radial distances (indicated (1–3) in (c)) and corresponding intensity profiles calculated by convolution with 2-D Gaussian PSF with FWHM = 0.9 µm. Good agreement with theory indicates that the resolution is 0.9 µm. e) Summary of resolution quantification for double-stripe objects performed using ball lenses with D = 0.5 mm, 1.0 mm, and 2.0 mm at λ = 430, 480, 546, 589, and 632 nm. a,b,e) Adapted under terms of the CC-BY license.[659] Copyright 2023, The Authors, published by Wiley-VCH. c,d) Reproduced under terms of the CC-BY license.[658] Copyright 2023, The Authors, published by Springer Nature.</figcaption></figure>

and FoV. Such systems were mainly used for imaging FL objects with a rather limited magnification and the best resolution about 1.5 µm.[652–655]

## 26.3. Advances in Science and Technology to Meet Challenges

A combination of smartphone microscopy with MSI methods allowed to suggest a radical way[646, 647, 658, 659] to dramatically increase the magnification and to exceed the resolution limitation determined by the pixilation of the cellphone sensor array. In the limit of geometrical optics, the lateral image magnification (M) of ball lens is determined by the following equation[656]:

$$M(n', D, g) = \frac{-n'}{2(n' - 1)\left(\frac{2g}{D} + 1\right) - n'} \quad (29)$$

where D is the diameter of the ball lens, $n' = n_{sp}/n_0$ is the refractive index contrast between the spherical ball lens and object space, and g is the gap between the object and ball lens. This equation shows that for zero gaps ($g = 0$) magnification increases

asymptotically as $n'$ approaches two. The ball lens creates a virtual image below the object for $n' < 2$ and real image above the object for $n' > 2$. Experimentally the latter situation was realized by using the ball lens made by LASFN35 glass with refractive index $n = 2.02$ at $\lambda = 600$nm and diameter of 2.0 mm separated from various biomedical structures with a sufficiently small distances ($g = 0.17$ µm), as it is illustrated in Figure 50a.

Comparisons of images of various biomedical samples in Figure 50a obtained by the conventional microscopy with 10× (NA = 0.25) objective and by the smartphone microscopy illustrate their comparable quality of imaging. In the case of imaging through individual ball lens, FoV is limited by $\approx D/10$. In practice, the ball lens can be manipulated to extend FoV, as illustrated by the pairs of cellphone images obtained at different position of the ball lens. The quality of imaging by the cellphone microscopy is sufficient for studying the distribution pattern and density of tumor-infiltrating lymphocytes (TILs), which makes possible the study of infiltration and disruption of TILs into tumor nests or direct contact between TILs and tumor cells required for melanoma diagnostics.[649, 650] It should be noted, however, that the cases illustrated in Figure 50a were suboptimal for achieving the maximal resolution because of the glass coverslip with the
thickness of $g = 0.17$ mm separating the ball lens from the biomedical samples. Under these conditions, the magnification of cellphone microscopy was limited at $M = 7$ level.

To study the maximal magnification and resolution of the proposed method, the LASFN35 ball lenses with different diameters were placed directly on top ($g = 0$) of the nanoplasmonic structures, as illustrated for a double-stripe object[647, 659] and ‘Siemens star’ target[658] in Figure 50b,c, respectively. We showed that because of the strong chromatic dispersion of the LASFN35 glass, magnification increases with the wavelength reaching extraordinarily large values $M > 50$ for the red wavelengths.[647, 659] The methodology of resolution quantification[656] was based on comparing intensity cross-sections indicated by dashed lines in Figure 50b,c with the calculated intensity profiles. This procedure is illustrated in Figure 50d for three cross-sections (1–3) taken at different radial distances from the center of the Siemens star object, as shown in Figure 50c by dashed lines. The experimental intensity profiles in Figure 50d (solid lines) were obtained after subtracting a uniform scattering background intensity. Calculated profiles (dashed lines) were obtained by convolution of the “perfect” drawn object with the 2-D Gaussian point-spread function (PSF). The full width at the half maximum (FWHM) of PSF providing the best fit to the experimental data was accepted as a resolution of the system based on the Houston resolution criterion. The results of a similar resolution quantification performed with the double-stripe object are summarized in Figure 50e for various visible wavelengths. It is seen that for $\lambda > 600$ nm the resolution capability of the ball lens reduces with $\lambda$ that represents a typical diffraction-limited behavior. However, for $\lambda < 500$ nm an opposite behavior was observed that indicates that at these wavelengths the resolution is still determined by the pixilation of the sensor array. Generally, a better resolution of $\approx 0.67$ µm was found for Au double-stripe object[659] compared to that for Cr Siemens star target.

## 26.4. Conclusions

With the advent of megapixel CCD and CMOS sensor arrays, the smartphone-based microscopy became attractive for developing medical diagnostics in low-resource settings, but its practical designs faced significant limitations. Several methods and optical designs have been proposed including shadow imaging, OFM, DIHM, combination with the microscope objectives, and microoptics solutions. However, the resolution of compact designs was limited at 1.5 µm level. An integration of millimeter-scale ball lenses with index close to two with a smartphone allows the developing pocket-size microscopes that can be used for biopsy-free diagnostics of skin diseases. In contrast to previous studies, the compound ball lens/cellphone camera objective has the best resolution $\approx 0.67$ µm at $\lambda = 589$ nm[646, 647, 658, 659] that corresponds to diffraction-limited imaging with NA $\approx 0.4$. Further increase of the resolution might become possible under two conditions: a) use of microspheres with diameters from several microns up to several tenths microns instead of millimeter-scale ball lenses, b) use of short-period nanoplasmonic arrays creating a “hot spot” plasmonic illumination for nanoscale objects.[632] It can help to demonstrate optical super-resolution imaging using the smartphones in future studies.

# 27. Imaging Beams for Subwavelength Resolution

(Constantin R. Simovski*)

The diffraction limit of resolution of two mutually non-coherent point sources equals 1/2 only for objects in free space when the objective lens has f-number larger than unity. When the imaged object is inside a mirrored glass microparticle, when the objective diaphragm has low f-number and in some other cases, the superresolution is achieved without involving evanescent waves.[60, 665] In the theory of a hyperlens,[126, 127, 666] the sub-wavelength information of the object is carried out to the far zone by imaging beams which do not contain evanescent spatial harmonics. If the object is subsurface the evanescent waves are absent in the model of the hyperlens imaging.[667, 668] Despite the common belief, the absence of evanescent waves does not prohibit the subwavelength accuracy of the point-wise source location in the image.

This also refers to a glass microsphere that grants the subwavelength far-field image similarly to a hyperlens. The non-resonant superresolution results from the feature of the wave beam created by a point dipole source normally polarized to the surface of a glass microsphere. This unique feature is zero field on the axis of the wave beam transmitted through the microsphere. This zero exactly determines the line connecting the real point dipole and the virtual dipole created in accordance with the geometrical optics (GO). This fine location of the real source is possible for a wide range of the sphere sizes and refractive indices. For a normally oriented dipole and only for it the role of the diffraction by a microsphere for the transmitted beam is not important beyond the Mie resonances. This transmission can be thought as a problem of the GO. This solution is presented in Figure 1 for two values of the refractive index of the sphere. Meanwhile, our numerical studies have shown that a tangentially polarized dipole such the solution makes no sense—in the phase distribution of the transmitted beam—the information of the exact location of the dipole source is lost. As to a normally polarized dipole the GO predicts the novel regime of subwavelength imaging which corresponds to specific refractive indices $n$ and radiuses $R$ of the microsphere.

In the case $n = 1.44$ GO predicts that all transmitted rays are parallel to the beam axis, and the ultimate angle of total internal reflection for the rays confined inside the sphere is $\theta_{TIR} \approx \pi/4$ as shown in Figure 51a, whereas for $n = <1.40$ the rays form a spherically divergent beam with the virtual center located at the axis $y$, as it is shown in Figure 51b. In Figure 51a, the intensity distribution is the function of the transverse coordinate $x$, and in Figure 51b, it is the function of the ray tilt angle. The situation illustrated by Figure 51b is known—all experimental studies of the superresolution granted by a microsphere refer to the case when the transmitted beam is conical. It is important that Figure 51 is applicable only to a normally polarized dipole. Consider the case $n = 1.44$ depicted in Figure 51a. The beam intensity is effectively concentrated in a ring of inner radius $R/2$ and outer radius $R$. The rays passing through this ring have nearly identical optical paths from the source to the plane $y > 0$. For $R = 10\lambda$ the difference of their phases does not exceed $\pi/4$. Meanwhile, for a tangential dipole the beam intensity is nearly uniform in the whole cross-section of radius $R$. For the same $R = 10\lambda$ the phase of the central