System and apparatus for dynamically shaped focal surface with a scanning microscope

Apparatuses, systems, and methods for dynamically shaped focal surface with a scanning microscope. A microscope (e.g., a scanning confocal microscope) may use a scanning element to scan an illumination beam to generate a focal region. The scanning element may include multiple degrees of freedom, such as rotation and translation along orthogonal axes. For example, if the illumination beam is a line focus, rotation along a first axis and translation along a second orthogonal axis may sweep the line focus across a focal surface which is substantially flat and normal to the second axis. In some embodiments, the microscope may be a dual-axis handheld microscope, where a single objective in a handheld housing is used to both direct the scanned illumination beam and receive the collected light.

BACKGROUND

Microscopy may generally involve directing light onto a sample, and then imaging the sample based on light received from the sample. Confocal scanning microscopes may use an element (such as a scanning mirror) to scan a focused beam of light across a sample to illuminate a region of interest. For example, a confocal microscope may raster scan a laser beam across a focal plane to illuminate and image it. Light may be collected from the sample and passed through an aperture (e.g., a pinhole) located at a confocal plane, which may limit the out of focus light received by a detector. This may generally be useful in situations where there is a desire to image a focal plane at or below the surface of a thick tissue specimen. The scanning may be achieved by rapidly rotating a reflective element back and forth about an axis. This rotation may introduce curvature in the focal surface imaged by the microscope. This may be undesirable, for example when trying to image thin, layered, structures or when trying to stitch multiple image frames together to form a larger mosaicked image.

SUMMARY

In at least one aspect, the present disclosure relates to an apparatus including an illumination source, a scanning module, and a detector. The illumination source provides illumination light. The scanning module scans an illumination beam based on the illumination light across a focal surface. The scanning module includes one or more scanning elements, and the one or more scanning elements rotate about a first axis and translate along a second axis. The detector receives light from the focal surface.

The one or more scanning elements may further rotate about a third axis. The focal surface may be a substantially flat field in a plane which is orthogonal to the second axis. The first axis may be orthogonal to the second axis. The scanning element and the detector may be contained by a hand-held housing.

The apparatus may also include illumination optics which may generate the illumination beam based on the illumination light. The illumination beam may have a line focus. The detector may be a linear array detector.

The one or more scanning elements may include a micro-electromechanical (MEMS) mirror. The MEMS mirror may perform the rotation about the first axis and the translation along the second axis.

The apparatus may also include a controller which may direct the scanning element to determine a shape of the focal surface. The apparatus may be a confocal microscope and the apparatus further also include a confocal aperture which may limit out of focus light received at the detector. The illumination light may include an excitation wavelength of a fluorophore, and the received light may include light emitted from the fluorophore.

In at least one aspect, the present disclosure relates to a system including a controller and a microscope. The controller includes an illumination source which generates illumination light and a scanning module controller which provides signals based on a selected shape of a focal surface. The microscope includes an illumination light path which generates an illumination light beam based on the illumination light, a collection light path, and a scanning module. The scanning module scans the illumination beam to generate the focal surface. The scanning module receives light from the focal surface and directs the received light to the collection light path. The scanning module includes one or more scanning elements which actuate based on the signals. The one or more scanning elements are rotate about a first axis and translate along a second axis.

The microscope may also include a detector which may generate an image based on the light received by the collection light path. The shape of the focal surface may be substantially flat. The shape of the focal surface may be arbitrary. The one or more scanning elements may include a micro-electromechanical system (MEMS) scanning mirror which may perform the rotation about the first axis and the translation about the second axis.

The illumination beam may have a line focus. The collection light path may include a confocal aperture. The controller may be positioned on a mobile platform. The microscope may be handheld. The one or more scanning elements may rotate about a third axis.

In at least one aspect, the present disclosure relates to a method. The method includes generating an illumination beam. The method includes scanning the illumination beam with a scanning module to generate a focal surface. The scanning module includes one or more scanning elements, and the scanning includes rotating at least one of the one or more scanning elements about a first axis and translating at least one of the one or more scanning elements along a second axis. The method includes collecting light from the local surface. The method includes generating an image based on the collected light.

The scanning module may include a micro-electromechanical system (MEMS) mirror which performs the rotation about the first axis and the translation about the second axis.

The method may also include collecting the light from the focal surface through the scanning module. The method may also include positioning a hand-held unit including the scanning module near a sample. The method may also include controlling the motions of the scanning module to determine a shape of the focal surface. The method may also include scanning the illumination beam to generate a local surface which is substantially flat.

Generating the illumination beam may include focusing illumination light into a line focus. The scanning further may also include rotating at least one of the one or more scanning elements about a third axis.

DETAILED DESCRIPTION

A scanning confocal microscope may include an illumination beam which is swept across a focal region of a sample. The shape of the illumination beam as well as the path across which the beam is swept may determine the size and shape of the focal region. The microscope may include illumination optics which direct the illumination beam onto a scanning element, such as a scanning mirror. The scanning element may oscillate to change the position and/or angle of the illumination beam with respect to the back focal plane of an objective lens. Accordingly, the illumination beam may also change position and/or angle as it exits the front focal plane of the objective lens towards the sample. In this manner, the focus of the illumination beam may be swept across a focal surface.

The oscillation of the scanning element (e.g., rotation about one or more axes) may introduce curvature into the shape of the focal surface. While this may be corrected by changing the optical path (e.g., the components of the objective lens) to account for and correct for this curvature, such a solution may be undesirable as it may require expensive custom optics, extensive design of the microscope, extremely tight tolerances, etc. It also may lock the microscope into a single shape of focal surface. It may be desirable to use a scanning element which can adjust the shape of the focal surface as needed.

The present disclosure is directed to a system and apparatus for dynamically shaped focal surfaces with a scanning microscope. The microscope may include a scanning module which includes one or more scanning elements which perform rotation about one or more axes, and translation along one or more axes. The rotation and translation may be distributed among multiple scanning elements (e.g., an element for translation and an element for translation) or may be combined into a scanning element with multiple degrees of freedom. For example, the scanning module may include a scanning mirror which can rotate about the x and y axes, and piston back and forth along a z axis. A controller may operate the scanning module to change the shape of the focal surface. For example, the translation along the z axis may be coordinated with rotation about the x and y axis to scan an illumination beam across a focal surface which is substantially flat (e.g., planar). Other shapes of the focal surface may be achieved by changing the pattern of rotation/translation instructed by the controller. The translation may also be useful in other situations, such as, for example, adjusting the depth of the focal surface (e.g., to change a level of the focal region within a sample).

One example embodiment using such a scanning element may be a handheld line-scanned dual-axis confocal microscope. The illumination optics may generate an illumination beam which is line (e.g., elongated in an x-axis). The scanning module may include a single scanning element which is a micro-electromechanical system (MEMS) mirror which may scan the line back and forth (e.g., in a y-axis) and also translate along an orthogonal axis (e.g., along the z-axis). The scanning element may be directed to generate a focal surface which is of arbitrary shape. For example, the scanning element may be directed to generate a focal surface which is substantially flat (e.g., along the x-y plane). The microscope includes a single objective lens, which may be used both to direct the scanned imaging beam onto the sample and also receive light from the sample. The scanning module also receives collected light from the focal region, and may descan the collected light and direct the descanned collected light onto a detector. Various components, such as the objective, scanning element, and/or detector may be packaged together into a hand-held system. This may be useful, for example, for in vivo imaging of tissue.

FIG.1is a schematic diagram of an optical system according to some embodiments of the present disclosure. The optical system100shows a layout of a particular example embodiment. It should be understood that details of the layout ofFIG.1may be different in other example embodiments. For example, while the example embodiment ofFIG.1shows a handheld microscope102coupled to a controller104, other embodiments may use a controller integrated into a handheld unit, or may use a microscope which is not handheld. In some embodiments, one or more parts of the controller104may be omitted, and the microscope102may be operated manually.

The optical system100includes a handheld microscope102(e.g., an optical unit) and a base-station controller104. The controller104includes an illumination source124, which generates light that may be coupled (e.g., via an optical fiber) to the handheld microscope102. The handheld microscope102includes imaging and collection optics106, which shape and direct the illumination beam towards a focal region (e.g., a focal surface). The optics106include a scanning module110, which scans the illumination beam across a back focal plane (BFP) of an objective lens112. The objective lens includes a number of optical elements (e.g., lenses) and directs the scanned illumination beam towards a front focal plane (FFP) of the objective. The scanning of the illumination beam across the FFP may generate the focal surface. The objective112may also receive light from the FFP, and direct the received light through the objective112to the BFP and onto the scanning module110. The scanning module110may direct the received light to the collection optics. For example, the scanning module110may descan the received light and direct the descanned light to the collection optics and detector120. The microscope102may include a confocal aperture to limit out of focus light. Example details of the optics106are described in more detail inFIG.2.

The handheld microscope102may be shaped so that it can be held by a user (e.g., in a user's hand) and manually positioned relative to a sample. For example, the sample may be a patient and the handheld microscope102may be used for in vivo imaging. The handheld microscope102may include a hand-held housing which contains the optics106and which is detachably coupled to the detector120and controller104. For example, the handheld microscope102includes an adaptor118and a fast release116. The adaptor118may be coupled to the detector120, and the fast release connector116may removably couple the adaptor118to the housing which holds the optics106. The adaptor118may, in some embodiments, be adjustable to control the positioning of the detector120relative to the optics106. For example, the adaptor118may adjust the relative position along the x, y, and or/axis, rotation about one or more of those axes, or combinations thereof. In some embodiments, the detector120may be more permanently coupled to the optics106. In some embodiments, the detector120may also be contained in a hand-held housing.

In some embodiments, the handheld microscope102may not necessarily be handheld. For example, the microscope102may be supported by an armature or other structure. A user may manually position the microscope102and/or actuators may position the microscope102. In some embodiments, the optical system100may be similar to an endoscope with the controller104as a base station and the microscope102extending in a housing which may be positioned (partially or entirely) inside a subject.

The handheld microscope102may include an optional lens cap114. The lens cap may be a transparent material which covers a front surface of the microscope102. The lens cap114may prevent the front of the microscope102from coming into direct contact with the sample as the lens cap114may contact the sample instead. The lens cap114may be removable so that it can be cleaned, sterilized, etc. In some embodiments, some form of immersion fluid (e.g., a transparent fluid such as ultrasound gel) may be positioned between the front surface of the lens cap114and the sample to improve optical coupling into and out of the sample.

The microscope102may be coupled to the controller104by one or more physical connections (e.g., wires, optical fibers, etc.), wireless connections (e.g., blue tooth, wifi, etc.), or combinations thereof. For example, an optical fiber may couple the illumination source124to the handheld microscope102, while various wires may be used to couple data to/from the detector120, and power and control signals to various components (e.g., the scanning module110). In some embodiments, commercial communication protocols (e.g., USB, PCIe, blue tooth, etc.), custom communication protocols or combinations thereof may be used.

The controller104may represent various components of the system100which are positioned in a base station separate from the handheld microscope102(although one or more of these components may be part of the handheld microscope102in other embodiments). In some embodiments, the controller104may not be handheld. The controller104may represent different components packaged together, or may represent separate components. In some embodiments, the controller104may be portable (e.g., packaged in a housing with a handle, positioned on a mobile cart, etc.). The controller104includes a scanning module controller122, an illumination source124, a detector interface126, a power supply128, a general purpose computer130, and an input/output system132.

The general purpose computer130may be used to coordinate the operation of one of more components of the controller104, which in turn may operate the handheld microscope102. In some embodiments, the computer130may directly operate one or more components of the microscope102. The general purpose computer130may include a processor and a memory. The memory may include various instructions which may be operated by the processor. The processor may operate the instructions to carry out various operations, such as operating the microscope102and or capturing/analyzing data from the microscope102. The general purpose computer130may be coupled to an input/output132, which may display information to a user (e.g., via a monitor) and receive commands from a user (e.g., via a keyboard and mouse). The instructions may be used to generate a user interface, which may allow a user to control various options of the imaging process. In some embodiments, the instructions may automate various options of the imaging process.

The controller104includes a scanning module controller122, such as a MEMS controller. The scanning module controller122may provide various signals which actuate the scanning module110. For example, the scanning module controller122may provide voltages, each of which may cause a displacement or rotation of one or more scanning elements of the scanning module110about/along an axis. The scanning module controller122may vary these voltages in time to scan the illumination light. For example, a first voltage may actuate the scanning module110in rotation about a first axis, while a second voltage may actuate the scanning module110in translation along a second axis. The computer130may include instructions which coordinate the signals provided by the scanning module controller122to control the shape of the focal region.

For example, the scanning module controller122may generate voltages based on a particular equation (or equations), which may be chosen to cause the scanning module110to generate a focal surface of a desired shape. This shape may be arbitrary, and may be any shape within the physical (and optical) limits of the system (e.g., the range of motion of the scanning elements in the scanning module110). For example, the focal surface may be flat, sloped, tilted, or curved across all or a portion of the field of view. In some embodiments, a shape of the focal surface may be chosen based on the sample to be imaged such that the focal surface lies within a structure in which imaging is desired. Since the focal surface may be generated based on signals (e.g., voltages) from the controller104, it may be possible to alter the shape of the focal surface during imaging.

In some embodiments, it may be useful to generate a focal surface which is substantially flat (e.g., which is at a plane orthogonal to an axis of the objective112). To generate a substantially flat focal surface, the scanning module110may have at least one component driven to rotate by a first voltage which is varied as a sinusoidal or triangular waveform, and at least one component driven to translate by a second voltage which is varied as a second sinusoidal or triangular wave. A focal surface may, in some embodiments, be considered substantially flat if its curvature is smaller than its depth of focus. Other waveforms may be used to produce a substantially flat focal surface, or may be used to produce other shapes of focal surface, in other embodiments.

The controller104includes an illumination source124. The source124may be a narrow band source, such as a laser or a light emitting diode (LED) which may emit light in a narrow spectrum. In some embodiments, the light may be a broadband source (e.g., an incandescent source, an arc source) which may produce broad spectrum (e.g., white) illumination. In some embodiments, one or more portions of the illumination light may be outside of the visible range. In some embodiments, a filter (not shown) may be used as part of the illumination path to further refine the wavelength(s) of the illumination light. For example, a bandpass filter may receive broadband illumination from the source118, and provide illumination light in a narrower spectrum. In some embodiments, the light source103may be a laser, and may generate collimated light.

In some embodiments, the sample imaged by the microscope102may be a biological sample. For example, the sample may be a tissue which includes a region of suspected disease (e.g., cancer). In some embodiments, the sample may be an in vivo tissue, and may be part of a subject (e.g., a patient). The user may position the microscope102along various regions of the subject to image different tissues (or different regions of the same tissue) in the subject. In some embodiments, the tissue may undergo various processing, such as optical clearance, exposure (e.g., via surgery), and/or labeling before being examined by the optical system100. In some embodiments, examination of the tissue with the optical system100may be used for diagnosis, to determine treatment progress, to monitor disease progression, etc. In some embodiments, the sample may be non-biological.

In some embodiments, the illumination light may illuminate one or more structures of the sample, and the collected light may be reflected (and/or scattered) from those structures. In some embodiments, labels (e.g., stains) may be applied to the sample. In some embodiments, the imaging may be label-free. In some embodiments, the sample may include one or more types of fluorophores. The fluorophores may be intrinsic to the sample (e.g., proteins in biological samples) or may be a fluorescent label (e.g., acridine orange, eosin) applied to the sample. Some samples may include a mix of intrinsic types of fluorophores and exogenously applied fluorescent labels. Each type of fluorophore may have an excitation spectrum, which may be centered on an excitation wavelength. When a fluorophore is excited by light in the excitation spectrum, it may emit light in an emission spectrum, which maybe centered on an emission wavelength which is different than (e.g., red-shifted from) the excitation wavelength.

In some embodiments, the optical system100may be used to image fluorescence in the sample. The illumination source124may include light at a particular excitation wavelength, which may excite fluorophores in the sample. The illumination source124may include a broad spectrum of light which includes the excitation wavelength, or may be a narrow band centered on the excitation wavelength. In some embodiments, the light source124may produce a narrow spectrum of light centered on (or close to) the excitation wavelength). In some embodiments, filter(s) (not shown) may be used in the illumination optics to limit the illumination beam to wavelengths near the excitation wavelength. Once excited by the illumination sheet, the fluorophores in the sample may emit light (which may be centered on a given emission wavelength). In some embodiments, the collection path (e.g., in the optics106) may include one or more filters which may be used to limit the light which reaches the detector120to wavelengths of light near the peak of the emission spectrum. In some embodiments the collection path may reject light at or near the excitation wavelength (e.g., by using a longpass filter).

The controller104may include a detector interface126, such as a frame grabber, which may be used to capture data provided by the detector120. The controller104may also include a power supply128, which may provide voltage(s) to power one or more components of the handheld microscope102. For example, the power supply128may power the detector120. In some embodiments, the power supply128may power one or more components of the controller104. In some embodiments, the components of the controller104may have independent power supplies (e.g., wall power).

The controller may present an image to a user (e.g., via the input output system132). For example, images may be presented on a monitor, along with various options which may allow the user to control the imaging and/or save/annotate the images. In some embodiments, the system100may capture new images at a relatively high frame rate to allow for video/real-time imaging. For example, the system may update the images on the screen at 4, 8, 16, or 32 Hz. Higher or lower frame rates may be used in other embodiments. In some embodiments, the frame rate may be user adjustable. In some embodiments, the user may choose to capture particular frames for later review/analysis.

The microscope102may capture an image across a particular field of view (FOV). The field of view may represent a portion of the focal surface from which light is captured and directed onto the detector120. In many applications, it may be desirable to capture images from an area which is larger than a single FOV. In some embodiments, this may be accomplished by having the user move the handheld microscope102relative to the sample. In some embodiments, the microscope102may include one or more controls which may be actuated to move the FOV relative to the sample without the need to reposition the microscope102relative to the sample.

In some embodiments, the controller104may mosaic several FOVs together to present an image of a larger region to a user. For example, the PC130may capture images of a first FOV and a second FOV, and then register those images to each other using one or more image registration algorithms. Once registered, the FOVs may be combined and presented to a user as a single image.

In some embodiments, the controller104may perform one or more image processing steps. For example, the PC130may include instructions which may be executed to remove background noise from the images. In some embodiments, the image processing may annotate the image. In some embodiments, the image processing may identify one or more regions of interest (e.g., by analyzing features of the image).

In some embodiments, the optical system100may be portable. For example, the controller104may be packaged onto a mobile platform (e.g., such as surgical cart) for ease of transportation. Accordingly, the system100may be transported near a subject (e.g., into a examination room) and the handheld microscope102may then be used to image particular regions of the subject.

In some embodiments, the microscope102may include a sample holder. For example, a clip or other holder may be used to attach a slide to a front end of the objective112. In some embodiments, the microscope102may not be handheld, and may include a sample holder such as a 3-axis stage which supports a sample. In some embodiments, the stage may be moved by the controller104to capture multiple fields of view.

In some embodiments, different layouts of the microscope102may be used. For example, the microscope102may include a fiber bundle which couples light to and from a distal scanning head which may be positioned near a sample. The scanning module110may produce a focal surface at a proximal end of the fiber bundle (e.g., a substantially flat focal surface) which may also generate a second focal surface at the distal end of the fiber bundle. In some embodiments, one or more components of the scanning module110may be located at a distal end of the fiber bundle.

FIG.2is a schematic diagram of a microscope according to some embodiments of the present disclosure. The microscope200may, in some embodiments, represent an optical path of the optical system100ofFIG.1. For example the microscope200may represent components of the handheld microscope102ofFIG.1, along with certain additional components such as the light source202(e.g., light source124ofFIG.1).

The microscope200includes a light source202, which produces illumination light. For example the light source202may be a laser, which may produce light at an excitation wavelength of a fluorophore in the sample214. The laser may be coupled into an optional optical fiber204, such as a single mode fiber (SMF), to illumination optics206. The illumination optics206may redirect the illumination light towards the scanning module230, and also alter one or more properties of the illumination light. For example, the illumination optics206may reshape the illumination light into a focused illumination beam. The scanning module230may scan the illumination beam across a back focal plane (BFP) of the objective212, which may focus the scanned illumination beam to a front focal plane (FFP) of the objective. For example, the FFP may be positioned in a sample214. The scanning of the illumination beam across the BFP may generate a focal surface at or near the FFP. Light from the sample214(e.g., from the focal surface) may be collected through the objective212to the BFP, and may be directed by the scanning module230through collection optics216. The collection optics216may direct the collected light to a detector224and may alter one or more properties of the collected light (e.g., changing the magnification, filtering, etc.). An optional filter222may limit the light which reaches the detector224to certain wavelengths.

The light source202may generate illumination light. The example microscope200may be a fluorescence confocal microscope. Accordingly, the light source202may generate light which has a spectrum which includes an excitation wavelength of one or more target fluorophores. For example, the light source202may be a laser with a wavelength of about 488 nm. Other types of sources, and other wavelengths of the illumination light, may be used in other example embodiments. In some embodiments, optional filters (not shown) may be used to limit the excitation light to a spectrum centered on the excitation wavelength.

The illumination light may be coupled into the illumination optics206through an optional optical fiber204. The optical fiber204may be useful in embodiments where it is not desirable to have the light source202in close proximity to the illumination optics206. For example, in embodiments such as the handheld microscope102ofFIG.1, it may not be practical (e.g., due to space, weight, and/or heat issues) to place the light source202in the handheld unit. Accordingly, the use of a fiber204may allow the light source202to be placed at a base station, while still coupling the illumination light into the illumination optics206. In some embodiments, the fiber204may be omitted, and the light source202may be directly coupled to the illumination optics206.

The illumination optics206focus the light onto the scanning module230, and also reshape the light into an illumination beam. The example microscope200is a line scan microscope, in which the illumination beam is focused to a line, which is scanned across the focal region of the FFP. For example, the illumination focal line may extend along the x-axis. Although idealized as a line, the illumination focal line may have a non-zero thickness (e.g., in the y-axis). Similarly, while idealized as a line, the illumination focal line may be cursed (e.g., due to the properties of optical elements such as lenses). The illumination focal line may be swept back and forth (e.g., along the y-axis) by the scanning module230to generate a local surface. Other example embodiments may use other shapes of the focus of the illumination beam and other scanning paths (e.g., a point illumination may be raster scanned in the X-Y plane, etc.).

The illumination optics206include a first lens208and a cylindrical lens210. The first lens208may collimate the light from the SMF204. For example, the first lens208may be a spherical lens. In some embodiments, the first lens208may have a focal length of about 9.0 mm. The cylindrical lens210may shape the light into a line focus. For example, the cylindrical lens210may have a focal length of about 12.0 mm. The illumination optics206may generate an illumination beam with a line focus (e.g., an illumination light sheet), which may have a full-width half-maximum (FWHM) length of 1.2 mm and a focusing NA of approximately 0.09. Other dimensions of illumination beam may be used in other example embodiments.

The illumination beam may be directed to the scanning module230. The scanning module includes a MEMS scanning mirror236which acts as the scanning element, as well as alignment mirrors232and234to direct the illumination and collection beams (respectively) into and out of the MEMS scanning mirror236. The alignment mirrors232and234may be tilted mirrors to redirect light to and from the mirror236. The alignment mirrors232and234may, in some embodiments, be adjustable (e.g., either manually and/or with an actuator) and adjustments may be made to align the system. The positioning of scanning elements such as the mirror236may generally be referred to in reference to a neutral position of that element. The view ofFIG.2shows a ‘top-down’ view of a scanning module230where the mirror236is aligned in an x-y plane. However, it should be understood that the mirror may have a neutral position where the mirror does not lie in the X-Y plane. For example, the neutral position of the mirror may be tilted and rather than rotate about the x-axis, the mirror236may rotate about an axis which is within the plane of the surface of the mirror236.

The example microscope200is a dual-axis system, where the illumination and collection light paths share the same scanning module230(and objective212). The MEMS scanning mirror236may be used to scan the illumination beam across a BFP of the objective212, and also direct collected light received through the objective212onto the collection optics216. The MEMS scanning mirror236may have multiple degrees of freedom. For example, the MEMS scanning mirror236may rotate about the x-axis to scan the illumination beam back and forth and may translate (e.g., piston) along the z-axis. The rotation and translation may be coordinated (e.g., by controller104ofFIG.1) to control the shape and/or location of the focal region. An example scanning module is discussed in more detail inFIG.3, and an example operation of a scanning module is discussed in more detail inFIG.4.

One example operation of the scanning module230may be to ‘flatten’ a focal region in the sample214. The rotation of the MEMS scanning mirror236about the x-axis may sweep the illumination light sheet (i.e. line focus) back and forth across the focal region. However, rotation alone may cause certain parts of the mirror to be further closer to the BFP, which may lead to a curved focal surface. However, when coupled with translation along the z-axis, the MEMS scanning mirror236may both change angle (via rotation) to sweep the illumination beam, and the translation may cause the scanned beam focus to remain within a substantially flat focal plane near the BFP. This may lead to a ‘flat’ or ‘planar’ swept focal surface within the sample. It should be understood that while the focal surface may be described and idealized as a surface (e.g., 2D), it may have a non-zero thickness (e.g., a depth of focus) due to the limitations of optics. A substantially flat focal surface may have a curvature which is smaller than the depth of focus across the imaged area. A flat focal surface may be particularly useful in situations when one wishes to image a sample with thin flat structures (e.g., imaging thin tissue layers) or when one wishes to stitch many such images together, while slowly changing the lateral position of each image, to form a larger mosaicked image.

Other shapes of focal surface may also be used, and since the shape of the planar region is controlled by the behavior of the MEMS scanning mirror236, additional shapes may be rapidly achieved by changing the programmed behavior of the MEMS scanning mirror236. For example, the focal surface may be flat-sloped, curved, etc.

In an example operation where flattening of the focal region214is desired, the rotation of the MEMS scanning mirror236may be controlled by a voltage driven in a sine wave fashion. The translation of the MEMS scanning mirror236along the z-axis may be controlled by a voltage varied using Equation 1, below:
Vz=|Asin(ωt+ϕ)|  (1)
Where VZis the driving voltage of the scanning module110along the z-axis (e.g., the axis of the objective212), A is the amplitude of the translational motion along that axis, ω is the frequency of the actuation, and ϕ is used to synchronize the actuation along the z axis to the rotation about a different axis (e.g., the x-axis). Other waveforms may be used to control the operation of the MEMS scanning mirror236in other embodiments.

In some embodiments, the MEMS scanning mirror236may have additional degrees of freedom which may go unused in certain scanning modes. For example, the MEMS scanning mirror236may generally rotate about the x-axis and translate along the z-axis in a ‘flat-field’ scanning mode. However, the MEMS scanning mirror236may have additional degrees of freedom, such as the ability to also rotate about the y-axis. This may be useful in additional imaging modes. For example, if the illumination beam has a point focus, then rotation about the y-axis may be used to raster scan the illumination beam across a focal surface. Other imaging modes with additional degrees of freedom in the MEMS scanning mirror236may have more complicated waveforms to produce various shapes of focal surfaces.

The example microscope200uses a single MEMS scanning mirror236. However, in other embodiments, multiple scanned optical elements may be used to control the operation of the scanning module230. For example, a first mirror may have rotational freedom while a second mirror may have translational freedom. The action of the scanning mirror236may be distributed over any number of elements. Similarly, while the example microscope200has a scanning module230which includes mirrors, other scanning modules230may have other forms of scanning optics.

The objective lens212may focus the illumination beam onto the sample214at a FFP, and receive collected light from the sample214and direct it to the scanning module230. The objective lens212may include a number of individual lenses. The objective lens may package these lenses together into a single optical element. For example, in some embodiments, the objective212may be a commercial objective lens. In some embodiments, the objective lens212may be a relay objective with a 3× magnification.

Inset240shows an expanded view of the illumination and collection light where they are focused at the BFP. Optical axes of the illumination and collection light paths cross at an angle θ (e.g., a half-crossing angle), while each of the light paths represents a beam with an NA represented by the angle α. Inset250shows an expanded view of the illumination and collection beams where they focus at the FFP (e.g., within a sample). The angles θ and α are both increased by the magnification of the objective212and the increased index of refraction of the tissue (e.g., as compared to the air at the BFP). For example, α at the BFP may be about 0.09 rad and 0.20 rad at the FFP, while θ at the BFP may be about 11.0 deg and about 23.9 deg at the FFP.

The collected light may pass back through the objective212and across the scanning module230. The same motion of the scanning module230which scans the illumination light of the target across the focal region may also descan the collected light into the collection optics216. The collection optics216may focus the light on the detector224, and may, in some embodiments, further magnify the image. The collection optics216includes a second lens220and a third lens218. The second lens220may be a spherical-doublet with a focal length of about 12.0 mm, while the third lens218may be a spherical doublet with a focal length of about 60.00 mm. In some embodiments, an additional descanning module may be used to descan the collected light onto the collection optics216. For example, the descanning module may include scanning elements which may be synchronized to the scanning elements of the scanning module230.

In some embodiments, an optional filter222may be used to limit the light reaching the detector224. This may be useful in fluorescence imaging, for example to prevent excitation light from reaching the detector Accordingly, a long pass filter, with a cutoff longer than the illumination wavelength may limit the illumination light reaching the detector. In the example embodiment where the illumination light is a 488 nm laser, a 495 nm long pass filter may be used.

In some embodiments, a confocal aperture, such as a confocal pinhole or slit may be used to limit out of focus light reaching the detector. The use of a confocal aperture may help to reduce the amount of undesirable background light that is detected. In some embodiments, the detector224may act as the confocal aperture. For example, the detector224may be a linear array detector, such as a CMOS linear array detector. The linear array of pixels of the detector may act as a confocal slit. The shape of the confocal aperture may be based, at least in part, on the shape of the focus of the illumination beam.

FIG.3is a schematic diagram of a scanning module according to some embodiments of the present disclosure. The scanning module300may, in some embodiments, be included in the scanning module230ofFIG.2. For the sake of brevity, features and details already explained with respect toFIG.2may not be repeated again with respect toFIG.3.

FIG.3shows a schematic diagram of a scanning module similar to the scanning module230ofFIG.2where a single MEMS scanning mirror302is used for both scanning an illumination beam310and for descanning the collected light320. In order to make the optical path more compact (e.g., so it can fit in a handheld housing), a pair of alignment mirrors306and308are used. The alignment mirror308directs the illumination light310from lens332(e.g., lens210ofFIG.2) onto the MEMS scanning mirror302. The collected light320bounces off the MEMS scanning mirror302and off alignment mirror306through lens322(e.g., lens220ofFIG.2).

Unlike the flattened views of the scanning modules of FIGS. and2,FIG.3shows a layout of the scanning module300in three dimensions. Here, rather than lie in an X-Y plane, the surface of the MEMS scanning mirror302is tilted. For example, the neutral position of the mirror302is rotated around the y-axis such that there is a non-zero angle between the surface of the mirror302and the x-axis. Accordingly, rather than rotate about the x-axis as previously described, the axis of rotation may lie at a tilt with respect to the x-axis (e.g., along axis lying along a centerline of the surface of the mirror302).

The illumination light is focused to a line focus304. The collected light is limited primarily to the signal generated at the line focus304by the confocal slit (e.g., the line detector). The oscillation of the MEMS scanning mirror302about the axis of rotation (as well as the translation of the MEMS scanning mirror along the z-axis) causes the line focus to be swept back and forth across a focal surface305, which may be located at a FFP of the system. The amount of rotation may, in part, determine the size of the focal region305. At a given instant, only the area at the position of line focus304may receive illumination light310and have collected light320collected from that line focus. However, the rapid scanning of the line focus304may effectively illuminate/collect from the entire focal surface305over time, in which an image is generated line by line.

The size and shape of the focal surface305may therefore be defined, in part, by the operation of the MEMS scanning mirror302. The rotation of the mirror about the axis of rotation may move the line focus304back and forth (e.g., along the scanning direction, y-axis. Pistoning of the MEMS mirror302may move the line focus back and forth along the z-axis (e.g., to change a focal depth of the system). Rotation of the MEMS mirror302about the y-axis may cause the line focus to move up and down along the x-axis. These movements may be coordinated in time to achieve various shapes of the focal region305.

In some embodiments, the MEMS scanning mirror302may be used to perform other operations. For example, the position of the focal surface relative to the sample may be changed. The waveforms which actuate the MEMS scanning mirror302may be given a DC offset, which may change the ‘default’ position of the focal surface. For example, a DC offset in the waveform which actuates the translation along the z axis may change a depth of the focal surface within the sample. This may be useful to change a depth of focus of the collected images.

FIGS.4A-4Bare graphs depicting example operations of a scanning mirror according to some embodiments of the present disclosure.FIGS.4A and4Bshow different modes which may be used by a microscope, such as the microscope102ofFIG.1,200ofFIG.2, and/or300ofFIG.3to achieve different shapes of focal region. In particular.FIGS.4A and4Brepresent a comparison of a scanning mode where only rotation of a scanning mirror402about an axis is used to scan a line focus across a focal surface404a, and a mode where both rotation and translation (e.g., pistoning) of the mirror402is used to generate focal surface404b. Graphs410aand410billustrate the shape of the focal surfaces404aand404brespectively in more detail.

Schematic400aofFIG.4Ashows an operational mode where the mirror402rotates about an axis, but does not piston, to generate a focal surface404a. The focal surface404ais curved (relative to a focal plane z0) due to the rotation of the mirror402. In contrast, schematic400ashows an operational mode where the mirror402both rotates (e.g., about the x-axis) and also translates along the z axis. These movements may be coordinated to generate a flat focal surface404b.

As may be seen by comparing graphs410aand410b, the focal surface404ahas substantially increased curvature compared to the focal surface404b. The focal surface404bmay, in some embodiments be substantially flat, whereas the focal surface404amay not be. In some embodiments, the focal surface404bmay have a slight curvature caused by the inherent behavior of the lenses in the system. For example, the focal surface at the back focal plane of the objective (e.g., BFP ofFIG.2) may be flat, but the objective may generate some level of curvature at the front focal plane (e.g., FFP ofFIG.2). In some embodiments, the microscope may have optics designed to correct for this (e.g., a set of objectives, which accounts for the flat focal plane) and the focal surface404bat the FFP may also be planar.

The focal surface404bmay have an amount of curvature which is smaller than a depth of focus of the focal surface404b. In some embodiments, the focal surface404bmay have a curvature which is smaller than a thickness of an imaging target (e.g., may have a smaller curvature than a thickness of a tissue layer).

FIG.5is a set of schematics and images demonstrating an example operation of a microscope according to some embodiments of the present disclosure. In some embodiments,FIG.5may represent operations performed by, and images collected by, one or more of the microscopes described inFIGS.1-4.

FIG.5illustrates the operation of a microscope in a first mode where a curved focal surface is generated and a second mode where a flat focal surface is generated, and compares operation in these modes to show the potential benefits of using a flat focal surface for imaging certain types of samples, such as tissues.

The schematic500ashows an example piece of tissue (e.g., human tissue) which has a layer of mesothelium (e.g., a monolayer of cells) stacked on top of a capsule (e.g., which includes connective tissue, muscles, etc.). It may be desirable to image a large region of the mesothelium, which has a large lateral extent, but is relatively thin (e.g., since the mesothelium is generally a monolayer of cells). The schematic500ashows an idealized imaging plane cutting through the mesothelium.

Schematic500bshows a ‘top-down’ view of the imaging plane from the schematic500a. The image plane of schematic500bshows how multiple fields of view (FOVs) may be combined to allow for imaging of a larger area. Each FOV may represent an area scanned by the microscope. For example, the FOV may represent an area which can be imaged as the illumination beam is scanned back and forth (e.g., an area which can be imaged without moving the sample and microscope relative to each other, adjusting the optics to point to a different area, etc.). In order to image a wider area, mosaicking may be used to combine multiple fields of view. For example, a controller (e.g., computer130ofFIG.1) may automatically combine multiple FOVs by registering the images and finding areas of overlap between the FOVs. However, finding such areas of overlap may be difficult when the FOVs are curved (e.g., due to a curvature of the focal surface).

Schematic500cshows an operational mode where the scanning element rotates, but does not translate (e.g., piston) and produces a curved focal surface. Images500dand500eshow example images of tissue collected using the scanning mode of schematic500c. Schematic500fshows an operational mode where the scanning element both rotates and pistons to generate a generally flat focal surface. Images500gand500hshow example images of tissue collected using the scanning mode of schematic500f.

As may be seen by comparing image500dto image500g, the use of pistoning to generate a flat focal surface may improve the image quality. The curved focal surface of image500dmay dip into the capsule, and thus fail to capture the cellular structure of mesothelium in certain regions. Meanwhile, the flat focal surface of image500gmay stay within the monolayer of the mesothelium and produce more accurate images.

As may be seen by comparing the image500eto the image500h, a flat focal surface may also be useful for mosaicking. The curved sur faces of image500emay tail to capture features which would have added in image registration. Thus the mosaicking of the image500efails. Meanwhile in the image500h, overlapping features are found and the two FOVs are properly registered to each other to generate a mosaic.

FIG.6is a flow chart of a method according to some embodiments of the present disclosure. The method600may, in some embodiments, be implemented by one or more of the microscopes or optical systems described inFIGS.1-5.

The method600may generally begin with box610, which describes generating an illumination beam. An illumination source (e.g., a laser) may generate illumination light. The illumination light may be directed through illumination optics to produce the illumination beam, in some embodiments, an optical fiber may couple the illumination light into the illumination optics. The illumination optics may change one or more properties (e.g., focus, wavelength, direction, etc.) of the illumination light to generate the illumination beam. In some embodiments, the illumination beam may have a line focus.

Box610may generally be followed by box620, which describes scanning the illumination beam with a scanning module to generate a focal surface. For example, the illumination beam may be scanned across as BFP of an objective to generate the focal surface at a FFP of the objective. In some embodiments, the method600may include controlling the motion(s) of the scanning module to determine the shape of the focal surface.

Box620may include box622, which describes rotating one or more scanning elements of the scanning module about a first axis and box624which describes translating one or more scanning elements about a second axis. The first and second axis may, in some embodiments be orthogonal to each other. In some embodiments, the first and second axis may be non-orthogonal. In some embodiments, additional axes of motion may be used. For example, in a point scan microscope, one or more scanning elements may rotate about a third axis. In some embodiments, various motions (e.g., rotation, translation) may be distributed among different ones of the scanning elements. In some embodiments, various motions may be combined into a scanning element. For example, in a point-scanned microscope, one scanning mirror may rotate about the first and third axis (e.g., an x and y axis) while a second scanning mirror may translate (e.g., piston) along the second axis (e.g., a z axis). In some embodiments, the motion of the scanning module may be controlled such that the focal surface is substantially flat (e.g., a flat-field). In some embodiments, the rotation and translation may be performed by different components or by a same component of the scanning element. For example, a MEMS mirror may perform both the rotation and the translation.

Box620may generally be followed by box630, which describes collecting light from the focal surface. For example, light may be collected through the objective, and the scanning module may direct the collected light to collection optics. The scanning module may descan the collected light onto the collection optics. In some embodiments, the collection optics may alter one or more properties of the collected light (e.g., by filtering) before directing the collected light onto a detector In some embodiments, collected light from the objective may be directed to a set of descanning optics separate from the scanning module. The descanning optics may descan the collected light, for example with one or more scanning elements which may be synchronized with the scanning module.

Box630may generally be followed by box640, which describes generating an image based on the collected light. For example, the detector may generate an image based on the collected light. The image may be presented to a user (e.g., on a display) and or saved.

In some embodiments, the method600may include positioning a hand-held unit which includes the scanning element near the sample. In some embodiments, the method600may include capturing multiple fields of view and mosaicking them together.

The example embodiments of the disclosure may be described with respect to a particular implementation or application, however other implementations or applications may also be used. For example, optical processes which were described with respect to lenses may be instead implemented with one or more curved mirrors, spectrums of illumination light described as narrowband sources may be substituted with broadband sources and filters, etc. Similarly, different layouts of the optical system may be used to alter the shape of the microscope to accommodate different constraints (e.g., a different shape of housing may require additional mirrors, fibers, and/or other components to reshape the optical path.

Various components and optical beams may be discussed herein with respect to their idealized behavior. For example, an illumination light sheet may be described as “flat” or “planar” to represent its idealized behavior, how ever it should be understood that such beams will in reality have a non-zero thickness and that the scanned surface may not be perfectly flat.

Of course, it is to be appreciated that any one of the examples, embodiments or processes described herein may be combined with one or more other examples, embodiments and or processes or be separated and/or performed amongst separate dev ices or dev ice portions in accordance with the present systems, devices and methods.