SYSTEMS AND METHODS FOR DETECTING AND IDENTIFYING CELLS

Examples described herein provide systems and methods for quantifying cells. An example method includes receiving at least one image, improving a contrast of the at least one image to generate a contrast image, and performing a fit operation on the contrast image to generate a processed image. The method includes applying a filter to the processed image to generate a filtered image, identifying cells within the filtered image, and providing an output image including an indication of the cells.

FIELD

The present disclosure generally relates to microscope image analysis. More specifically, the present disclosure relates to detecting and identifying cells in microscope images.

BACKGROUND

Cell viability counting is traditionally a manual, time-intensive process. Advances in the field of image processing have enabled imaging systems to automate some of these manual tasks or otherwise to reduce the amount of time and manual effort associated with determining cellular concentrations in a sample. Existing automated systems and methods for analyzing cell viability within a sample, however, suffer from many drawbacks. For example, many cell viability systems require use of dyes, labels, or other compounds to determine the viability of cells within a sample. The use of many of these compounds often require specialized, expensive, and/or bulky equipment to retrieve a readout of the results.

SUMMARY

Examples described herein receive microscope image of cells as input and output a list of cell locations. The image of cells may be associated with Raman measurements, fluorescence measurements, or the like. To convert the image of cells to a quantitative description of a list of cells, example systems and methods include performing operations such as attribute morphology, exponential histogram fit for image auto-thresholding, connected component identification, watershed segmentation, and combinations thereof.

One example provides a method of identifying cells. The method includes receiving at least one image, improving a contrast of the at least one image to generate a contrast image, and performing a fit operation on the contrast image to generate a processed image. The method includes applying a filter to the processed image to generate a filtered image, identifying cells within the filtered image, and providing an output image including an indication of the cells.

Another examples provides one or more hardware storage devices storing instructions executable by one or more processing devices of an imaging system, the instructions including receiving at least one image, improving a contrast of the at least one image to generate a contrast image, and performing a fit operation on the contrast image to generate a processed image. The instructions include applying a filter to the processed image to generate a filtered image, identifying cells within the filtered image, and providing an output image including an indication of the cells.

Another example provides an imaging apparatus including a stage assembly operable to receive a cell counting slide, and a controller including an electronic processor and a memory. The controller is configured to capture the cell counting slide to generate at least one image, improve a contrast of the at least one image to generate a contrast image, and perform a fit operation on the contrast image to generate a processed image. The controller is configured to apply a filter to the processed image to generate a filtered image, identify cells within the filtered image, and provide an output image including an indication of the cells.

DETAILED DESCRIPTION

Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. In case of conflict, the present document, including definitions, will control. Example methods and systems are described below, although methods and systems similar or equivalent to those described herein can be used in practice or testing of the present disclosure. All publications, patent applications, patents and other references mentioned herein are incorporated by reference in their entirety. The systems, methods, and examples disclosed herein are illustrative only and not intended to be limiting.

The terms “comprise(s),” “include(s),” “having,” “has,” “can,” “contain(s),” and variants thereof, as used herein, are intended to be open-ended transitional phrases, terms, or words that do not preclude the possibility of additional acts or structures. The singular forms “a,” “an” and “the” include plural references unless the context clearly dictates otherwise.

In addition, unless otherwise indicated, numbers expressing quantities, constituents, distances, or other measurements used in the specification and claims are to be understood as being modified by the term “about.” The terms “about,” “approximately,” “substantially,” or their equivalents, represent an amount or condition close to the specific stated amount or condition that still performs a desired function or achieves a desired result. For example, the terms “approximately,” “about,” and “substantially” may refer to an amount or condition that deviates by less than 10%, or by less than 5%, or by less than 1%, or by less than 0.1%, or by less than 0.01% from a specifically stated amount or condition.

The present disclosure is described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numbers of specific details are set forth in order to provide an improved understanding of the present disclosure. It may be evident, however, that the systems and methods of the present disclosure may be practiced without one or more of these specific details. In other instances, well-known structures and devices are shown in block diagram form to facilitate describing the systems and methods of the present disclosure. There is no specific requirement that a system, method, or technique relating to microscope image analysis include all of the details characterized herein to obtain some benefit according to the present disclosure. Thus, the specific examples characterized herein are meant to be example applications of the techniques described and alternatives are possible.

Cell viability counting was traditionally a manual, time-intensive process. Advances in the field of image processing have enabled imaging systems to automate some of these manual tasks or otherwise to reduce the amount of time and manual effort associated with determining cellular concentrations in a sample, particularly with respect to ascertaining the proportional number of live/dead cells (or viability counting) within a given sample.

Existing automated systems and methods for analyzing cell viability within a sample, however, suffer from many drawbacks. For example, many cell viability systems require use of dyes, labels, or other compounds to determine the viability of cells within a sample. The use of many of these compounds often require specialized, expensive, and/or bulky equipment to retrieve a readout of the results. As such, the equipment is unlikely to be readily available and/or positioned within the lab space as to make it accessible or convenient to use. Also, systems and methods utilizing machine learning techniques require large training datasets and can be computationally expensive.

Accordingly, to address these and other issues, the systems and methods disclosed herein provide accurate and automatic conversion of an image of cells to a quantitative description of a list of cells. In this manner, cell viability counting is provided automatically and without the manual, time-intensive process.

FIG.1is a perspective view of an imaging system100configured to perform one or more of the methods disclosed herein. The imaging system100ofFIG.1is operable to facilitate the method associated with flow diagram200of automated cell viability count disclosed by the example flow diagram ofFIG.2. As shown, the imaging system100includes a housing102, which encloses and protects the microscope and computing systems used for cell viability counts. The housing102includes a slide port/stage assembly106operable to receive a cell counting slide into the imaging system (at step202). Once received therein, the imaging system100determines a target focus position for imaging cells on the cell counting slide (at step204) and performs automated cell viability counting using the target focus position (at step206). A representation of the cell viability count is displayed (at step208) at the imaging system100using, for example, display104. This representation and/or other data associated with the automated cell viability count can be removed from the imaging system100and/or saved to a separate device through user interaction with the communications module108, which in some embodiments can include a USB port or other data exchange port known in the art.

One will appreciate, in view of the present disclosure, that the principles described herein may be implemented utilizing any suitable imaging system and/or any suitable imaging modality. The specific examples of imaging systems and imaging modalities discussed herein are provided by way of example and as a means of describing the features of the disclosed embodiments. Thus, the embodiments disclosed herein are not limited to any particular microscopy system or microscopy application and may be implemented in various contexts, such as brightfield imaging, fluorescence microscopy, flow cytometry, confocal imaging (e.g.,3D confocal imaging, or any type of3D imaging), and/or others. For example, principles discussed herein may be implemented with flow cytometry systems to provide or improve cell counting capabilities. As another example, cell count and/or viability data obtained in accordance with techniques of the present disclosure may be used to supplement fluorescence data to improve accuracy in distinguishing among different cells.

Furthermore, one will appreciate, in view of the present disclosure, that any number of principles described herein may be implemented in various fields. For example, a system may implement the cell counting techniques discussed herein without necessarily also implementing techniques such as cell viability determination.

FIG.3is a schematic of various example components within the imaging system100ofFIG.1, in accordance with one or more embodiments of the present disclosure. As illustrated inFIG.3, the imaging system100may include a computer system110and a microscopy system120included therewith.FIG.3conceptually represents the computer system110and the microscopy system120as disposed within the housing102of the imaging system100. However, one will appreciate, in view of the present disclosure, that any portion of the computer system110or the microscopy system120may be disposed at least partially outside of the housing102within the scope of the disclosed embodiments.

FIG.3shows that the computer system110of the imaging system100can comprise various components, such as electronic processor(s)112, hardware storage device(s)114, controller(s)116, and communications module(s)108.

The electronic processor(s)112may comprise one or more sets of electronic circuitry that include any number of logic units, registers, and/or control units to facilitate the execution of computer-readable instructions (e.g., instructions that form a computer program). Such computer-readable instructions may be stored within the hardware storage device(s)114, which may comprise physical system memory and which may be volatile, non-volatile, or some combination thereof.

The controller(s)116may comprise any suitable software components (e.g., set of computer-executable instructions) and/or hardware components (e.g., an application-specific integrated circuit, or other special-purpose hardware component(s)) operable to control one or more physical apparatuses of the imaging system100, such as portions of the microscopy system120(e.g., the positioning mechanism(s)128).

The communication module(s)108may comprise any combination of software or hardware components that are operable to facilitate communication between on-system components/devices and/or with off-system components/devices. For example, the communications module(s)108may comprise ports, buses, or other physical connection apparatuses for communicating with other devices (e.g., USB port, SD card reader, and/or other apparatus). Additionally, or alternatively, the communications module(s)108may comprise systems operable to communicate wirelessly with external systems and/or devices through any suitable communication channel(s), such as, by way of non-limiting example, Bluetooth, ultra-wideband, WLAN, infrared communication, and/or others.

As shown inFIG.3, the imaging system100includes a microscopy system120having an image sensor122, an illumination source124, an optical train126, the slide port/stage assembly106for receiving the sample slide, and a positioning mechanism128.

The image sensor122is positioned in the optical path of the microscopy system and configured to capture images of the samples, which will be used in the disclosed methods to identify a target focus position and subsequently for performing automated cell viability counting. As used herein, the term “image sensor” or “camera” refers to any applicable image sensor compatible with the apparatuses, systems and methods described herein, including but not limited to charge-coupled devices, complementary metal-oxide-semiconductor devices, N-type metal-oxide-semiconductor devices, Quanta Image Sensors, combinations of the foregoing such as scientific complementary metal-oxide-semiconductor devices, and the like.

The optical train126may include one or more optical elements configured to facilitate viewing of the cell counting slide by directing light from the illumination source124toward the received cell counting slide. The optical train126may also be configured to direct light scattered, reflected, and/or emitted by a specimen within the cell counting slide toward the image sensor122. The illumination source124may be configured to emit various types of light, such as white light or light of one or more particular wavelength bands. For example, the illumination source124can include a light cube, which may be installed and/or exchanged within the housing to any of a desired set of illumination wavelengths.

The positioning mechanism128can include any of an x-axis motor, a y-axis motor, and a z-axis motor that are operable to adjust the components of the optical train126and/or image sensor122, accordingly.

FIG.3furthermore illustrates that, in some instances, the imaging system100includes a display104.FIG.3indicates that the display104may be in communication, whether directly or indirectly, with various other components of the imaging system100, such as the computer system110or the microscopy system120thereof (e.g., as indicated inFIG.3by the tri-headed arrow). For example, the imaging system may capture images using components of the microscopy system120and captured images may be processed and/or stored using components of the computer system110(e.g., electronic processor(s)112, hardware storage device(s)114, etc.), and the processed and/or stored images may be displayed on the display104for observation by one or more users.

As described herein, the components of the imaging system100may facilitate cell viability counting on the sample contained on the cell counting slide. In some instances, a representation of results of cell viability counting may be displayed on the display104of the imaging system100within a short time period after initiating cell viability counting processing for a cell counting slide inserted into the imaging system100(e.g., within a time period of about 20 seconds or less, or within about 10 seconds or less).

One will appreciate, in view of the present disclosure, that an imaging system may comprise additional or alternative components relative to those shown and described with reference toFIG.3, and that such components may be organized and/or distributed in various manners.

FIG.4is a flowchart of an example method400for performing automated cell viability counting. The method400may be performed, for example, by the controller(s)116, the electronic processor(s)112, or a combination thereof.

The method400includes receiving an at least one image (for example, an input image) (at step402). For example, the image sensor122captures an image of a sample. In another example, an image is acquired from, for example, a server or a memory device. Accordingly, the image may be a previously-captured image.

The method400includes improving the contrast of the at least one image to generate a contrast image (at step404) by, for example, performing area attribute operations (e.g., area attribute opening, area attribute closing), greyscale operations, and the like.

The method400includes performing a fit operation on the contrast image to generate a processed image (at step406). For example, as described below in more detail with respect toFIGS.8A-8D, an exponential histogram fit operation may be performed on the contrast image (e.g., the improved-contrast image). Other fit operations may alternatively be performed, such as a curve fit operation. In another example, a Gaussian function may be implemented for the histogram fit operation, as defined by Equation (1):

f⁡(x)=1σ⁢2⁢π⁢e-(x-μ)2/(2⁢σ2)Equation⁢(1)Where:σ is the standard deviation; andμ is the weighted average.

The method400includes applying a filter to the processed image to generate a filtered image (at step408). For example, one or more of a binary opening operation followed by a binary closing operation with square kernel of 2×2 pixels, a binary mask, a median filter, a low-pass filter, and the like may be applied to the third image to remove noise, trim tendrils, and fill small holes within the third image.

The method400includes identifying cells within the filtered image (at step410). For example, a connected component operation (e.g., an eight-connected component operation) may be performed to identify cells within the filtered image, as described below in more detail.

The method400includes providing an output image including an indication of cells (at step412). For example, a representation of the cells within the input image is displayed at the imaging system100using, for example, display104. In this manner, cell viability count is performed.

FIG.5is a flowchart of one example implementation of the method400for performing automated cell viability counting. The method500may be performed, for example, by the controller(s)116, the electronic processor(s)112, or a combination thereof.

The method500includes receiving an input image (at step502). For example, the image sensor122captures an image of a sample.FIG.6Aillustrates an example input image600discussed with respect to the method500.FIG.6Billustrates a highlighted portion650of the input image600in greater detail. The input image600may be received by the controller(s)116.

The method500includes performing an area attribute grayscale opening operation on the input image600to generate a second image (at step504). In some implementations, the area attribute opening is performed with a radius of size between 7.4 μm and 7.8 μm. In some implementations, the area attribute opening is performed with a radius of size 7.6 μm.

The method500includes performing an area attribute grayscale closing operation on the second image to generate a third image (at step506). In some implementations, the area attribute closing is performed with an area between 4 pixels and 6 pixels. In some implementations, the area attribute closing is performed with an area of 5 pixels.

Step504, step506, or the combination thereof increases the contrast of the input image600. For example,FIG.7Aillustrates an example third image700.FIG.7Billustrates a highlighted portion750of the third image700in greater detail. The third image700has increased contrast between the background and foreground compared to the input image600.

Returning toFIG.5, the method500includes performing an exponential histogram fit operation on the third image700to generate a fourth image (at step508). For example,FIGS.8A-8Dillustrate histograms representing the third image700during a histogram fit operation. The histograms illustrate the relationship between pixel count and pixel intensity.FIG.8Aillustrates a histogram of the third image700counting numbers of pixels at each of a plurality of pixel intensity values (grey scale).FIG.8Billustrates the histogram ofFIG.8Anormalized by total pixel count (i.e., divide the number of pixels for each intensity value by the total number of pixels).FIG.8Cillustrates the normalized histogram ofFIG.8Bwith identified bin values greater than a trim rate.FIG.8Dillustrates the histogram ofFIG.8Cwith an overlaid exponential function and normalized by maximum bin value. The exponential function is provided by

where tau is the fitting parameter of the exponential function. Tau may be empirically defined for each individual histogram. In the example ofFIG.8D, the tau value is 0.654612.

In some implementations, the log trim rate is between a value of −1 and −2. In the examples ofFIGS.8A-8D, the log trim rate is −1.3. Accordingly, to identify the trim rate:

The trim rate limits the bins used in the analysis to a select number of top bins (i.e., intensity values having the highest number of counts) in the histogram. For example, in the above example, a trim rate of 0.05 indicates that 5% of the pixels in the lowest bins should be trimmed. Accordingly, to apply the trim rate, the normalized number of pixels in each bin (each intensity value) starting from the smallest bin (i.e., the intensity value with the smallest normalized count) is added until the sum exceeds the trim rate. As illustrated for the example bins inFIG.8C, the normalized number of pixels starting with bin7are summed. The sum of bins7,6, and5is less than the trim rate (0.05), but the sum of bins7,8,5, and4(0.09) is greater than the trim rate (0.05). Thus, in this example, bins1,2,3, and4are retained but bins7,6, and5are trimmed (ignored). As illustrated inFIG.8D, the retained bins are normalized by dividing each retained bin by the maximum value among the retained bins before performing the exponential histogram fit operation. In some implementations, the exponential histogram fit operation has a log false-alarm rate of between −1 and −7. In the examples ofFIGS.8A-8D, the log false-alarm rate is −4.2. The log false-alarm rate is used to determine a global threshold value:

In the example ofFIGS.8A-8D, where tau=0.654612, the global threshold is 6.3307.

Automatic thresholding by histogram fitting provides overall robustness as compared to other methodologies. Histogram fitting is unique for each input image, and accordingly the global threshold is adapted for each input image.

Returning toFIG.5, the method500includes applying the global threshold on the fourth image to generate a fifth image (at step510). For example, the global threshold may be applied to apply a binary mask to highlight areas of interest within the fourth image. In other words, every pixel having a value above the threshold to set to one value and all other pixels are set to a different value.FIG.9Aillustrates an example fifth image900including highlighted areas of interest.FIG.9Billustrates a highlighted portion950of the fifth image900in greater detail

As illustrated inFIG.5, the method500includes applying a small binary morphology operation on the fifth image900to generate a sixth image (at step512). For example, a small binary morphology operation may be performed to remove background noise within the fifth image900, as well as trim tendrils around the areas of interest and fill small holes within the fifth image900.FIG.10Aillustrates an example sixth image1000following the small binary morphology operation.FIG.10Billustrates a highlighted portion1050of the sixth image1000in greater detail. Example of binary morphology operation includes a morphological opening followed by a morphological closing with square kernel of 2×2 pixels for both opening and closing operators.

The small binary morphology operation assists with removing foreground holes. Foreground holes may appear in images based on the appearance of cells from different focus positions (e.g., the different z-height focus positions from which the image is captured by image sensor122). Thus, in some instances, foreground hole fill operations may be performed to expand connected pixels in a way that fills the foreground holes. In some instances, after performing morphological operations on the image (e.g., the small binary morphology operation), the imaging system100may define the connected components within the image in preparation for additional processing.

The method500includes applying a watershed operation on the sixth image1000to generate a seventh image and segment cell candidates (at step514). For example, two or more cells within the sixth image1000may be touching. The watershed operation may be implemented to identify and logically separate the touching cells.

The method500includes applying an eight-connected component operation on the seventh image to identify cell candidates (at step516). For example, connected components within the seventh image may be identified to label candidate cells within the seventh image. In some implementations, a four-connected component operation is performed.FIG.11Aillustrates an example seventh image1100including identified cell candidates.FIG.11Billustrates a highlighted portion1150of the seventh image1100in greater detail.

As used herein, “connectedness” as used in “connected components,” refers to which pixels are considered neighbors of a pixel of interest. A connected component is a set of pixels of a single value, for example, the value representing black, wherein a path can be formed from any pixel of the set to any other pixel in the set without leaving the set, for example, by traversing only black pixels. In general terms, a connected component may be either “four-connected” or “eight-connected.” In the four-connected case, the path can move in only horizontal or vertical directions, so there are four possible directions. In the eight-connected case, the path between pixels may also proceed diagonally.

The method500includes determining acceptable cell candidates based on the contrast and length of the cell candidates (at step518). For example, the contrast of the cell candidates and the length of the cell candidates can be compared to a contrast threshold and a length threshold, respectively. The contrast threshold may be, for example, between 15 and 25 bytes. The length threshold may be, for example, a minor-axis length between 1.5 μm and 3.0 μm.FIG.12Aillustrates an eighth image1200including accepted cell candidates.FIG.12Billustrates a highlighted portion1250of the eighth image1200in greater detail. In the example ofFIG.12A, the contrast threshold is 20 bytes, and the length threshold is 2.3 μm.

The method500includes outputting an image (e.g., the eighth image1200) including the accepted cell candidates (at step520). For example, an indication or representation of the cells within the input image is displayed (e.g., by the controller(s)116) at the imaging system100using, for example, display104. In this manner, cell viability count is performed. Cells may be identified by being highlighted, circled, indicated with an ellipse, or the like. The ellipses may be fit to each connected component defined within the eighth image1200. In some instances, the controller(s)116provide data regarding the accepted cell candidates on the display1104, such as an ellipse center coordinate (in pixels), an ellipse semi-major axis (in pixels), an ellipse semi-minor axis (in pixels), an ellipse angle (in degrees), cell viability (for example, whether cells are dead or alive), cell brightness (for example, average grayscale intensity within the ellipse), circularity of the cells (for example, ranging from 0 to 1, where 1 is a perfect circle), and the like. It should be understood that the identified cells and associated cell count information may be output in various formats and forms. For example, information regarding identified cells may be presented graphically (e.g., as generally illustrated inFIGS.12A-12B), textually, or a combination thereof.

As described above in the detailed description, reference is made to the accompanying drawings that form a part hereof wherein like numerals designate like parts throughout, and in which is shown, by way of illustration, implementations that may be practiced. It is to be understood that other implementations may be utilized, and structured or logical changes may be made, without departing from the scope of the present disclosure. Therefore, the detailed description as described above is not to be taken in a limiting sense.