Patent Description:
Light microscopy (e.g., fluorescence microscopy) often involves the parallel analysis of several species of emitters, e.g., different fluorophores, which have a distinct fluorescence emission spectrum or fluorescence lifetime, or the same fluorophore bound to different target molecules resulting in an altered emission spectrum or lifetime. In addition, in STED microscopy, where the sample is illuminated by a spatial intensity distribution of STED light, the emitters display different fluorescence lifetimes depending on their location relative to the local minimum of the STED light intensity distribution.

The amount of light generated by each species of the emitters for each location in the sample can be assigned to a specific channel which can be displayed as an image representing the distribution of the respective emitter species in the sample. In case of scanning microscopy, each of these locations are represented by a pixel of the scanning image.

However, different emitter species often co-exist at the same location in the sample, which makes it difficult to determine the relative contribution of the species to the light detected for a certain location.

In case of different emission spectra, a combined spectrum results for each pixel, and determining the relative contribution of the detected light to each channel (spectral unmixing) is often complicated, especially since reference spectra of the pure components are sometimes not available or differ from the actual spectra under the conditions in the sample.

Likewise, in fluorescence lifetime imaging, mixed species with different lifetimes result in multiexponential decay curves. Obtaining the lifetimes of the pure components from these curves is often difficult and prone to errors and fitting artefacts.

In certain cases, lifetime components may be separated by time-gated detection, but this technique reduces the signal-to-noise ratio since a fraction of the detected photons are simply cut off from the data.

An approach for unmixing of lifetime components based on time gating (<NPL>) involves fast, hardware-implemented binning of photon arrival time data to generate several subsequent time gates. From the photon counts in each time gate, a lifetime component distribution is obtained and fitted with a multi-exponential function. The user can select and allocate certain time gates to lifetime channels from the component distribution to separate the signals from different dyes ("TauSeparation") or visualize the lifetime distribution in an imaged cell ("TauScan").

However, this technique requires dedicated hardware. In addition, valuable time information is lost by the binning/time gating process, which may be problematic especially when the difference of the two species in lifetimes or spectra is relatively small.

Another powerful solution to signal unmixing in fluorescence microscopy while preserving the signal-to-noise ratio is phasor analysis, which has been described both for lifetime imaging (<NPL>)) and spectral imaging (<NPL>)).

In phasor analysis, the measured spectra (in case of spectral imaging) or fluorescence time traces (in case of lifetime imaging) for each pixel are Fourier transformed and the real and imaginary parts of the Fourier coefficients are displayed in a two-dimensional phasor plot, where each pixel is represented by a vector.

Signals from a single emitter species are localized around a single location on the phasor plot, and mixed signals from two species are scattered around a line connecting the locations of the pure species (channels). There is a linear dependence between the location of a pixel relative to this line and the contribution of the detected light to a channel representing a species of emitters. , if <NUM> % of the light detected at a pixel belongs to channel A and <NUM> % to channel B, the pixel is located at the center of the line, mid-way between the points representing pure channel A and pure channel B. In case of three channels, the pixels are located within a triangle formed by the pure emitter species, and the relative contributions of the three channels to the light detected for a pixel may be determined, e.g., by calculating the normalized barycentric coefficients of the pixel relative to the three points.

An example of three-component phasor plot analysis is described in the article "<NPL>. According to this publication, the relative contribution of a lifetime species can be analyzed in the phasor plot graphically by drawing a line from one corner of a triangle representing the lifetime of a first component through the phasor pixel and projecting this line onto the connection between the corners representing the other two components. The relative contribution of the first component can be deduced from the distance of the phasor pixel from the first component relative to the distance between the component and the line intersection.

Phasor analysis has also been used to improve the resolution in STED microscopy by a method termed "STED-SPLIT" (<NPL>)). According to this approach, time traces are acquired for each pixel during STED microscopy, Fourier transformed and displayed in a phasor plot. Based on the phasor plot, the data is then split into a first group of pixels from a central region of low STED intensity representing high-resolution information and a second group of pixels from a peripheral region of the diffraction limited PSF which are exposed to higher STED intensities and thus represent lower resolution information (so-called "SPLIT algorithm") In addition, uncorrelated background fluorescence may be removed using the STED-SPLIT method.

In a variant of this method termed "mSTED" (modulation-enhanced STED), STED images are acquired while modulating the power of the STED beam, and fluorescence intensity values are determined for each pixel and each STED power (<NPL>), <CIT>). Fourier coefficients are then calculated from the stack of images acquired at different STED powers. Due to the increasing resolution as a result of the increasing STED power, additional spatial resolution is encoded in this series of images, which can be used to apply the SPLIT algorithm and improve the resolution of the final processed STED image. An advantage of the mSTED method is that it does not require time-resolved detection of fluorescence.

Phasor analysis has been included in a number of commercial microscopy software packages for spectral imaging and fluorescence lifetime analysis. In the corresponding graphical user interfaces, the phasor plot is displayed and users are sometimes able to select and separately display partial data sets representing spectral or lifetime channels.

However, understanding the concept of the phasor plot and effectively working with phasor plots to analyze microscopy data requires background knowledge and experience. Therefore, many unexperienced users are reluctant to use these tools or do not exploit their full potential.

Thus, the problem underlying the present invention is to provide a method for analyzing microscopy data involving determining the relative contribution of different species of emitters to detected emittance light in an intuitive and comprehensive manner.

This objective is attained by the subject matter of the independent claims <NUM> (method), <NUM> (device), and <NUM> (computer program). Embodiments of the invention are specified in dependent claims <NUM> to <NUM> and claim <NUM> and described hereafter.

A first aspect of the invention relates to a method for analyzing microscopy data, as defined with appended independent claim <NUM>, comprising - inter alia and as further defined with claim <NUM> - the steps of receiving, by a processor, microscopy data obtained from a sample comprising a plurality of emitters which emit emittance light in response to excitation light, wherein the microscopy data comprise emittance light intensities for a plurality of pixels, determining, by the processor, for each of the pixels a relative contribution to a plurality of channels representing emittance light emitted by different species of the emitters, displaying, by a display device, the microscopy data in at least one histogram comprising a first axis representing the relative contributions to at least one of the channels, displaying by the display device a manipulation tool, receiving, by an input device, a user input indicating a manipulation of the manipulation tool to select a subset of the microscopy data from the histogram or to adjust the relative contribution to the channels, and displaying, by the display device, at least one image based on the subset of the microscopy data that have been selected based on the histogram and/or the relative contributions to the channels that have been adjusted based on the histogram.

The histogram displays numbers of pixels assigned to a plurality of classes indicating respective ranges or bins of the relative contribution to the plurality of channels. In particular, pixels which have the same relative contribution to a channel are counted and depicted in the histogram as a number of occurrences in certain ranges of the relative contribution.

The relative contribution to the channels is adjusted by determining weights based on the relative contributions to the channels and scaling the intensities of the pixels by the weights.

By displaying a histogram of the microscopy data on a scale representing the relative contribution to the channels, an intuitive and comprehensive tool for data selection and allocation to channels is provided. The histogram representation of the microscopy data is easily understood also by inexperienced users, e.g., compared to a phasor plot representation.

In particular, the emitters in the sample are fluorophores, which are excited by excitation light to emit fluorescence light. For example, the emitters may be fluorescent dye molecules, which are, e.g., conjugated to antibodies or similar molecules which bind to target biomolecules in a cell, or fluorescent proteins, which may be covalently or non-covalently linked to the target molecules.

The pixels of the microscopy data may be obtained by scanning microscopy, e.g., by scanning a focused excitation light beam through the sample and de-scanning the fluorescence light, as known from confocal laser scanning microscopy. In other words, each pixel may relate to a scanning position of the excitation light and/or the detection light.

The expression 'relative contribution to a channel' as used herein describes the amount of the detected emission light signal of a respective pixel stemming from a certain emitter species constituting the channel. For instance, <NUM> % of the light of a given pixel may have been generated by fluorophore A (first channel) and <NUM>% by fluorophore B (second channel), and <NUM>% by background light. The relative contribution may be expressed in per cent but may also be displayed in various other analogous ways, e.g., real numbers between <NUM> and <NUM>.

For pixels comprising measured light from two or more emitters, the relative contribution to a channel may be determined by an unmixing algorithm, such as, e.g., phasor analysis or multi-exponential fitting.

The microscopy data are displayed in a histogram comprising a first axis representing the relative contributions. The term 'histogram' as used herein should be construed broadly as a diagram displaying the absolute or relative occurrence of events (e.g., photon counts) relative to the first axis. For example, histograms are commonly displayed as column diagrams, wherein each column represents the occurrence of events in a certain bin. However, other analogous representations are possible within the scope of the invention, e.g., a continuous line plot representing a fitting function of the absolute or relative occurrence values.

The first axis displaying the relative contributions may be scaled, e.g., by per cent or numbers between <NUM> and <NUM>, as discussed above. However, other units representing the relative contribution to a channel are also possible. For instance, for lifetime data, the relative contribution to a channel may be expressed in the unit time (expressing an apparent lifetime or average lifetime of the data). Furthermore, in case of most lifetime data sets, the phase angle of the data points in a phasor plot also indicates the relative contribution to a channel in many cases and can thus be used as the unit of the first axis, in case data from phasor analysis are available. Furthermore, in certain situations (e.g., the mSTED method described above), phasors are distributed on a straight line through the origin of the phasor plot. In this case, the phasor radius/amplitude indicates the relative contribution to a channel and can thus be used as the unit of the first axis.

Various implementations of the manipulation tool are envisioned within the scope of the present invention. Examples include movable grid points or sliders on a graphical user interface. Alternatively, the manipulation tool may also be implemented, e.g., as one or several entry fields, in which numbers indicating certain ranges of the microscopy data in the histogram may be entered.

In certain embodiments, the first axis comprises a linear scale.

In certain embodiments, the microscopy data have been obtained by or the method comprises generating, in a focal plane in or on the sample, a first light distribution comprising excitation light and a second light distribution comprising depletion or switching light, wherein the depletion or switching light is capable of transferring the emitters from an active state, in which the emitters emit light in response to the excitation light to an inactive state, in which the emitters do not emit light in response to the excitation light or emit less light in response to the excitation light, and wherein the second light distribution comprises at least one local intensity minimum adjacent to at least one intensity maximum. Examples of this embodiment include STED microscopy and RESOLFT microscopy. For example, the second light distribution may comprise a single local minimum, such as, e.g., for a donut-shaped or bottle-beam-shaped light distribution, or the second light distribution may comprise several local minima, such as e.g., in a standing wave pattern. The first light distribution particularly comprises a local maximum which is aligned with the local minimum of the second light distribution.

In certain embodiments, the microscopy data comprise, for each of a plurality of pixels, a respective series of emittance light intensities changing in response to a change of a first parameter. For example, the series of emittance light intensities may be a time series or a stack of images (particularly from the same focal plane and the same area of the sample) obtained at different light intensities (e.g., excitation light intensities and/or depletion/switching light intensities).

In certain embodiments, a plot of the series of emittance light intensities relative to the first parameter is displayed, wherein a selection tool (which is particularly comprised in the manipulation tool) configured to select at least one subset of the microscopy data according to a sub-range of the first parameter upon receiving user input is displayed, particularly wherein the relative contribution of the pixels to the channels is determined based on the at least one selected subset of the microscopy data. For example, in a time series displaying a fluorescence decay obtained by pulsed STED microscopy, a first sub-range of the time axis corresponding to the time of a STED pulse and a second sub-range corresponding to the time after the STED pulse may be defined. These sub-ranges may be separately evaluated, since the decay rate is significantly faster during the STED pulse than after the STED pulse.

In certain embodiments, a second parameter of the microscopy data is obtained from at least two selected subsets, wherein particularly the contribution of the pixels to the channels is determined based on the second parameter. In case of a time series, the second parameter may be, e.g., a decay rate or a lifetime.

In certain embodiments, for each of a plurality of the pixels, a ratio between the second parameters from at least two of the selected subsets is obtained, wherein particularly the contribution of the pixels to the channels is determined based on the second parameter or the ratio. , a ratio between the decay rate of the subset measured during the STED pulse and the decay rate of the subset measured after the STED pulse may be obtained.

In certain embodiments, the first parameter is a time. In other words, according to this embodiment, the microscopy data comprise a respective time trace of emittance light intensities for each pixel.

In certain embodiments, the different species of the emitters are characterized by different emittance lifetimes. Therein, the different species may have a different molecular structure, such as e.g., for different dye molecules. Alternatively, the different species may be identical molecules but may be exposed to different conditions in the sample, e.g., due to their subcellular location in a biological cell.

The term "lifetime" as used herein describes a characteristic time parameter describing a decay of light emission of an ensemble of emitters, or, in case of single emitters, a characteristic time after which a single emitter emits light with a certain probability.

In certain embodiments, the emitters belonging to the different species are located within different areas or volumes around the local minimum (or a respective local minimum in case of more than one minimum) of the second light distribution, such that the emitters of the different species have different distances from the local minimum, wherein particularly the depletion or switching light results in different emittance lifetimes of the emitters of the different species. In this situation, an improved spatial resolution may be obtained, e.g., by applying the STED-SPLIT algorithm (see reference cited above).

In certain embodiments, the series of emittance light intensities has been obtained (or is obtained) by illuminating the sample with the second light distribution at varying intensities of the depletion or switching light, wherein the first parameter is an intensity of the depletion or switching light. , based on a phasor analysis of a stack of images obtained at varying light intensities of depletion or switching light, the resolution may be improved by the SPLIT algorithm as previously described for the mSTED method (see reference cited above).

In certain embodiments, the emitters of the different species comprise different emission spectra. Therein, in particular, determining the relative contribution to the channels is performed by a spectral unmixing algorithm.

In certain embodiments, the method further comprises illuminating the sample with a plurality of excitation pulses of the excitation light to excite the at least one emitter, detecting a plurality of photons emitted by the at least one emitter in response to the excitation pulses, and determining an arrival time relative to a preceding excitation pulse for each of the photons. In particular, the method further comprises determining the series of emittance light intensities from a histogram of the determined arrival times (which is different from the histogram of the microscopy data comprising a first axis representing the relative contributions to the channels). In particular, a time trace of photon counts is determined from the arrival times for each pixel. The displayed histogram comprising the first axis is particularly generated based on the time traces for each pixel, for example by generating a phasor plot from the time traces and performing linear unmixing in the phasor plot.

In certain embodiments, the method comprises illuminating the sample with time-modulated excitation light to generate a time-modulated emittance signal. The time-modulated emittance signal can be used, e.g., for lifetime analysis, particularly using phasor analysis.

In certain embodiments, the relative contribution of the pixels to the channels is determined by a phasor analysis of the microscopy data, or by means of a fitting function, particularly an exponential function or a sum of exponential functions, particularly of the series of emittance intensities.

In certain embodiments, the relative contribution of the pixels to the channels is determined by generating a phasor plot from the microscopy data, wherein a data point is displayed in the phasor plot for each pixel.

In certain embodiments, the data points are orthogonally projected onto a line connecting a first point on the phasor plot representing a first channel (of the above-described plurality of channels) and a second point on the phasor plot representing a second channel (of the above-described plurality of channels), wherein the relative contribution of a respective pixel to the first channel is determined from the distance of the respective projected data point from the first point divided by the distance between the first point and the second point, and/or the relative contribution of the respective pixel to the second channel is determined as the distance of the respective projected data point from the second point divided by the distance between the first point and the second point. This process may also be designated herein as 'linear unmixing'. The first point and the second point may be phasors representing the pure first and second emitter species. These may be determined experimentally, e.g., from samples containing only a single fluorophore, or may be derived from the phasor plot, e.g., by defining end points of the phasor distribution, particularly end points on a semicircle representing mono-exponential decays. Alternatively, arbitrary end points may be defined by the user.

In certain embodiments, the phasor plot is be displayed in addition to the histogram representing the relative contribution to the channels, particularly on the graphical user interface, wherein particularly a further manipulation tool is displayed, wherein an analysis of the phasor data is performed based on user inputs indicating a manipulation of the further manipulation tool. Alternatively, the phasor analysis may be performed automatically in the background without displaying the phasor plot. In particular, the further manipulation tool is configured to define and/or arrange a line in the phasor plot to determine the relative contribution to the first channel and/or the second channel (more particularly as described above).

When defining a line in the displayed phasor plot, either all phasors or a subset of phasors may be included in the phasor analysis. For example, to include all phasors in the orthogonal projection on the line, the phasors located further outside the line may be projected on an elongation of the line. Alternatively, only the phasors which can be orthogonally projected on the displayed line segment may be used in the analysis and other phasors may be discarded.

In certain embodiments, normalized barycentric coordinates are determined for each data point with respect to a first point on the phasor plot representing a first channel (of the above-described plurality of channels), a second point on the phasor plot representing a second channel (of the above-described plurality of channels), and a third point on the phasor plot representing a third channel (of the above-described plurality of channels), wherein the relative contribution of the pixels to the first channel, the second channel and the third channel is determined from the normalized barycentric coordinates.

In certain embodiments, a partial section of the first axis is selected in response to the user input indicating the manipulation of the manipulation tool, wherein the subset of the microscopy data selected by the manipulation tool comprises values of the relative contribution to at least one of the channels in a range defined by the partial section of the first axis.

The method further comprises determining weights, based on the relative contributions to the channels, wherein the intensities of the pixels of the at least one image are scaled by the weights, wherein particularly a second axis representing the weights is displayed, particularly on the graphical user interface.

In certain embodiments, at least one function defining the weights is adjusted by the user input indicating the manipulation of the manipulation tool, particularly wherein the at least one function comprises a first function which is monotonously increasing and a second function which is monotonously decreasing. In particular, the at least one function (particularly the first function and/or the second function) is a linear function, a section-wise linear function, a polynomial or a logistic function. In certain embodiments, a plot of the at least one function (particularly the first function and/or the second function) is displayed relative to the first axis and/or the second axis, particularly by the graphical user interface.

In certain embodiments, the manipulation of the manipulation tool comprises selecting and/or moving at least one grid point, wherein the at least one grid point is displayed relative to the at least one function. In particular, the at least one grid point defines the at least one function.

In certain embodiments, the relative contribution for each pixel to the channels is determined by an artificial intelligence module, wherein the artificial intelligence module has been trained based on a training data set comprising microscopy data and corresponding relative contributions to the channels. The artificial intelligence module operates based on an artificial intelligence algorithm involving, e.g., an artificial neural network, a support vector machine or similar means. In the training data set, the actual relative contributions of each pixel to the channels may be obtained by independent methods.

In certain embodiments, a subset of the microscopy data is selected from the histogram or the relative contribution to the channels is adjusted by an artificial intelligence module. In this manner, initial parameters may be provided which may be further modified by the user as needed.

In certain embodiments, the artificial intelligence module is further trained based on user inputs manipulating the manipulation tool and corresponding images.

A second aspect of the invention relates to a device for analyzing microscopy data, for executing the method according to the first aspect, as defined with appended independent claim <NUM>, wherein the device comprises - inter alia and as further defined with claim <NUM> - a processor configured to receive microscopy data obtained from a sample comprising a plurality of emitters which emit emittance light in response to excitation light, wherein the microscopy data comprise emittance light intensities for a plurality of pixels, wherein the processor is further configured to determine a relative contribution to a plurality of channels (i.e., at least two channels) representing emittance light emitted by different species of the emitters for each of the pixels, a display device configured to display the microscopy data in at least one histogram comprising a first axis representing the relative contributions to the channels, wherein the display device is further configured to display a manipulation tool. The device further comprises an input device configured to receive a user input indicating a manipulation of the manipulation tool to select of a subset of the microscopy data from the histogram and/or to adjust the relative contribution to the channels, wherein the display device is configured to display at least one image based on the subset of the microscopy data that have been selected based on the histogram and/or the relative contributions to the channels that have been adjusted based on the histogram.

In particular, the processor is configured to adjust the relative contribution to the channels based on the user input received by the input device.

The histogram displays numbers of pixels assigned to a plurality of classes indicating respective ranges of the relative contribution to the plurality of channels. In particular, pixels which have the same relative contribution to a channel are counted and depicted in the histogram as a number of occurrences in certain ranges of the relative contribution.

The processor is configured to adjust the relative contribution to the channels by determining weights based on the relative contributions to the channels and scaling the intensities of the pixels by the weights.

In certain embodiments, the processor comprises an artificial intelligence module configured to determine the relative contribution for each pixel to the channels and/or select a subset of the microscopy data from the histogram and/or adjust the relative contribution to the channels, wherein the artificial intelligence module is configured to be trained or is trained based on a training data set comprising microscopy data and corresponding relative contributions to the channels, wherein particularly the artificial intelligence module is further configured to be trained or is trained based on user inputs manipulating the manipulation tool and corresponding images.

An aspect (which may be combined with any of the other aspects of the present invention as defined with the independent claims) relates to an artificial intelligence module configured to receive microscopy data obtained from a sample comprising a plurality of emitters which emit emittance light in response to excitation light, wherein the microscopy data comprise emittance light intensities for a plurality of pixels, and wherein the artificial intelligence module is configured to determine for each of the pixels a relative contribution to a plurality of channels representing emittance light emitted by different species of the emitters. In particular, the artificial intelligence module is configured to be trained or is trained based on a training data set comprising microscopy data and corresponding relative contributions to the channels and/or trained based on user inputs manipulating the manipulation tool and corresponding images.

In certain embodiments, the sample is stained with a plurality of fluorophores having distinct fluorescence lifetimes, particularly wherein the fluorophores comprise similar emission spectra and/or excitation spectra.

In certain embodiments, the microscopy data are obtained by fluorescence lifetime imaging (FLIM).

In certain embodiments, the artificial intelligence module is configured to implement an artificial intelligence algorithm, particularly involving an artificial neural network, a support vector machine or a cluster analysis algorithm.

In certain embodiments, the artificial intelligence module is configured to allocate the microscopy data into a plurality of clusters.

In certain embodiments, each of the clusters represents an emittance lifetime (e.g., associated with a certain emitter species).

In certain embodiments, the microscopy data comprises a time series displaying a fluorescence decay obtained by pulsed STED microscopy, wherein the artificial intelligence module is configured to analyze the microscopy data from a first sub-range of a time axis corresponding to the time of a STED pulse and/or a second sub-range of the time axis corresponding to the time after the STED pulse. In particular, the artificial intelligence module is configured to define the first sub-range and/or the second sub-range.

Based on the result generated by the artificial intelligence module, the channels may be displayed as separate images or overlaid in one image.

According to an embodiment of the invention as defined with appended dependent claim <NUM>, there is provided a microscope comprising a device for analyzing microscopic data according to appended independent claim <NUM>.

A further aspect of the invention relates to a computer program according to appended claim <NUM>.

The invention is only limited by the scope of the appended claims.

Further embodiments of the invention may be derived from the claims, the description and the drawings, limited only by the scope of the appended claims.

Further embodiments may be drawn from features stated in the description or derivable from the drawings which may be singly or cumulatively applied, limited only be the scope of the appended claims.

The invention is further elucidated and described hereafter with reference to the exemplary embodiments displayed in the figures. These embodiments are non-restrictive examples which are not meant to limit the scope of the invention which is defined in the appended claims.

<FIG> is a schematic diagram showing four subsequent steps (<FIG>) of the method according to an embodiment of the present invention, specifically involving fluorescence lifetime analysis.

<FIG> illustrates microscopy data <NUM> comprising a plurality of pixels <NUM> (e.g., from a confocal scan of the sample <NUM>) each containing a time trace of emittance light intensities relative to an initial excitation light pulse. Although only <NUM> pixels <NUM> are shown for simplicity, the actual number of pixels <NUM> may be much higher.

The time traces may be obtained, e.g., by recording arrival times of single photons emitted by emitters <NUM> in the sample <NUM> relative to a corresponding excitation light pulse. For detection, e.g., a confocal point detector, such as an avalanche photodiode or a photomultiplier, coupled to evaluation electronics (e.g., a time-correlated single photon counting module, TCSPC) may be used. The evaluation electronics typically generate a photon histogram indicating the frequency of occurrence of photon detection events versus time.

These time traces display a fluorescence decay curve, which when light from more than one emitter species contributes to the respective pixel, typically follows multiexponential decay kinetics.

In the next step of the example according to <FIG>, a phasor analysis is performed. In other words, Fourier coefficients are calculated from the time traces, resulting in a complex phasor having a real part g and an imaginary part s for each pixel <NUM>. These phasors may be plotted in a phasor plot <NUM> as shown in <FIG>. The phasor plot <NUM> comprises data points <NUM>, wherein each data point <NUM> represents a pixel <NUM>. Each data point <NUM> comprises a phase angle φ and a radius r. The fluorescence decay of the pure first species of emitters <NUM> (associated with the first channel A) is represented by a first point (or phasor) <NUM>, and the decay of the pure second species associated with channel B is represented by a second point (or phasor) <NUM>. These points <NUM>, <NUM> are depicted on a semi-circle of radius <NUM>,<NUM> and an origin at the coordinates (<NUM>,<NUM>;<NUM>), meaning that the pure first and second species decay with mono exponential kinetics. Of course, certain fluorophores may also exhibit multiexponential decays, and the corresponding phasors would therefore be located inside of the semicircle.

In the data set shown in <FIG>, the data points/phasors <NUM> scatter around a line <NUM> connecting the first point <NUM> and the second point <NUM>, wherein data points <NUM> comprising light from both emitter species, and thus characterized by a mixture of the corresponding lifetimes, would be expected to be located on the line, wherein the relative contribution to each of the species, i.e., the first channel A and the second channel B can be determined from the distance of the respective data point <NUM> from the first point <NUM> and the second point <NUM>. , a pixel comprising <NUM>% of signal from the first and second emitter species would result in a data point <NUM> located on the line <NUM> located half-way between the first point <NUM> and the second point <NUM>. A pixel comprising <NUM>% of signal from the first emitter species and <NUM>% of signal from the second emitter species, would result in a data point <NUM> located at a distance from the first point <NUM> equal to <NUM>% of the total distance between the first point <NUM> and the second point <NUM>.

A possible linear unmixing procedure to determine the relative contributions to the first channel A and the second channel B includes projecting the data points <NUM> on the line <NUM>, measuring the distance of the projection from the first point <NUM> and the second point <NUM>, and determining the relative contribution to the channels A, B from the measured distance.

An allocation to a first channel A, a second channel B and a third channel C may be performed based on a phasor plot <NUM> as shown in <FIG> by determining the barycentric coefficients of each data point/phasor <NUM> with respect to the first point <NUM>, the second point <NUM> and a third point <NUM> forming a triangle in the phasor plot <NUM>. Essentially, determining the barycentric coefficient for a given data point <NUM> with respect to channel A is equivalent to calculating the fraction between the area of the partial triangle formed by the data point <NUM>, the second point <NUM> and the third point <NUM> and the total area of the triangle formed by the first point <NUM>, the second point <NUM> and the third point <NUM>. The other barycentric coefficients relative to the second channel B and the third channel C can be determined in a similar manner using the areas of the corresponding partial triangles.

<FIG> shows a histogram <NUM> of the microscopy data <NUM> plotted against a first axis <NUM> representing the relative contribution of the microscopy data <NUM> to the first channel A and the second channel B. In the depicted example, the histogram <NUM> was derived from the phasor plot <NUM> shown in <FIG>, and the first axis <NUM> represents the distances of the data points <NUM> from the first point <NUM> and the second point <NUM> on the line <NUM>.

The histogram <NUM> is displayed on a graphical user interface together with a manipulation tool <NUM>. According to the example illustrated in <FIG>, the manipulation tool <NUM> comprises two partial segments 11a, 11b of the first axis <NUM>, the length of which can be changed by a user input, particularly by means of dragging (e.g., by a computer-mouse drag-and-drop operation or by a swipe on a touchscreen) the grid points <NUM> along the first axis <NUM>. The two partial segments 11a, 11b represent the data points <NUM> of the histogram <NUM> to be displayed for the first channel A and the second channel B, respectively. The first partial segment 11a and the second partial segment 11b may be selected and deselected using check boxes <NUM>. For instance, if only the check box <NUM> below the first partial segment 11a is activated, only the selection of data points <NUM> allocated to the first channel A is used and an image is only generated from the selected data points <NUM>.

<FIG> shows a first image <NUM> representing the first channel A and a second image <NUM> representing the second channel B, wherein the first image <NUM> and the second image <NUM> are generated from the data points <NUM> selected using the manipulation tool <NUM>, using the stored scanning positions of the respective pixels <NUM>.

As schematically indicated in <FIG>, the selection results in separation of first structures <NUM> in the sample <NUM> labeled by the first species of emitters <NUM> (first image <NUM>) and second structures <NUM> labeled by the second species of emitters <NUM> (second image <NUM>). The first image <NUM> and the second image <NUM> may also be overlaid to form a combination image which simultaneously displays the information from the first channel A and the second channel B. To this end, e.g., the information from the two different channels may be color-coded.

<FIG> shows further embodiments of histograms <NUM> and selection tools <NUM> which can be used in the method according to the invention. <FIG> show manipulation tools <NUM> configured to select data points and adjust the weights determining the relative contribution only with respect to the first channel A. In contrast <FIG> - <FIG> display cases where selection and adjustment with respect to both the first channel A and the second channel B may be achieved by the manipulation tool <NUM>. In general, in these cases, the selection and adjustment with respect to the first channel A and the second channel B may be performed independently, or, e.g., the adjustment/selection of points with respect to one of the channels may be performed according to a user input via the manipulation tool <NUM>, whereas the selection/adjustment with respect to the remaining channel (or channels) is performed automatically based on the selection/adjustment with respect to the first channel A.

In all depicted histograms <NUM>, a first function <NUM> and/or a second function <NUM> is plotted above the histogram <NUM>, wherein the functions <NUM>, <NUM> define weights which are used to determine the relative contribution of the data points to the first channel A and the second channel B, respectively. In the histograms <NUM> shown in <FIG>, only the first function <NUM> is displayed. In all depicted examples, grid points <NUM> are provided along the plot of the first function <NUM> and/or the second function <NUM> as part of the manipulation tool <NUM>, wherein the grid points <NUM> may be moved by a user input (e.g., by a drag-and drop operation or by a swipe on a touchscreen as described above).

The functions <NUM>, <NUM> may be linear or stepwise linear functions as displayed in <FIG> or non-linear functions as shown in <FIG>.

<FIG> shows a microscope <NUM> comprising an embodiment of the device <NUM> for analyzing microscopy data <NUM> according to the invention.

The microscope <NUM> is a laser scanning microscope comprising a first light source <NUM> (e.g., an excitation laser) and an optional second light source <NUM>. The beams from the first light source <NUM> and the second light source <NUM> are combined by a beam combiner <NUM>.

The light from the first light source <NUM> and optionally the second light source <NUM> is focused into a sample <NUM> comprising emitters <NUM> which are excited by the light produced by the first light source <NUM>, and emit, e.g., fluorescence light when excited.

The light beams generated by the first light source <NUM> and the second light source <NUM> may be scanned over/through the sample <NUM> by a scanning device <NUM>, e.g., a galvanometric scanner.

A beam shaping device <NUM> (e.g., a phase plate or a spatial light modulator) may be used to generate a light distribution with a local minimum, e.g., a donut-shaped light distribution, from the light generated by the second light source <NUM> at the focus in the sample <NUM>. To this end, additional polarizing elements may be provided in the beam path (not shown). If the second light source <NUM> generates depletion light or switching light (e.g., STED light), this setup may be used to achieve super-resolution.

Alternatively, e.g., the second light source <NUM> may provide excitation light of a different wavelength than the first light source <NUM>, e.g., to simultaneously excite two different species of emitters <NUM> in the sample <NUM>.

The light (e.g., fluorescence light) emitted by the emitters <NUM> in the sample <NUM> is picked up through the objective <NUM> and de-scanned by the scanning device <NUM>. The emitted light is then separated from the excitation light (and optionally depletion or switching light) by a dichroic mirror <NUM> and picked up by a detector <NUM>, e.g., a confocal point detector such as an avalanche photodiode or a photomultiplier.

The detector <NUM> is coupled to a processor <NUM> (provided as a single processing unit or several connected processing units), which is configured to determine a relative contribution to the channels A,B, particularly the first channel A representing emittance light emitted by a first species of the emitters <NUM> and/or the second channel B representing emittance light emitted by a second species of the emitters <NUM> for each of the pixels <NUM>. To this end, the processor <NUM> is optionally configured to count single photons arriving at the detector <NUM>, determine arrival times of individual photons relative to a corresponding excitation pulse, and/or evaluate lifetime data or STED microscopy data as described above. For instance, the processor <NUM> may be configured to perform phasor analysis of the microscopy data <NUM> received from the detector <NUM> and/or configured to generate a histogram <NUM> comprising the first axis <NUM> representing the relative contributions to the first channel A and/or the second channel B.

Furthermore, the device <NUM> comprises a display device <NUM> and an input device <NUM> which are both connected to the processor <NUM>.

The display device <NUM> is configured to display the microscopy data <NUM> in the histogram <NUM> and display the manipulation tool <NUM>. The input device <NUM> is configured to receive a user input indicating a manipulation of the manipulation tool <NUM> to select of a subset of the microscopy data <NUM> from the histogram <NUM> and/or to adjust the relative contribution to the channels A, B. The display device <NUM> is configured to display the images <NUM>,<NUM>, particularly the first image <NUM> based on the selected subset of the microscopy data <NUM> and/or the adjusted relative contributions to the first channel A and/or a second image <NUM> based on the selected subset of the microscopy data <NUM> and/or the adjusted relative contributions to the second channel B.

Claim 1:
A method for analyzing microscopy data (<NUM>), comprising the steps of
a. receiving, by a processor (<NUM>), microscopy data (<NUM>) obtained from a sample (<NUM>) comprising a plurality of emitters (<NUM>) which emit emittance light in response to excitation light, wherein the microscopy data (<NUM>) comprise emittance light intensities for a plurality of pixels (<NUM>),
b. determining, by the processor (<NUM>) for each of the pixels (<NUM>) a relative contribution to a plurality of channels (A, B, C) representing emittance light emitted by different species of the emitters (<NUM>),
c. displaying, by a display device (<NUM>), the microscopy data (<NUM>) in at least one histogram (<NUM>) comprising a first axis (<NUM>) representing the relative contributions to at least one of the channels (A, B, C), wherein the histogram (<NUM>) displays numbers of pixels (<NUM>) assigned to a plurality of classes indicating respective ranges of the relative contribution to the plurality of channels (A, B, C),
d. displaying, by the display device (<NUM>), a manipulation tool (<NUM>),
e. receiving, by an input device (<NUM>), a user input indicating a manipulation of the manipulation tool (<NUM>) to select a subset of the microscopy data (<NUM>) from the histogram (<NUM>) or to adjust the relative contribution to the channels (A, B, C) by determining weights based on the relative contributions to the channels (A, B, C), and scaling the intensities of the pixels (<NUM>) by the weights, and
f. displaying, by the display device (<NUM>) at least one image (<NUM>, <NUM>) based on the selected subset of the microscopy data (<NUM>) and/or the adjusted relative contributions to the channels (A, B, C).