Abstract:
The invention discloses a system utilized in a scanning microscope and a method for providing user guidance. The system for providing user guidance comprises an illumination source for producing a light beam and optical means for shaping and guiding the light beam. A scanning device scans the light beam across a sample. At least one detector is used to detect the fluorescent or reflected light from the sample. The position signal of the light beam on the sample is also detected. A control and processing unit with digitizing means processes the intensity data received from a sample and generates an online representation of various image quality parameters to be observed by the user on a display of a computer.

Description:
RELATED APPLICATIONS  
       [0001]    This application is a Continuation of patent application Ser. No. 09/476,588 which was filed on Dec. 31, 1999 and is incorporated by reference herein in its entirety. 
     
    
     
       FIELD OF THE INVENTION  
         [0002]    The present invention generally relates to a method and system for user guidance in the field of scanning microscopy. More particularly, the invention relates to a user guidance system and method allowing an inexperienced user to operate a scanning microscope successfully.  
         BACKGROUND OF THE INVENTION  
         [0003]    In confocal microscopy a specimen is scanned with a focused laser beam. The focus of the laser beam is moved in a section plane of a specimen by tilting two scan mirrors around their respective axes. The axes of the scan mirrors are perpendicular to each other. For example one scan mirror diverts the laser light in the x-direction and the other scan mirror diverts the laser light in the y-direction. The intensity of the reflected or the fluorescent light is measured for each scanning point. Each measured intensity value relates to an x, y and z-position of the specimen, therefore, providing a user with a three-dimensional image of the specimen.  
           [0004]    “Tutorial on Practical Confocal Microscopy and Use of the Confocal Test Specimen” by Victoria Centozone and James Pawley (the Handbook of Confocal Microscopy, edited by James B. Pawley, Plenum Press, New York, 1995, pp. 549-569) contains an overview of the parameters affecting the process of image capturing in confocal microscopy. To utilize a confocal microscope properly, it is necessary for a user to be familiar with basic concepts and principles of operation of a confocal microscope. Numerous parameters affecting the image quality and proper settings of a confocal microscope are essential to good image quality and have to be set by the user. Presently, no system furnishes a user with suggestions regarding automatic parameter settings of a confocal microscope providing an optimal parameter set.  
         SUMMARY OF THE INVENTION  
         [0005]    It is, therefore, an object of the present invention to provide a scanning microscope with a user guidance system allowing an inexperienced user to efficiently use the scanning microscope. The user guidance system enables a user to view optimal quality images in a short period of time. This object of the invention is achieved by a system for user guidance in scanning microscopy. The system comprises a control and processing unit having digitizing means for providing online digital data of scan signals. The signals are formed by fluorescent or reflected light from a sample. Additionally, a light beam scanning the sample generates a position signal. A control and processing means generated the relevant image quality parameters to be displayed to the user. The system also comprises computer means for image processing, and a display unit for displaying information about the relevant image quality parameters to the user.  
           [0006]    It is also an object of the present invention to provide a scanning microscope with a user guiding system for allowing inexperienced users to efficiently use the scanning microscope after a short period of time. This object of the invention is accomplished by a scanning microscope with a user guidance system comprising an illumination source generating a light beam, optical means for shaping and guiding the light form the illumination source. Also provided is a scanning device for scanning a light beam across a sample and at least one detector registering fluorescent and/or reflected light from the sample. The system also comprises registration means for determining the position signal of the light beam on the surface of the sample, a control and processing unit with digitizing means for generating online digital signals of scan data. The signals are generated by fluorescent or reflected light from the sample, the signals including the position signal of the light beam on the surface of the sample. Control and processing means of the present invention provides online relevant image quality parameters, computer means performs data processing, and a display unit displays visualized information about the plurality of relevant image quality parameters.  
           [0007]    Another object of the present invention is to provide a user guidance method enabling inexperienced users to operate a scanning microscope and obtain high quality images without the need to make adjustments to the microscope settings. This object is accomplished by a method comprising collecting sequential image data of a first and a second image of a sample, statistically processing the data independently of the image structure and location. The method further comprises calculating a plurality of relevant image quality parameters and using the calculated image quality parameters for subsequent image registration.  
           [0008]    A further object of the present invention is to provide a user guidance method allowing inexperienced users to obtain high quality images in a confocal microscope without the need to make adjustments to the microscope settings. That object is accomplished by a guidance method comprising recording sequential image signals of a first and a second image of a sample, wherein the image signals correspond to fluorescent or reflected light from a sample. A related pixel position signal corresponds to a position of the scanning beam on the surface of the sample. The method further comprises online digitizing of the sequentially registered image signals, statistically processing the received data independently of the image structure and location, calculating a plurality of relevant image quality parameters and using the calculated image quality parameters for subsequent image registration.  
           [0009]    In scanning microscopy a specimen is scanned with a suitably focused laser beam. The reflected or fluorescent light propagating from the specimen is detected by a suitable detector. Particularly in confocal scanning microscopy, there are many parameters affecting the quality of a resulting image, therefore, making it desirable to have a way to control such parameters to receive the optimal image result. Examples of such parameters are a bleaching rate of a sample, a bleaching rate of sample dyes, background noise, over- and undersampling, over- and underflow of detectors, saturation, a scanning rate, and the dimensions of the scanned part of the sample. Getting a perfect image in a scanning microscope is often a difficult task not only for an inexperienced microscope user, but also for a user with a lot of experience in operating such a microscope. When analyzing fluorescent samples, a user has to take into account that the fluorochromes of the specimen bleach when illuminated. Plus, the user has to avoid saturation effects occurring when the specimen is illuminated with a high intensity light beam.  
           [0010]    The present invention displays the main image quality parameters, therefore helping inexperienced and experienced users to adjust interactively the optimal microscope settings. Such microscope settings may include the degree of noise in the image, the actual bleaching rate and the degree of saturation as well as the actual setting of the scanning speed, scanning rate, size of the scanned part of the specimen, over- or undersampling. The present invention utilizes the intensities of two sequentially taken images to determine important image quantity and quality parameters. For example, one possibility of such determination is to measure the bleaching rate by detecting and analyzing the shift of the center of the two intensity diagrams H(I 1 ) and H(I 2 ).  
           [0011]    Another way of determining the bleaching rate is to plot a diagram of intensities with respect to the corresponding points of two different images. Intensity pairs (I 1 ,I 2 ) of a sample with no bleaching will fall on a straight line with a slope of 45°. Intensity pairs of a sample with bleaching will fall on a straight line having a higher slope. A straight line represents an ideal case unaffected by noise. In reality the intensity pair data (I 1 ,I 2 ) will be scattered around the ideal line. The distribution of the intensity pair data (I 1 ,I 2 ) in a cross section perpendicular to the ideal line allows the determination of the actual degree of noise. In most cases this distribution will be a Gaussor Poisson distribution.  
           [0012]    To determine the saturation behavior of the specimen dyes, the illumination intensity setting will have to be changed after obtaining the first image. If the ratio of intensities corresponding to two different images stays the same, it means there is no saturation. If the intensity ratio changes, then it means that there is saturation and the degree of saturation can be calculated from the changes of the intensity ratio.  
           [0013]    Over- and undersampling can be calculated from the chosen settings.  
           [0014]    In one embodiment of the present invention, the calculated settings will be automatically saved and applied to forming subsequent images. In another embodiment of the invention various image quality parameters are displayed to a user, so that the user can utilize the displayed parameters to adjust the microscope settings. The displayed parameters are preferably presented to the user visualization in the form of graphical indicators, for example, such as a scroll-bar.  
           [0015]    The user guidance method and system of the present invention use the approach of digitizing signals received from a specimen and from a detector as soon as possible in order to accelerate processing of the data by programmable digital logic (for example, a field programmable gate array FPGA). The advantage of the FPGA electronics is its ability to processing data up to a nanosecond accuracy in real time. 
       
    
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0016]    The subject matter of the present invention is described with reference to the embodiments shown in the following drawings.  
         [0017]    [0017]FIG. 1 is a schematic illustration of a confocal microscope and a system for achieving three dimensional specimen images.  
         [0018]    [0018]FIG. 2 is a schematic illustration of a calculation method for displaying the pixel intensity of sequential images in a histogram.  
         [0019]    [0019]FIG. 3 is a schematic illustration of a calculation method comparing the frequency of pixel intensities of sequential images.  
         [0020]    [0020]FIG. 4 is a schematic illustration of a method for determining the image noise.  
         [0021]    [0021]FIG. 5 is a schematic illustration of an embodiment of a visual user interface.  
         [0022]    [0022]FIG. 6 is a schematic illustration of an embodiment of the visual user interface in more detail.  
         [0023]    [0023]FIG. 7 is a graph illustrating a cross talk between two detection channels. 
     
    
     DETAILED DESCRIPTION OF THE INVENTION  
       [0024]    [0024]FIG. 1 shows a schematic illustration of a confocal microscope system reducing the complexity of parameter settings and image quality adjustments and eliminating the need of extensive use of the complex microscope user manuals. An illumination system  2  generates a light beam L. A beam splitter  4  divides the incident light beam L into a first path L 1  and into a second path L 2 . The light from the illumination system is directed to a scanning device  6  along the first path L 1 . Scanning device  6  comprises a scanning mirror system  7  moveable in such a way that the light propagating along the first path L 1  is scanned across a specimen  10 . The light propagating along the first path L 1  passes through an optical system  8  before reaching specimen  10 . The light reflected from the specimen travels along the first path L 1  back to beam splitter  4  and then to a first detector  12  positioned to receive the light reflected from the specimen  10 . First detector  12  converts that light into a first electrical signal I, wherein the first signal I is proportional to the intensity of the light reflected from the specimen. A position signal P generated by scanning device  6  is fed to a second input port  162  via an electrical connection  17 . The embodiment illustrated in FIG. 1 shows two different analog signals I and P which are fed to a control and processing unit  16  via ports  16   1  and  16   2 , respectively. These distorted and disturbed incoming analog signals I and P are converted into corrected digital signals in control and processing unit  16 . The digital signals are then sent to a computer  18  for image processing and for displaying an image on a display unit  20 . Display unit  20  provides a user with a simultaneous visual presentation of the relevant image quality parameters, such as, for example, the bleaching rate of specimen  10 , saturation behavior, over- an undersampling, pinhole-size, etc. Control and processing unit  16  is implemented with a plurality of FPGA-units (Field Programmable Gate Array). To implement online processing of scan signals, the analog signals I and P are digitized as soon as possible and the resulting digital data are processed by programmable digital logic, allowing real time processing with a nanosecond accuracy.  
         [0025]    An embodiment of a process implemented in control and processing unit  16  is shown schematically in FIG. 2. Pixel intensities of two sequentially captured images are displayed as a histogram. A first image B(t 1 ) is captured at time t 1  and a second image B(t 2 ) is captured at time t 2 . The signals corresponding to the images are fed to control and processing unit  16  and to computer  18 . The measured pixel intensities of the first and second images B(t 1 ) and B(t 2 ) are plotted as a histogram  22 . The abscissa of each data point represents an intensity I and the ordinate represents the frequency of the intensity H(I). The measured pixel data (intensities H(I)) are sent to control and processing unit  16  and to computer  18  to run the necessary calculations. The process of plotting the intensity histograms is used to calculate the bleaching rate from the histogram data. A first histogram H(I 1 ) is obtained from the pixel intensities of the first image B(t 1 ) and a second histogram H(I 2 ) is obtained from the pixel intensities of the second image B(t 2 ). For each of the first and the second histograms H(I 1 ) and H(I 2 ), a center of gravity representing the average pixel intensity of every image B(t 1 ) and B(t 2 ) is calculated. Therefore, the existing scan parameters have to be taken into consideration. For example, detector  12  may be operated near the overflow region or near the registration limit. (Those facts are taken into account and displayed to the user.) The center of gravity I of a histogram is calculated according to equation (1):  
             I   =       ∑       H        (   I   )          I         ∑     H        (   I   )                   (   1   )                               
 
         [0026]    The center of gravity I 1  of the first histogram H(I 1 ) of the first image B(t 1 ) may be at a different location as compared to the center of gravity I 2  of the second histogram H(I 2 ) of the second image B(t 2 ). A bleach factor B can be calculated from the shift of the center of gravity according to equation (2):  
             B   =     1   -       I   1       I   2                 (   2   )                               
 
         [0027]    If the light intensity of a scan differs from one scan to the next, the center of gravity shift of a histogram allows one to calculate the saturation of the specimen dyes. To calculate the saturation of the specimen dyes, a previously calculated bleaching rate may be taken into account.  
         [0028]    Another method of determining the bleaching rate and image noise is schematically illustrated in FIG. 3. The method compares the pixel intensity frequencies of sequentially captured a first image B(t 1 ) and a second image B(t 2 ). According to this method, the pixel intensities I 1  of the first image B(t 1 ) captured at time t 1  and the pixel intensities I 2  of the second image B(t 2 ) captured at time t 2  are determined and the corresponding image signals are fed to control and processing unit  16  and computer  18 . The pixel intensities corresponding to the first and second images B(t 1 ) and B(t 2 ) are presented as a graph  24 . The abscissa of that graph represents the pixel intensity I 2  of the second image B(t 2 ), the ordinate represents the intensity I 1  of first image B(t 1 ) at the same pixel. A graph corresponding to a sample without bleaching is a straight line  30  (solid) having a slope of 45° with respect to the abscissa. In contrast, a sample with bleaching produces a straight line  32  (dashed), having a slope larger than 45° with respect to the abscissa. It has to be noted, of course, that lines  30  and  32  can be produced only in a case of perfect measurements, while in reality the pixel intensity measurements are affected by noise.  
         [0029]    A method for determining the image background noise is schematically illustrated in FIG. 4. A straight line  40  corresponds to an ideal case with no noise. In reality the intensity pair data (I 1 ,I 2 ) corresponding to the pixel intensities I 1  and I 2  of the two images B(t 1 ) and B(t 2 ) are scattered around straight line  40 . A distribution  42  of the intensity pair data (I 1 ,I 2 ) in a cross section perpendicular to straight line  40  can be related to the actual degree of noise. In most cases distribution  42  is a Gauss or Poisson distribution  44 . The standard deviation represents the average pixel noise, which is displayed to a user. In another embodiment of the invention the diagram shown in FIG. 4 is displayed to a user on display  20 . According to the described method, a user can determine the existing noise from the width of distribution  42 , plus, the user can directly determine the bleaching rate from the slope of distribution  42 .  
         [0030]    In one embodiment of the invention a user can adjust the scanning parameters of the microscope while observing the corresponding image quality parameters on display  20 , as shown in FIG. 5. The parameters can be displayed as a graphical window  50  of any size located anywhere on the display.  
         [0031]    A more detailed view of graphical window  50  is shown in FIG. 6. Graphical window  50  displays the actual value of various parameters and a plurality of click buttons  65 ,  66 ,  67  and  68  in individual sections  61 ,  62 ,  63  and  64 . The noise of the specimen is displayed as a diagram  70  in the first section  61 . The height of a noise-bar  71  corresponds to the degree of noise within the range from 0% to 100% of the measured pixel intensities. The bleaching of the sample is displayed in the second section  62 . The bleaching is presented as a diagram  73  in which the height of a bleaching-bar  74  corresponds to the degree of a bleaching within the range from 0% to 10% per 1 scan of the scanning beam across the specimen. The saturation of the sample is displayed in the third section  63  by a number showing the saturation in per cent. Click button  67  is also displayed in the third section  63  of the graphical window  50 . Click button  67  has an “optimize” label which opens an additional window  69 . Additional window  69  provides the user with an explanation and a hint regarding how to adjust the parameters to a lower level of saturation. In a particular embodiment shown in FIG. 6, the explanation reads as “Useful illumination intensity is limited by the dye. This will reduce resolution.” and the hint reads as “Use lower power for less saturation.” The sampling settings of the specimen are monitored and displayed in the fourth section  64 . The current sampling setting is displayed as a highlighted button  64   a . The sampling settings reflect three possible sampling rates: oversampling, o.k. an undersampling. Below the first section  61  is a first click button  65  for adjusting the microscope settings to optimize the noise/signal ratio. Below the second section  62  is a second click button  66  for setting and maintaining the optimal bleaching of the specimen from one scan to another. Below the fourth section  64  is a fourth click button  68  for automatically adjusting all of the parameters necessary to capture a good quality specimen image.  
         [0032]    [0032]FIG. 7 illustrates a cross talk between two detection channels. The abscissa shows a intensity I(Ch 1 ) of a signal generated by a first spectral detection channel and the ordinate shows an intensity I(Ch 2 ) of a signal generated by a second detection channel. When there is no cross talk between the two channels, a first distribution  80  defining a first line  81  and a second distribution  82  defining a second line  83  are observed. A cross talk, for example, from the second channel to the first channel will rotate the first spectral detection channel away from the abscissa, as shown by arrows in FIG. 7. The graph of FIG. 7 can be shown to a user on display  20  to allow the user to view a possible cross talk.  
         [0033]    The present invention has been described in detail with particular reference to the illustrated embodiments thereof, but it is to be understood that variations and modifications can be practiced without departing from the spirit and scope of the invention.