Microscope apparatus chronologically storing different types of image information

A microscope apparatus organizes and stores a plurality of types of image information acquired by a plurality of image acquisition methods at different timings. The microscope apparatus includes a time counting unit for counting time, a plurality of different image acquisition units, and a storage unit for storing image information, when acquired by any one of the image acquisition units, and timing information counted by the time counting unit, by having them associated with each other. Even if the image acquisition units acquire different types of image information at different timings, the timing information can be used to call up the image information stored in the storage unit in a chronological order.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a microscope apparatus.

This application is based on Japanese Patent Application No. 2009-159721, the content of which is incorporated herein by reference.

2. Description of Related Art

Heretofore, there is known a microscope apparatus which makes an association of a plurality of image data acquired by different observation methods with each other and stores them as a single image data file (for example, refer to Japanese Unexamined Patent Application, Publication No. 2005-149397).

However, this microscope apparatus involves an inconvenient situation in which the acquirable image data by different observation methods are limited only to those acquired at the same time, and it is not possible to process a plurality of images if acquired by different image acquisition methods at different timings.

BRIEF SUMMARY OF THE INVENTION

The present invention provides a microscope apparatus capable of organizing and storing a plurality of types of image information acquired by a plurality of image acquisition methods at different timings.

One aspect of the present invention is a microscope apparatus comprising a time counting unit for counting time, a plurality of different image acquisition units, and a storage unit for storing image information, when acquired by any one of the image acquisition units, and timing information counted by the time counting unit, by having them associated with each other.

According to the above-mentioned aspect of the present invention, different types of image information acquired by the plurality of different image acquisition units are stored in association with a common time line counted by the time counting unit, in the storage unit. Therefore, even if the image acquisition units acquire different types of image information at different timings, it is possible, by using the timing information as a clue, to call up the image information stored in the storage unit in a chronological order. For example, if a Laser Scanning Microscope device (“LSM”) moves a laser beam to scan an object by using a scanner and detects fluorescent light therefrom and a Charge Coupled Device (“CCD”) which captures two dimensional images of the object are provided as the image acquisition units, although the respective images are acquired at different timings, the respective images can be stored in association with a same type of timing information along a single time line, by which these images can be observed in mutual relation with each other.

In the above-mentioned aspect, the storage unit may also separately store image information acquired by each of the image acquisition units and timing information counted by the time counting unit at the time when the concerned image information has been acquired, by respectively providing them with each serial number and each identification information which indicates the type of the image acquisition unit that has acquired the image information.

By so doing, similarly to the case where the image information is stored in direct association with the timing information, images can be observed through indirect association between the image information and the timing information by using the serial number and the identification information which indicates the type of the image acquisition unit.

In addition, in the above-mentioned aspect, the storage unit may also collectively store image information acquired by the image acquisition units and collectively store timing information counted by the time counting unit at the time when the concerned image information has been acquired, by respectively providing them with each serial number and each identification information which indicates the type of the image acquisition unit that has acquired the image information.

Moreover, in the above-mentioned aspect, the storage unit may also store pairs of image information acquired by each of the image acquisition units and timing information counted by the time counting unit at the time when the concerned image information has been acquired, in an arrangement according to the order of the timing information.

Furthermore, the above-mentioned aspect may also comprise an image display unit for displaying the image information, an image specifying unit for specifying a certain type of image information displayed on the image display unit, and an image retrieval unit for retrieving, based on the timing information stored in association with the concerned type of image information specified by the image specifying unit, a different type of image information acquired by a different image acquisition unit other than the image acquisition unit that has acquired the specified type of image information, so that the image display unit displays the different type of image information retrieved by the image retrieval unit.

By so doing, if a certain type of image is specified by the image specifying unit during a state where the concerned type of image information is being displayed by the image display unit, the image retrieval unit retrieves a different type of image information acquired by a different image acquisition unit by using, as a clue, the timing information stored in association with the specified type of image information, and the different type of image information is displayed by the image display unit. By so doing, different types of related image information acquired by different image acquisition units can be readily called up, displayed, and observed.

The present invention can offer an effect of enabling to organize and store a plurality of types of image information acquired by a plurality of image acquisition methods at different timings.

DETAILED DESCRIPTION OF THE INVENTION

Hereunder is a description of a microscope apparatus1according to one embodiment of the present invention with reference to drawings.

As shown inFIG. 1, the microscope apparatus1of this embodiment comprises an LSM device2, a planar illumination light source3attached to the LSM device2, a CCD (imaging part, image acquisition unit)4, a control unit (image retrieval unit)5for controlling the LSM device2and the CCD4, an input unit (image specifying unit)6for entering inputs into the control unit5, a monitor (image display unit)7for displaying images or such information acquired by the LSM device2and the CCD4, a timer (time counting unit)8for counting the observation time, and a storage unit9for storing image information and other information acquired by the LSM device2and the CCD4.

The LSM device2is a confocal microscope comprising: a laser light source10for emitting a laser beam; a scanner11for moving the laser beam emitted from the laser light source10to scan in a two dimensional manner; an object lens12for focusing the laser beam moved by the scanner11on a specimen A and collecting fluorescent light generated in the specimen A; a dichroic mirror13for branching the fluorescent light that has been collected by the object lens12and is returning back through the scanner11, out from the optical path of the laser beam; a condenser lens14for condensing the fluorescent light branched out by the dichroic mirror13; a confocal pinhole15positioned at the focal point of the condenser lens14; and a photodetector (PMT: photomultiplier tube, image acquisition unit)16for detecting the fluorescent light passing through the confocal pinhole15. In the drawing, the reference signs17and18denote dichroic mirrors.

The scanner11is, for example, a so-called proximal galvano mirror comprising a pair of independently oscillatable galvano mirrors (not shown). The control unit5makes a command to the scanner11regarding the position to irradiate the laser beam so that the scanner11can drive the respective galvano mirrors to be arranged at an appropriate oscillation angle according to the command from the control unit5.

In addition, when receiving fluorescence brightness information from the photodetector16of the LSM device2, the control unit5sequentially stores the brightness information in association with the position and the oscillation angle of the scanner11, so as to thereby generate two-dimensional LSM image information.

Moreover, when receiving a picture signal from the CCD4, the control unit5generates two-dimensional CCD image information to be displayed on the monitor7.

Furthermore, as shown inFIG. 2, when the reception of the picture signal from the CCD4is started, the control unit5starts to associate the timing information t received from the timer8at the same time of the reception, with the generated CCD image information. Similarly, when the reception of the fluorescence brightness information from the PMT16of the LSM device2is started, the control unit5starts to associate the timing information t received from the timer8at the same time of the reception, with the generated LSM image information.

InFIG. 2, the CCD image information is composed of a set of frames which respectively comprises a plurality of two-dimensional images, while the LSM image information is respectively composed of a single two-dimensional image.

The timer8starts to count the time in response to the observation start command which has been input from the input unit6.

As shown inFIG. 3, the storage unit9stores pairs of timing information t and image information which have been associated by the control unit5, in an arrangement according to the order of the timing information. In this case, it is sufficient only if the timing information contains information which indicates the time counted by the timer8from the start of the observation.

The input unit6comprises a keyboard, a mouse, a joystick, or the like, which are handled by the user. It is possible for the user, by handling the input unit6, to make a command to start the observation and a command to specify a certain image displayed on the monitor7.

Moreover, when a particular image is specified by the user through the input unit6, the control unit5reads out a piece of timing information stored in relation with the specified piece of image information, then retrieves nearby piece(s) of timing information which is(are) close to the read out piece of timing information, and reads out piece(s) of a different type of image information stored in relation with the concerned nearby piece(s) of timing information, and displays it(them) on the monitor7. By so doing, different type of image(s) related with the specified image can be displayed.

Hereunder is a description of the operation of the thus configured microscope apparatus1according to this embodiment.

In order to observe the specimen A by using the microscope apparatus1of this embodiment, as shown inFIG. 4, firstly, the user handles the input unit6to set the acquisition conditions for LSM image and the acquisition conditions for CCD images (step S1). The acquisition conditions can be exemplified by the range of the oscillation angle of the scanner11which serves as the range of acquisition for LSM images, the timing to acquire LSM images (interval), the number of LSM images to be acquired, the timing to acquire CCD images (interval), the number of CCD images to be acquired, and the like.

Next, the user handles the input unit6to make a command to start the observation (step S2), by which the timer8starts to count the time (step S3, t=0 inFIG. 2), and the acquisitions of LSM images and CCD images according to the respective acquisition conditions are started in parallel.

LSM images are acquired by operating the laser light source10to emit a laser beam, and moving the laser beam irradiated on the specimen A through the scanner11and the object lens12to thereby scan the specimen A in a two dimensional manner by the operation of the scanner11. In the specimen A irradiated with the laser beam, fluorescent substances are excited to generate fluorescent light. The generated fluorescent light is collected by the object lens12. The returning light on the way back through the dichroic mirrors17and18and the scanner11is branched by the dichroic mirror13. The branched light is condensed by the condenser lens14. The light passing through the confocal pinhole15is detected by the photodetector16.

When the acquisition of LSM images is started (step S4), the timing information read out by the timer8at the same time of the image acquisition is stored in the storage unit9(step S5). Thereafter, time is allowed until the acquisition of LSM images is completed (step S6), and the image information regarding the acquired LSM images is stored in association with the timing information stored in the step S5(step S7). Then, it is determined whether or not a predetermined number of LSM images according to the acquisition condition have been acquired (step S8). If the number is not achieved, the steps S4to S8are repeated again. If the predetermined number of LSM images have been acquired, the process is completed.

On the other hand, CCD images are acquired by operating the planar illumination light source3to irradiate the specimen A with planar illumination light through the object lens12. Fluorescent substances existing in the specimen A are excited by the irradiation of the planar illumination light, and thereby fluorescent light is generated. The generated fluorescent light is collected by the object lens12and captured by the CCD.

When the acquisition of CCD images is started (step S9), the timing information read out by the timer8at the same time of the image acquisition is stored in the storage unit9(step S10). Thereafter, time is allowed until the acquisition of CCD images is completed (step S11), and the image information regarding the acquired CCD images is stored in association with the timing information stored in the step S10(step S12). Then, it is determined whether or not a predetermined number of CCD images according to the acquisition condition have been acquired (step S13). If the number is not achieved, the steps S9to S13are repeated again. If the predetermined number of CCD images have been acquired, the process is completed.

In this way, according to the microscope apparatus1of this embodiment, different types of image information acquired by different image acquisition means such as the LSM device2and the CCD4at different timings are stored in association with a same type of timing information along a single time line based on each time point when the acquisition of each image information has been started. Therefore, even if image information is individually acquired at totally unrelated acquisition timings, such as the manner of the LSM device2and the CCD4, the acquired image information can be readily observed in a chronological order afterward.

In addition, according to the microscope apparatus1of this embodiment, for example, upon the observation of CCD images by sequentially displaying them on the monitor7, when the user specifies a particular CCD image by handling the input unit6, the control unit5reads out a piece of timing information stored in association with the specified CCD image, from the storage unit9, and retrieves LSM image(s) whose timing information is close to the readout peace of timing information in the storage unit9. Then, the control unit5displays the retrieved LSM image(s) on the monitor7.

For example, in the example shown inFIG. 2, when a CCD image (CCD2) is specified as the image information, a piece of timing information (CCDt2) associated with the concerned image (CCD2) is read out. Then, the control unit5retrieves pieces of timing information (LSMt2and LSMt3) close to this piece of timing information (CCDt2), and displays LSM images (LSM2and LSM3) associated with them, on the monitor7.

By so doing, an advantage is given in which: if the specimen A makes some change, the detail of which is particularly desired to be observed, during the observation of the entire specimen A by CCD images, or in such a situation, LSM image(s) which is(are) the closest to the concerned time point is(are) displayed on the monitor7so that the specimen A can be observed more in detail.

This embodiment is designed to specify a CCD image displayed on the monitor7so as to call up nearby LSM image(s). However, instead of this, it is also possible to specify an LSM image displayed on the monitor7so as to call up nearby CCD image(s).

Moreover, in this embodiment, the storage unit9stores associated pairs of timing information counted by the timer8and either LSM image or CCD image. However, instead of this, as shown inFIG. 5, the storage unit9may also collectively store pieces of timing information and collectively store pieces of image information, respectively. In this case, in order to associate the timing information and the image information with each other, it is necessary to store them by respectively providing each piece of the timing information and the image information with each serial number indicating the chronological order and each identification information indicating the type of the image, that is, whether the information is of an LSM image or a CCD image.

In addition, if the identification information and the serial number are stored in respective association with the timing information and the image information in such a manner, it is also possible as shown inFIG. 6to store the timing information and the image information in separate files. Furthermore, it is also possible as shown inFIG. 7to respectively store the timing information associated with LSM images, the timing information associated with CCD images, the LSM image information, and the CCD image information in separate files. Moreover, all pieces of the timing information and the image information may also be stored in separate files.

In addition, although the CCD and the PMT are exemplified as the image acquisition units, the image acquisition units are not limited to them. It is also possible to employ any other types of image acquisition units having different image acquisition methods. For example, a CMOS or resonant scanner may be employed instead of the CCD.