Microscope apparatus

Provided with a time-lapse imaging unit which repeatedly captures a specimen at predetermined time intervals and generates a plurality of images, and a recording unit which records at least one of an image group including one or more of the images captured during a predetermined period among a period of a time-lapse capturing performed by the time-lapse imaging unit or an image group including one or more of the images picked at predetermined time intervals among the period of the time-lapse capturing performed by the time-lapse imaging unit. Thus, data generated in time-lapse photography are managed favorably in a microscope apparatus provided with a time-lapse imaging unit which repeatedly captures a specimen at predetermined time intervals and generates a plurality of images.

BACKGROUND

The present application relates to a microscope apparatus including an imaging unit which captures an image of a specimen repeatedly and generates plurality of images.

2. Description of the Related Art

A microscope system which enlarges and photographs cultured cells being grown in a culture medium while controlling an environment such as a culture medium in a culture vessel are in practical use (see Patent Document 1: Japanese Unexamined Patent Application Publication No. 2005-326495, etc.). In this microscope system, time-lapse photography is effective for visually capturing gradual changes over time which occur in cultured cells (see Patent Document 2: Japanese Unexamined Patent Application Publication No. 2006-220904, etc.).

In the above-described microscope system, when the time-lapse photography is performed for a long period of time, an enormous amount of data (mainly image data) is generated. Accordingly, it is quite difficult to pull out, for example, target image data from the enormous amount of data.

A proposition of a microscope system according to the present application is to favorably manage data generated during time-lapse photography in a microscope apparatus including an imaging unit which captures an image of a specimen repeatedly and generates plurality of images.

SUMMARY

A microscope apparatus according to the present embodiments includes a time-lapse imaging unit which repeatedly captures a specimen at predetermined time intervals and generates a plurality of images, and a recording unit which records at least one of an image group including one or more of the images captured during a predetermined period among a period of a time-lapse capturing performed by the time-lapse imaging unit or an image group including one or more of the images picked at predetermined time intervals among the period of the time-lapse capturing performed by the time-lapse imaging unit.

Note that, preferably, the microscope apparatus may include a selecting unit which selects the images to be recorded from the plurality of images as the image group according to a state change of the specimen.

Further, preferably, the microscope apparatus may include an observation member for a fluorescence observation of the specimen, in which the selecting unit detects the state change of the specimen by obtaining an intensity ratio in fluorescence based on the images each generated by the time-lapse imaging unit when the fluorescence observation is performed by at least two types of different wavelengths.

Another microscope apparatus according to the present embodiments includes a time-lapse imaging unit which repeatedly captures a specimen at predetermined time intervals and generates a plurality of images, and a recording unit which clips an image of a predetermined area from each of the plurality of images generated by the time-lapse imaging unit, and records an image group including one or more images clipped.

Note that, preferably, the microscope apparatus may include an accepting unit which accepts a user instruction to specify an area of performing the clipping from at least two images of the plurality of images, in which the recording unit determines the area of performing the clipping for each of the plurality of images based on the user instruction.

Further, preferably, the microscope apparatus may include a selecting unit which selects the area of performing the clipping for each of the plurality of images according to a state change of the specimen.

Further, preferably, the microscope apparatus may include an observation member for a fluorescence observation of the specimen, in which the selecting unit detects the state change of the specimen by obtaining an intensity ratio in fluorescence based on the images each generated by the time-lapse imaging unit when the fluorescence observation is performed by at least two types of different wavelengths.

Another microscope apparatus according to the present embodiments includes a time-lapse imaging unit which repeatedly captures a specimen at predetermined time intervals and generates a plurality of images, and a recording unit which generates either a first image group including one or more of the images captured during a predetermined period among a period of a time-lapse capturing performed by the time-lapse imaging unit or a second image group including one or more of the images picked at predetermined time intervals among the period of the time-lapse capturing performed by the time-lapse imaging unit, and further clips an image of a predetermined area from each of the plurality of images forming the generated image groups and records a third image group including one or more images clipped.

With a microscope system according to the present application, data generated during time-lapse photography can be managed favorably in a microscope apparatus including an imaging unit which captures an image of a specimen repeatedly and generates plurality of images.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Hereinafter, an embodiment of the present application will be explained using drawings.

FIG. 1is a structural diagram of a microscope apparatus1of this embodiment. As shown inFIG. 1, the microscope apparatus1is provided with an apparatus body10, a computer170, a monitor160, and an input device180. The apparatus body10is provided with a microscope part150, a transmission lighting part40, a cold camera300, an excitation light source70, and an optical fiber7.

The microscope part150is provided with a stage23, an objective lens part27, a fluorescence filter part34, an imaging lens part38, a deflecting mirror452, a field lens411, and a collector lens41. The transmission lighting part40is provided with a transmission light source47, a field lens44, and a deflecting mirror45.

A chamber100accommodating a transparent culture vessel20is mounted on the stage23. The culture vessel20is filled with a culture medium, and cultured cells marked by a fluorescent material are grown in this culture medium. For observing the cultured cells from the outside of the chamber100, a part a of a bottom face of the chamber100and a part b of an upper part of the chamber100are made transparent. Here, for brevity, the culture vessel20with an open upper face will be explained, but the upper face of the culture vessel20is covered as necessary with a lid made of the same material as the culture vessel20.

In the objective lens part27, plural types of objective lenses are fitted along an X-direction ofFIG. 1. When the objective lens part27is driven in the X-direction by a not-shown mechanism, the type of the objective lens disposed in an optical path of the apparatus body10is switched. This switching is performed under control by the computer170.

In the fluorescence filter part34, plural types of filter blocks are fitted along the X-direction ofFIG. 1. When the fluorescence filter part34is driven in the X-direction by a not-shown mechanism, the type of the filter block disposed in the optical path of the apparatus body10is switched. This switching is also performed under control by the computer170.

The computer170switches a combination of the type of the objective lens disposed in the optical path and the type of the filter block disposed in the optical path according to an observation method to be set in the apparatus body10. Hereinafter, it is considered that this switching causes the observation method of the apparatus body10to switch between phase difference observation and two types of fluorescence observation.

Between the phase difference observation and the fluorescence observation among them, both the type of the filter block disposed in the optical path and the type of the objective lens disposed in the optical path are different. Between the two types of fluorescence observation, only the type of the filter block disposed in the optical path is different. Further, also an illumination method is different between the phase difference observation and the fluorescence observation.

The computer170turns on the transmission light source47for activating an optical path of the transmission lighting part40when performing the phase difference observation, and turns on the excitation light source70for activating an optical path of an epi-illumination unit (optical path passing through the excitation light source70, the optical fiber7, the collector lens41, the field lens411, the deflecting mirror452, the fluorescence filter part34, and the objective lens part27in this order) when performing the fluorescence observation. In addition, when the transmission light source47is turned on, the excitation light source70is turned off, and when the excitation light source70is turned on, the transmission light source47is turned off.

When performing the phase difference observation, a light emitted from the transmission light source47illuminates an observation point c in the culture vessel20via the field lens44, the deflecting mirror45, and the transparent part b of the chamber100. The light passed through the observation point c reaches a light receiving surface of the cold camera300via a bottom face of the culture vessel20, the transparent part a of the chamber100, the objective lens part27, the fluorescence filter part34, and the imaging lens38, thereby forming a phase-contrast image of the observation point c. When the cold camera300is driven in this state, the phase-contrast image is captured and image data is generated. This image data (image data of the phase-contrast image) is taken into the computer170.

When performing the fluorescence observation, a light emitted from the excitation light source70illuminates the observation point c in the culture vessel20via the optical fiber7, the collector lens41, the field lens411, the deflecting mirror452, the fluorescence filter part34, the objective lens part27, the transparent part a of the chamber100, and the bottom face of the culture vessel20. Thus, a fluorescent material existing at the observation point c is excited and emits fluorescence. This fluorescence reaches the light receiving surface of the cold camera300via the bottom face of the culture vessel20, the transmission part a of the chamber100, the objective lens part27, the fluorescence filter part34, and the imaging lens38, thereby forming a fluorescence image of the observation point c. When the cold camera300is driven in this state, the fluorescence image is captured and image data is generated. This image data (image data of the fluorescence image) is taken into the computer170.

In addition, the computer170controls X, Y, Z coordinates of the observation point c in the culture vessel20by controlling X, Y coordinates of the stage23and a Z coordinate of the objective lens part27.

Further, a not-shown humidifier is coupled to the chamber100via a not-shown silicone tube, and humidity and CO2concentration in the chamber100are both controlled to be close to predetermined values. Further, an ambient atmosphere of the chamber100is circulated appropriately by a not-shown heat exchanger, and thereby the internal temperature of the chamber100is controlled to be close to a predetermined value. The humidity, CO2concentration, and temperature in the chamber100are measured by not-shown sensors. Then the measurement results are taken into the computer170. On the other hand, the cold camera300is housed in a cabinet separated from the other parts of the apparatus body10, and is kept at approximately the same temperature as the air temperature outside the apparatus body10, irrespective of the internal temperature of the chamber100.

Next, operation of the computer170related to time-lapse photography will be explained.

It is assumed that a program for observation is installed in the computer170, and the computer170operates according to this program. It is also assumed that any input of information to the computer170by the user is performed via the input device180.

FIG. 2is an operation flowchart of the computer170. As shown inFIG. 2, the computer170first sets the observation method of the apparatus body10to the phase difference observation (more precisely, the phase difference observation at a low magnification), obtains image data of a bird view image in this state, and displays it on the monitor160(step S11). The bird view image refers to an image of a relatively wide area in the culture vessel20.

When obtaining the image data of the bird view image, the computer170repeatedly obtains image data of a phase difference image while moving the observation point c in the culture vessel20in the x, y directions, and combines obtained plurality of image data into image data of one image. Individual Image data are image data of what are called tile images, and image data after being combined is image data of the bird view image.

While observing the bird view image displayed on the monitor160, the user determines conditions (interval, round number, observation point, observation method, and the like) of the time-lapse photography.

When conditions are input by the user (steps S11, YES), the computer170creates a recipe in which the conditions are written (step S13). The destination of storing the recipe is, for example, a hard disk in the computer170. Hereinafter, the interval, round number, observation point, and observation method written in the recipe are called “specified interval”, “specified round number”, “specified point”, and “specified observation method”, respectively.

Thereafter, when a start instruction for the time-lapse photography is input by the user (step S14), the computer170starts the time-lapse photography (step S15).

In the time-lapse photography, the computer170matches the observation point c of the culture vessel20with the specified point, and sets the observation method of the apparatus body10to the specified observation method and obtains image data in this state. When there are three types of specified observation methods, three types of image data are obtained sequentially while switching the observation method of the apparatus body10among the three types of specified observation methods. Thus, a first round of photography finishes.

Thereafter, the computer170allows a standby time by the specified interval to elapse from the time of starting the first round of photography, and then starts a second round of photography. The method of the second round of photography is the same as that of the first round.

Furthermore, the photography thereafter is repeated until the number of the already performed photography reaches the specified round number (until YES in step S16becomes true).

Now, in the period of time-lapse photography (period in which NO in step S16becomes true), the computer170sequentially stores image data taken in from the apparatus body10in an implementation progress file. The destination of storing this implementation progress file is, for example, the hard disk in the computer170.

Furthermore, when a check instruction is input by the user during the period of time-lapse photography or after the time-lapse photography is finished, the computer170refers to the contents of the implementation progress file at this moment, and displays an operation progress check screen, which will be explained below, on the monitor160based on the contents.

FIG. 3AandFIG. 3Bare views showing the operation progress check screen. As shown inFIG. 3A, in the operation progress check screen, display areas101,102,103,104and a reproduction control part105are disposed.

The display area101is an area where a moving image of a phase difference image is to be displayed. The display area102is an area where a moving image of one (first fluorescence image) of two types of fluorescence images is to be displayed. The display area103is an area where a moving image of the other (second fluorescence image) of the two types of fluorescence images is to be displayed. Further, the display area104is an area where a moving image of a combined image of the phase difference image and the fluorescence images is to be displayed.

Prior to displaying of the moving images, the computer170reads the three types of image data obtained in each round from the implementation progress file. The computer170then creates a moving image file of the phase difference image by coupling frames of image data of the phase difference image among the read data in time-series order, creates a moving image file of the first fluorescence image by coupling frames of image data of the first fluorescence image in time-series order, and creates a moving image file of the second fluorescence image by coupling frames of image data of the second fluorescence image in time-series order. Further, the computer170combines the image data of the phase difference image and the image data of the fluorescence images frame by frame, and couples frames after being combined in time-series order to thereby create a moving image file of the combined images. The destination of storing these four types of moving image files is, for example, the hard disk in the computer170.

When displaying the moving images, the computer170reads the created four types of moving image files in a simultaneous and parallel manner, generates moving image signals for individually displaying the four types of moving images in the four display areas101,102,103,104, and transmits the signals to the monitor160in the order of generation. Hereinafter, this series of processing by the computer170including reading moving image files, generating moving image signals, and transmitting the moving image signals is called “reproduction of moving image files”.

The reproduction control part105shown inFIG. 3Ais a GUI image for the user to input an instruction related to reproduction of moving image files to the computer.

FIG. 3Bis an enlarged view of the reproduction control part105. As shown inFIG. 3B, on the reproduction control part105, there are disposed a stop button52, a skip button53, a play button54, a fast forward button55, a clipping button56(details of which will be described later), a time line50, and so on.

When the user selects the play button54, reproduction of moving image files is started, and display of moving images in the display areas101,102,103, and104is started. Further, when the user selects the stop button52, the moving images displayed in the display areas101,102,103, and104are stopped.

The reproducing point in the moving image files is reflected on the time line50. A left end (1) of the time line50indicates the beginning of the moving image files (start time of the time-lapse photography), and a right end (2) of the time line50indicates the end of the moving image files (end time of the time-lapse photography). In addition, when the time-lapse photography is not ended, the right end (2) of the time line50indicates the current time. A slider bar60is disposed on this time line50, and the reproducing point in the moving image file is represented in real time by the position of the slider bar60in a left and right direction.

The position of this slider bar60in the left and right direction can be changed freely by the user. When the position of the slider bar60in the left and right direction is changed, the reproducing point in the moving image file is changed accordingly.

Incidentally, when the time-lapse photography described above is performed for a long period of time, an enormous amount of moving image files is generated. Accordingly, in this embodiment, a different moving image file including target image data is created using two methods, time clipping and space clipping.

(1) Time Clipping

Time clipping includes two types of methods. The first method is to create a moving image file including plurality of images generated in a predetermined period during the period of time-lapse photography. Further, the second method is to create a moving image file including plurality of images which are picked at every predetermined time interval from plurality of images generated during the period of time-lapse photography.

These methods will be explained in detail usingFIGS. 4A and 4B.FIG. 4Ais a diagram explaining the first method, andFIG. 4Bis a diagram explaining the second method. In the first method, as shown inFIG. 4A, plurality of images generated during a predetermined period, which is specified by a method that will be described later, are pulled out from plurality of images generated during the period of time-lapse photography, and a moving image file including the pulled out plurality of images is created. Further, in the second method, as shown inFIG. 4B, there is created a moving image file including plurality of images which are picked at every predetermined time interval, which is specified by a method that will be described later, from plurality of images generated during the period of time-lapse photography.

There are following two methods for specifying the period in which time clipping is performed and the picking interval.

(1)-[1] Manual Method

When the user selects the clipping button56, the computer170displays a clipping specification screen shown inFIG. 5Aon the monitor160. The user can input the period in which time clipping is performed and the picking interval via the input device180so as to specify them. Note that the picking interval may not necessarily be even. For example, the picking interval may be specified such as every 10 minutes in a first hour, every 20 minutes in a next hour, every 10 minutes in a next hour thereafter, and so on.

Further, when the user selects the clipping button56, a clipping frame56-1is displayed on the time line50as shown inFIG. 5B, by which the user may specify an arbitrary period on the time line50via the input device180as a period to perform time clipping.

Furthermore, as shown inFIG. 5C, a marking button57may be provided instead of the clipping button56on the reproduction control part105. Then the user may specify a desired time by selecting the marking button57via the input device180during the period of time-lapse photography and after the time-lapse photography is finished. In this case, as shown inFIG. 5C, marks (57-1,57-2) may be displayed on the time line50.

Time clipping is then performed according to the marks. For example, another moving image file may be created by pulling out only marked images, or another moving image file may be created by pulling out a few or few tens of images around marked images.

In either case, the user can mark a target image via the input device180to thereby perform time clipping around this image.

(1)-[2] Automatic Method

Time clipping may be performed automatically according to a state change while monitoring a state of a specimen by the computer170. Specifically, the state of the specimen is monitored based on images generated in the period of time-lapse photography, and processing similar to that by the marking button57explained in (1)-[1] may be performed based on a monitoring result. For example, an intensity ratio in fluorescence of the above-described first fluorescence image and second fluorescence image can be obtained, and a state change of the specimen can be monitored based on this intensity ratio in fluorescence.

Space clipping is a method to clip an image of a predetermined area from each of plurality of images generated in a predetermined period in the period of time-lapse photography, and generate a moving image file including clipped plurality of images.

There are following two methods for specifying the clipping area in space clipping.

(2)-[1] Manual Method

When the user selects the clipping button56(seeFIG. 3B), the computer170displays a clipping frame58-1on a display area as shown inFIG. 6A. Then the user specifies a clipping area on the display area via the input device180in one image (first image in the example ofFIG. 6A) among plurality of images generated in the period of time-lapse photography. Then, space clipping is performed by the same clipping area for all the images.

Further, as shown inFIG. 6B, the user may specify clipping areas (58-2,58-3, and58-4) on display areas via the input device180in plurality of images (the first image, a middle image, and the last image in the example ofFIG. 6B). In this case, space clipping is performed by calculating a clipping area in every image by an interpolation method or the like.

In either case, the user can specify a target area as a clipping area via the input device180to thereby perform space clipping around this area.

(2)-[2] Automatic Method

Space clipping may be performed automatically according to a state change while monitoring a state of a specimen by the computer170. Specifically, the user specifies a clipping area via the input device180at the time of starting the time-lapse photography. Then fluorescence intensity distributions of a first fluorescence image and a second fluorescence image generated during the time-lapse photography period thereafter are obtained, and the clipping area specified by the user can be traced based on the fluorescence intensity distributions. In addition, the user may specify a clipping area via the input device180in middle of the time-lapse photography, and thereafter the clipping area may be traced based on an image generated during the time-lapse photography period.

(3) Application Examples

The time clipping explained in (1) and the space clipping explained in (2) may be performed in combination. Note that the space clipping may be performed after the time clipping is performed, or the time clipping may be performed after the space clipping is performed. Further, the time clipping and the space clipping may be performed at the same time. Particularly, when the space clipping is performed after the time clipping is performed, processing by the computer170can be made lighter.

As has been explained above, according to this embodiment, in the microscope apparatus including a time-lapse imaging unit which captures a specimen repeatedly at predetermined time intervals and generates plurality of images, pulling out target data can be facilitated by performing at least one of the time clipping and the space clipping on generated data. Thus, data generated during time-lapse photography can be managed preferably.

Note that in this embodiment, a moving image file generated by performing at least one of the time clipping and the space clipping may be recorded by associating with a moving image file including all the images generated in the period of time-lapse photography. Further, after checking the user's demand, the moving image file generated by performing at least one of the time clipping and the space clipping may be recorded instead of the moving image file including all the images generated in the period of time-lapse photography.