Patent Description:
Conventionally, in capturing a sample by using a microscope, there presents a technology that determines a focusing point of an object lens on the basis of a size and a position of a spot of a focus detecting light that is changed by driving the object lens of the microscope in a light-axis direction thereof.

<CIT> discloses a method for auto focusing a microscopic imaging system using a machine learned regression system such as Convolutional Neural Network (CNN). The method comprises receiving a first image of a sample under review and a second image of sample wherein the first image is captured at first focus position and second image is captured at second focus position. The CNN is trained using the plurality of historic difference images along with direction of focus and optimal focus position. The difference of two images are obtained in terms of difference in pixel values. The direction of focus and optimal focus position for difference image is identified based on plurality of historic difference images along with direction of focus and optimal focus position. The method enables automated stage comprising sample to move towards direction of focus and position at optimal focus position for capturing a focused image.

However, in the conventional technology, it is difficult to quickly execute automatic focusing. For example, the above-mentioned conventional technology assumes that a bottom surface of a container is planar and uniform, in a case of a threedimensionally cultured cell container, a focus detecting light is not normally reflected, a spot of a focus detecting light which is acquired by an image capturing device is accordingly deformed, and thus it may be impossible to easily determine a focusing point in some cases.

One aspect of the present disclosure is made in view of the aforementioned, and an object of an embodiment is to quickly execute automatic focusing.

According to an aspect of the embodiments, an information processing apparatus includes, a first acquisition unit that acquires pieces of image data obtained by capturing an image of a target object to be observed by using a microscope at respective positions of an object lens of the microscope while driving the object lens in a light-axis direction of an illumination with which the target object is irradiated, the image being obtained by condensing light by using the object lens, a first specifying unit that specifies a focusing position of the object lens at which a focus of the target object is determined based on the pieces of image data, and a second acquisition unit that in a case where the object lens is arranged in the focusing position, acquire, as a reference light receiving pattern, a light receiving pattern obtained by receiving, via the object lens, reflected light of a focus detecting light emitted to the target object via the object lens, and store the reference light receiving pattern in a storage.

According to an aspect of the embodiments, an information processing system includes, an image capturing device, a microscope, and, an information processing apparatus, wherein the image capturing device captures a plurality of images of a target object to be observed by using the microscope at respective positions of an object lens of the microscope while driving the object lens in a light-axis direction of an illumination with which the target object is irradiated, the image being obtained by condensing light by using the object lens, the microscope includes a light emitting device that emits a focus detecting light toward the target object via the object lens, and the information processing apparatus includes, a first acquisition unit that acquires pieces of image data obtained by capturing the image of the target object to be observed by using the image capturing device, a first specifying unit that specifies, based on the pieces of image data, a focusing position of the object lens at which a focus of the target object is determined, and a second acquisition unit that in a case where the object lens is arranged in the focusing position, acquire, as a reference light receiving pattern, a light receiving pattern obtained by receiving, via the object lens, reflected light of the focus detecting light emitted to the target object via the object lens, and store the reference light receiving pattern in a storage.

According to an aspect of the embodiments, an information processing method includes, acquiring pieces of image data obtained by capturing an image of a target object to be observed by using a microscope at respective positions of an object lens of the microscope while driving the object lens in a light-axis direction of an illumination with which the target object is irradiated, the image being obtained by condensing light by using the object lens, based on the pieces of image data, specifying a focusing position of the object lens at which a focus of the target object is determined, in a case where the object lens is arranged in the focusing position, acquiring, as a reference light receiving pattern, a light receiving pattern obtained by receiving, via the object lens, reflected light of a focus detecting light emitted to the target object via the object lens, and storing the reference light receiving pattern in a storage.

According to an aspect of the embodiments, an information processing program that causes a computer to execute a process includes, acquiring pieces of image data obtained by capturing an image of a target object to be observed by using a microscope at respective positions of an object lens of the microscope while driving the object lens in a light-axis direction of an illumination with which the target object is irradiated, the image being obtained by condensing light by using the object lens, based on the pieces of image data, specifying a focusing position of the object lens at which a focus of the target object is determined, in a case where the object lens is arranged in the focusing position, acquiring, as a reference light receiving pattern, a light receiving pattern obtained by receiving, via the object lens, reflected light of a focus detecting light emitted to the target object via the object lens, and storing the reference light receiving pattern in a storage.

Hereinafter, an embodiment of an information processing apparatus, an information processing system, an information processing method, and an information processing program according to the present disclosure will be described in detail with reference to the accompanying drawings. The present disclosure is not limited to the embodiment described in the following.

Hereinafter, a configuration of an information processing system according to the embodiment, configurations of an information processing apparatus according to the embodiment and the like, flows of processes according to the embodiment will be explained in this order, and further effects of the embodiment will be finally explained.

A configuration of an information processing system <NUM> according to the embodiment will be specifically explained with reference to <FIG> is a diagram illustrating a configuration example of the information processing system <NUM> according to the embodiment. Hereinafter, a configuration example of the whole information processing system <NUM>, processes of the information processing system <NUM>, and effects the information processing system <NUM> will be explained in this order.

The information processing system <NUM> includes a container <NUM>, a microscope <NUM>, a camera <NUM> that is an image capturing device, and an information processing apparatus <NUM>. The information processing system <NUM> illustrated in <FIG> may include the plurality of containers <NUM>, the plurality of microscopes <NUM>, the plurality of cameras <NUM>, or the plurality of information processing apparatuses <NUM>. Note that the information processing apparatus <NUM> may have a configuration in which the information processing apparatus <NUM> is integrated with one or more of the microscope <NUM> and the camera <NUM>. Hereinafter, the container <NUM>, the microscope <NUM>, the camera <NUM>, and the information processing apparatus <NUM> will be explained.

The container <NUM> includes a plurality of partitions <NUM> each of which stores therein a sample such as a culturing cell to be observed by the microscope <NUM>. In the following explanation, the plurality of partitions <NUM> may be indicated as a first partition <NUM>-<NUM>, a second partition <NUM>-<NUM>, and the like. The container <NUM> is placed on a not-illustrated pedestal so that the microscope <NUM> is capable of observing a sample.

The microscope <NUM> includes an object lens <NUM>, a focusing detection device <NUM>, a driving device <NUM>, and an imaging lens <NUM>. Herein, the object lens <NUM> is arranged between the container <NUM> and the focusing detection device <NUM> in a light-axis direction (may be referred to as "Z-direction") and a position thereof can be changed by the driving device <NUM>. The focusing detection device <NUM> includes a light emitting device <NUM>, a first lens <NUM>, a first light splitting element <NUM>, a second light splitting element <NUM>, a second lens <NUM>, and a two-dimensional light sensor <NUM>. The driving device <NUM> is connected to the information processing apparatus <NUM> to be controllable so as to drive the object lens <NUM> in the Z-direction on the basis of a signal transmitted from the information processing apparatus <NUM>. The imaging lens <NUM> is arranged between the focusing detection device <NUM> and the camera <NUM> in the Z-direction.

The camera <NUM> is arranged so as to receive an observation light <NUM> emitted from a sample via the microscope <NUM>. Hereinafter, the camera <NUM> captures a reception light image on the basis of a signal transmitted from the information processing apparatus <NUM> so as to transmit image data of the reception light image to the information processing apparatus <NUM>.

The information processing apparatus <NUM> is connected to the microscope <NUM> so as to control a driving process of the driving device <NUM> included in the microscope <NUM>. The information processing apparatus <NUM> is connected to the microscope <NUM> so as to acquire a pattern (may be referred to as "light receiving pattern") of a focus detecting light (reflected light) <NUM> via the two-dimensional light sensor <NUM> included in the microscope <NUM>. The information processing apparatus <NUM> is connected to the camera <NUM>, and further controls a capturing process and the camera <NUM> so as to acquire image data of a captured sample.

Processes of the above-mentioned whole information processing system <NUM> will be explained. Note that the following processes can be executed in a different order. A part of the following processes may be skipped.

Hereinafter, with reference to <FIG>, a process for acquiring a light receiving pattern of the focus detecting light (reflected light) <NUM> emitted from the container <NUM> will be explained.

The focusing detection device <NUM> causes the light emitting device <NUM> to emit light so as to emit a focus detecting light. In this case, a focus detecting light (irradiation light) <NUM> emitted from the light emitting device <NUM> is converted into a parallel light by the first lens <NUM>, is reflected from the first light splitting element <NUM>, is reflected from the second light splitting element <NUM>, is condensed by the object lens <NUM>, and further is led to the container <NUM>. The focus detecting light (reflected light) <NUM> reflected from a bottom surface of the container <NUM> goes through the object lens <NUM> again, is reflected from the second light splitting element <NUM>, passes through the first light splitting element <NUM>, is condensed by the second lens <NUM>, and further is received by the two-dimensional light sensor <NUM>. The information processing apparatus <NUM> acquires a light receiving pattern from the two-dimensional light sensor <NUM> as image data.

In a case where a position (may be referred to as "Z-position") in the Z-direction of the object lens <NUM> is changed by the driving device <NUM> and further the above-mentioned process is executed, a state of the focus detecting light (reflected light) <NUM> is changed depending on difference in a Z-position of the object lens <NUM>, in other words, difference in an irradiation state of the focus detecting light (irradiation light) <NUM> with respect to the container <NUM>, and thus a light receiving pattern to be projected on the two-dimensional light sensor <NUM> is changed. In this case, in a case where a bottom surface of the container <NUM> is flat (planar) as illustrated in <FIG> (<NUM>), a light receiving pattern of the focus detecting light (reflected light) <NUM> is regular, and thus focus detection is easy. However, in a case where a bottom surface of the container <NUM> is a curved surface as illustrated in <FIG> (<NUM>), a light receiving pattern of the focus detecting light (reflected light) <NUM> is irregular, and thus execution of focus detection using the above-mentioned light receiving pattern alone is difficult.

Herein, with reference to <FIG>, a light receiving pattern acquired by the information processing apparatus <NUM> will be explained. Hereinafter, a light receiving pattern <NUM> in a case where a bottom surface of the container <NUM> is planar and a light receiving pattern <NUM> in a case where a bottom surface of the container <NUM> is a curved surface will be explained.

With reference to <FIG>, the light receiving pattern <NUM> in a case where a bottom surface of the container <NUM> is planar will be explained. <FIG> is a diagram illustrating one example of the light receiving pattern <NUM> of a focus detecting light according to the embodiment.

With reference to <FIG> (<NUM>), a relation will be explained between the partition <NUM> of a container and a reflection position of the focus detecting light (irradiation light) <NUM> caused by difference in a Z-position of the object lens <NUM>. Z<NUM> that is a Z-position of the object lens <NUM> is a position in which a bottom surface of the partition <NUM> in a container coincides with a focus. As indicated in an example illustrated in <FIG> (<NUM>), in a case where the object lens <NUM> is at Z<NUM>, the focus detecting light (irradiation light) <NUM> reflects from a bottom surface of the partition <NUM> in the container. Furthermore, Z+<NUM> that is a Z-position of the object lens <NUM> is a position obtained by moving the object lens <NUM> close to the container <NUM> in the Z-direction. As indicated in the example illustrated in <FIG> (<NUM>), in a case where the object lens <NUM> is at Z+<NUM>, the focus detecting light (irradiation light) <NUM> reflects from a position in the Z-direction that is farther from the object lens <NUM> than a bottom surface of the partition <NUM> in the container. Z-<NUM> that is a Z-position of the object lens <NUM> is a position obtained by moving the object lens <NUM> away from the container <NUM> in the Z-direction. As indicated in the example illustrated in <FIG> (<NUM>), in a case where the object lens <NUM> is at Z-<NUM>, the focus detecting light (irradiation light) <NUM> reflects from a position in the Z-direction that is closer to the object lens <NUM> than a bottom surface of the partition <NUM> in the container.

With reference to <FIG> (<NUM>), a light receiving pattern of the focus detecting light (reflected light) <NUM> caused by difference in a Z-position of the object lens <NUM>, which is projected on the two-dimensional light sensor <NUM>, will be explained. As indicated in the example illustrated in <FIG> (<NUM>), in a case where the object lens <NUM> is at Z<NUM>, a light receiving pattern of the focus detecting light (reflected light) <NUM> is obtained as a spot that is clear and further the smallest. On the other hand, as indicated in the example illustrated in <FIG> (<NUM>), in a case where the object lens <NUM> is at Z+<NUM> or Z-<NUM>, a light receiving pattern of the focus detecting light (reflected light) <NUM> is obtained as an unclear spot. In other words, in a case where a bottom surface of the container <NUM> is planar as illustrated in <FIG> (<NUM>), a light receiving pattern of the focus detecting light (reflected light) <NUM> is regular, and thus focus detection is easy.

With reference to <FIG>, the light receiving pattern <NUM> in a case where a bottom surface of the container <NUM> is a curved surface will be explained. <FIG> is a diagram illustrating one example of the light receiving pattern <NUM> of a focus detecting light according to the embodiment.

With reference to <FIG> (<NUM>), a relation will be explained between the partition <NUM> of a container and a reflection position of the focus detecting light (irradiation light) <NUM> caused by difference in a Z-position of the object lens <NUM>. Herein, Z<NUM> that is a Z-position of the object lens <NUM> is a position in which a bottom surface of the partition <NUM> in a container coincides with a focus. As indicated in an example illustrated in <FIG> (<NUM>), in a case where the object lens <NUM> is at Z<NUM>, the focus detecting light (irradiation light) <NUM> reflects from a bottom surface of the partition <NUM> in the container. Furthermore, Z+<NUM> that is a Z-position of the object lens <NUM> is a position obtained by moving the object lens <NUM> close to the container <NUM> in the Z-direction. As indicated in the example illustrated in <FIG> (<NUM>), in a case where the object lens <NUM> is at Z+<NUM>, the focus detecting light (irradiation light) <NUM> reflects from a position in the Z-direction that is farther from the object lens <NUM> than a bottom surface of the partition <NUM> in the container. Z-<NUM> that is a Z-position of the object lens <NUM> is a position obtained by moving the object lens <NUM> away from the container <NUM> in the Z-direction. As indicated in the example illustrated in <FIG> (<NUM>), in a case where the object lens <NUM> is at Z-<NUM>, the focus detecting light (irradiation light) <NUM> reflects from a position in the Z-direction that is closer to the object lens <NUM> than a bottom surface of the partition <NUM> in the container.

With reference to <FIG> (<NUM>), a light receiving pattern of the focus detecting light (reflected light) <NUM> caused by difference in a Z-position of the object lens <NUM>, which is projected on the two-dimensional light sensor <NUM>, will be explained. As indicated in the example illustrated in <FIG> (<NUM>), regardless of Z-positions of the object lens <NUM> at Z<NUM>, Z+<NUM>, and Z-<NUM>, a light receiving pattern of the focus detecting light (reflected light) <NUM> by the object lens <NUM> is obtained as an unclear spot. In other words, in a case where a bottom surface of the container <NUM> is a curved surface as illustrated in <FIG> (<NUM>), a light receiving pattern of the focus detecting light (reflected light) <NUM> becomes irregular, and thus execution of focus detection using the above-mentioned light receiving pattern alone is difficult.

Herein, with reference to <FIG>, in a case where a bottom surface of the container <NUM> is a curved surface and further a sample <NUM> is stored in the partition <NUM>, a process for acquiring a reference pattern (reference light receiving pattern) of a light receiving pattern will be explained. <FIG> is a diagram illustrating one example of a reference light receiving pattern specifying process according to the embodiment. Hereinafter, a reflection position of the focus detecting light (irradiation light) <NUM>, a light receiving pattern of the focus detecting light (reflected light) <NUM>, and image data of the sample <NUM> will be explained, and then a reference light receiving pattern acquiring process will be explained.

With reference to <FIG> (<NUM>), a relation will be explained between the partition <NUM> of a container and a reflection position of the focus detecting light (irradiation light) <NUM> caused by difference in a Z-position of the object lens <NUM>. Z<NUM> that is a Z-position of the object lens <NUM> is a position in which the sample <NUM> of the partition <NUM> in a container coincides with a focus. As indicated in an example illustrated in <FIG> (<NUM>), in a case where the object lens <NUM> is at Z<NUM>, the focus detecting light (irradiation light) <NUM> reflects from the sample <NUM> of the partition <NUM> in the container. Furthermore, Z+<NUM> that is a Z-position of the object lens <NUM> is a position obtained by moving the object lens <NUM> close to the container <NUM> in the Z-direction. As indicated in the example illustrated in <FIG>, in a case where the object lens <NUM> is at Z+<NUM>, the focus detecting light (irradiation light) <NUM> reflects from a position in the Z-direction that is farther from the object lens <NUM> than the sample <NUM> of the partition <NUM> in the container. Z-<NUM> that is a Z-position of the object lens <NUM> is a position obtained by moving the object lens <NUM> away from the container <NUM> in the Z-direction. As indicated in the example illustrated in <FIG> (<NUM>), in a case where the object lens <NUM> is at Z-<NUM>, the focus detecting light (irradiation light) <NUM> reflects from a position in the Z-direction that is closer to the object lens <NUM> than the sample <NUM> of the partition <NUM> in the container.

With reference to <FIG> (<NUM>), a light receiving pattern of the focus detecting light (reflected light) <NUM> caused by difference in a Z-position of the object lens <NUM>, which is projected on the two-dimensional light sensor <NUM>, will be explained. As indicated in the example illustrated in <FIG> (<NUM>), regardless of Z-positions of the object lens <NUM> at Z<NUM>, Z+<NUM>, and Z-<NUM>, a light receiving pattern of the focus detecting light (reflected light) <NUM> by the object lens <NUM> is obtained as an unclear spot. In other words, in a case where a bottom surface of the container <NUM> is a curved surface as illustrated in <FIG> (<NUM>), similar to the above-mentioned case illustrated in <FIG>, a light receiving pattern of the focus detecting light (reflected light) <NUM> becomes irregular, and thus execution of focus detection using the above-mentioned light receiving pattern alone is difficult.

With reference to <FIG> (<NUM>), image data of the sample <NUM> due to difference in a Z-position of the object lens <NUM>, which is captured by the camera <NUM>, will be explained. As indicated by the example illustrated in <FIG> (<NUM>), in a case where the object lens <NUM> is at Z<NUM>, the sample <NUM> is captured as a clear image. On the other hand, as indicated in the example illustrated in <FIG> (<NUM>), in a case where the object lens <NUM> is at Z+<NUM> or Z-<NUM>, the sample <NUM> is captured as an unclear image.

With reference to <FIG>, a process for acquiring a reference light receiving pattern will be explained in a case where a bottom surface of the container <NUM> exemplified in the example illustrated in <FIG> is a curved surface and the container <NUM> stores the sample <NUM> in the partition <NUM> thereof. Note that details of the reference light receiving pattern acquiring process will be mentioned later in the following <<NUM>. Flow of Process of Information Processing System <NUM>> and <<NUM>-<NUM>. Flow of Reference Light Receiving Pattern Acquiring Process>.

First, an illuminating device (not illustrated) irradiates the sample <NUM> stored in the partition <NUM> of the container <NUM> with an illumination. In this case, the observation light <NUM> emitted from the sample <NUM> which is caused by irradiation of the illumination passes through the object lens <NUM>, passes through the second light splitting element <NUM>, is condensed by the imaging lens <NUM>, and is made incident on the camera <NUM>.

Next, the driving device <NUM> of the microscope <NUM> drives the object lens <NUM> in accordance with a signal of the information processing apparatus <NUM> so as to change a Z-position of the object lens <NUM>. The camera <NUM> repeats a capturing process, and further transfers the captured image data to the information processing apparatus <NUM>. In this case, the information processing apparatus <NUM> associates the transferred image data with a corresponding Z-position of the object lens <NUM>.

Subsequently, the information processing apparatus <NUM> analyzes the acquired image data, selects image data in which a focus coincides most with the sample <NUM>, and determines a Z-position of the object lens <NUM> corresponding to the selected image data to be a focusing position Zf. In this case, the information processing apparatus <NUM> uses a feature amount (e.g., luminance value, contrast value) of image data so as to specify the focusing position Zf.

After lights-out of an illuminating device (not illustrated), the driving device <NUM> of the microscope <NUM> drives the object lens <NUM> in accordance with a signal of the information processing apparatus <NUM> so as to move the object lens <NUM> to the focusing position Zf.

Finally, the information processing apparatus <NUM> acquires a light receiving pattern on the two-dimensional light sensor <NUM> of the focus detecting light (reflected light) <NUM> returned from the container <NUM>, which is caused by light emission of the light emitting device <NUM> of the microscope <NUM>, and further determines the light receiving pattern to be a focused reference pattern, in other words, a reference light receiving pattern. In this case, the information processing apparatus <NUM> stores therein the acquired reference light receiving pattern and the focusing position Zf in association with each other.

Note that the information processing apparatus <NUM> may execute the above-mentioned reference light receiving pattern acquiring process every time capturing the container <NUM>. The information processing apparatus <NUM> may preliminarily execute the reference light receiving pattern acquiring process for each combination of types of the object lens <NUM> and the container <NUM>, may store reference light receiving patterns of the respective combinations in a storage <NUM> of the information processing apparatus <NUM> which is to be mentioned later, and may read and use it in a sample capturing process as needed.

With reference to <FIG>, a process will be explained for capturing a sample on the basis of a reference light receiving pattern obtained by the above-mentioned reference light receiving pattern acquiring process. Note that details of the sample capturing process will be mentioned later in the following <<NUM>. Flow of Process of Information Processing System <NUM>> and <<NUM>-<NUM>. Flow of Sample Capturing Process>.

First, the information processing apparatus <NUM> acquires a light receiving pattern on the two-dimensional light sensor <NUM> of the focus detecting light (reflected light) <NUM> returned from the container <NUM>, which is caused by light emission of the light emitting device <NUM> of the microscope <NUM>. In this case, the information processing apparatus <NUM> transmits a signal to the driving device <NUM> of the microscope <NUM> so as to acquire a plurality of light receiving patterns while driving the object lens <NUM> in the Z-direction.

Next, the information processing apparatus <NUM> specifies a light receiving pattern that coincides with a reference light receiving pattern, determines a Z-position at which the above-mentioned light receiving pattern is obtained to be a focusing position (capturing reference position) Zf1. In this case, the information processing apparatus <NUM> may execute a coincidence determination of the light receiving pattern by using a cross-correlation value of images, or may calculate a similarity degree by using a machine learning model so as to execute the coincidence determination.

After lights-out of the light emitting device <NUM>, an illuminating device (not illustrated) irradiates the sample <NUM> stored in the partition <NUM> of the container <NUM> with an illumination. The driving device <NUM> of the microscope <NUM> drives the object lens <NUM> in accordance with a signal of the information processing apparatus <NUM> so as to move the object lens <NUM> to a focusing position Zf1. Finally, the camera <NUM> captures the sample <NUM> while determining the focusing position Zf1 to be a reference Z-position, and further transfers the captured image data to the information processing apparatus <NUM>.

Hereinafter, a problem of a technology that is used as general automatic focusing and a general capturing process will be explained as a reference technology, and then effects of the information processing system <NUM> will be explained.

In a reference technology disclosed in Patent Literature <NUM>, a focus detecting light emitted from a light source is converted into a parallel light by a first lens, is reflected from a beam splitter, is condensed by an object lens, is reflected from a plate, passes through the object lens again, passes through the beam splitter, passes through an aperture, is condensed by a second lens, and is made incident on an image capturing device. In this case, if the object lens is driven in a light-axis direction, a position and a size of a spot of a focus detecting light which is captured by the image capturing device changes. A focusing point with respect to the plate of the object lens is determined on the basis of a size and a position of a spot of the focus detecting light. The above-mentioned technology has the following problem.

First, in the recent research in biology and pharmacy, it is important to execute a cell experiment under a condition closer to in vivo by not plane culturing but three-dimensional culturing. In such a tide, generation of a cell aggregate is generally employed. For example, in Patent Literature <NUM>, a cell container whose cell culturing surface is a curved surface is used for facilitating formation of a cell aggregate. In Patent Literature <NUM>, a cell container in which a three-dimensional structure is formed on a cell culturing surface thereof is used for facilitating formation of a cell aggregate.

On the other hand, a technology of automatic focusing in a reference technology disclosed in Patent Literature <NUM> has a hypothesis that a bottom surface of a cell container is planar and uniform, in a case of a cell container (see Patent Literature <NUM>, for example) whose cell culturing surface is a curved surface or a cell container (see Patent Literature <NUM>, for example) in which a three-dimensional structure is formed on a cell culturing surface thereof, a focus detecting light is not normally reflected, a spot of a focus detecting light which is acquired by an image capturing device is deformed, and thus there presents a problem that easy determination of a focusing point may be impossible. Specifically, even if an object lens is focused with respect to a cell container, a spot of a focus detecting light in an image capturing device is deformed due to aberration of a focus detecting light caused by a curved surface of a cell culturing surface in a case of a cell container whose cell culturing surface is a curved surface, or due to diffusion of a focus detecting light caused by a three-dimensional structure on a cell culturing surface in a case of a cell container in which a three-dimensional structure is formed on a cell culturing surface thereof, and thus determination of a size and a position may be impossible.

In another reference technology, there presents a method in which a cell image to be a target is captured every time a position of an object lens is changed along a light axis thereof, and a focusing point is determined by using a feature amount (e.g., luminance value, contrast value) of the image. According to the above-mentioned method, it is possible to search a focusing point regardless of a state of a bottom surface of a cell container.

However, in the above-mentioned technology, a cell image must be captured for each position of an object lens, and thus there presents a problem that a focus searching time interval is increased due to a capturing exposure time interval, a transferring time interval of data from an image capturing device to a processing device, an analysis time interval of an image, and the like. In the above-mentioned technology, in a case where a capturing type of a cell is an epifluorescent type, a fluorescence confocal type, or the like; a cell is irradiated with excitation light, and thus there presents a problem leading to color deterioration of a fluorescent dye, optical damage against a cell, etc..

The information processing system <NUM> is configured to: acquire pieces of image data obtained by capturing an image of the sample <NUM> to be observed by using the microscope <NUM> at respective positions while driving the object lens <NUM> of the microscope <NUM> in a light-axis direction of an illumination with which the sample <NUM> is irradiated, the image being obtained by condensing light by using the object lens <NUM>; specify a focusing position Zf of the object lens <NUM> at which a focus of the sample <NUM> is determined based on the pieces of the image data; in a case where the object lens <NUM> is arranged in the focusing position Zf, acquire a light receiving pattern obtained by receiving, via the object lens <NUM>, the focus detecting light (reflected light) <NUM> emitted to the sample <NUM> via the object lens <NUM> as a reference light receiving pattern; and store the reference light receiving pattern in the storage <NUM>. In this case, in the information processing system <NUM>, capturing of the sample <NUM> is executed by using the microscope <NUM> in which the object lens <NUM> is arranged at the focusing position Zf at which the reference light receiving pattern is acquired.

The information processing system <NUM> is configured to: in a case where the sample <NUM> is captured again, acquire a plurality of light receiving patterns at respective positions of the object lens <NUM> driven along the light-axis direction in a case where the sample <NUM> is irradiated with the focus detecting light; specify a capturing reference position (focusing position) Zf1 of the object lens <NUM> that captures the sample <NUM> by using a similarity degree between the reference light receiving pattern stored in the storage <NUM> and each of the plurality of light receiving patterns; and capture the sample <NUM> by using the microscope <NUM> in which the object lens is arranged at the capturing reference position Zf1.

The information processing system <NUM> is capable of executing focus detection based on a light that is reflected from a bottom surface of a sample container, even in a case of a cell container for forming a cell aggregate, which has a special shape and thus provides irregular reflection from the bottom surface. In other words, regardless of a case where a bottom surface of a sample container is planar or a curved surface, the information processing system <NUM> is capable of specifying a focusing position of the object lens <NUM>, so that it is possible to quickly execute automatic focusing of the microscope <NUM>.

The information processing system <NUM> does not need a long focus searching time interval for an exposure time interval of capturing, a transferring time interval of data from an image capturing device to a processing device, or a time interval of image analysis, and thus in a case where a capturing type of a cell is an epifluorescent type or a fluorescence confocal type, there presents little risk of occurrence of color deterioration of a fluorescent dye or optical damage against a cell. In other words, the information processing system <NUM> is capable of specifying a focusing position regardless of a type of a sample (capturing target) so as to execute capturing.

With reference to <FIG>, functional configurations of devices included in the information processing system <NUM> illustrated in <FIG> will be explained. Hereinafter, details of a configuration example of the information processing apparatus <NUM> according to the embodiment, a configuration example of the microscope <NUM> according to the embodiment, and a process to be executed by the camera <NUM> according to the embodiment will be explained in this order.

With reference to <FIG>, a configuration example of the information processing apparatus <NUM> illustrated in <FIG> will be explained. <FIG> is a block diagram illustrating a configuration example of devices in the information processing system <NUM> according to the embodiment. The information processing apparatus <NUM> includes an input unit <NUM>, an output unit <NUM>, a communication unit <NUM>, the storage <NUM>, and a control unit <NUM>.

The input unit <NUM> is in charge of input of various kinds of information into the above-mentioned information processing apparatus <NUM>. For example, the input unit <NUM> is realized by a mouse, a keyboard, and the like so as to receive an input such as setting information for the above-mentioned information processing apparatus <NUM>.

The output unit <NUM> is in charge of output of various kinds of information from the above-mentioned information processing apparatus <NUM>. For example, the output unit <NUM> is realized by a display or the like so as to output setting information and the like stored in the above-mentioned information processing apparatus <NUM>.

The communication unit <NUM> is in charge of data communication with other devices. For example, the communication unit <NUM> executes data communication with other communication devices via a router. The communication unit <NUM> is capable of executing data communication with a terminal (not illustrated) of an operator.

The storage <NUM> stores therein various kinds of information to be referred in operation of the control unit <NUM>, various kinds of information that is acquired in operation of the control unit <NUM>, etc. The storage <NUM> includes a light receiving pattern storage <NUM>. Herein, the storage <NUM> may be realized by using a semiconductor memory element such as a Random Access Memory (RAM) and a flash memory, a storage such as a hard disk and an optical disk, or the like. Note that in the example illustrated in <FIG>, the storage <NUM> is arranged inside of the information processing apparatus <NUM>; however, the storage <NUM> may be arranged outside of the information processing apparatus <NUM>, and further a plurality of storages may be arranged.

The light receiving pattern storage <NUM> stores therein a reference light receiving pattern that is acquired by a second acquisition unit <NUM> of the control unit <NUM>. For example, the light receiving pattern storage <NUM> stores therein a reference light receiving pattern for each combination of a type of the container <NUM> and a type of the object lens <NUM>.

The control unit <NUM> is in charge of overall control of the above-mentioned information processing apparatus <NUM>. The control unit <NUM> includes a first acquisition unit <NUM>, a first specifying unit <NUM>, the second acquisition unit <NUM>, a third acquisition unit <NUM>, a second specifying unit <NUM>, an image capturing unit <NUM>, and a drive unit <NUM>. Herein, the control unit <NUM> may be realized by using an electric circuit such as a Central Processing Unit (CPU) and a Micro Processing Unit (MPU), or an integrated circuit such as an Application Specific Integrated Circuit (ASIC) and a Field Programmable Gate Array (FPGA).

The first acquisition unit <NUM> acquires pieces of image data obtained by capturing an image of a target object (sample <NUM>) to be observed by using the microscope <NUM> at respective positions while driving the object lens <NUM> of the microscope <NUM> in a light-axis direction of an illumination with which the sample <NUM> is irradiated, the image being obtained by condensing light by using the object lens <NUM>. In this case, the first acquisition unit <NUM> acquires image data captured by the camera <NUM>.

The first acquisition unit <NUM> includes the plurality of partitions <NUM> each of which stores therein the sample <NUM>, and acquires image data with respect to the first partition <NUM>-<NUM> of the container <NUM>. In this case, the first acquisition unit <NUM> may acquire image data of a control sample that is different from the sample <NUM> stored in the first partition <NUM>-<NUM> of the container <NUM>. Note that the first acquisition unit <NUM> may store the acquired image data in the storage <NUM>.

The first specifying unit <NUM> specifies a focusing position Zf of the object lens <NUM> at which a focus of the sample <NUM> is determined on the basis of image data. For example, the first specifying unit <NUM> specifies a focusing position Zf by using a feature amount of each piece of image data. In other words, the first specifying unit <NUM> specifies image data in which a focus coincides from among pieces of image data by using a luminance value and/or a contrast value as a feature amount of the image data, and further specifies a position of the object lens <NUM> of the corresponding piece of image data as a focusing position Zf. Note that a specifying method of the first specifying unit <NUM> is not limited to the above-mentioned method.

In a case where having arranged the object lens <NUM> at a focusing position Zf, the second acquisition unit <NUM> acquires, as a reference light receiving pattern, a light receiving pattern obtained by receiving, via the object lens <NUM>, the focus detecting light (reflected light) <NUM> with which the sample <NUM> is irradiated via the object lens <NUM>, and further stores the reference light receiving pattern in the storage <NUM>. For example, the second acquisition unit <NUM> acquires a reference light receiving pattern in a case where the object lens <NUM> specified from a feature amount of each piece of image data of the first partition <NUM>-<NUM> in the container <NUM> is arranged at a focusing position Zf, and further stores the reference light receiving pattern in the light receiving pattern storage <NUM> of the storage <NUM>.

The second acquisition unit <NUM> stores a reference light receiving pattern in the light receiving pattern storage <NUM> of the storage <NUM> for each combination of a type of the container <NUM> and a type of the object lens <NUM>. The second acquisition unit <NUM> stores reference light receiving patterns in the light receiving pattern storage <NUM> of the storage <NUM>, for example, "pattern AA" is a reference light receiving pattern with respect to a combination of a type "container A" of the container <NUM> and a type "object lens A" of the object lens <NUM>, "pattern AB" is a reference light receiving pattern with respect to a combination of a type "container A" of the container <NUM> and a type "object lens B" of the object lens <NUM>, and the like.

The second acquisition unit <NUM> may store a reference light receiving pattern in the light receiving pattern storage <NUM> of the storage <NUM> for each type of the container <NUM>, or may store a reference light receiving pattern in the light receiving pattern storage <NUM> of the storage <NUM> for each type of the object lens <NUM>. The second acquisition unit <NUM> stores reference light receiving patterns in the light receiving pattern storage <NUM> of the storage <NUM>, for example, types "container A", "container B", and "container C", ··· , of the container <NUM> are respectively associated with reference light receiving patterns "pattern A-<NUM>", "pattern B-<NUM>", and "pattern C-<NUM>", ···. The second acquisition unit <NUM> stores reference light receiving patterns in the light receiving pattern storage <NUM> of the storage <NUM>, for example, types "object lens A", "object lens B", and "object lens C", ··· , of the object lens <NUM> are respectively associated with reference light receiving patterns "pattern A-<NUM>", "pattern B-<NUM>", and "pattern C-<NUM>", ···.

In a case where capturing the sample <NUM> again, the third acquisition unit <NUM> acquires light receiving patterns when the sample <NUM> is irradiated with a focus detecting light at respective positions obtained by driving the object lens <NUM> in a light-axis direction thereof. For example, in a case where capturing the sample <NUM> of which a type of the container <NUM> is the same and a type of the object lens <NUM> is the same, the third acquisition unit <NUM> is capable of acquiring light receiving patterns for a part or a whole of the sample <NUM> to be captured. The third acquisition unit <NUM> acquires light a receiving pattern with respect to the second partition <NUM>-<NUM> of the container <NUM>. In other words, in a case where capturing the sample <NUM> again, when capturing the samples <NUM> of the second partition <NUM>-<NUM> and the following thereof after acquisition of a reference light receiving pattern in the first partition <NUM>-<NUM> of the container <NUM>, the third acquisition unit <NUM> is capable of acquiring a light receiving pattern with respect to a part or a whole of the sample <NUM> to be captured.

The second specifying unit <NUM> specifies a capturing reference position Zf1 of the object lens <NUM> for capturing the sample <NUM> by using a similarity degree between a reference light receiving pattern stored in the storage <NUM> and each light receiving pattern. For example, the second specifying unit <NUM> is capable of specifying a capturing reference position Zf1 by using a similarity degree between already-acquired reference light receiving pattern and each light receiving pattern acquired in capturing. The second specifying unit <NUM> specifies a capturing reference position Zf1 by using a similarity degree between a reference light receiving pattern acquired in the first partition <NUM>-<NUM> and each light receiving pattern acquired in the second partition <NUM>-<NUM>. In other words, in the container <NUM> storing therein a plurality of partitions and the sample <NUM> is stored in each of the partitions, the second specifying unit <NUM> specifies a capturing reference position Zf1 by using a similarity degree between a reference light receiving pattern acquired in the first partition <NUM>-<NUM> for specifying the reference light receiving pattern and each light receiving pattern acquired in the second partition <NUM>-<NUM> for capturing the sample <NUM>.

In this case, for example, the second specifying unit <NUM> calculates a plurality of cross-correlation values between the reference light receiving pattern and the light receiving patterns, and specifies, as the capturing reference position Zf1, a position of the object lens <NUM> where a light receiving pattern corresponding to the highest cross-correlation value of the cross-correlation values is acquired. Specifically, in a case where calculating that a cross-correlation value between a reference light receiving pattern and "light receiving pattern A" is "<NUM>", a cross-correlation value between the reference light receiving pattern and "light receiving pattern B" is "<NUM>", and a cross-correlation value between the reference light receiving pattern and "light receiving pattern C" is "<NUM>"; the second specifying unit <NUM> specifies, as a capturing reference position Zf1, a position of the object lens <NUM> at which "light receiving pattern C" whose cross-correlation value, in other words, similarity degree approximates to one. Note that a method of the second specifying unit <NUM> for calculating a cross-correlation value is not particularly limited.

The second specifying unit <NUM> inputs each of the plurality of light receiving patterns and the reference light receiving pattern into a machine learning model that is trained to output a similarity degree between a pair of light receiving patterns in response to an input of the pair of light receiving patterns; acquires a plurality of similarity degrees respectively corresponding to the plurality of light receiving patterns; and specifies, as a capturing reference position Zf1, a position of the object lens <NUM> where a light receiving pattern corresponding to the highest similarity degree of the plurality of similarity degrees is acquired. Specifically, in a case where a similarity degree between a reference light receiving pattern and "light receiving pattern A" is output to be "<NUM>", a similarity degree between the reference light receiving pattern and "light receiving pattern B" is output to be "<NUM>", and a similarity degree between the reference light receiving pattern and "light receiving pattern C" is output to be "<NUM>" by the machine learning model, the second specifying unit <NUM> specifies, as a capturing reference position Zf1, a position of the object lens <NUM> where "light receiving pattern C" whose similarity degree approximates to one is acquired. Note that a method of the second specifying unit <NUM> for calculating a similarity degree is not particularly limited.

The image capturing unit <NUM> executes capturing on the sample <NUM> by using the microscope <NUM> in which the object lens <NUM> is arranged at a focusing position Zf where a reference light receiving pattern is acquired. For example, in a case where capturing the sample <NUM> whose type of the container <NUM> is the same and further whose type of the object lens <NUM> is the same, the image capturing unit <NUM> transmits a signal including a capturing instruction to the camera <NUM> (image capturing device) to be capable of executing capturing on the sample <NUM>. The image capturing unit <NUM> executes capturing on the sample <NUM> that is stored in the second partition <NUM>-<NUM> by using the microscope <NUM> in which the object lens <NUM> is set at a capturing reference position Zf1. In other words, in a case where capturing the samples <NUM> in the second partition <NUM>-<NUM> and the following after acquiring a reference light receiving pattern in the first partition <NUM>-<NUM> of the container <NUM>, the image capturing unit <NUM> transmits a signal including a capturing instruction to the camera <NUM> (image capturing device) to be capable of execute capturing on the samples <NUM> stored in respective partitions, for example, the second partition <NUM>-<NUM>, the third partitions <NUM>-<NUM>, ···.

The drive unit <NUM> controls drive of the driving device <NUM> of the microscope <NUM>. For example, the drive unit <NUM> transmits a signal to the driving device <NUM> so as to drive the object lens <NUM> that is connected with the driving device <NUM>. Specifically, the drive unit <NUM> transmits a signal to the driving device <NUM> in order to acquire a light receiving pattern so as to drive the object lens <NUM> connected with the driving device <NUM> in a light-axis direction by ΔZ for each step. The drive unit <NUM> transmits a signal to the driving device <NUM> in order to capture a capturing reference position Zf1 so as to drive the object lens <NUM> connected with the driving device <NUM> to a capturing reference position Zf1.

With reference to <FIG>, a configuration example of the microscope <NUM> illustrated in <FIG> will be explained. The microscope <NUM> includes the object lens <NUM>, the focusing detection device <NUM>, the driving device <NUM>, and the imaging lens <NUM>.

The object lens <NUM> is arranged between the container <NUM> and the focusing detection device <NUM> in a light-axis direction thereof, is driven by the driving device <NUM>, and transmits or condenses an illumination light emitted from an illuminating device (not illustrated) in capturing, a focus detecting light emitted from the light emitting device <NUM> of the focusing detection device <NUM>, and a focus detecting light reflected from a bottom surface of the container <NUM> and/or the sample <NUM>.

The focusing detection device <NUM> is arranged between the object lens <NUM> and the imaging lens <NUM> in a light-axis direction, and includes the light emitting device <NUM>, the first lens <NUM>, the first light splitting element <NUM>, the second light splitting element <NUM>, the second lens <NUM>, and the two-dimensional light sensor <NUM>.

The light emitting device <NUM> irradiates the sample <NUM> in the container <NUM> with a focus detecting light via the object lens <NUM>. In this case, the light emitting device <NUM> irradiates the sample <NUM> in the container <NUM> with a focus detecting light on the basis of a signal including a light emitting instruction, which is transmitted from the information processing apparatus <NUM>. The light emitting device <NUM> stops irradiation of a focus detecting light on the basis of a signal including a light emission stopping instruction, which is transmitted from the information processing apparatus <NUM>.

The first lens <NUM> converts the focus detecting light (irradiation light) <NUM> emitted from the light emitting device <NUM> into a parallel light so as to lead it to the first light splitting element <NUM>.

The first light splitting element <NUM> reflects a parallel light of the focus detecting light (irradiation light) <NUM> obtained by conversion of the first lens <NUM> so as to lead it to the second light splitting element <NUM>. The first light splitting element <NUM> transmits the focus detecting light (reflected light) <NUM> that is reflected from the second light splitting element <NUM> so as to lead it to the second lens <NUM>.

The second light splitting element <NUM> reflects the focus detecting light (irradiation light) <NUM> that is reflected from the first light splitting element <NUM> so as to lead it to the object lens <NUM>. The second light splitting element <NUM> reflects the focus detecting light (reflected light) <NUM> passed through the object lens <NUM> so as to lead it to the second lens <NUM>.

The second lens <NUM> condenses the focus detecting light (reflected light) <NUM> reflected from the second light splitting element <NUM>, and further leads it to the two-dimensional light sensor <NUM>.

The two-dimensional light sensor <NUM> receives the focus detecting light (reflected light) <NUM> that is condensed by the second lens <NUM>. The two-dimensional light sensor <NUM> outputs image data of a received light receiving pattern to the information processing apparatus <NUM>.

The driving device <NUM> drives the object lens <NUM> connected therewith. For example, the driving device <NUM> receives a signal transmitted from the drive unit <NUM> of the information processing apparatus <NUM> so as to drive the connected object lens <NUM>.

The imaging lens <NUM> is arranged between the focusing detection device <NUM> and the camera <NUM> in a light-axis direction thereof, and transmits or condenses the observation light <NUM> emitted from the sample <NUM>.

With reference to <FIG>, the camera <NUM> that is the image capturing device illustrated in <FIG> will be explained. The camera <NUM> captures images of the sample <NUM> obtained by condensing light by using the object lens <NUM>, at positions while driving the object lens <NUM> of the microscope <NUM> in a light-axis direction of an illumination with which a target object (sample <NUM>) is irradiated, the images are observed by using the microscope <NUM>. For example, the camera <NUM> receives a signal transmitted from the image capturing unit <NUM> of the information processing apparatus <NUM> so as to capture the sample <NUM> of the container <NUM>.

The camera <NUM> may capture a light receiving pattern of the focus detecting light (reflected light) <NUM>. For example, the camera <NUM> may receive a signal transmitted from the image capturing unit <NUM> of the information processing apparatus <NUM> so as to capture a light receiving pattern of light received by the two-dimensional light sensor <NUM>.

Capturing the sample <NUM> in the container <NUM> by using the camera <NUM> and execution of a process for acquiring a light receiving pattern by using the two-dimensional light sensor <NUM> achieves the following advantage. First, a general focus detecting light uses light whose wavelength is different from that of light used in capturing a cell sample, and thus in a case where a focus detecting light is to be received by the camera <NUM>, restriction arises in a microscope capturing optical system up to the camera <NUM>; however, the above-mentioned process is free from such a disadvantage.

Next, in a focus detecting process in capturing a cell sample, it is ideal to dynamically execute matching of light receiving patterns while continuously moving the object lens <NUM>, and thus a logic circuit for image processing may be connected to the two-dimensional light sensor <NUM> and the following so as to realize a configuration and a process for quickly executing pattern matching. In this case, a built-in ready-made camera for high-sensitivity capturing is generally employed for the camera <NUM> for capturing a cell sample, and thus addition of the above-mentioned circuit is difficult so that it is preferable to employ one other than a cell-sample capturing camera for a light-receptive sensor of a focus detecting light.

With reference to <FIG>, a flow of a process of the information processing system <NUM> according to the embodiment will be explained. Hereinafter, a flow of the reference light receiving pattern specifying process and a flow of the sample capturing process will be explained.

With reference to <FIG>, a flow of a reference light receiving pattern specifying process according to the embodiment will be explained. <FIG> is a flowchart illustrating one example of a flow of a reference light receiving pattern specifying process according to the embodiment. Steps S101 to S110 to be explained hereinafter may be executed in a different order. Furthermore, a part of Steps S101 to S <NUM> to be explained hereinafter may be skipped.

First, the drive unit <NUM> of the information processing apparatus <NUM> controls drive of the driving device <NUM> in the microscope <NUM>, and further moves the object lens <NUM> to a predetermined position Z<NUM> (Step S101).

Next, an illuminating device start to irradiate the sample <NUM> with an illumination (Step S102). In this case, the illuminating device may start irradiation using an illumination on the basis of a signal including an irradiation starting instruction that is transmitted from the control unit <NUM> of the information processing apparatus <NUM>.

Next, the camera <NUM> captures an image of the sample <NUM>, and further records the captured image (Step S103). In this case, the camera <NUM> may capture an image of the sample <NUM> on the basis of a signal including a capturing instruction that is transmitted from the image capturing unit <NUM> of the information processing apparatus <NUM>.

Next, the image capturing unit <NUM> of the information processing apparatus <NUM> executes confirmation whether or not the predetermined number of recordings by the camera <NUM> is completed (Step S104). In this step, in a case where the predetermined number of recordings is completed (Step S104: Yes), the image capturing unit <NUM> shifts the processing to Step S106. On the other hand, in a case where the predetermined number of recordings is not completed (Step S104: No), the image capturing unit <NUM> shifts the processing to Step S105.

Next, the drive unit <NUM> of the information processing apparatus <NUM> controls the driving device <NUM> of the microscope <NUM> to shift the object lens <NUM> by ΔZ (Step S105), and returns the processing to Step S104.

Next, the illuminating device stops irradiation of the sample <NUM> with the illumination (Step S106). In this step, the illuminating device may stop emitting the illumination in accordance with a signal including an emission stopping instruction that is transmitted from the control unit <NUM> of the information processing apparatus <NUM>.

Next, the first specifying unit <NUM> of the information processing apparatus <NUM> specifies a focusing position Zf on the basis of an image of the sample <NUM> captured by the camera <NUM> (Step S107).

Next, the drive unit <NUM> of the information processing apparatus <NUM> controls the driving device <NUM> of the microscope <NUM> to move the object lens <NUM> to the focusing position Zf (Step S108).

Next, the light emitting device <NUM> of the microscope <NUM> starts to emit the focus detecting light (irradiation light) <NUM> (Step S109). In this step, the light emitting device <NUM> may start to emit the focus detecting light (irradiation light) <NUM> in accordance with a signal including an emission starting instruction that is transmitted from the control unit <NUM> of the information processing apparatus <NUM>.

Next, the two-dimensional light sensor <NUM> of the microscope <NUM> receives the focus detecting light (reflected light) <NUM> (Step S110). In this step, the second acquisition unit <NUM> of the information processing apparatus <NUM> obtains and records image data received by the two-dimensional light sensor <NUM> as a reference light receiving pattern, and ends the processing.

With reference to <FIG>, a flow of a sample capturing process according to the embodiment will be explained. <FIG> is a flowchart illustrating one example of a flow of a sample capturing process according to the embodiment. Steps S201 to S208 to be explained hereinafter may be executed in a different order. Furthermore, a part of Steps S201 to S208 to be explained hereinafter may be skipped.

First of all, the drive unit <NUM> of the information processing apparatus <NUM> controls the driving device <NUM> of the microscope <NUM> to move the object lens <NUM> to a predetermined position Z<NUM> (Step S201).

Next, the light emitting device <NUM> of the microscope <NUM> starts to emit the focus detecting light (irradiation light) <NUM> (Step S202). In this step, the light emitting device <NUM> may start to emit the focus detecting light (irradiation light) <NUM> in response to a signal including an emission starting instruction that is transmitted from the control unit <NUM> of the information processing apparatus <NUM>.

Next, the two-dimensional light sensor <NUM> of the microscope <NUM> receives the focus detecting light (reflected light) <NUM> (Step S203). In this step, the third acquisition unit <NUM> of the information processing apparatus <NUM> obtains and records image data received by the two-dimensional light sensor <NUM> as a light receiving pattern.

Next, the third acquisition unit <NUM> of the information processing apparatus <NUM> executes confirmation whether or not the predetermined number of recordings is completed (Step S204). In this step, in a case where the predetermined number of recordings is completed (Step S204: Yes), the third acquisition unit <NUM> shifts the processing to Step S206. On the other hand, in a case where the predetermined number of recordings is not completed (Step S204: No), the third acquisition unit <NUM> shifts the processing to Step S205.

Next, the drive unit <NUM> of the information processing apparatus <NUM> controls the driving device <NUM> of the microscope <NUM> to shift the object lens <NUM> by ΔZ (Step S205), and returns the processing to Step S203.

Next, the light emitting device <NUM> of the microscope <NUM> stops emitting the focus detecting light (irradiation light) <NUM> (Step S206). In this step, the light emitting device <NUM> may stop emitting the focus detecting light (irradiation light) <NUM> in response to a signal including an emission stopping instruction that is transmitted from the control unit <NUM> of the information processing apparatus <NUM>.

Next, the second specifying unit <NUM> of the information processing apparatus <NUM> defines, as a capturing reference position Zf1, a position of the object lens <NUM> where a light receiving pattern of the focus detecting light (reflected light) <NUM> is the closest to the reference light receiving pattern (Step S207).

Next, the camera <NUM> captures an image of the sample <NUM> on the basis of the capturing reference position Zf1 and records it (Step S208), and further ends the processing. In this step, the camera <NUM> may capture the image of the sample <NUM> on the basis of a signal including a capturing instruction that is transmitted from the image capturing unit <NUM> of the information processing apparatus <NUM>.

Finally, effects of the embodiment will be explained. Hereinafter, effects <NUM> to effects <NUM> corresponding to the processes according to the embodiment will be explained.

First, the process according to the above-mentioned embodiment includes:.

Next, the process according to the above-mentioned embodiment includes: capturing the sample <NUM> by using the microscope <NUM> in which the object lens <NUM> is arranged at the focusing position Zf where the reference light receiving pattern is acquired. Thus, the process according to the embodiment is capable of quickly executing automatic focusing, and further of capturing the sample <NUM>.

Next, the process according to the above-mentioned embodiment includes: in a case where the sample <NUM> is captured again, acquiring a plurality of light receiving patterns at respective positions of the object lens <NUM> driven along the light-axis direction in a case where the sample <NUM> is irradiated with the focus detecting light; specifying a capturing reference position Zf1 of the object lens <NUM> that captures the sample <NUM> by using a similarity degree between the reference light receiving pattern stored in the storage <NUM> and each of the plurality of light receiving patterns; and capturing the sample <NUM> by using the microscope <NUM> in which the object lens <NUM> is arranged at the capturing reference position Zf1. Thus, the process according to the embodiment is capable of quickly executing automatic focusing, and further of effectively capturing the sample <NUM>.

Next, the process according to the above-mentioned embodiment includes: acquiring the pieces of image data with respect to the first partition <NUM>-<NUM> of the container <NUM> that includes the plurality of partitions <NUM> and stores the respective samples <NUM> in the plurality of partitions <NUM>; specifying the focusing position Zf by using a feature amount of each of the pieces of image data; acquiring the reference light receiving pattern in a case where the object lens <NUM> is arranged at the focusing position Zf; storing the reference light receiving pattern in the storage <NUM>; acquiring the plurality of light receiving patterns with respect to the second partition <NUM>-<NUM> of the container <NUM>; specifying the capturing reference position Zf1 by using the similarity degrees between the reference light receiving pattern and the plurality of light receiving patterns; and capturing the sample <NUM> stored in the second partition <NUM>-<NUM> by using the microscope <NUM> in which the object lens <NUM> is arranged at the capturing reference position Zf1. Thus, the process according to the embodiment is capable of quickly executing automatic focusing on the container <NUM> including the plurality of partitions <NUM>, and further of effectively capturing the samples <NUM>.

Next, the process according to the above-mentioned embodiment includes: storing the reference light receiving pattern in the storage <NUM> for each of combinations of a type of the container <NUM> and a type of the object lens <NUM>. Thus, in a case where a type of the container <NUM> and a type of the object lens <NUM> are the same, the process according to the embodiment is capable of more quickly executing automatic focusing, and further of effectively capturing the sample <NUM>.

Next, the process according to the above-mentioned embodiment includes: calculating a plurality of cross-correlation values between the reference light receiving pattern and the plurality of light receiving patterns; and specifying, as the capturing reference position Zf1, a position of the object lens <NUM> where a light receiving pattern corresponding to a highest value of the cross-correlation values is acquired. Thus, the process according to the embodiment is capable of more effectively and more quickly executing automatic focusing, and further of effectively capturing the sample <NUM>.

Next, the process according to the above-mentioned embodiment includes: inputting each of the plurality of light receiving patterns and the reference light receiving pattern into a machine learning model that is trained to output a similarity degree between a pair of light receiving patterns in response to an input of the pair of light receiving patterns; acquiring a plurality of similarity degrees respectively corresponding to the plurality of light receiving patterns; and specifying, as the capturing reference position Zf1, a position of the object lens <NUM> where a light receiving pattern corresponding to a highest value of the plurality of similarity degrees is acquired. Thus, the process according to the embodiment is capable of more effectively and more quickly executing automatic focusing by using a machine learning model, and further of effectively capturing the sample <NUM>.

Any of the processing procedures, the controlling procedures, the specific appellations, and the information including various data and parameters, which are described in the specification and the accompanying drawings, may be arbitrarily changed if not otherwise specified.

The illustrated components of the devices are functionally conceptual, and thus they are not to be physically configured as illustrated in the drawings. Specific forms of distribution and integration of the configuration elements of the illustrated devices are not limited to those illustrated in the drawings, and all or some of the devices can be configured by separating or integrating the apparatus functionally or physically in any unit, according to various types of loads, the status of use, etc..

Moreover, all or arbitrary part of the various processing functions, which are to be executed by each of the devices, may be executed by a CPU or a program that is analyzed and executed by the CPU, or may be realized as hardware by a wired logic.

Next, a hardware configuration example of the information processing apparatus <NUM> will be explained. <FIG> is a diagram illustrating a hardware configuration example. As illustrated in <FIG>, the information processing apparatus <NUM> includes a communication device 4a, a Hard Disk Drive (HDD) 4b, a memory 4c, and a processor 4d. The units illustrated in <FIG> are connected to each other by using a bus or the like.

The communication device 4a includes a network interface card or the like so as to communicate with another server. The HDD 4b stores therein a program and a database (DB) for operating the function illustrated in <FIG>.

The processor 4d reads out a program that executes processes similar to the processing units illustrated in <FIG> from the HDD 4b and the like and expands the program into the memory 4c so as to operate a program that executes functions having described with reference to <FIG> and the like. For example, the process executes functions similar to each processing unit included in the information processing apparatus <NUM>. Specifically, the processor 4d reads out, from the HDD 4b and the like, a program including functions similar to those of the first acquisition unit <NUM>, the first specifying unit <NUM>, the second acquisition unit <NUM>, the third acquisition unit <NUM>, the second specifying unit <NUM>, the image capturing unit <NUM>, the drive unit <NUM>, and the like. The processor 4d executes a process for executing processes similar to those of the first acquisition unit <NUM>, the first specifying unit <NUM>, the second acquisition unit <NUM>, the third acquisition unit <NUM>, the second specifying unit <NUM>, the image capturing unit <NUM>, the drive unit <NUM>, and the like.

As described above, the information processing apparatus <NUM> reads out and executes a program so as to operate as an information processing apparatus that executes various processing methods. The computer <NUM> may read out the program from a recording medium by using a medium reading device, and executes the above-mentioned read program so as to realize functions similar to those according the above-mentioned embodiment. Note that the above-mentioned program is not limitedly executed by the information processing apparatus <NUM> alone. For example, the present disclosure may similarly apply to a case where a computer or a server having another hardware configuration executes a program and a case where a computer and a server cooperate to execute a program.

The program may be distributed via a network such as the Internet. The program is recorded in a computer-readable recording medium such as a hard disk, a flexible disk (FD), a CD-ROM, a Magneto-Optical disk (MO), and a Digital Versatile Disc (DVD); and is read out from the recording medium by a computer to be executed.

Claim 1:
An information processing apparatus (<NUM>) comprising:
a first acquisition unit (<NUM>) that is configured to acquire pieces of image data obtained by capturing an image of a target object to be observed by using a microscope (<NUM>) at respective positions of an object lens (<NUM>) of the microscope while driving the object lens (<NUM>) in a light-axis direction of an illumination with which the target object is irradiated, the image being obtained by condensing light by using the object lens (<NUM>);
a first specifying unit (<NUM>) that is configured to specify a focusing position of the object lens (<NUM>) at which a focus of the target object is determined based on the pieces of image data; and characterized by
a second acquisition unit (<NUM>) that is configured to
in a case where the object lens (<NUM>) is arranged in the focusing position, acquire, as a reference light receiving pattern, a light receiving pattern obtained by receiving, via the object lens, reflected light of a focus detecting light (<NUM>) emitted to the target object via the object lens (<NUM>), and
store the reference light receiving pattern in a storage.