CELL IMAGE ANALYSIS METHOD

A cell image analysis method according to this invention includes a step of acquiring a cell image (10) including a cell (90); a step of inputting the cell image to a learned model (6) that has learned classification of the cell into one of two or more types; a step of acquiring an index value (20) indicating accuracy of the classification of the cell that is included in the cell image into one of two or more types based on an analysis result of each of pixels of the cell image output from the learned model; and a step of displaying the acquired index value.

TECHNICAL FIELD

The invention relates to a cell image analysis method, in particular to a cell analysis method for analyzing cells by using a learned model.

BACKGROUND ART

Cell analysis methods for analyzing cells by using a learned model are known in the art. Such a cell analysis method is disclosed in International Publication No. WO 2019-171546, for example.

International Publication No. WO 2019-171546 discloses a cell image analysis method for analyzing images of cells captured by an imaging apparatus. Specifically, International Publication No. WO 2019-171546 discloses a configuration in which images of cells cultivated on a cultivation plate are captured by an imaging device such as a microscope whereby acquiring cell images. The analysis method for analyzing cell images disclosed in International Publication No. WO 2019-171546 classifies the cells in the cell image into normal and abnormal cells based on analysis results of the learned model. Also, International Publication No. WO 2019-171546 discloses a configuration in which each cell is classified by segmentation for classifying each of pixels of the cell into one of categories.

PRIOR ART

Patent Document

Patent Document 1: International Publication No. WO 2019-171546

SUMMARY OF THE INVENTION

Problems to be Solved by the Invention

Although not stated in International Publication No. WO 2019-171546, in a case in which cells captured in a cell image are classified based on analysis results of a learned model, each pixel is classified according to the highest value of an analysis result of the pixel. However, if a cell is classified according to its highest value, the same result is obtained irrespective of whether a difference between the highest value and the second highest value is small or not. In other words, in a case in which a cell captured in a cell image are classified, the same result is obtained based on the highest value of its analysis result irrespective of whether accuracy of classification is high or not. For this reason, user cannot easily immediately grasp accuracy of classification of cells in the cell image when viewing the cell image. Accordingly, it is desired to allow a user to easily grasp accuracy of classification of a cell in a cell image in a cell image analysis method.

The present invention is intended to solve the above problem, and one object of the present invention is to provide a cell image analysis method capable of allowing a user to easily grasp accuracy of classification of a cell in a cell image.

Means for Solving the Problems

In order to attain the aforementioned object, a cell image analysis method according to an aspect of the present invention includes a step of acquiring a cell image(s) including a cell(s); a step of inputting the cell image to a learned model that has learned classification of the cell into one of two or more types; a step of acquiring an index value indicating accuracy of the classification of the cell that is included in the cell image into one of two or more types based on an analysis result of each of pixels of the cell image output from the learned model; and a step of displaying the acquired index value.

Effect of the Invention

In the cell image analysis method according to the aforementioned aspect, as discussed above, a step of acquiring an index value indicating accuracy of the classification of the cell image into one of two or more types based on an analysis result of each of pixels of the cell image output from the learned model; and a step of displaying the acquired index value are included. Consequently, because the index value indicating accuracy of the classification of the cell that is included in the cell image into one of two or more types is displayed, users can easily grasp a probability of classification of a cell in a cell image by seeing the index value. Therefore, it is possible to provide a cell image analysis method capable of allowing users to easily grasp a probability of classification of a cell in a cell image.

MODES FOR CARRYING OUT THE INVENTION

Embodiments embodying the present invention will be described with reference to the drawings.

A configuration of a cell image analysis apparatus100according to an embodiment is now entirely described with reference toFIG.1.

(Configuration of Cell Image Analysis Apparatus)

The cell image analysis apparatus100includes an image acquirer1, a processor2, a storage3, display4, and an input acceptor5as shown inFIG.1.

The image acquirer1is configured to acquire cell images10. Each cell image10includes cells90(seeFIG.2). Specifically, the cell image10is an image including cultivated cells90cultivated in a cultivation container80(seeFIG.3) filled with cultivation solution81(seeFIG.3). In this embodiment, the image acquirer1is configured to acquire the cell image10from a device that is configured to capture the cell image10such as a microscope8to which an imaging apparatus is attached, for example. The image acquirer1includes an input/output interface, for example.

The processor2is configured to analyze the acquired cell images10. The processor2can include a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), a graphics processing unit (GPU), a field-programmable gate array (FPGA) configured for image processing, etc. Also, the processor2, is constructed of a CPU as hardware, etc., includes a controller2a, an image analyzer2b, an image processor2cand a superimposed cell image generator2das functional blocks of software (programs). The processor2can serve as the controller2a, the image analyzer2b, the image processor2c, and the superimposed cell image generator2dby executing programs stored in the storage3. The controller2a, the image analyzer2b, the image processor2c, and the superimposed cell image generator2dmay be individually constructed of a dedicated processor (processing circuit) as hardware.

The controller2ais configured to control the cell image analysis apparatus100. The controller2ais also configured to acquire index values20indicating accuracy of classification of the cells90that are included in the cell image10into one of two or more types. Specifically, the controller2ais configured to acquire representative values20aof probability values21obtained based on the probability values21(seeFIG.4) output by a learned model6as the index value20. The index values20are real values indicating accuracy of classification of the cells90that are included in the cell image10into one of two or more types. In this embodiment, each index value20falls within a range 0 to 100. In this embodiment, the controller2aoutputs one cell image10with one index value20.

More specifically, the controller2ais configured to obtain an index value(s)20of at least one of classification of the cell whether a focus of the cell image10is correct when the cell image is captured, classification of the cell whether a coating material on the cultivation container80(seeFIG.3) of the cell is proper, and classification of the cell whether the number of cultivation days of the cell is proper is acquired. The probability value21is an estimation value of the classification output by the learned model6as an analysis result. As a result of analysis, the learned model6is configured to output the probability value21for every pixel of the cell image10.

Also, the controller2ais configured to direct the display4to display a superimposed cell image50. A configuration in which the index value20is acquired by the controller2a, and the details about the superimposed cell image50will be described later.

In this embodiment, the image analyzer2bis configured to classify each cell90(seeFIG.2) into one of two or more types. Specifically, the image analyzer is configured to use the learned model6, which has learned classification of the cell90into one of two or more types, to classify the cells90that are included in the cell image10into one of two or more types. The learned model6includes a first learned model6a, a second learned model6band a third learned model6c, which are configured to classify the cells into one of two or more types that relate to imaging and cultivation conditions. Normal cells, abnormal cells, the first learned model6a, the second learned model6b, and the third learned model6cwill be described in detail later.

The image processor2cis configured to generate a probability distribution image12(seeFIG.6), which will be described later. Also, the image processor2cis configured to acquire cell areas that are areas of the cells90in the cell image10based on the probability distribution image12. A configuration in which the probability distribution image12is generated by the image processor2c, and a configuration in which the cell areas are acquired by the image processor will be described in detail later.

The superimposed cell image generator2dis configured to generate the superimposed cell image50by superimposing a distribution of probabilities21(seeFIG.4) on the cell image10. A configuration in which the superimposed cell image50is generated by the superimposed cell image generator2dwill be described in detail later.

The storage3is configured to store the cell images10, the first learned model6a, the second learned model6b, and the third learned model6c. Also, the storage3is configured to store various programs to be executed by the processor2. The storage3includes an HDD (Hard Disk Drive) or a storage such as SSD (Solid State Drive), for example.

The display4is configured to display the superimposed cell image50generated by the superimposed cell image generator2d, the index values20, and the frequency distribution22. The display4includes a display device such as an LCD monitor, for example.

The input acceptor5is configured to accept operating inputs from a user. The input acceptor5includes an input device such as a computer mouse, keyboard, etc., for example.

The cell image10is described with reference toFIG.2. Each cell image10includes cultivated cells90. In this embodiment, the cell image10is a microscopic image captured by a microscope8to which an imaging apparatus attached. Each cell image10includes cells90that be able to differentiate (cell potency) as the cultivated cells90. For example, the cells90include IPS cells (Induced Pluripotent Stem cells) and ES cells (Embryonic Stem cells). Undifferentiated cells refer to cells that have cell potency. Deviated cells refer to cells that are already differentiated cell and do not have cell potency. In this embodiment, an undifferentiated cell is referred to as a normal cell. Also, a deviated cell is referred to as an abnormal cell.

(Cultivated Cells Cultivated in Cultivation Container)

The cultivated cells90cultivated in the cultivation container80is now described with reference toFIG.3.

As shown inFIG.3, the cells90are cultivated cells cultivated in the cultivation container80filled with the cultivation solution81. In this embodiment, the cell image10includes cultivated cells90cultivated in the cultivation container80. A coating material for culturing the cells90is applied onto a bottom80aof the cultivation container80. The coating material includes proteins required for the cells90to settle in the cultivation container80.

A method for analyzing the cell images10by using the cell image analysis method according to this embodiment is now described with reference toFIG.4. A configuration in which the cells90included in the cell image10are classified into one of two or more types by analyzing the cell image10by using the cell image analysis apparatus100(seeFIG.1) in this embodiment. In this embodiment, the cell image analysis apparatus100analyzes the cell image10by using the learned model6(seeFIG.1) to determine into which type the cells90included in the cell image10is classified from the two or more types. The learned model6is configured to input the probability value21for every pixel of the cell image10when receiving the cell image10. The probability value21is an estimation value of the classification.

FIG.4is a block diagram showing a flow of image processing in this embodiment. As shown inFIG.4, in this embodiment, the cell image analysis method roughly includes an image analysis method101, and a production method102of the learned model6(seeFIG.1).

(Generation of Learning Model)

In the production method102of the learned model6in this embodiment, the learned model6is produced by learning a learning model7by using the cell images10. Specifically, the learned model6is produced by learning to output the probability value21for every pixel of the cell image10as an analysis result. As shown inFIG.4, the production method102of the learned model6includes a step102aof inputting teacher cell images30into the learning model7, and a step102bof learning the learning model7to output teacher correct images31. The learned model6is a convolutional neural network (CNN) shown inFIG.4or a learning model partially including a convolutional neural network. The learned model6produced by learning the learning model7is stored in the storage3(FIG.1) of the cell image analysis apparatus100.

In this embodiment, the learned model6is produced by leaning at least one of classification of the cell whether a focus of the cell image10is correct when the cell image is captured, classification of the cell whether a coating material on the cultivation container80of the cell is proper, and classification of the cell whether the number of cultivation days of the cell is proper. In the production method102of the learned model6, the learned model6is produced by using the teacher cell images30that are the cell image10, and teacher correct images31that are generated adding the cell image10with a label value relating to at least two imaging conditions corresponding to the classification or a label value relating to at least two cultivation conditions corresponding to the classification.

In this embodiment, the learned model6includes the first learned model6a, the second learned model6band the third learned model6c. The first learned model6ais a learned model that has learned classify the cells90included in the cell image10into one of the images captured under two or more imaging conditions based on the cell image10. In other words, the teacher cell images30used to produce the first learned model6aare cell images10that are captured under different imaging conditions.

Also, teacher correct images31are images that are added with different label values for pixels depending on the difference between the imaging conditions. Specifically, the teacher correct images31are images added with label values relating to two or more types of imaging conditions for pixels. The imaging conditions are a condition in which a focus of the cell image10(teacher and cell image30) is correct, and a condition in which the focus is incorrect. Correspondingly, the teacher correct image31is an image that is added with a label value corresponding to the condition in which a focus of the cell image10is correct when the cell image is captured for every pixel, or an image that is added with a label value corresponding to the condition in which the focus is incorrect for every pixel. In other words, the teacher correct images31includes images classified into two classes, which are a correct focus class and an incorrect focus class. As a result, the first learned model6acan be produced by learning the learning model7to classify each pixel of the input image into one of two or more types relating to the imaging conditions.

The second learned model6band the third learned model6care learned models that have learned classify the cells90included in the cell image10into one of the images including cell90cultivated under two or more types of cultivation conditions based on the cell image10. Specifically, the second learned model6band the third learned model6care produced by using the cell images10including the cells cultivated under different cultivation conditions the teacher cell images30. Also, images that are added with different label values for pixels depending on the difference between the cultivation conditions are used as the teacher correct images31. Specifically, the teacher correct images31are images added with label values relating to two or more types of cultivation conditions for pixels. The cultivation conditions include conditions relating to different coating materials on the cultivation container80(seeFIG.3) in which the cells90are cultivated, and conditions relating to different numbers of cultivation days of the cells90.

In other words, the second learned model6bis produced by using the teacher correct images31added with at least two types of label values relating to the coating materials on the cultivation container80in which the cells90are cultivated. Specifically, the second learned model6bis produced by using the teacher correct images31that are added with a label value indicating a coating material A used as the coating material on the cultivation container80in which the cells90are cultivated, and the teacher correct images31that are added with a label value indicating a coating material different from the coating material A. That is, the teacher correct images31includes images classified into two classes, which are a class of the coating material A and a class of the coating material B.

Also, the third learned model6cis produced by using the teacher correct images31added with at least two types of label values relating to the numbers of cultivation days of the cells90. Specifically, the third learned model6cis produced by using the teacher correct images31that are added with a label value indicating a predetermined number of days as the number of cultivation days of the cells90, and a label value indicating one other number of days different from the predetermined number of days as the number of cultivation days of the cells90. In this embodiment, the predetermined number of days is 5, for example. That is, the teacher correct images31includes images classified into two classes, which are a class of 5 cultivation days and a class of cultivation days other than 5 days.

In this embodiment, the learned model6is produced by learning classification of cell images into two or more types of classification relating to imaging conditions or cultivation conditions to learn classification of the cell images whether the cell90is suitable for analysis whether the cell is a normal or abnormal cell. In this embodiment, the learned model6is produced by learning classification of the cell whether cells90of a common type are suitable for analysis whether each cell is a normal or abnormal cell.

In this embodiment, the image analysis method101classifies the cells90included in the cell image10acquired by the image acquirer1from the microscope8(seeFIG.1), etc., into one of two or more types. The cell image analysis method101according to this embodiment includes a step of acquiring a cell image10of cells90(seeFIG.2); a step of inputting the cell image10to a learned model6; a step of acquiring an index value20indicating accuracy of the classification of the cells90that is included in the cell image10into one of two or more types based on an analysis result of each of pixels of the cell image10output from the learned model6; and a step of displaying the acquired index value20. The steps in the image analysis method101will be described in detail later.

In this embodiment, the step of acquiring the cell image10is executed by the image acquirer1. The image acquirer1is configured to acquire the cell images10from the imaging device such as the microscope8(seeFIG.1). Also, the image acquirer1is configured to output the acquired cell image10to the image analyzer2b. In addition, the image acquirer1is configured to output the acquired cell image10to the superimposed cell image generator2d.

In this embodiment, the step of analyzing the cell image10is executed by the image analyzer2b. The image analyzer2bis configured to input the cell image10to the learned model6whereby obtaining the index value20. Specifically, the image analyzer2bis configured to input the cell image10to any of the first learned model6a, the second learned model6b, an the third learned model6cwhereby obtaining the index value20. The controller2ais configured to determine which of the first learned model6a, the second learned model6b, and the third learned model6cis used for analysis by the image analyzer2b. The image analyzer2bis configured to output the acquired index value20to the controller2aand the superimposed cell image generator2d. Specifically, the image analyzer2bis configured to output the probability values21as the index value20to the controller2aand the superimposed cell image generator2d.

The controller2ais configured to determine which of the first learned model6a, the second learned model6b, and the third learned model6cis used for analysis in accordance with an operating input from a user. Specifically, the controller2ais configured to determine which of the first learned model6a, the second learned model6b, and the third learned model6cis used for analysis in accordance with an operating input that select which condition is used to analyze the cell image10.

The controller2ais configured to acquire a representative value20aof the probability values21based on the probability values21. In this embodiment, the controller2ais configured to output one cell image with one representative value20abased on the probability values21acquired for pixels of the cell image10. In this embodiment, the controller2ais configured to acquire an average value of probability values21as the representative value20a.

Also, the controller2ais configured to acquire a frequency distribution22of probability values21based on the probability values21. In addition, the controller2ais configured to display the acquired representative value20aand the acquired frequency distribution22on the display4. A configuration in which the representative value20aand the frequency distribution22are acquired by the controller2awill be described in detail later.

The superimposed cell image generator2dis configured to generate the superimposed cell image50based on the cell image10and the index value20. Also, the superimposed cell image generator2dis configured to display the generated superimposed cell image50on the display4.

(Difference Between Cell Images Captured Under Different Imaging Conditions)

Difference between cell images10captured under different imaging conditions is now described with reference toFIG.5. In this embodiment, difference between the imaging conditions is that a focus of the cell image10is correct or in correct when the cell image is captured. A common part of the cultivation container80(seeFIG.3) is captured at different focus points as cell images10ato10cshown inFIGS.5(A) to5(C).

The cell image10ashown inFIG.5(A)is a cell image10captured at a correct focus point. In other words, the cell image10aincludes high contrast images of the cells90. That is, outlines of the images of the cells90are clear in the cell image10a. Here, the term a focus of an cell image is correct does not means that all cells in the cell image10are in focus but means that a central part of the cell image10is in focus. In other words, focus degrees of the cells90in the cell image10aare not constant but a deviation of a focus degree of one cell90from the correct focus increases as a distance between the cell and the center of the image. That is, some cells90may be out of focus in the cell image10a.

The cell image10bshown inFIG.5(B)is a cell image10captured at an incorrect focus point. In other words, the cell image10bshown inFIG.5(B)includes low contrast images of the cells90. That is, outlines of the images of the cells90are unclear in the cell image10b. A deviation of a focus degree of the cell image10bfrom the correct focus (out-of-focus degree) is smaller than the cell image10cshown inFIG.5(C). Here, an indication “focus deviation −1” shown inFIG.5(B)means that the deviation of a focus degree of the cell image10bfrom the correct focus is smaller than the cell image10cshown inFIG.5(C). In the cell image10binFIG.5(B), dashed lines indicating the outlines of the cells90represent the focus deviation from the correct focus. Also, focus degrees of the cells90in the cell image10bshown inFIG.5(B)are not constant but a deviation of a focus degree of one cell from the correct focus increases as a distance between the cell and the center of the image.

The cell image10cshown inFIG.5(C)is a cell image10captured at an incorrect focus point. In other words, the cell image10cshown inFIG.5(C)includes low contrast images of the cells90. That is, outlines of the images of the cells90are unclear in the cell image10c. A deviation of a focus degree of the cell image10cfrom the correct focus (out-of-focus degree) is larger than the cell image10b. That is, outlines of the images of the cells90are unclear in the cell image10c. Here, an indication “focus deviation −2” shown inFIG.5(C)means that the deviation of a focus degree of the cell image10cfrom the correct focus is larger than the cell image10bshown inFIG.5(B). In the cell image10cinFIG.5(C), no outline of the cells90is shown to represent that the deviation of a focus degree of the cell image10cfrom the correct focus is larger than the cell image10b. Also, focus degrees of the cells90in the cell image10bshown inFIG.5(B)are not constant but a deviation of a focus degree of one cell from the correct focus increases as a distance between the cell and the center of the image.

In this embodiment, the image analyzer2bis configured to classify cells90included in each of the cell images10ato10cinto one of two or more types by using the first learned model6a.

Specifically, the image analyzer2bis configured to input the cell images10ato10cto the first learned model6awhereby generating the probability distribution images12based on the probability values21output the probability from the first learned model6a.

In an exemplary case shown inFIG.6, the image analyzer2binputs the cell image10ain focus to the first learned model6awhereby acquiring the probability values21. In other words, in the exemplary case shown inFIG.6, the image analyzer2bacquires the probability value21for every pixel of the cell image10a. The image analyzer2bis configured to output the probability the acquired probability values21to the image processor2c. In the exemplary case shown inFIG.6, the image analyzer2bacquires index values corresponding to a class that represents correct focus for pixels of the cell image10as the probability values21.

As shown inFIG.6, the image processor2cis configured to generate the probability distribution image12representing a distribution of the probability values21. The probability values21that are estimation values of the classification are distributed as pixel values in the probability distribution image12. The probability distribution image12shown inFIG.6representing the distribution of the probability values21that are estimation values corresponding to a class that represents correct focus for pixels of the cell image10. In the exemplary case shown inFIG.6, different probability values21are represented by difference hatching patterns. The probability values21decrease in order of a black hatching pattern, a dark hatching pattern, and a light hatching pattern as shown in legends8. In addition, as shown in the legends8, the probability value21of each pixel is not indicated by an area of one hatching pattern but a certain area corresponding to a common probability value21is indicated by an area of one hatching pattern.

Although not shown inFIG.6, the image analyzer2balso inputs the cell image10to the first learned model6awhereby acquiring the probability distribution image12that represents the distribution of the probability values21that are estimation values corresponding to a class that represents incorrect focus for pixels of the cell image.

The image processor2csimilarly acquires the probability distribution image12corresponding to a class that represents correct focus, and the probability distribution image12corresponding to a class that represents incorrect focus for the out-of-focus cell image10b(seeFIG.5) and the out-of-focus cell image10c(seeFIG.5).

(Superimposed Cell Image and Difference Between Superimposed Cell Images Due to Different Imaging Conditions)

Superimposed cell image50(seeFIG.1) and differences between superimposed cell images50due to different imaging conditions are now described with reference toFIG.7. The superimposed cell image generator2dis configured to generate the superimposed cell image50based on the cell image10and the probability distribution image12. Specifically, the superimposed cell image generator2dis configured to generate the superimposed cell image50by using the cell image10and the probability distribution images12that are acquired for at least two different label values.

Specifically, the superimposed cell image generator2dis configured to generate the superimposed cell image50by superimposing marks that allow users to distinguish difference between different probability values21on the cell image10based on the probability distribution images12. In this embodiment, the superimposed cell image generator2dis configured to superimpose the marks that allow users to distinguish between the probability values21of label values corresponding to two or more classification types. Specifically, the superimposed cell image generator2dis configured to superimpose the marks that allow users to distinguish between the probability values21of label values corresponding to two or more types of imaging conditions. More specifically, the superimposed cell image generator2dis configured to superimpose the marks that allow users to distinguish between the probability value21of the label value corresponding to correct focus and the probability value21of the label value corresponding to in correct focus on the cell image10. For example, the superimposed cell image generator2dsuperimposes a blue mark51for the probability value21of the label value corresponding to correct focus. Also, the superimposed cell image generator2dsuperimposes a red mark52for the probability value21of the label value corresponding to incorrect focus. In an exemplary case shown inFIG.7, as shown in legends9, the blue mark51is indicated by a tightest hatching pattern. Also, in the exemplary case shown inFIG.7, as shown in legends9, the red mark52is indicated by a loosest hatching pattern.

The superimposed cell image50ashown inFIG.7(A)is an image that is generated by superimposing a distribution of the probability values21that are acquired by inputting the cell image10ato the first learned model6aon the cell image10ain focus (seeFIG.5(A)). Also, the superimposed cell image50bshown inFIG.7(B)is an image that is generated by superimposing a distribution of the probability values21that are acquired by inputting the cell image10bto the first learned model6aon the cell image10bout of focus (seeFIG.5(B)). Also, the superimposed cell image50cshown inFIG.7(C)is an image that is generated by superimposing a distribution of the probability values21that are acquired by inputting the cell image10cto the first learned model6aon the cell image10cout of focus (seeFIG.5(C)). In the exemplary case shown inFIG.7, the blue mark51is superimposed on parts of the probability value21corresponding to correct focus. Also, the red mark52is superimposed on parts of the probability value21corresponding to incorrect focus. Accordingly, in the exemplary case shown inFIG.7, in a mixed area in which the probability value21corresponding to correct focus and the probability value21corresponding to incorrect focus are included, a gradation mark53of blue and red is superimposed. In the exemplary case shown inFIG.7, as shown in the legends9, the gradation mark53of blue and red is indicated by a middle tight hatching pattern.

The superimposed cell image50ain focus is largely occupied by parts on which the blue mark51indicating the probability value21corresponding to correct focus is superimposed. Also, the superimposed cell image50cwhose deviation of a focus degree from the correct focus is the largest is largely occupied by parts on which the red mark52indicating the probability value21corresponding to incorrect focus is superimposed. The parts on which the blue mark51is superimposed have the largest share of the superimposed cell image50bwhose deviation of a focus degree from the correct focus is smaller than the superimposed cell image50c, and the parts on which the gradation mark53of blue and red is superimposed have the second largest share of the superimposed cell image50b. The parts on which the red mark52is superimposed are also included in the superimposed cell image50b.

(Representative Value and Difference Between

Representative Values Due to Different Imaging Conditions) In this embodiment, the controller2ais configured to acquire the representative values20aof the probability values21as shown inFIG.8. Specifically, the controller2ais configured to acquire numerical data of the representative values20aof the probability values21. In other words, in this embodiment, the controller2ais configured to acquire one representative value20afrom the probability values21acquired for every pixel of the cell image10a. Also, in this embodiment, the controller2ais configured to acquire the representative value20aof the probability values21in cell areas as the representative value20aof the probability values21. The cell area is acquired by the image processor2c. Specifically, the image processor2cadds the probability distribution images12of at least two different label values to each other, and acquires an area of not smaller than a predetermined probability value21as the cell area in the probability distribution image12that is generated by the addition.

In this embodiment, the controller2aacquires the representative value20abased on the probability values21of the label value corresponding to one of two or more types of imaging conditions. Specifically, the controller2ais configured to acquire the representative value20abased on the probability values21of the label value corresponding to correct focus. That is, the controller2ais configured to acquire the representative value20abased on the probability values21of the label value suitable for analysis whether the cell is a normal or abnormal cell.

In this embodiment, the controller2ais configured to acquire a graph that collectively indicates numerical data of a plurality of representative values20aas shown in a graph40a. In the graph40a, its horizontal axis indicates deviations of a focus degree from the correct focus for each cell image10, and its vertical axis indicates the representative value20a. In other words, “0” on the horizontal axis indicates the cell image10ain focus in the graph40a. Also, “−1” on the horizontal axis indicates the cell image10bout of focus in the graph40a. Also, “−2” on the horizontal axis indicates the cell image10cout of focus in the graph40a. As shown in the graph40a, the representative value20adecreases as the deviation of a focus degree from the correct focus increases.

(Frequency Distribution and Difference Between Frequency Distributions Due to Different Imaging Conditions)

The frequency distribution22acquired by the controller2a(seeFIG.4) and difference between the frequency distributions22due to different imaging conditions (seeFIG.4) are now described with reference toFIGS.9and10.

A frequency distribution22ashown inFIG.9is a frequency distribution acquired based on the probability values21of the cell image10ain focus. In the frequency distribution22a, its horizontal axis indicates the probability value21, and its vertical axis indicates a frequency. In other words, the frequency distribution22ais a graph of the frequency of probability values21of pixels in the cell image10a(seeFIG.5). In the frequency distribution22a, the probability value21of a first type in two or more types of label values is hatched. In other words, in the frequency distribution22a, the probability value21corresponding to a class that represents correct focus is hatched. In the frequency distribution22a, the probability value21of a second type, which is different from the first type, in two or more types of label values is not hatched and indicated by a white bar. In other words, in the frequency distribution22a, the probability value21corresponding to a class that represents incorrect focus is indicated by the white bar.

As shown inFIG.9, because frequencies of pixels that correspond to high probability values21are high in the frequency distribution22aof the cell image10ain focus, a number of pixels are distributed in a right part of the frequency distribution22a. Also, because frequencies of pixels that correspond to low probability values21are also high in the frequency distribution22aof the cell image10ain focus, a number of pixels are distributed also in a left part of the frequency distribution22a.

A frequency distribution22bshown inFIG.10is a frequency distribution acquired based on the probability values21of the cell image10bout of focus. In the frequency distribution22b, its horizontal axis indicates the probability value21, and its vertical axis indicates the frequency. In other words, the frequency distribution22bis a graph of the frequency of probability values21of pixels in the cell image10b(seeFIG.5). Also, in the frequency distribution22b, the probability value21corresponding to a class that represents correct focus is hatched, while the probability value21corresponding to a class that represents incorrect focus is not hatched but indicated by the white bar.

As shown inFIG.10, because frequencies of pixels that correspond to high probability values21are low in the frequency distribution22bof the cell image10bout of focus as compared to the frequency distribution22aof the cell image10ain focus (seeFIG.9), and frequencies of pixels that correspond to low probability values21corresponding to a class that represents correct focus are high, a distribution of pixels that correspond to high probability values21corresponding to a class that represents correct focus is not shifted toward the right side of the frequency distribution22abut entirely spreads. Also, because frequencies of pixels that correspond to low probability values21corresponding to a class that represents incorrect focus are low, and frequencies of pixels that correspond to low probability values21corresponding to a class that represents incorrect focus are high in the frequency distribution22aof the cell image10ain focus, their distribution is not shifted toward the left side of the frequency distribution22abut entirely spreads. In other words, it is possible to easily immediately classify the cell90included in the cell image10into one of two or more types of imaging conditions by seeing a shape of the frequency distribution22.

(Display of Superimposed Cell Images, Representative Value, and Frequency Distribution)

In this embodiment, the controller2a(seeFIG.1) is configured to display numerical data of the representative value20a(seeFIG.4) of the probability values21(seeFIG.4), and the superimposed cell image50that is generated by superimposing the distribution of probability values21on the cell image10on the display4as shown inFIG.11. In this embodiment, the controller2a(seeFIG.4) is configured to display the frequency distribution22on the display4of the probability values21together with numerical data of the representative value20aof the probability values21, and the superimposed cell image50. In an exemplary case shown inFIG.11, the controller2adisplays superimposed cell images50ato50cas the superimposed cell image50. In the exemplary case shown inFIG.11, the controller2adisplays a graph40aas the numerical data of the representative value20a. In the exemplary case shown inFIG.11, the controller2adisplays a frequency distribution22aas the frequency distribution22.

(Difference Between Superimposed Cell Images Due to Different Coating Materials)

Difference between the superimposed cell images50(seeFIG.1) due to different coating materials is now described with reference toFIG.12. A superimposed cell image50dshown inFIG.12(A)is generated based on the probability distribution image12that is generated based on the index values20acquired by analyzing the cell image10by using by the second learned model6b, and the cell image10. Specifically, the superimposed cell image50dis an image generated based on the cell image10of cells90that are cultivated in the cultivation container80with a coating material A applied to a bottom80aof the cultivation container80. A configuration in which the superimposed cell image50dis generated is similar to a configuration in which the superimposed cell images50ato50care generated by first learned model6a, except that the second learned model6bis used instead of the first learned model6a, and its description is omitted.

Also, a superimposed cell image50eshown inFIG.12(B)is generated based on the probability distribution image12that is generated based on the index values20acquired by analyzing the cell image10by using the second learned model6b, and the cell image10. Specifically, the superimposed cell image50eis an image generated based on the cell image10of cells90that are cultivated in the cultivation container80with a coating material B applied to a bottom80aof the cultivation container80. In this embodiment, the second learned model6bis produced by learning to output a probability that a coating material applied to the bottom80aof the cultivation container80is the coating material A as the probability value21. Accordingly, parts (tightest hatching part) on which the blue mark51is superimposed have a large share of the superimposed cell image50dshown inFIG.12(A). Also, parts (loosest hatching part) on which the red mark52is superimposed have a large share of the superimposed cell image50eshown inFIG.12(B).

(Difference Between Representative Values Due to Different Coating Materials)

The controller2ais configured to acquire the representative value20afor each cell image10based on the probability values21output by the second learned model6b. In this embodiment, the controller2ais configured to acquire a graph that collectively indicates a plurality of representative values20aas shown in a graph40bofFIG.13. The graph40billustrates difference between the representative values20adue to different coating materials. In the graph40b, its horizontal axis indicates types of coating materials, and its vertical axis indicates the representative value20a. A configuration in which the superimposed cell image40bis generated by the controller2ais similar to a configuration in which the graph40ais generated, except that the probability values21output by the second learned model6bis used instead of the probability values21output by the first learned model6a, and its description is omitted.

It can be seen from the graph40bthat the representative value20aof the cell image10of the cells90cultivated in the cultivation container80coated with the coating material A is greater than the representative value20aof the cell image10of the cells90cultivated in the cultivation container80coated with the coating material B.

(Difference Between Superimposed Cell Images Due to Difference in Cultivation Days)

Difference between the superimposed cell images50due to difference in cultivation days is now described with reference toFIG.14. A superimposed cell image50fshown inFIG.14(A)is an image generated based on the cell image10of cells90that are cultivated for 5 days as the cultivation days. Specifically, the superimposed cell image50fis generated based on the probability distribution image12that is generated based on the index values20acquired by analyzing the cell image10aby using the third learned model6c, and the cell image10. A configuration in which the superimposed cell image50fis generated is similar to a configuration in which the superimposed cell images50ato50care generated by first learned model6a, except that the third learned model6cis used instead of the first learned model6a, and its description is omitted.

Also, a superimposed cell image50gshown inFIG.14(B)is an image generated based on the cell image10of cells90that are cultivated for 4 days as the cultivation days. Also, a superimposed cell image50hshown inFIG.14(C)is an image generated based on the cell image10of cells90that are cultivated for 6 days as the cultivation days. Also, a superimposed cell image50ishown inFIG.14(D)is an image generated based on the cell image10of cells90that are cultivated for 7 days as the cultivation days.

The third learned model6cis produce by learning to output the probability_values21relating to determination whether the cultivation days is 5. Accordingly, parts (tightest hatching part) on which the blue mark51is superimposed have a large share of the superimposed cell image50fshown inFIG.14(A). Also, parts (loosest hatching part) on which the red mark52is superimposed have a larger share of the superimposed cell image50gshown inFIG.14(B)as compared to the superimposed cell image50gshown inFIG.14(A). As shown inFIGS.14(C) and14(D), as the number of cultivation days increases, share of the superimposed cell image by the parts on which the red mark52is superimposed increases, and share of the superimposed cell image by the parts on which the gradation mark53of blue and red (middle tight hatching part) is superimposed increases.

(Difference Between Representative Values Due to Difference in Cultivation Days)

The controller2ais configured to acquire the representative value20afor each cell image10based on the probability values21output by third learned model6c. In this embodiment, the controller2ais configured to acquire a graph that collectively indicates a plurality of representative values20aas shown in a graph40cofFIG.15. The graph40cillustrates difference between the representative values20adue to difference in cultivation days. In the graph40c, its horizontal axis indicates cultivation days, and its vertical axis indicates the representative value20a. A configuration in which the graph40cis generated by the controller2ais similar to a configuration in which the graph40ais generated, except that the probability values21output by the third learned model6cis used instead of the probability values21output by the first learned model6a, and its description is omitted.

As shown in the graph40c, the representative value20aof the cell image10of cells90that are cultivated for 5 days is the highest. Also, it can be seen that the representative values20aof the cell images10of cells90that are cultivated not for 5 days are smaller than the representative value20aof the cell image10of cells90that are cultivated for 5 days. Also, in comparison between the representative value20aof the cell image10of cells90that are cultivated for 4 days and the representative value20aof the cell image10of cells90that are cultivated for 6 days, it can be seen that the representative value20aof the cell image10of cells90that are cultivated for 6 days is higher.

(Threshold Processing of Representative Value)

In this embodiment, as shown inFIG.11, a plurality of superimposed cell images50, the numerical data (graph) of the representative values20a, and the frequency distribution22aare displayed on the display4. Accordingly, a user can classify the cells90included in the cell image10into one of two or more types based on the plurality of superimposed cell images50, the numerical data (graph) of the representative values20aand the frequency distribution22adisplayed on the display4. In other words, the user can determine whether the cells90included in the cell image10are suitable for analysis of determination whether they are normal or abnormal cells under two or more types of imaging conditions or two or more types of cultivation conditions. That is, the user can determine whether the cells90included in the cell image10are suitable for analysis of determination whether they are normal or abnormal cells. In this case, the user can specify a threshold for determination whether the cells90included in the cell image10are suitable for analysis of determination whether they are normal or abnormal cells based on the numerical data of the representative values20a. In this embodiment, the controller2acan determine whether the index value20is greater than the threshold. Specifically, the controller2adetermines whether the representative value20aof the probability values21is greater than the threshold specified by the user whereby determining whether the cells90included in the cell image10are suitable for analysis of determination whether they are normal or abnormal cells. For example, the controller2adetermines that the cell image10is suitable for analysis of determination whether the cells90included in the cell image10are normal or abnormal cells if the representative value20ais not smaller than 50%. Also, for example, the controller2adetermines that the cell image10is not suitable for analysis of determination whether the cells90included in the cell image10are normal or abnormal cells if the representative value20ais smaller than 50%.

(Display of Superimposed Cell Image, Representative Value, and Frequency Distribution)

Processes of displaying the superimposed cell image50, the representative value20a, and the frequency distribution22in the cell image analysis apparatus100is now described with reference toFIG.16.

In step200, the image acquirer1acquires a cell image10including cells90.

In step201, the image analyzer2binputs the cell image10to the learned model6, which has leaned to classify the cells90into one of two or more types.

In step202, the image processor2cacquires cell areas that are areas of the cells90included in the cell image10. In this embodiment, the image processor2cacquires the cell areas based on the probability distribution image12(seeFIG.6).

In step203, the controller2aacquire an index value20indicating accuracy of the classification of the cells90included in the cell image10into one of two or more types based on an analysis result of each of pixels of the cell image10output from the learned model6. In this embodiment, in a process of step203, the controller2aacquired the representative value20aof the probability values21obtained based on the probability values21output by the learned model6as the index value20. Specifically, the controller2ais configured to obtain the index values20of at least one of classification of the cell whether a focus of the cell image10is correct when the cell image is captured, classification of the cell whether a coating material on the cultivation container80of the cell is proper, and classification of the cell whether the number of cultivation days of the cell is proper is acquired. Also, in this embodiment, in the processing of step203, the controller2aacquires a value representing accuracy of suitability for analysis whether the cells90that are included in the cell image10are normal or abnormal cells based on the probability values21as the index value20.

Also, in this embodiment, the controller2ais configured to acquire the representative value20aof the probability values21in the cell areas as the representative value20aof the probability values21in a process of step203. Specifically, the controller2aacquires an average value of the probability values21as the representative value20a.

In step204, the controller2aacquires the frequency distribution22. Specifically, the controller2aacquires the frequency distribution22based on the probability values21output from the learned model6.

In step205, the superimposed cell image generator2dgenerates the superimposed cell image50. Specifically, the superimposed cell image generator2dgenerates the superimposed cell image50based on the cell image10and the probability distribution image12(seeFIG.6) acquired based on the probability values21.

In step206, the controller2adisplays the acquired index value20. In this embodiment, in a processing of step206, the controller2adisplay numerical data of the representative value20aof the probability values21, and the superimposed cell image50that is generated by superimposing the distribution of probability values21on the cell image10. In this embodiment, the controller2adisplays the frequency distribution22of the probability values21together with numerical data of the representative value20aof the probability values21, and the superimposed cell image50. After that, the procedure ends.

Any one of the process in step204and the process in step205may be executed before another process.

(Generation of Learned Model)

Processes of generating the learned model6is now described with reference toFIG.17.

In step300, the image acquirer1acquires a teacher cell image30. The teacher cell image30is the cell image10.

In step301, the image acquirer1acquires a teacher correct image31. The teacher correct image31is a label image that is the cell image10generated by adding the cell image with a label value relating to at least two imaging conditions corresponding to the classification or a label value relating to at least two cultivation conditions corresponding to the classification.

In this embodiment, when the first learned model6ais generated, the cell image10that is added with a label value relating to at least two imaging conditions corresponding to the classification is acquired as the teacher correct image31in a process of step301. Specifically, as the label value relating to the imaging conditions, the cell image10that is added with a label value corresponding to correct focus and label values corresponding to incorrect focus when the cell image10is captured for every pixel is acquired as the teacher correct image31. The label values corresponding to incorrect focus include a plurality of label representing values representing degrees of out-of-focus degree. In this embodiment, the label values corresponding to incorrect focus include two label values.

Also, when the second learned model6bis generated, the image acquirer1acquires the cell image10that is added with a label value relating to at least two cultivation conditions corresponding to the classification as the teacher correct image31in a process of step301. Specifically, the image acquirer1acquires the cell image10that is added with at least two types of label values relating to the coating materials on the cultivation container80in which the cells90are cultivated as the teacher correct image31. In this embodiment, the image acquirer1acquires the teacher correct images31that are added with two label values for pixels, which are a label value of the coating material A and the label value of the coating material B, as the label values relating to the coating materials.

Also, when the third learned model6cis generated, the image acquirer1acquires the cell image10that is added with a label value relating to at least two cultivation conditions corresponding to the classification as the teacher correct image31in a process of step301. Specifically, the image acquirer1acquires the cell image10that is added with at least two types of label values relating to the cultivation days of the cells90as the teacher correct image31. In this embodiment, the image acquirer1acquires the teacher correct images31that are added with a label value of 5 cultivation days and a label value of cultivation days excluding 5 for pixels as the label values relating to the cultivation days.

In step302, the image processor2cproduces the learned model6by using the teacher cell images30that are the cell image10, and teacher correct images31that are generated by adding the cell image10with a label value relating to at least two imaging conditions corresponding to the classification or a label value relating to at least two cultivation conditions corresponding to the classification.

In this embodiment, in a process of step302, the image processor2cproduces the learned model6by using the teacher correct images31that are added with two types of label values corresponding to whether a focus of the cell image10is correct when the cell image is captured as the label value relating to the imaging conditions, or at least two types of label values relating to coating materials on the cultivation container80in which the cells90are cultivated, and the number of cultivation days of the cell as the label value relating to the cultivation conditions. After that, the procedure ends.

(Classification of Cell Image)

Processes of classifying the cell image10in the cell image analysis apparatus100is now described with reference toFIG.18.

In step400, the controller2aacquires the index value20. In this embodiment, the controller2aacquires the index value20acquired by the image analyzer2bby using the cell image10and the learned model6.

In step401, the controller2aacquires a threshold. Specifically, the controller2aacquires a threshold that is previously specified by a user and stored in the storage3.

In step402, the controller2adetermines whether the index value20is greater than the threshold. In other words, the controller2adetermines whether the index value20is greater than the threshold for each cell image10. If the index value20is greater than the threshold, the procedure goes to step403. If the index value20is not greater than the threshold, the procedure goes to step404.

In step403, the controller2aclassifies the cell image10whether it is suitable for analysis whether the cells90are normal or abnormal cells. In addition, the controller2astores the cell image10that is classified into the image that is suitable for analysis whether the cells90are normal or abnormal cells in the storage3. After that, the procedure ends.

If the procedure goes from step402to step404, the controller2aclassifies the cell image10into the image that is not suitable for analysis whether the cells90are normal or abnormal cells in step404. In this case, the controller2adoes not store the cell image10in the storage3. After that, the procedure ends.

Advantages of the Embodiment

In this embodiment, the following effects can be acquired.

In this embodiment, as discussed above, a cell image analysis method includes a step of acquiring a cell image10including a cell90; a step of inputting the cell image10to a learned model6that has learned classification of the cell90into one of two or more types; a step of acquiring an index value20indicating accuracy of the classification of the cell90that is included in the cell image10into one of two or more types based on an analysis result of each of pixels of the cell image10output from the learned model6; and a step of displaying the acquired index value20.

Consequently, because the index value20indicating accuracy of the classification of the cell90that is included in the cell image10into one of two or more types is displayed, users can easily grasp a probability of classification of the cell90in the cell image10by seeing the index value20. Therefore, it is possible to provide a cell image analysis method capable of allowing users to easily grasp a probability of classification of the cell90in the cell image10.

In addition, following additional advantages can be obtained by the aforementioned embodiment added with configurations discussed below.

That is, in this embodiment, as discussed above, the learned model6has been learned to output the probability values21that are estimation values of the classification as the analysis result; and a representative value20aof the probability values21obtained based on the probability values21output by the learned model6is acquired as the index value20in the step of acquiring an index value20. Consequently, dissimilar to a configuration in which probability values21for pixels of the cell image10are displayed, it is possible to easily grasp accuracy of classification of the cells90included in the cell image10based on the representative value20aof the probability values21in each cell image10.

In this embodiment, as discussed above, the cell image10includes cultivated cell90that is cultivated in a cultivation container80; the learned model6is produced by leaning at least one of classification of the cell whether a focus of the cell image is correct when the cell image10is captured, classification of the cell whether a coating material on the cultivation container80of the cell is proper, and classification of the cell whether the number of cultivation days of the cell is proper; and index values20of at least one of classification of the cell whether a focus of the cell image is correct when the cell image10is captured, classification of the cell whether a coating material on the cultivation container80of the cell is proper, and classification of the cell whether the number of cultivation days of the cell is proper are acquired as the index value in the step of acquiring the index values20. Consequently, users can easily grasp at least one of classification of the cell whether a focus of the cell image is correct when the cell image10is captured, classification of the cell whether a coating material on the cultivation container80of the cell is proper, and classification of the cell whether the number of cultivation days of the cell is proper are acquired as the index value by seeing the index values20.

In this embodiment, as discussed above, the learned model6is produced by learning classification of the cell whether the cell90is suitable for analysis whether the cell is a normal or abnormal cell; and a value representing accuracy of suitability for analysis whether the cells90that are included in the cell image10are normal or abnormal cells based on the probability values21is acquired as the index value20in the step of acquiring the index values20. Accordingly, because the index values20representing accuracy of suitability for analysis whether the cells90that are included in the cell image10are normal or abnormal cells are displayed, users can easily grasp whether the cells90included in the cell image10are suitable for analysis of determination whether they are normal or abnormal cells by seeing the index values20. Consequently, it is possible to provide a cell image analysis method capable of allowing users to easily grasp whether the cells90included in the cell image10are suitable for analysis of determination whether they are normal or abnormal cells.

In this embodiment, as discussed above, the learned model6is produced by learning classification of the cell whether cells90of a common type are suitable for analysis whether each cell is a normal or abnormal cell. Consequently, based on analysis of the cell image10by using the learned model6, it is possible to classify images including cells90of a common type whether it is suitable for analysis whether each cell is a normal or abnormal cell.

In this embodiment, as discussed above, a step of acquiring cell areas that are an area of the cell90included in the cell image10is further provided; and the representative value20aof the probability values21in the cell areas as the representative value20aof the probability values21is acquired in the step of obtaining the representative value20aof the probability values21. Accordingly, it is possible to prevent increase of a processing load as compared to a configuration in which the representative value20ais acquired based on the probability values21of pixels of the entire cell image10.

In this embodiment, as discussed above, numerical data of the representative value20aof the probability values21, and the superimposed cell image50that is generated by superimposing the distribution of probability values21on the cell image10are displayed in the step of displaying the representative value20aof the probability values21. Accordingly, because the representative value20aof the probability values21is displayed, it is possible to easily grasp accuracy of classification of the cells90included in the cell image10based on the numerical data of the representative value20aof the probability values21in each cell image10. In addition, because superimposed cell image50is displayed, it is possible to grasp accuracy of classification of each cell90included in the cell image10based on the superimposed cell image50.

In this embodiment, as discussed above, the frequency distribution22of the probability values21is displayed together with numerical data of the representative value20aof the probability values21, and the superimposed cell image50in the step of displaying the representative value20aof the probability values21. Consequently, it is possible to grasp accuracy of classification of the cells90included in the cell image10from different viewpoints by confirming the frequency distribution22together with numerical data of the representative value20aof the probability values21, and the superimposed cell image50in each cell image10.

In this embodiment, as discussed above, an average value of the probability values21as the representative value20ais acquired in the step of obtaining the representative value20aof the probability values21. For example, as compared to a configuration in which a median value in the probability values21is acquired as the representative value20a, in a case in which the cell image10includes an area that is small but corresponds to very high accuracy (probability value21) of a first type of classification of two or more types, the value of the first type of classification is acquired as the representative value20a. In this case, even if the cells90included in the cell image10should be classified as a second type of classification different from the first type of classification from its entire view, the cells90included in the cell image10are classified into the first type of the two or more types based on the probability values21of a part of the cell image10. To address this, the average value of the probability values21is acquired as the representative value20aas discussed above whereby preventing that the cell image10is classified into the first type of the two or more types based on the probability values21of a part of the cell image10when the cell image10is classified by classifying the cells90included in the cell image10. Consequently, it is possible to prevent reduction of classification accuracy when the cell image10is classified by classifying the cells90included in the cell image10.

In this embodiment, as discussed above, a step of producing the learned model6by using the teacher cell images30that are the cell image10, and teacher correct images31that are generated by adding the cell image10with a label value relating to at least two imaging conditions corresponding to the classification or a label value relating to at least two cultivation conditions corresponding to the classification is further included. Accordingly, the learned model6can be produced to be able to classify the cell image10into one of images in which the cells90included in the cell image10are captured under two or more types of imaging conditions by using the teacher correct images31that are generated by adding the cell image10with a label value relating to at least two imaging conditions corresponding to the classification. Also, the learned model6can be produced to be able to classify the cell image10into one of images including the cultivated cells90in which the cells90included in the cell image10are cultivated under two or more types of cultivation conditions by using the teacher correct images31that are generated by adding the cell image with a label value relating to at least two cultivation conditions corresponding to the classification.

In this embodiment, as discussed above, the learned model6is produced by using the teacher correct images31that are added with two types of label values corresponding to whether a focus of the cell image10is correct when the cell image is captured as the label value relating to the imaging conditions, or at least two types of label values relating to coating materials on a cultivation container80in which the cells90are cultivated, and the number of cultivation days of the cell as the label value relating to the cultivation conditions in the step of producing the learned model6. Consequently, the learned model6can be produced to be used to classify an image into one of two or more types of classes relating to any of conditions whether a focus of the cell image is correct when the cell image10is captured, conditions whether a coating material on the cultivation container80of the cell is proper, and conditions whether the number of cultivation days of the cell is proper.

In this embodiment, as discussed above, a step of determining whether the index value20is greater than a threshold is further provided. Accordingly, the cell image10whose index value20is greater than the threshold can be determined as an image that is suitable for analysis of determination whether the cells90included in the cell image10are normal or abnormal cells if the Also, the cell image10whose index value20is not greater than the threshold can be determined as an image that is not suitable for analysis of determination whether the cells90included in the cell image10are normal or abnormal cells if the Accordingly, in a case in which a plurality of cell images10are used for analysis whether the cells90that are included in the cell image10are normal or abnormal cells, only the cell image10that is suitable for analysis of determination whether the cells90included in the cell image10are normal or abnormal cells can be used foe the analysis. Consequently, it is possible to prevent reduction of accuracy of analysis of determination whether the cells90included in the cell image10are normal or abnormal cells.

Modified Embodiments

While the example in which the controller2aacquires an average value of the probability values21as the representative value20ahas been shown in the aforementioned embodiment, the present invention is not limited to this. For example, the controller2amay acquire any of a median value, the maximum value, the minimum value, and the most frequently appearing value of the probability values21as the representative value20a.

While the example in which the learned model6is produced by leaning at least one of classification of the cell whether a focus of the cell image is correct as imaging conditions, classification of the cell whether a coating material on the cultivation container80of the cell is proper, and classification of the cell whether the number of cultivation days of the cell90is proper has been shown in the aforementioned embodiment, the present invention is not limited to this. For example, the learned model6may be produced by learning classification of conditions other than classification whether a focus of the cell image is correct as imaging conditions. For example, the learned model6may be produced by learning classification whether a type of imaging device is proper as imaging conditions. Also, the learned model6may be produced by learning classification of conditions other than classification of the cell whether a coating material on the cultivation container80of the cell is proper, and classification of the cell whether the number of cultivation days of the cell is proper. For example, the learned model6may be produced by learning classification whether a type of cultivation device is proper as imaging conditions. Any imaging conditions and any cultivation conditions can be used for classification by the learned model6.

While the example in which the controller2ais configured to acquire the representative value20aof the probability values21in cell areas has been shown in the aforementioned embodiment, the present invention is not limited to this. For example, the controller2amay be configured to acquire the representative value20abased on the probability values21of all pixels in the cell image10. However, if the controller2ais configured to acquire the representative value20aof the probability values21of all pixels in the cell image10, the processing load of acquiring the representative value20aincreases. For this reason, the controller2ais preferably configured to acquire the representative value20aof the probability values21in cell areas.

While the example in which the controller2ais configured to display the superimposed cell image50, numerical data of the representative value20a, and the frequency distribution22on the display4has been shown in the aforementioned embodiment, the present invention is not limited to this. For example, the controller2amay be configured to display only the numerical data of the representative value20aon the display4. Also, the controller2amay be configured to display the numerical data of the representative value20aand the superimposed cell image50on the display4.

While the example in which the controller2adisplays numeric data of the representative value20aon the display4by collectively displaying numeric data of a plurality of representative values20ain the graph40a, the graph40b, or the graph40chas been shown in the aforementioned embodiment, the present invention is not limited to this. For example, the controller2amay be configured to display the numerical data of the representative values20aby displaying Arabic figures of the representative values20ainstead of the graph.

While the example in which the cell image analysis apparatus100produces the learned model6has been shown in the aforementioned embodiment, the present invention is not limited to this. For example, the cell image analysis apparatus100may be configured to use the learned model6that is produced by an image analysis apparatus other than the cell image analysis apparatus100.

While the example in which the superimposed cell image generator2dgenerates the superimposed cell image50including the blue mark51that is superimposed on parts of the probability value21corresponding to correct focus a first type of classification of two or more types of classification, and the red mark52that is superimposed on parts of the probability value21corresponding to a second type of classification different from the first type of classification has been shown in the aforementioned embodiment, the present invention is not limited to this. The superimposed cell image generator2dmay superimpose any color marks on parts of the probability values21of label values corresponding to two or more types of classification as long as the probability values21can be distinguished from each other.

While the example in which the image processor2cgenerates the first learned model6aby using teacher correct images31that are added with two types of label values corresponding to classification whether a focus of the cell image is correct has been shown in the aforementioned embodiment, the present invention is not limited to this. For example, the image processor2cmay be configured to generate a learning model7by using teacher correct images that are added with three or more label values corresponding to in-focus degrees.

While the example in which the image processor2cgenerates the second learned model6bby using teacher correct images31that are added with two types of label values corresponding to classification whether a type of the coating material is the coating material A has been shown in the aforementioned embodiment, the present invention is not limited to this. For example, the image processor2cmay be configured to generate a learning model7by using teacher correct images that are added with three or more label values corresponding to types of coating materials.

While the example in which the image processor2cgenerates the third learned model6cby using teacher correct images31that are added with two types of label values corresponding to classification whether the number of cultivation days is 5 has been shown in the aforementioned embodiment, the present invention is not limited to this. For example, the image processor2cmay be configured to generate a learning model7by using teacher correct images that are added with three or more label values corresponding to cultivation days.

While the example in which the image acquirer1acquires the cell image10in step201has been shown in the aforementioned embodiment, the present invention is not limited to this. For example, the image processor2cmay be configured to acquire the cell image10that is previously acquired by the image acquirer1and stored in the storage3.

While the example in which the controller2adetermines whether the index value20is greater than the threshold has been shown in the aforementioned embodiment, the present invention is not limited to this. For example, the controller2amay not determine whether the index value20is greater than the threshold

Modes

The aforementioned exemplary embodiment will be understood as concrete examples of the following modes by those skilled in the art.

A cell image analysis method includes a step of acquiring a cell image(s) includes a cell(s); a step of inputting the cell image to a learned model that has learned classification of the cell into one of two or more types; a step of acquiring an index value indicating accuracy of the classification of the cell that is included in the cell image into one of two or more types based on an analysis result of each of pixels of the cell image output from the learned model; and a step of displaying the acquired index value.

In the cell image analysis method according to mode item 1, the learned model has been learned to output a probability value(s) that is/are an estimation value(s) of the classification as the analysis result; and a representative value of the probability value(s) obtained based on the probability value(s) output by the learned model is acquired as the index value in the step of acquiring an index value.

In the cell image analysis method according to mode item 2, the cell image includes cultivated cell that is cultivated in a cultivation container; the learned model is produced by leaning at least one of classification of the cell whether a focus of the cell image is correct when the cell image is captured, classification of the cell whether a coating material on the cultivation container of the cell is proper, and classification of the cell whether the number of cultivation days of the cell is proper; and an index value(s) of at least one of classification of the cell whether a focus of the cell image is correct when the cell image is captured, classification of the cell whether a coating material on the cultivation container of the cell is proper, and classification of the cell whether the number of cultivation days of the cell is/are proper is acquired as the index value in the step of acquiring an index value.

In the image analysis method according to mode item 2 or 3, the learned model is produced by learning classification of the cell whether the cell is suitable for analysis whether the cell is a normal or abnormal cell; and a value representing a suitability accuracy for analysis whether the cell that is included in the cell image is a normal or abnormal cell is acquired based on the probability value(s) as the index value in the step of acquiring an index value.

In the cell image analysis method according to mode item 4, the learned model is produced by learning classification of the cell whether cells of a common type are suitable for analysis whether each cell is a normal or abnormal cell.

In the cell image analysis method according to any of mode items 2 to 5, a step of acquiring a cell area that is an area of the cell included in the cell image is further provided, wherein the representative value of the probability value(s) in the cell area is obtained as the representative value of the probability value(s) in the step of obtaining the representative value of the probability value(s).

In the cell image analysis method according to any of mode items 2 to 6, a superimposed cell image that is generated by superimposing numerical data of the representative value of the probability values and a distribution of the probability values on the cell image is displayed in the step of displaying the representative value of the probability value(s).

In the cell image analysis method according to mode item 7, a frequency distribution of the probability values is displayed together with the numerical data of the representative value of the probability values and the superimposed cell image in the step of displaying the representative value of the probability value(s).

In the cell image analysis method according to any of mode items 2 to 8, an average value of the probability values is obtained as the representative value in the step of obtaining the representative value of the probability value(s).

In the cell image analysis method according to any of mode items 1 to 9, a step of producing the learned model by using teacher cell images that are the cell images, and teacher correct images that are generated by adding the cell images with a label value relating to at least two imaging conditions corresponding to the classification or a label value relating to at least two cultivation conditions corresponding to the classification is further provided.

In the cell image analysis method according to mode item 10, the learned model is produced by using the teacher correct images that are added with two types of label values corresponding to whether a focus of the cell image is correct when the cell image is captured as the label value relating to the imaging conditions, or at least two types of label values relating to coating materials on a cultivation container in which the cell is cultivated, and the number of cultivation days of the cell as the label value relating to the cultivation conditions in the step of producing the learned model.

In the cell image analysis method according to any of mode items 1 to 11, a step of determining whether the index value is greater than a threshold is further provided.

DESCRIPTION OF REFERENCE NUMERALS