Production method of learned model, brightness adjustment method, and image processing apparatus

This production method of a trained model (10) includes a step of optimizing a training model (11) so that a value of a loss function (A) becomes small. The loss function (A) is configured to output a relatively large value in a case where the contrast of the predetermined area (PA) in a brightness-adjusted image adjusted in brightness becomes high with respect to training data (20) than in a case where the contrast of the predetermined area in the brightness-adjusted image (3) becomes low with respect to the training data.

TECHNICAL FIELD

The present invention relates to a brightness adjustment method of a radiographic image.

BACKGROUND ART

Conventionally, it is known to perform a brightness adjustment on a medical radiographic image reflecting a subject so as to have brightness and contrast suitable for a diagnosis. Such a brightness adjustment of a radiographic image is disclosed in, for example, Japanese Unexamined Patent Application Publication No. 2015-181543.

In Japanese Unexamined Patent Application Publication No. 2015-181543, it is disclosed to perform a brightness adjustment of an input image by extracting a main subject area from an input image reflecting a subject, acquiring a representative value representing the brightness value in the area, calculating a gradation conversion coefficient for emphasizing or suppressing the contrast close to the representative value, and converting the brightness value (pixel value) of each pixel of the input image in accordance with a predetermined algorithm by using the calculated gradation conversion coefficient.

PRIOR ART DOCUMENT

Patent Document

Patent Document 1: Japanese Unexamined Patent Application Publication No. 2015-181543

SUMMARY OF THE INVENTION

Problems to be Solved by the Invention

In the brightness adjustment of the radiographic image, it is desirable to improve the visibility by enhancing the contrast in the area related to a diagnosis of a subject or the like in the image while preventing the occurrence of saturation of pixel values in the area. Saturation of a pixel value means that the information of the pixel is lost by the signal of the level exceeding the upper or lower limit of the pixel value. A pixel exceeding the upper limit is called “halation”, and a pixel falling below the lower limit is called “black defects”. As the contrast is increased by the brightness adjustment, the saturation of the pixel value more likely occurs. Therefore, it is preferable that the contrast of the pixel value in the area related to a diagnosis is increased as large as possible under the condition that no saturation occurs in the area related to the diagnosis of the subject.

In the technique disclosed in Japanese Unexamined Patent Application Publication No. 2015-181543, since the brightness adjustment is performed according to a certain algorithm, it is difficult to cope with exceptional cases, such as, e.g., the occurrence of the extraction error of the area and an artificial joint in which the pixel value distribution differs greatly from a usual distribution. Thus, halation or black defects may occur in the area related to the diagnosis of a subject or the like. For this reason, in recent years, an attempt has been made to directly output a brightness adjustment parameter from an input image using a trained model by machine learning. If the machine learning functions effectively, it is expected that the contrast can be increased as much as possible while suppressing the occurrence of halation and black defects in an area related to a diagnosis of a subject or the like, and that it is possible to deal with exceptional cases, such as, e.g., an artificial joint.

However, in machine learning, statistical learning is performed using a large amount of training data. Therefore, the brightness adjustment parameter to be output varies in a direction of increasing the contrast and in a direction of decreasing the contrast with respect to the optimal value (the value of increasing the contrast as much as possible under the condition that no saturation of a pixel value occurs in the area related to a diagnosis of a subject or the like). Therefore, in the trained model machine-trained such that the optimal value is output, when the brightness adjustment parameter to be output varies in the direction of increasing the contrast with respect to the optimal value, there is a problem that halation or black defects (saturation of a pixel value) may occur.

The present invention has been made to solve the above-mentioned problems. An object of the present invention is to provide a production method of a trained model, a brightness adjustment method, and an image processing apparatus capable of suppressing the occurrence of halation or black defects in an area related to a diagnosis of a subject or the like, in a brightness adjustment using a trained model by machine learning.

Means for Solving the Problem

In order to attain the above-described object, the production method of a trained model according to a first aspect of the present invention, a production method of a trained model by machine learning in which a radiographic image reflecting a subject is input and a brightness adjustment parameter of the radiographic image is output, the production method includes:a step of outputting the brightness adjustment parameter of an input image by a training model with the input image as an input, the input image being included in training data;a step of acquiring a value of a loss function related to the brightness adjustment parameter using the brightness adjustment parameter by the training model; anda step of optimizing the training model so that the value of the loss function becomes small,wherein in a case where contrast of a predetermined area in a brightness-adjusted image adjusted in brightness based on the brightness adjustment parameter becomes high with respect to the training data than in a case where the contrast of the predetermined area in the brightness-adjusted image becomes low with respect to the training data, the loss function is configured to output a relatively large value to bias the machine learning in a direction of decreasing the contrast.

In the trained model production method according to the first aspect of the present invention, by the above-described configuration, in a case where the contrast of the predetermined area in the brightness-adjusted image becomes high with respect to the training data, the loss function outputs a relatively large value. Therefore, it is possible to acquire a trained model in which it is suppressed that the brightness adjustment parameter is output to become a large contrast beyond the optimal value. That is, in the produced trained model, it is possible to bias the predictive distribution of the brightness adjustment parameter to be output to the radiographic image in the direction of decreasing the contrast with respect to the optimal value (the value that increases the contrast as much as possible under the condition that no saturation of the pixel value occurs in the predetermined area related to a diagnosis of a subject or the like). Thus, it is possible to suppress the occurrence of halation or black defects (saturation of a pixel value) in the predetermined area in the brightness-adjusted image by excessively increasing the contrast in the predetermined area. Thus, in the brightness adjustment using the trained model by the machine learning, it is possible to suppress the occurrence of halation or black defects in the area (predetermined area) related to a diagnosis of a subject or the like in the image.

In the production method of a trained model according to the first aspect of the present invention, in a case where halation or black defects occur due to saturation of a pixel value in the predetermined area in the brightness-adjusted image, the loss function is configured to output a relatively large value than in a case where saturation of a pixel value does not occur in the predetermined area. With this configuration, in a case where halation or black defects occur in the predetermined area, it is biased so that the value of the loss function becomes large. Therefore, it is possible to acquire a trained model trained so that the occurrence of halation or black defects in the predetermined area is suppressed.

In the production method of a trained model according to the first aspect of the present invention, in a case where the contrast of the brightness-adjusted image becomes high with respect to the training data than in a case where the contrast of the brightness-adjusted image becomes low with respect to the training data, the loss function includes a largely weighted weighting function. With this configuration, there is no need to prepare a plurality of loss functions to bias the output value according to the contrast, and therefore the value of the loss function can be easily biased by the weighting function.

In the production method of a trained model according to the first aspect of the present invention, the loss function is a function based on an error between a training parameter included in the training data and the brightness adjustment parameter with respect to the input image included in the training data. In the machine learning using a loss function based on an error of a brightness adjustment parameter, it is considered that errors vary symmetrically in the direction of decreasing the contrast and the direction of increasing the contrast with respect to the optimal value. Therefore, when learning is performed by an unbiased loss function, there is a possibility that the half of the predictive distribution of the brightness adjustment parameter to be output from the trained model varies in the direction of increasing the contrast with respect to the optimal value, causing halation or black defects (saturation of a pixel value). According to the above-described configuration, even by a loss function based on the error between the training parameter and the brightness adjustment parameter, it is possible to suppress that the error is evaluated symmetrically with respect to the optimal value. Thus, when the contrast becomes high with respect to the optimal value, it is possible to increase the value of the loss function relatively. As a result, it is possible to effectively suppress the occurrence of saturation of a pixel value in the predetermined area.

In the production method of a trained model according to the first aspect of the present invention, the loss function is a function based on a comparison between a pixel value a training image included in the training data and a pixel value of the brightness-adjusted image in which the input image included in the training data has been adjusted in the brightness based on the brightness adjustment parameter. Here, the training image is a brightness-adjusted image adjusted in the brightness based on the brightness adjustment parameter considered to be the optimal value. In the machine learning using a lost function based on a comparison of pixel values, when there is halation or black defects (saturation of a pixel value) in the brightness-adjusted image, it is underestimated by the amount of saturation of the pixel value in the area, which is likely to be biasedly trained in the direction of increasing the contrast. Consequently, in the trained model trained by an unbiased loss function, there is a possibility that the output brightness adjustment parameter varies in the direction of increasing the contrast with respect to the optimal value, and therefore halation or black defects (saturation of a pixel value) occurs. Therefore, according to the above-described configuration, even by the loss function based on the comparison of pixel values, it is possible to relatively increase the value of the loss function when the contrast becomes high with respect to the optimal value and suppress the underestimation of errors due to the saturation of the pixel value. Therefore, it is possible to effectively suppress the occurrence of saturation of the pixel value in the predetermined area.

In a case where the loss function is a function based on a comparison between the pixel value of the training image and the pixel value of the brightness-adjusted image, in a case where saturation of a pixel value occurs in the predetermined area in the image, the loss function is configured to output a value larger than in a case where saturation of a pixel value occurs in an area other than the predetermined area. Here, for example, the diagnosis importance level differs between the predetermined area reflecting a subject and the area of the background other than the predetermined area in the image. Therefore, in a case where saturation of a pixel value occurs in the predetermined area, by relatively increasing the value of the loss function and relatively decreasing the value of the loss function in the area other than the predetermined area, it is possible to suppress that the learning is affected by the error in the area other than the predetermined area while suppressing the occurrence of saturation of the pixel value in the predetermined area.

In this case, the loss function is set in magnitude of bias for each position in the input image, based on position information of the predetermined area included in the training data and bias information of the loss function. With this configuration, it is possible to distinguish between the predetermined area and the area other than the predetermined area based on the position information of the predetermined area and also possible to specify the magnitude relation of the bias of the loss function between the predetermined area and the area other than the predetermined area, based on the bias information of the loss function. Therefore, it is possible to easily differentiate the magnitude of the bias of the loss function according to the area in the image.

In the production method of a trained model according to the first aspect of the present invention, the predetermined area includes at least a part of the area reflecting the subject. With this configuration, in the image, a trained model in which the learning is biased in the direction of decreasing the contrast in the area reflecting a subject related to a diagnosis in the image. Therefore, it is possible to more assuredly suppress the occurrence of halation or black defects in the area reflecting a subject using the brightness adjustment parameter output by the trained model.

In the production method of a trained model according to the first aspect of the present invention, the brightness adjustment parameter includes a window level that defines a median of a pixel value range to be converted to the brightness-adjusted image in the radiographic image and a window width that defines magnitude of the pixel value range. In such a brightness adjustment by a window level and a window width, the magnitude of the contrast of the pixel value in the predetermined area can be adjusted. Therefore, the contrast can be easily evaluated by the loss function, as compared with a case where a large number of parameters affect the contrast.

A brightness adjustment method according to a second aspect of the present invention is a brightness adjustment method using machine learning in which a radiographic image reflecting a subject is input as an input and a brightness adjustment parameter of the radiographic image is output. The brightness adjustment method includes:a step of acquiring the radiographic image;a step of acquiring the brightness adjustment parameter with respect to the radiographic image using a trained model by the machine learning; anda step of acquiring a brightness-adjusted image by adjusting brightness of the radiographic image based on the brightness adjustment parameter,wherein the trained model is generated by learning that optimizes a loss function related to the brightness adjustment parameter, andwherein in a case where the contrast of a predetermined area in the brightness-adjusted image becomes high with respect to the training data of the machine learning than in a case where the contrast of the predetermined area in the brightness-adjusted image becomes low with respect to the training data,the loss function is configured to output a relatively large value to bias the machine learning in a direction of decreasing the contrast.

In the brightness adjustment method according to the second aspect of the present invention, with the above-described configuration, in a case where the contrast of the predetermined area in the brightness-adjusted image becomes high with respect to the training data, a value in which the loss function is relatively large is output. Consequently, in the trained model machine-trained by the loss function, the predictive distribution of the brightness adjustment parameter to be output to the radiographic image can be biased in the direction of decreasing the contrast with respect to the optimal value. Therefore, even by the trained model learned to output the optimal value, it is suppressed to output the brightness adjustment parameter to become high contrast exceeding the optimal value. Therefore, even if the brightness adjustment is performed using the output brightness adjustment parameter, it is possible to suppress the occurrence of halation or black defects (saturation of a pixel value) of the predetermined area in the brightness-adjusted image by excessively increasing the contrast of the predetermined area. Thus, in the brightness adjustment using a trained model by machine learning, it is possible to suppress the occurrence of halation or black defects in the area (predetermined area) related to a diagnosis of a subject or the like in the image.

In the brightness adjustment method according to the second aspect, the brightness adjustment method as recited in claim10, further includes:a step of acquiring information related to an imaging site or an imaging purpose of the subject,wherein in the step of acquiring the brightness adjustment parameter, the trained model for use in the brightness adjustment from a plurality of the trained models is selected based on the information related to the imaging site or the imaging purpose

Note that the term “imaging site” denotes an individual site to be imaged, such as, e.g., a head, an abdomen, a lumbar region, and limbs of a subject. The imaging purpose denotes a diagnosis content desired to be grasped based on the captured image or an application of the image. For example, even in the case of a chest image, which lesion is to be grasped between the heart, the lungs, and the bones. Here, in a medical radiographic image, since the target to be depicted differs depending on the imaging site and the imaging purpose, there is a possibility that the optimal value of the brightness adjustment parameter may differ greatly. Therefore, by preparing a plurality of trained models separately trained according to the imaging site or the imaging purpose and selecting the trained model based on the imaging site or the imaging purpose, it is possible to perform the optimal brightness adjustment according to the imaging site or the imaging purpose.

An image processing apparatus according to a third aspect of the present invention, includes:an image acquisition unit configured to acquire a radiographic image;a storage unit configured to store a trained model by machine learning in which a radiographic image reflecting a subject is input and a brightness adjustment parameter of the radiographic image is output;a parameter acquisition unit configured to acquire the brightness adjustment parameter with respect to the radiographic image using the trained model; andan adjustment processing unit configured to acquire a brightness-adjusted image by adjusting brightness of the radiographic image based on the brightness adjustment parameter, andwherein in a case where contrast of a predetermined area in the brightness-adjusted image adjusted in brightness becomes high with respect to training data of the machine learning than in a case where the contrast of the predetermined area in the brightness-adjusted image becomes low with respect to the training data,the loss function is configured to output a relatively large value to bias the machine learning in a direction of decreasing the contrast of the predetermined area in the brightness-adjusted image.

In the image processing apparatus according to the third aspect, with the above-described configuration, in the same manner as in the brightness adjustment method according to the second aspect, in the brightness adjustment using the trained model by the machine learning, it is possible to suppress the occurrence of halation or black defects in the area (predetermined area) related to a diagnosis of a subject or the like in the image.

Effects of the Invention

According to the present invention, as described above, it is possible to suppress the occurrence of halation or black defects in an area related to a diagnosis of a subject or the like in an image, in a brightness adjustment using a trained model by machine learning.

EMBODIMENTS FOR CARRYING OUT THE INVENTION

Hereinafter, embodiments in which the present invention is embodied will be described with reference to the attached drawings.

Referring toFIG.1toFIG.15, a production method of a trained model, a brightness adjustment method, and an image processing apparatus according to a first embodiment will be described.

(Outline of Production Method of Trained Model, Brightness Adjustment Method, and Image Processing Apparatus)

The brightness adjustment method according to the first embodiment shown inFIG.1is a brightness adjustment method using machine learning in which a captured radiographic image1reflecting a subject is input and a brightness adjustment parameter2of the radiographic image1is output.

The radiographic image1is a medical image used for, e.g., diagnose of a patient by a doctor or the like, and is, for example, an X-ray image as shown inFIG.2. Specifically, the radiographic image1is a still image acquired by imaging a diagnosis region (lesion) of a subject by so-called simple roentgenography. The radiographic image1is a monochrome digital image in which the shading (light and dark) of each pixel is expressed by, for example, pixel values sectioned by a predetermined gradation number.FIG.2shows an example of a radiographic image1in which the periphery of the femur in the lumbar region of the subject is imaged. In the diagnosis by the radiographic image1, the brightness adjustment of the radiographic image1is performed in order to more accurately grasp the state of the affected part which is the object of the diagnosis. The brightness adjustment is also referred to as a gradation conversion.

Specifically, as shown inFIG.1, the brightness adjustment method of the first embodiment includes: Step S1of acquiring a radiographic image1; Step S2of acquiring the brightness adjustment parameter2with respect to the radiographic image1, using a trained model10by machine learning; and Step S3for acquiring a brightness-adjusted image3by adjusting the brightness of the radiographic image1, based on the brightness adjustment parameter2.

In Step S1, for example, by the imaging unit210constituted by, e.g., an X-ray imaging device, a radiographic image1reflecting a subject is acquired. In Step S1, a radiographic image1previously stored in an image server250for storing images may be acquired from the image server250.

In Step S2, when the radiographic image1to be subjected to a brightness adjustment is input, the trained model10outputs a brightness adjustment parameter2for the input image. The trained model10has been machine-trained to output the optimal brightness adjustment parameter2for the diagnosis with respect to the input image (radiographic image1). The machine learning method is not particularly limited as long as it is suitable for the processing of outputting the brightness adjustment parameter2to the input image.

As one example, in the first embodiment, as shown inFIG.3, the trained model10includes a front stage composed of a convolutional neural network CNN and a rear stage composed of a fully connected layer FC. Such a trained model10is considered to have a function of recognizing an image of a predetermined area which is crucial for the diagnosis from the radiographic image1by the convolutional neural network CNN of the front stage, and a function of converting the output of the front stage to a brightness adjustment parameter2by the fully connected layer FC of the rear stage. The trained model10is provided with a high-precision image recognition function of the convolutional neural network CNN. Therefore, it is expected to effectively suppress the saturation of a pixel value, such as, e.g., halation and black defects, in the area which is crucial for a diagnosis. It is also expected that the multi-layering (deepening) of the convolutional neural network CNN will be able to cope with exceptional cases, such as, e.g., individual differences of the individual radiographic image1and artificial joints.

As shown inFIG.1, in order to generate the trained model10for performing the processing of outputting the brightness adjustment parameter2, machine learning using a large amount of training data20is performed. In this specification, the state in the middle of learning by machine learning at the production stage of the trained model10is referred to as a training model11. When the training model11has been machine-trained, it is referred to as a trained model10.

In the first embodiment, the training data20used for machine learning includes an input image21and a training parameter22. The input image21is a radiographic image (seeFIG.2) prepared in advance for learning and is an original image before the brightness adjustment is performed. The training parameter22is ground truth data generated as the optimal value of the brightness adjustment parameter output to the input image21.

In a medical radiographic image, in order to improve the visibility of the region of interest, it is desirable to acquire the contrast as large as possible in the predetermined area PA (seeFIG.2) related to a diagnosis of a subject or the like in an image. On the other hand, in a medical radiographic image, it is desired that saturation of a pixel value, such as, e.g., so-called halation and black defects, does not occur in the predetermined area PA related to a diagnosis of a subject or the like in the image. On the other hand, in a medical radiographic image, for an area PB (seeFIG.2) other than the predetermined area PA of a subject in an image, such as, e.g., a background area and an area not related to the diagnosis, the decrease in the contrast or the occurrence of saturation in the pixel value is acceptable. Therefore, the optimal value of the brightness adjustment parameter in a medical radiographic image can be defined as a value that increases the contrast of the predetermined area PA as much as possible under the condition that saturation of a pixel value does not occur in the predetermined area PA related to a diagnosis. The training parameter22is an optimal value derived with respect to each input image21by, for example, the parameter adjustment operation by a training data generator.

When producing a trained model, an input image21is input and a brightness adjustment parameter2is output, and the learned model is made to learn the conversion processing from the input image21to the training parameter22. In machine learning, a loss function for evaluating the difference between the output result (brightness adjustment parameter2) of the training model11which is being trained and the training data (training parameter22) is defined. The machine learning makes the inner parameter of the training model11optimize (train) to output an output result that minimizes the difference from the training data20by the optimization operation that reduces the loss function. The trained model10is produced by performing the optimization (learning) of the training model11over a sufficient number of times to converge the learning by using a sufficient amount of training data20to converge the learning. As described above, the trained model10is generated by performing the learning for optimizing a loss function related to the brightness adjustment parameter2.

Here, in a case where the contrast of the predetermined area PA in the brightness-adjusted image3becomes high with respect to the training data20than in a case where the contrast of the predetermined area PA (seeFIG.2) in the brightness-adjusted image3becomes low with respect to the training data20of the machine learning, the loss function A of the first embodiment is configured to output a relatively large value to bias the learning in a direction of decreasing the contrast.

The predetermined area PA (seeFIG.2) is, as described above, an area related to a diagnosis of a subject or the like in the image in the medical radiographic image. The predetermined area PA includes at least a part of the area reflecting a subject. The predetermined area PA can be set to only a part of the area reflecting the subject, not the entire area reflecting the subject. The region of interest related to a diagnosis is a part of an area in which a disorder is considered to exist. Thus, the predetermined area PA may be a part of an area including the region of interest in the area reflecting the subject.

In Step S3, the processing of the brightness adjustment based on the brightness adjustment parameter2is performed. Consequently, a brightness-adjusted image3in which the brightness of the input radiographic image1has been adjusted is acquired. The brightness-adjusted image3is output to, e.g., a display device and is used for a diagnosis by a doctor or the like.

Next, the brightness adjustment parameter2output by the trained model10will be described. In the first embodiment, the content of the brightness adjustment (gradation conversion) processing is not particularly limited, and the brightness adjustment parameter2is not particularly limited as long as it corresponds to the brightness adjustment processing of an image. In the first embodiment, as one example, a brightness adjustment by windowing processing for converting a partial range of a gradation range of the input image into a gradation range for displaying will be described.

FIG.4is a conceptual diagram of a histogram HG of pixel values (gradations) in the radiographic image1and an output gradation after the brightness adjustment. The horizontal axis of the histogram HG represents a pixel value of the input image, and the vertical axis of the histogram HG represents the number of pixels (frequency) having its pixel value.

In the window process, a part of the gradation range specified by the brightness adjustment parameter2in the input image is extracted and converted to match the output gradation range after the brightness adjustment. With this, in the window processing, the pixel value range of the image portion constituting the predetermined area PA (seeFIG.2) related to a diagnosis is specified by the brightness adjustment parameter2, so that the contrast of the predetermined area PA can be adjusted. Further, in the window processing, it is possible to extract a part of the gradation range in the original radiographic image1and convert it to a gradation range that the display device can output. As shown inFIG.4, the gradation number of the medical radiographic image is generally greater than the gradation number that the display device can display. For example, a medical radiographic image is image data having a pixel value of 4,096 gradations (12 bit) ranging from 0 to 4,095. On the other hand, a typical display device displays 256 gradations (8 bit) ranging from 0 to 255. Hereinafter, an example of the window processing for converting an input image of 4,096 gradations to a gradation range of 256 gradations will be described.

The brightness adjustment parameter2includes: a window level WL that defines the median of the pixel value range to be converted to the brightness-adjusted image3in the radiographic image1; and a window width WW that defines the size of the pixel value range. In the brightness adjustment processing, a brightness adjustment is performed by assigning the gradation range of a window width WW centered on the window level WL to output gradations (0 to 255). At this time, in the radiographic image1, a pixel having a pixel value (0 to VL) that is equal to or lower than the lower limit VL of the window width WW is converted to the lowest gradation (0 gradations) in the brightness-adjusted image3. In the radiographic image1, a pixel having a pixel value (VH to 4,095) which is equal to or higher than the upper limit VH of the window width WW is converted to the maximum gradation (255 gradations) in the brightness-adjusted image3. With this, the brightness-adjusted image3becomes an image represented by the gradation number corresponding to the output gradation. The gradation range of the window width WW may be larger or smaller than the output gradation range (256 gradations).

FIG.5shows examples of conversion characteristics (density conversion curves) when the gradation range of the window width WW is converted to the output gradations (0 to 255). In the brightness adjustment, as shown in (A) ofFIG.5, the gradation range of the window width WW may be converted to the output gradation by a linear expression (linear transformation characteristic), or as shown in (B) ofFIG.5, the gradation range of the window width WW may be converted to the output gradation by a nonlinear transformation characteristic, such as, e.g., a quadratic and a logarithmic function. In the case as shown in (B) ofFIG.5, the brightness adjustment parameter2may include an additional parameter that determines the transformation characteristics (shape of the concentration conversion curve).

As shown in (A) ofFIG.6, it is assumed that the image portion of the predetermined area PA is composed of pixels included in, e.g., the hatched area of the histogram HG. At this time, it is assumed that the window width WW is set in three patterns: the WWa which is larger than the gradation range of the predetermined area PA; the WWb which coincides the gradation range of the predetermined area PA; and the WWc which is smaller than the gradation range of the predetermined area PA.

When the window width WW is set by WWa, as shown in (B) ofFIG.6, the gradation range WWb of the predetermined area PA is expressed by a part of the output gradation range (0 to 255) in the brightness-adjusted image3. The predetermined area PA is expressed by a range narrower than 256 gradations.

When the window width WW is set by WWb, as shown in (C) ofFIG.6, the gradation range WWb of the predetermined area PA is expressed by the entire range of the output gradation range (0 to 255) in the brightness-adjusted image3. Therefore, the contrast increases as compared with the case where the window width WW is set by WWa.

When the window width WW is set by WWc, as shown in (D) ofFIG.6, the gradation range WWb of the predetermined area PA becomes wider than the window width WWc, and therefore, among the pixels constituting the predetermined area PA, as for the pixels having pixel values outside the window width WWc (pixel values equal to or larger than the upper limit VH of the window width WWc and pixel values equal to or lower than the lower limit VL of the window width WWc), the pixel value is saturated in the brightness-adjusted image3, and therefore it is expressed as 0 gradation or 255 gradations. As for the pixels belonging to the inside of the window width WWc in the predetermined area PA, it is expressed by the entire range of the output gradation range (0 to 255) in the brightness-adjusted image3, and therefore the contrast increases.

Thus, in the brightness adjustment parameter2, as the window width WW becomes smaller, the contrast of the predetermined area PA in the brightness-adjusted image3increases. In other words, as the window width WW is reduced, the shading (light and dark) of the pixels included in the predetermined area PA can be expressed with more gradation numbers in the brightness-adjusted image3. Therefore, the contrast resolution increases. On the other hand, if the window width WW is set too small, the saturation of pixel values, such as, e.g., halation and black defects, occurs in the predetermined area PA of the brightness-adjusted image3. The larger the window width WW, the smaller the contrast of the predetermined area PA in the brightness-adjusted image3. As the window width WW is increased, the contrast of the predetermined area PA in the brightness-adjusted image3is decreased. As the window width WW increases, the shading (light and dark) of the pixels included in the predetermined area PA is expressed with the smaller number of gradations, and therefore the contrast resolution decrease.

Next, the loss function will be described. As shown inFIG.1, the training data20(training parameter22) used in machine learning is to provide the optimal value (ground truth) of the brightness adjustment parameter2. Therefore, the gradation range WWb (seeFIG.6) is designed to be the window width WW as much as possible. The trained model10is configured to output the value of the window width WW corresponding to the gradation range Wb as the brightness adjustment parameter2with respect to the input radiographic image1by the optimization using the loss function. Therefore, in a case where the window width WW as the brightness adjustment parameter2varies in a direction in which the window width WW as the brightness adjustment parameter2is larger than the optimal value (WWb), the contrast becomes low as compared with the optimal value. Thus, the brightness adjustment is performed in the direction in which the saturation of the pixel value in the predetermined area PA is less likely to occur. On the other hand, in a case where the window width WW as the brightness adjustment parameter2varies in a direction in which the window width is smaller than the optimal value (WWb), as compared with the optimal value, the contrast becomes high. Thus, the brightness adjustment is performed in a direction in which the saturation of the pixel value, such as, e.g., halation and black defects, is likely to occur in the predetermined area PA.

Therefore, as shown inFIG.1, in the first embodiment, in a case where the contrast of the predetermined area PA in the brightness-adjusted image3becomes high with respect to the training data20than in a case where the contrast of the predetermined area PA in the brightness-adjusted image3becomes low with respect to the training data20of the machine learning, the loss function A is configured to output a relatively large value to bias the learning in a direction of decreasing the contrast.

That is, the loss function A is a function designed such that in a case where the brightness adjustment parameter2output from the training model11which is being trained has an error in a direction of decreasing the contrast with respect to the training data20which is the optimal value, the error is evaluated relatively small, and in a case where the brightness adjustment parameter2output from the training model11which is being trained has an error in a direction of increasing the contrast with respect to the training data20which is the optimal value, the error is evaluated relatively large. In the machine learning, the training model11is optimized so that the loss function A is reduced. Therefore, the output value of the loss function A increases with respect to the error in the direction of increasing the contrast. As a result, the training model11is biased in a direction of suppressing the contrast.

Therefore, in a case where the brightness adjustment parameter2output from the training model11which is being trained outputs a window width WW smaller than the training parameter22which is the optimal value (WWb) and saturation of a pixel value (halation or black defects) occurs in the predetermined area PA, the loss function A is estimated relatively large. As described above, in the first embodiment, in a case where halation or black defects occur due to saturation of a pixel value in the predetermined area PA in the brightness-adjusted image3, the loss function A is configured to output a value relatively larger than in a case where no saturation of a pixel value occurs in the predetermined area PA.

Specific examples of the loss function A are shown. In the first embodiment, the loss function A is a function based on the error of the training parameter22included in the training data20and the brightness adjustment parameter2with respect to the input image21included in the training data20.

Further, in a case where the contrast of the brightness-adjusted image3becomes high with respect to the training data20than in a case where the contrast of the brightness-adjusted image3becomes low with respect to the training data20, the loss function A includes a highly weighted weighting function.

For example, the loss function A is a weighted mean squared error shown in the following Expression (1).

In the above Expression (1), WL and WW are a window level and a window width (brightness adjustment parameters2), respectively, which are output by the training model11which is being trained to the input image21. WL0, WW0are a window level and a window width (training parameters22) included in the training data20. WW0is a value corresponding to the optimal value (WWb) illustrated inFIG.6f(r) is a weighting function. w is a value of the weight (w>1).

In the case where the error (WW−WW0) between the output result of the training model11and the training data20becomes negative (in the case where the window width WW is smaller than the optimal value and the contrast is high), the weighting function f(r) estimates the error relatively large by adding the value of the weight w. In the case where the error (WW−WW0) between the output result of the training model11and the training data20is positive (in the case where the window width WW is larger than the optimal value and the contrast is low), the weighting function f(r) outputs the error as it is without adding the weight w. The weight w is not particularly limited, but is, for example, 10. In this case, the loss function A evaluates the error 10 times larger when the error (WW−WW0) becomes negative as compared with the case when the error (WW−WW0) becomes positive.

As a result, in a case where the contrast of the predetermined area PA in the brightness-adjusted image3becomes high with respect to the training data20(when the error (WW−WW0) becomes negative), the loss function A outputs a relatively large value to bias the learning in a direction of decreasing the contrast.

Next, referring toFIG.7, the effect of the learning bias by the loss function A in the first embodiment on the brightness adjustment parameter2output by the trained model10will be described. (A) ofFIG.7is a graph showing the change in the output value (vertical axis) of the loss function A with respect to the window width WW (horizontal axis). (B) ofFIG.7is a graph showing the predictive distribution PD of the window width WW output by the trained model10optimized using the loss function A. The predictive distribution PD indicates the frequency (probability) that the trained model10outputs the value of each window width WW.

As Comparative Example, a case using a loss function RMSE by a simplified mean square error without weighting is shown in dashed lines. The loss function RMSE of Comparative Example is represented by the following Expression (2).
RMSE=√{square root over ((WL−WL0)2+(WW−WW0)2)}  (2)

As can be seen from the above Expression (2) and (A) ofFIG.7, the loss function RMSE according to Comparative Example is symmetrical with respect to the error direction (the magnitude of the window width WW). Therefore, it is considered that the predictive distribution CD of the output result of the trained model optimized using the loss function RMSE is symmetrically distributed with respect to the optimal value WW0. As described above, in a case where the window width WW to be output is shifted to a smaller side with respect to the optimal value WW0, it is considered that saturation of a pixel value occurs in the predetermined area PA. In the trained model optimized using the loss function RMSE according to Comparative Example, the saturation of a pixel value may occur in the predetermined area PAs in approximately half of the cases of the window width WW to be output due to the symmetry of the predictive distribution CD of the output result.

On the other hand, the loss function A shown by the solid line inFIG.7is asymmetrically biased because the window width WW is estimated to be larger by the weighting factor w in the direction smaller than the optimal value (training data20) while taking the minimum value by the optimal value (WW0).

For this reason, the predictive distribution PD of the output result of the trained model10optimized by using the loss function A of the first embodiment becomes a distribution biased in a direction of decreasing the contrast (a direction of increasing the window width WW) with respect to the optimal value WW0than in a direction of increasing the contrast (a direction of decreasing the window width WW).

As a result, as can be seen from the predictive distribution PD, in the trained model10optimized using the loss function A of the first embodiment, the window width WW to be output is suppressed from varying in the direction of increasing the contrast (the direction of decreasing the window width WW) beyond the optimal value WW0, and the possibility of generating saturation of a pixel value in the predetermined area PA is suppressed.

Next, referring toFIG.8andFIG.9, the image processing apparatus100according to the first embodiment will be described. The image processing apparatus100is configured to perform brightness adjustment processing on a radiographic image1reflecting a subject using a trained model10by machine learning that outputs a brightness adjustment parameter2of a radiographic image1. That is, the image processing apparatus100is configured as a device for performing the brightness adjustment method according to the first embodiment described above.

In the example ofFIG.8, the image processing apparatus100is configured as a part of an imaging system200for capturing a radiographic image1of a subject. The imaging system200is provided with an imaging unit210, a controller220, an image processing apparatus100, an operation unit230, and a display unit240. The operation unit230is an input device, such as, e.g., a keyboard and a mouse. The display unit240is constituted by an image display device, such as, e.g., a liquid crystal display device.

The imaging unit210includes a top board211, an X-ray irradiation unit212, and an X-ray detection unit213. The top board211is configured to support a subject SB (human). The X-ray irradiation unit212is configured to emit X-rays toward the subject SB. The X-ray detection unit213is constituted by, for example, an FPD (Flat Panel Detector) and is configured to detect the X-rays emitted from the X-ray irradiation unit212and transmitted through the subject SB. By the imaging unit210, the X-ray image (seeFIG.2) reflecting the region of interest of the subject is acquired as a radiographic image1.

The controller220is a computer configured to control the imaging operation of the imaging unit210. The controller220performs various controls, such as, e.g., the control of the moving mechanism (not shown) for moving the top board211, the X-ray irradiation unit212, and the X-ray detection unit213, the control of the X-ray irradiation by the X-ray irradiation unit212, and the setting of the imaging condition.

Further, the image processing apparatus100is connected to the imaging unit210, the operation unit230, and the display unit240. The image processing apparatus100acquires the detection signal of X-rays from the X-ray detection unit213of the imaging unit210and acquires the radiographic image1based on the acquired detection signal. The image processing apparatus100performs the brightness adjustment of the radiographic image1(seeFIG.1) to acquire the brightness-adjusted image3(seeFIG.1).

As illustrated inFIG.9, the image processing apparatus100is configured by a computer including a processor110, such as, e.g., a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a dedicated FPGA (field-programmable gate array), and a storage unit120, such as, e.g., a ROM, a RAM, a volatile or a non-volatile storage device (HDD, SSD (solid state drive)). The image processing apparatus100may be configured by a computer (a personal computer, a workstation, a supercomputer, etc.) on which predetermined software (program) is installed in a storage unit, or a computer system consisting of a plurality of computers.

In the first embodiment, the image processing apparatus100is provided with an image acquisition unit111, a parameter acquisition unit112, and an adjustment processing unit113. The image acquisition unit111, the parameter acquisition unit112, and the adjustment processing unit113are configured as, for example, functional blocks of the processor110. That is, the processor110of the computer functions as the image acquisition unit111, the parameter acquisition unit112, and the adjustment processing unit113by executing the programs121stored in the storage unit120.

The image acquisition unit111is configured to acquire the radiographic image1. The image acquisition unit111generates the radiographic image1, based on, for example, the detection signal acquired from the X-ray detection unit213. The image acquisition unit111can acquire the radiographic image1by reading out the radiographic image1stored in the storage unit120from the storage unit120or by receiving the radiographic image1from an external image server250via a network. The image acquisition unit111outputs the acquired radiographic image1to the parameter acquisition unit112and the adjustment processing unit113.

The parameter acquisition unit112is configured to acquire the brightness adjustment parameter2with respect to the radiographic image1using the trained model10stored in the storage unit120. That is, the parameter acquisition unit112acquires the brightness adjustment parameter2including the window level WL and the window width WW by inputting the radiographic image1acquired from the image acquisition unit111to the trained model10. The parameter acquisition unit112outputs the acquired brightness adjustment parameter2to the adjustment processing unit113.

The adjustment processing unit113is configured to acquire the brightness-adjusted image3by adjusting the brightness of the radiographic image1based on the brightness adjustment parameter2. The adjustment processing unit113performs the window processing based on the brightness adjustment parameter2(the window level WL, the window width WW) acquired from the parameter acquisition unit112to acquire the brightness-adjusted image3. The adjustment processing unit113outputs the acquired brightness-adjusted image3to the display unit240and makes the storage unit120store it. Further, the adjustment processing unit113receives an input operation from the operator via the operation unit230. The adjustment processing unit113can receive a direct input of the brightness adjustment parameter2by the operator through the operation unit230. When the input of the brightness adjustment parameter2is received, the adjustment processing unit113performs the brightness adjustment based on the input brightness adjustment parameter2.

The storage unit120stores the trained model10by machine learning in which the radiographic image1reflecting a subject is input and the brightness adjustment parameter2of the radiographic image1is output. That is, as shown inFIG.1, the storage unit120stores the trained model10generated in advance by the learning for performing the optimization of the loss function A related to the brightness adjustment parameter2.

The storage unit120also stores the program121to be executed by the processor110and the radiographic image1. Further, the storage unit120stores the acquired brightness-adjusted image3.

Instead of constructing the image acquisition unit111, the parameter acquisition unit112, and the adjustment processing unit113of the image processing apparatus100realized by executing software by a processor as functional blocks, hardware dedicated for performing the respective processing may be constructed.

Referring toFIG.10toFIG.12, the brightness adjustment processing according to the image processing apparatus100will be described. The brightness adjustment processing shown inFIG.10toFIG.12performs the brightness adjustment method of the first embodiment by the image processing apparatus100.

In Step S11ofFIG.10, for example, by performing imaging by the imaging unit210, the image acquisition unit111acquires the radiographic image1. In Step S12, the parameter acquisition unit112performs the processing of acquiring the brightness adjustment parameter2of the radiographic image1by the trained model10stored in the storage unit120. In the first embodiment, the processing of acquiring the brightness adjustment parameter2includes Steps S21to S23inFIG.11.

That is, the brightness adjustment method of the first embodiment includes Step S21for acquiring the information5related to the imaging site or the imaging purpose of the subject, as shown inFIG.11.

Specifically, as shown inFIG.12, the parameter acquisition unit112receives the information input via the operation unit230to acquire the information5related to the imaging site or the imaging purpose of the subject.

The information of the imaging site is information indicating which part is imaged, such as, e.g., the front chest, the side, the front abdomen (lumbar part), the front, limb bones (e.g., the front of the knee joint, the side, the hand part), and the like. The information on the imaging purpose is information about the diagnosis of fractures, the diagnosis of a certain disease, or the like. For example, the optimal value of the brightness adjustment parameter2differs between the case of examining the condition of the pulmonary and the cardiac tissue (e.g., presence or absence of tumors, bronchial and vascularization) in the radiographic image of the front chest and examining the condition of the femur and other bone conditions (e.g., fractures, dislocations, bone deformities) in the radiographic image1in the lumbar region as shown inFIG.2.

Therefore, in the first embodiment, the storage unit120is storing in advance a plurality of trained models10corresponding to the information5about the imaging site or the imaging purpose. Each trained model10has been trained to output the optimal brightness adjustment parameter2with respect to the radiographic image1corresponding to the imaging site and the imaging purpose. For example, in the trained model10for the front chest, the brightness adjustment parameter2for the image of the front chest is trained by using the training data20for performing the optimal brightness adjustment for confirming the state of the pulmonary tissue and the cardiac tissue. The storage unit120stores the respective trained models10in association with the information5related to the imaging site or the imaging purpose.

Then, in Step S22ofFIG.11, the parameter acquisition unit112selects the trained model10used for the brightness adjustment from the plurality of trained models10, based on the information5related to the imaging site or the imaging purpose. As a result, the parameter acquisition unit112acquires the trained model10according to the imaging site or the imaging purpose.

In Step S23, the parameter acquisition unit112acquires the brightness adjustment parameter2of the radiographic image1according to the selected trained model10. Thus, in the brightness adjustment method of the first embodiment, in the step of acquiring the brightness adjustment parameter2, based on the information5related to the imaging site or the imaging purpose, the trained model10used for the brightness adjustment is selected from the plurality of trained models10. The brightness adjustment parameter2of the radiographic image1is acquired by the selected trained model10.

Returning toFIG.10, in Step S13, the adjustment processing unit113performs the brightness adjustment of the radiographic image1based on the acquired brightness adjustment parameter2. Thus, the adjustment processing unit113acquires the brightness-adjusted image3. In Step S14, the adjustment processing unit113makes the acquired brightness-adjusted image3store in the storage unit120and outputs it to the display unit240.

As described above, the brightness adjustment method of the first embodiment according to the image processing apparatus100is performed.

(Production Method of Trained Model)

Next, referring toFIG.13toFIG.15, the production method of the trained model according to the first embodiment will be described.

As shown inFIG.13, the production (machine learning) of the trained model10can be performed by a training data processing unit50provided separately from the image processing apparatus100. The training data processing unit50is configured by a computer (e.g., a personal computer, a workstation, or a supercomputer) in which a predetermined software (program) is installed in a storage unit or a computer system comprising a plurality of computers.

The production method of the trained model10is a method of producing a trained model by machine learning in which a radiographic image1reflecting a subject is input and the brightness adjustment parameter2of the radiographic image1is output. The training data processing unit50stores the training data20including the input image21and the training parameter22, the loss function A, and the training model11for learning in a storage unit and produces the trained model10by machine learning. The training data processing unit50stores a leaning data set of a plurality of training data20required for converging the machine learning.

As shown inFIG.14, in Step S31, the training data processing unit50acquires the training data20for performing learning from the training data set. In Step S32, the training data processing unit50receives the input image21included in the training data20and acquires the brightness adjustment parameter2of the input image21by the training model11.

In Step S33, the training data processing unit50acquires the value of the loss function A related to the brightness adjustment parameter2using the brightness adjustment parameter2by the training model11. In the first embodiment, the training data processing unit50acquires the output value of the loss function A according to the above Expression (1), based on the training parameter22(the window level WL0, the window width WW0) included in the training data20and the brightness adjustment parameter2(the window level WL, the window width WW) as the output result of the training model11.

In Step S34, the training data processing unit50optimizes the training model11so that the loss function A becomes smaller. The training data processing unit50learns the internal parameter of the training model11for outputting the brightness adjustment parameter2(the window level WL, the window width WW) that minimizes the output value of the loss function A. Machine learning for one training data20is performed by Steps S31to S34.

In Step S35, the training data processing unit50determines whether or not there is following (untrained) training data20in the training data set. When there is the following training data20, in Step S31, the following training data20is acquired and machine learning is performed. In Step S35, when all the learning of the training data20included in the training data set is completed (when there is no subsequent training data20), the processing proceeds to Step S36.

In Step S36, the training data processing unit50determines whether or not the learning of the training data20included in the training data set has been performed a predetermined number of times set in advance. When the number of times of learning has not reached the predetermined number of times, the training data processing unit50repeats the processing of Steps S31to S35until the number of times of learning has reached the predetermined number of times. When the number of learning times has reached the predetermined number, the machine learning is completed and the trained model10is generated. The training data processing unit50stores the generated trained model10in the storage unit in Step S37and terminates the processing.

In this way, the production method of the trained model10is carried out. The produced trained model10is transmitted to the image processing apparatus100via, e.g., a network and is stored in the storage unit120. In a case where as shown inFIG.12, a plurality of trained models10corresponding to the imaging site or the imaging purpose is produced, a data set for learning is prepared for each imaging site and imaging purpose, and Steps S31to S37shown inFIG.14are executed using the training data set for each imaging site and imaging purpose, respectively. Thus, a plurality of trained models10trained to output the optimal brightness adjustment parameter2to the radiographic image1corresponding to the imaging site or the imaging purpose is produced.

Note that the training data set may include a plurality of input images21generated by performing various image modification processing for one input image21. For example, as shown inFIG.15, image modification processing, such as, e.g., a rotation (A), an inversion (B), a deformation (C) such as an affine transformation, a change (D) in brightness and/or contrast, enlargement or reduction (not shown), etc., of the input image21are performed by randomly changing parameters. A training parameter22is set in each input image21, and each is included in the training data set as separate training data20. As a result, a large number of different input images21can be acquired from one input image21, and the amount of data for converging the learning can be easily expanded.

Further, in the first embodiment, the production method of the trained model10may be performed by the image processing apparatus100in shownFIG.13. As shown inFIG.9, the storage unit120of the image processing apparatus100stores the training data set of the training data20, and the processor110of the image processing apparatus100includes the training unit114as a functional block. The training unit114executes each processing of Steps S31to S37shown inFIG.14.

At this time, the training unit114may generate new training data20using the radiographic image1acquired from the imaging unit210as the input image21and perform additional learning of the trained model10. For example, in a case where an input (manual resetting) of the brightness adjustment parameter is performed by the operator via the operation unit230to the radiographic image1adjusted in the brightness using the brightness adjustment parameter2output by the parameter acquisition unit112, the training data20in which the input brightness adjustment parameter is the training parameter22may be generated using the radiographic image1as the input image21. In such a case, it is considered such that the result of the automatic brightness adjustment by the image processing apparatus100deviated from the optimal value, and therefore, the optimal adjustment parameter was manually set by the operator. Therefore, further optimization of the brightness adjustment processing can be achieved by performing additional learning of the trained model10using the training data20in which the manually input parameter is set as the training parameter22.

Note that the additional learning of the trained model10may be performed by the training data processing unit50connected to the image processing apparatus100via a network. In this instance, the image processing apparatus100may transmit the training data20generated from the radiographic image1to the training data processing unit50and may perform additional learning of the trained model10in the training data processing unit50.

Effects of First Embodiment

In this first embodiment, the following effects can be obtained.

In the production method of the trained model10, the brightness adjustment method, and the image processing apparatus100according to the first embodiment, as described above, in a case where the contrast of the predetermined area PA in the brightness-adjusted image3becomes high with respect to the training data20, the value in which the loss function A is relatively large is output. As a result, in the trained model10machine-trained by the loss function A, the predictive distribution PD of the brightness adjustment parameter2to be output to the radiographic image1can be biased in a direction of decreasing the contrast with respect to the optimal value WW0. Therefore, even by the trained model10trained to output the optimal value WW0, it is possible to suppress the output of the brightness adjustment parameter2so as to have a large contrast exceeding the optimal value WW0. Therefore, it is possible to suppress the occurrence of halation or black defects (saturation of a pixel value) of the predetermined area PA in the brightness-adjusted image3. As described above, in the brightness adjustment using the trained model10by machine learning, it is possible to suppress the occurrence of halation or black defects in the area (predetermined area PA) related to a diagnosis of a subject or the like in the image.

Note that in order to bias the predictive distribution PD of the brightness adjustment parameter2in the direction of decreasing the contrast with respect to the optimal value WW0, for example, it is conceivable to bias the training parameter22in the training data20in a direction of decreasing the contrast with respect to the optimal value WW0. In this situation, however, it is trained such that the peak of the predictive distribution PD deviates from the optimal value WW0and the brightness adjustment parameter2simply decreased in the contrast than the optimal value WW0is output. On the other hand, in the first embodiment, the error is biased with the training parameter22set as the optimal value WW0. Therefore, the minimum value of the loss function A remains as the optimal value WW0, and it is possible to perform the learning such that the peak of the predictive distribution PD matches the optimal value WW0as much as possible.

Further, in the first embodiment, as described above, it is configured such that in a case where halation or black defects occur due to the saturation of the pixel value in the predetermined area PA in the brightness-adjusted image3, the loss function A outputs a relatively large value than in a case where no saturation of the pixel value occurs in the predetermined area PA. With this, in a case where halation or black defects occur in the predetermined area PA, the value of the loss function A becomes large. Therefore, it is possible to acquire the trained model10that has been trained in a manner biased to suppress the occurrence of halation or black defects in the predetermined area PA.

Further, in the first embodiment, as described above, in a case where the contrast of the brightness-adjusted image3becomes high with respect to the training data20than in a case where the contrast of the brightness-adjusted image3becomes low with respect to the training data20, the loss function A includes a largely weighted weighting function f(r). This allows the value of the loss function A to be easily biased by the weighting function.

In the first embodiment, as described above, the loss function A is a function based on the error between the training parameter22included in the training data20and the brightness adjustment parameter2with respect to the input image21included in the training data20. With this, even by the loss function A based on the error between the training parameter22and the brightness adjustment parameter2, it is possible to suppress that the error is evaluated symmetrically with respect to the optimal value WW0and relatively increase the value of the loss function A in a case where the contrast becomes high with respect to the optimal value WW0. Therefore, it is possible to effectively suppress the occurrence of the saturation of the pixel value in the predetermined area PA.

Further, in the first embodiment, as described above, the predetermined area PA includes at least a part of the area reflecting the subject. With this, by using the brightness adjustment parameter2output by the trained model10, it is possible to assuredly suppress the occurrence of halation or black defects in the area reflecting the subject.

Further, in the first embodiment, as described above, the brightness adjustment parameter2includes the window level WL that defines the median of the pixel value range to be converted to the brightness-adjusted image3in the radiographic image1and the window width WW that defines the size of the pixel value range. As a result, the magnitude of the contrast of the pixel value in the predetermined area PA can be adjusted by the value of the window width WW. Therefore, it is possible to easily evaluate the contrast by the loss function A, as compared with the case where a large number of parameters affect the contrast.

Further, in the first embodiment, as described above, it is further provided with a step of acquiring the information5related to the imaging site or the imaging purpose of the subject. In the step of acquiring the brightness adjustment parameter2, based on the information5related to the imaging site or the imaging purpose, the learned model10for adjusting the brightness among from the plurality of the trained model10is selected. As described above, by preparing a plurality of trained models10separately trained according to the imaging site or the imaging purpose and selecting the trained model10based on the information5related to the imaging site or the imaging purpose, the optimum brightness adjustment according to the imaging site or the imaging purpose can be performed.

Second Embodiment

Next, referring toFIG.16toFIG.18, the production method of the trained model and the brightness adjustment method according to a second embodiment will be described. In the second embodiment, a loss function A will be described with reference to an example about a function based on the comparison between the training data20and the brightness-adjusted image3unlike a function based on the error between the training parameter22and the brightness adjustment parameter2in the above-described first embodiment. In the second embodiment, the same components as those of the first embodiment are denoted by the same reference numerals, and the description thereof are omitted.

As shown inFIG.16, in the second embodiment, the trained model10is generated by learning of performing the optimization of the loss function B related to the brightness adjustment parameter2. The loss function B is a function based on the comparison between the pixel value of the training image23included in the training data20and the pixel value of the brightness-adjusted image3in which the input image21included in the training data20is adjusted in the brightness based on the brightness adjustment parameter2.

Specifically, in the second embodiment, in the machine learning, the brightness-adjusted image3is acquired based on the brightness adjustment parameter2output by the training model11which is being trained by inputting the input image21included in the training data20. The training image23adjusted in the brightness is acquired based on the input image21included in the training data20and the training parameter22. The brightness-adjusted image3and the training image23are images respectively acquired by adjusting the brightness with respect to the same input image21included in the training data20. The training image23is a ground truth image of the machine learning adjusted in the brightness based on the optimal value of the brightness adjustment parameter.

Then, the loss function B is configured as a function based on the comparison between the pixel value of the training image23and the pixel value of the brightness-adjusted image3in the same pixel of both images. With this, in the second embodiment, based on the resulting image actually adjusted in the brightness by the brightness adjustment parameter2, the difference between the output result of the training model11and the training data20is evaluated.

For example, the loss function B is a function based on the difference of the pixel value of the corresponding pixel between the brightness-adjusted image3and the training image23. Specifically, as shown in the following Expression (3), the loss function B is a mean square error acquired by integrating the weight to the difference in the pixel value of the corresponding pixel between the brightness-adjusted image3and the training image23.

where x and y are coordinates of the respective images (the brightness-adjusted image3, the training image23), respectively. k (x, y) is the weight at each coordinate (x, y). I (x,y) is a pixel value at the coordinate (x, y) of the brightness-adjusted image3, and Jo (x, y) is a pixel value at the coordinate (x, y) of the training image23. f(r) is a weighting function. w is the value of the weight (w>1). Imaxis the maximum value a pixel value (=255).

Similar to the first embodiment, in a case where the contrast of the brightness-adjusted image3becomes high with respect to the training data20(I (x, y)=0, or Imax) than in a case where the contrast of the brightness-adjusted image3becomes high with respect to the training data20(I (x, y)=0, or other than Imax), the loss function B includes a heavily weighted weighting function

In other words, in a case where the window width WW becomes smaller than the optimal value WW0(when the contrast becomes high), a pixel in which the pixel value I is saturated occurs in the predetermined area PA. In other words, in the pixel in which the optimal pixel value I0becomes 0<I0<Imax, the pixel value I (x, y) becomes the lowest value 0 (black defects) or the highest value Imax(halation). In this case, the loss function B estimates the error relatively large by integrating the weight w to the error (I−I0). As a result, in a case where the contrast of the predetermined area PA of the brightness-adjusted image3becomes high with respect to the training data20(when I (x, y)=0 or Imax), the loss function B of the second embodiment outputs a relatively large value to bias the learning in a direction of decreasing the contrast. In a case where the window width WW becomes larger than the optimal value WW0(when the contrast becomes low), it is considered that no pixel in which the pixel value I is saturated in the predetermined area PA occurs. Therefore, the loss function B estimates the error relatively small without the accumulation of the weight w.

Furthermore, in the second embodiment, the loss function B may include a weight k (x, y) set for each pixel (position coordinate) in the image. Therefore, in the second embodiment, different weights k can be set for each area in the image. For example, in the second embodiment, as shown inFIG.17, in a case where saturation of the pixel value in the predetermined area PA in the image occurs by the weight k, the loss function B is configured to output a value larger than in a case where the saturation of the pixel value occurs in the area PB other than the predetermined area PA. Thus, in a case where the contrast of the brightness-adjusted image3becomes high with respect to the training data20(training mage23) (when saturation of the pixel value occurs) in the predetermined area PA, the value of the loss function B is estimated relatively large, and the learning is performed strongly in the direction of suppressing the occurrence of the saturation of the pixel value by suppressing the contrast. On the other hand, in the area PB other than the predetermined area PA, in a case where the contrast of the brightness-adjusted image3becomes high with respect to the training data20(training image23), the value of the loss function B is estimated relatively small, and it is trained so as not to suppress the contrast unnecessarily.

As described above, in order to set the weight k for each pixel in the image (position coordinate), in the training data20of the second embodiment, as shown inFIG.16, the position information24ain the predetermined area PA and the bias information24bof the loss function B are included. The position information24ais the information of the position coordinate in the image. The bias information24bis a set value of the weight k.

Specifically, the training data20includes a weight map24including the position information24ain the input image21and the bias information24b. The weight map24includes, as the position information24a, the area to which the common weight k is applied in the input image21. The weight map24includes, as the bias information24b, the values of the weight k applied to the area partitioned as the position information24ain the input image21.

For example, as shown in (B) ofFIG.17, the position information24acan be set so as to partition the input image21into the area61reflecting the subject SB and the background area62. In this case, the bias information24bincludes the weight k different for the area61reflecting the subject SB and the background area62, in response to the position information24a. In the example of (B) ofFIG.17, the area reflecting the subject SB is the predetermined area PA. The weight k of the area61reflecting the subject SB is set to be higher than the weight of the background area62. The area61may be set to a weight k=10, and the background area62may be set to a non-weight (k=1).

The position information24amay be set to further segment the interior of the area61reflecting the subject SB, for example, as shown in (C) ofFIG.17. That is, among the area61reflecting the subject SB, it is possible to distinguish the area having a high diagnosis importance from the area having a relatively low diagnosis importance. In addition, the ease of saturation of a pixel value may differ for each site of the subject depending on the morphological condition, such as, e.g., the anatomy of the subject. In this instance, the position information24amay be set by dividing the inside of the area61reflecting the subject SB into a plurality of partial areas, and the bias information24b(weight k) may be set for each partial area. In the example shown in (C) ofFIG.17, an example is shown in which the same or different weights k are respectively set to the respective partial areas of the femoral neck61a, the greater trochanter61b, the diaphyseal portion61c, and the ward triangle61din accordance with the anatomy of the lumbar portion of the subject.

By the weight map24, for the pixel p (x, y) at any position coordinate of the input image21, the weight k (x, y) set in the area to which the pixel p belongs is acquired. As a result, in the loss function B, the magnitude of the bias is set for each position in the input image21, based on the position information24ain the predetermined area PA included in the training data20and the bias information24bof the loss function B.

Consequently, in a case where saturation of a pixel value occurs in the predetermined area PA in the image, the loss function B is configured to output a value larger than in a case where saturation of a pixel value occurs in an area other than the predetermined area PA.

The effect of the learning bias by the loss function B in the second embodiment on the brightness adjustment parameter2output by the trained model10is described.

FIG.18is a graph showing the changes in the output value of the loss function B and the predictive distribution PD of the window width WW output by the trained model10optimized using the loss function B, with respect to the window width WW.

As Comparative Example, a case in which without performing weighting, a loss function RMSE by the mean square error of the pixel value of the corresponding pixel of the brightness-adjusted image3and the training image23is shown with dashed lines. The loss function RMSE of Comparative Example is represented by the following Expression (4).
RMSE=(√{square root over (ΣyΣx(I(x,y)−I0(x,y))2)}  (4)

In the above Expression (4), in the loss function RMSE according to Comparative Example, when halation or black defects (saturation of a pixel value) occur in the brightness-adjusted image, it is underestimated by the amount that the pixel value of the area is saturated. That is, as shown in (A) ofFIG.18, the output value of the loss function in a case where the window width WW is smaller than the optimal value WW0and saturation of the pixel values is occurring becomes relatively smaller than the output value of the loss function in a case where the window width WW is larger than the optimal value WW0and no saturation of the pixel value is occurring. Therefore, as shown in (B) ofFIG.18, it is considered that the predictive distribution CD of the output result of the trained model output optimized by using the loss function RMSE becomes asymmetrically distributed in the direction in which halation or black defects (saturation of pixel value) occur with respect to the optimal value WW0(in a direction of decreasing the window width WW). In the trained model optimized using the loss function RMSE according to Comparative Example, there is a possibility that saturation of a pixel value occurs in the predetermined area PA in more than half of the window width WW to be output.

On the other hand, in the loss function B shown by the solid line inFIG.18, when halation or black defects (saturation of a pixel value) occur in the brightness-adjusted image (I=0 or Imax), the weight w is estimated relatively large by integrating the weight W to the error of the pixel value in the area. In a case where no halation or black defects (saturation of pixel value) is occurring (0<I<Imax), the weight w is not accumulated, and the error of the pixel value of the area is estimated to be relatively small. Note that for the pixel for which both the optimal pixel value I0and the pixel value I are 0 or Imax, the weight w is integrated in the calculation, but since the error becomes 0, it does not affect the loss function B. Therefore, the predictive distribution PD of the output result of the trained model10optimized using the loss function B of the first embodiment becomes a distribution biased such that the predictive distribution PD is smaller in the direction in which the pixel value is likely to saturate and the contrast becomes high (the direction in which the window width WW becomes smaller) and larger in the direction in which the contrast becomes low (the direction in which the window width WW becomes larger) with respect to the optimal value WW0.

As a result, as can be seen from the predictive distribution PD, in the trained model10optimized using the loss function B of the first embodiment, it is suppressed that the window width WW to be output varies in the direction of increasing the contrast exceeding the optimal value WW0(in the direction of decreasing the window width WW to be output). Thus, the possibility of occurrence of saturation of the pixel value in the predetermined area PA is suppressed.

The rest of the configuration of the second embodiment is the same as that of the first embodiment. In the second embodiment, the production method of the trained model10, the brightness adjustment method, and the image processing apparatus100are the same except that the loss function B differs from the first embodiment, and therefore the descriptions thereof are omitted.

Effects of Second Embodiment

In this second embodiment, the following effects can be acquired.

In the second embodiment, in the same manner as in the first embodiment, it is possible to suppress the occurrence of halation or black defects in the area (predetermined area PA) related to a diagnose of a subject or the like in the image in the brightness adjustment using the trained model10by machine learning.

Further, in the second embodiment, as described above, the loss function B is a function based on the comparison between the pixel value of the training image23included in the training data20and the pixel value of the brightness-adjusted image3in which the input image21included in the training data20is adjusted in the brightness based on the brightness adjustment parameter2. With this, even by the loss function B based on the comparison of the pixel value between the training image23and the brightness-adjusted image3, it is possible to relatively increase the value of the loss function B in a case where the contrast becomes high with respect to the optimal value and suppress the underestimation of the error due to the saturation of the pixel. Therefore, it is possible to effectively suppress the occurrence of the saturation of the pixel value in the predetermined area PA.

Further, in the second embodiment, as described above, the loss function B is configured such that, when saturation of the pixel value occurs in the predetermined area PA in the image, a value larger in the case where saturation of the pixel value occurs in an area other than the predetermined area PA is output. With this, it is possible to suppress the learning from being affected by the error in the area PB with less importance other than the predetermined area PA while suppressing the occurrence of saturation of a pixel value saturation in the predetermined area PA.

In the second embodiment, as described above, in the loss function B, the magnitude of the bias is set for each position in the input image21, based on the position information24ain the predetermined area PA included in the training data20and the bias information24bof the loss function B. This makes it possible to distinguish between the predetermined area PA and the area PB other than the predetermined area PA based on the position information24ain the predetermined area PA and specify the magnitude relation of the bias of the loss function B between the predetermined area PA and the area PB other than the predetermined area PA based on the bias information24bof the loss function B. Therefore, the magnitude (weight k) of the bias of the loss function B can be easily changed according to the area in the image.

The other effects of the second embodiment are the same as those of the first embodiment.

It should be understood that the embodiments disclosed here are examples in all respects and are not restrictive. The scope of the present invention is shown in the claims rather than by the above-described embodiments, and the scope of the present invention includes all modifications (modified examples) within the meanings and ranges equivalent to the claims.

For example, in the above-described first and second embodiments, as the example of the loss function, the loss function A and the loss function B according to the mean square error are shown, but the present invention is not limited thereto. In the present invention, the loss function may be configured by a function other than the mean square error. For example, as the loss function B of the second embodiment, a function, such as, e.g., a PSNR (Peak signal-to-noise ratio) or an SSIM (Structural SIMilarity) may be used. However, for example, the SSIM can evaluate the similarity between images, but it is an index that also evaluates the structural element unrelated to the brightness adjustment, such as, e.g., the misalignment of images. Therefore, the mean square error that simply evaluates only the pixel value (brightness) between images is more suitable as the loss function for use in the training model for the brightness adjustment.

Further, in the first and second embodiments described above, an example is shown in which a large weight w (>1) is applied when the contrast in the predetermined area PA becomes high with respect to the training data20, but the present invention is not limited thereto. It may be configured such that a smaller weight w (<1) is applied when the contrast in the predetermined area PA becomes low with respect to the training data20so that the output value when the contrast in the predetermined area PA becomes larger is set to be relatively larger.

Further, in the first and second embodiments, an example is shown in which the predetermined area PA includes at least a part of the area reflecting a subject, but the present invention is not limited thereto. The predetermined area PA may be set according to the application of the radiographic image1and may be an area other than an area reflecting a subject.

Further, in the first and second embodiments, an example is shown in which the output value of the loss function is biased by including the weighting function in the loss function, but the present invention is not limited to thereto. For example, the output of the loss function may be biased by applying different loss functions when the contrast in the predetermined area PA becomes high with respect to the training data20and when the contrast of the predetermined area PA becomes low with respect to the training data20.

Further, in the second embodiment described above, an example is shown in which the weight k is differentiated between the predetermined area PA and an area PA other than the predetermined area PA using the weight map24, but the present invention is not limited thereto. In the present invention, the weight k may not be set differentiated for each area. In the loss function B, the weight k may not be included. In this case, the training data20is not required to include the weight map24.

Further, in the second embodiment, an example is shown in which the training image23is acquired based on the training parameter22included in the training data20, but the present invention is not limited thereto. In the present invention, the training data20may necessarily include the training image23. In this instance, the training data20may not necessarily be required to include the training parameter22.

Further, in the first and second embodiments, an example is shown in which the trained model10to be used for the brightness adjustment is selected from a plurality of trained models10by acquiring the information5related to the imaging site or the imaging purpose, but the present invention is not limited thereto. In the present invention, the information5about the imaging site or the imaging purpose is not always required to be acquired. That is, it is not necessary to select the trained model10.

Note that the acquisition of the information5about the imaging site or the imaging purpose is not limited to the method using the operation unit230, and the information5about the imaging site or the imaging purpose may be acquired by recognizing, e.g., the image of the input radiographic image1.

Further, in the first and second embodiments, an example is shown in which the brightness adjustment parameter2includes the window level WL and the window width WW, but the present invention is not limited thereto. For example, instead of the window width WW, the brightness adjustment parameter2may include the upper limit VH and the lower limit VL of the window width.

DESCRIPTION OF SYMBOLS

1: Radiographic image2: Brightness adjustment parameter3: Brightness-adjusted image5: Information related to imaging site or imaging purpose10: Trained model11: Training model20: Training data21: Input image22: Training parameter23: Training image24a: Position information24b: Bias information100: Image processing apparatus111: Image acquisition unit112: Parameter acquisition unit113: Adjustment processing unit120: Storage unitA, B: Loss functionPA: Predetermined areaPB: Area other than predetermined areaWL: Window levelWW: Window widthf(r): Weighting function