Patent Description:
In general, a technique in which a doctor interprets a radiographic image captured by a mammography apparatus to perform diagnosis of a lesion or the like of the breast is known. As this type of radiographic images, a plurality of projection images obtained by the so-called tomosynthesis imaging, and a tomographic image generated by performing reconstruction processing on a plurality of projection images are known. As the main radiographic image, a composite two-dimensional image generated from a plurality of images selected from among a plurality of projection images and a plurality of tomographic images is known.

In the image interpretation, diagnosis is performed by mutually referring to a tomographic image and a two-dimensional radiographic image such as a composite two-dimensional image, in some cases. For example, <CIT> discloses a technique of creating a depth map in which each position on a composite two-dimensional image and depth information indicating a position in a depth direction of a tomographic plane corresponding to each position are associated, and extracting and displaying a tomographic image corresponding to the depth of a specific position in a case where the specific position is designated on the composite two-dimensional image. <CIT> discloses displaying computer-aided detection information with associated breast tomosynthesis image information.

In the technique disclosed in <CIT>, in case of image interpretation, a plurality of tomographic images that can be referred to by an interpreter are required. In general, since tomographic images contain a large amount of information in the image and the number of images is relatively large, a large storage capacity is required in order to store a plurality of tomographic images which are prepared to be referable, in some cases. Further, for example, in a case where such a plurality of tomographic images are generated in real time according to the image interpretation of the interpreter, the processing time becomes increased, or a processing load of an image processing apparatus that generates tomographic images becomes increased in some cases.

In this manner, in the technique in the related art, in a case where diagnosis is performed by mutually referring to the composite two-dimensional image and the tomographic image, the load associated with the handling of tomographic images is large in some cases.

The present disclosure is made in view of the above-described circumstances, and an object of the present disclosure is to provide an image processing apparatus, an image processing method, and an image processing program which can reduce a load associated with the handling of tomographic images.

According to an aspect of the present invention, there is provided an image processing system, as claimed in claim <NUM>.

In an image processing system according to a second aspect of the present disclosure, in the image processing system according to the first aspect, the designated tomographic image is a tomographic image of a portion corresponding to the designated region, in the tomographic plane at the depth specified according to a position of the designated region.

In an image processing system according to a third aspect of the present disclosure, in the image processing system according to the first aspect, the designated tomographic image is a tomographic image including an image of the entire breast, in the tomographic plane at the depth specified according to a position of the designated region.

In an image processing system according to a fourth aspect of the present disclosure, in the image processing system according to the first aspect, the designated tomographic image is a tomographic image of a portion corresponding to a region in a predetermined range including the designated region, in the tomographic plane at the depth specified according to a position of the designated region.

In an image processing system according to a fifth aspect of the present disclosure, in the image processing system according to any one of the first to fourth aspects, the designated tomographic image generation unit further generates tomographic images in tomographic planes at predetermined depths before and after the depth of the designated tomographic image, and the display controller further performs control of causing the display device to display the generated tomographic images in the tomographic planes at the depths before and after the depth of the designated tomographic image.

In an image processing system according to a sixth aspect of the present disclosure, in the image processing system according to any one of the first to fifth aspects, the display controller performs control of causing the display device to display the composite two-dimensional image and the designated tomographic image side by side.

In an image processing system according to a seventh aspect of the present disclosure, in the image processing system according to any one of the first to fifth aspects, the display controller performs control of causing the display device to display the composite two-dimensional image and the designated tomographic image in a state in which the composite two-dimensional image and the designated tomographic image are at least partially superimposed on each other.

The image processing system according to the present disclosure further comprises a storage controller that performs control of deleting the plurality of tomographic images from a storage unit, in a case where the plurality of tomographic images are stored in the storage unit after the composite two-dimensional image and the correspondence relationship information are generated.

An image processing system according to a eighth aspect of the present disclosure, in the image processing system according to any one of the first to seventh aspects, further comprises an image processing apparatus that includes a transmission unit that transmits the composite two-dimensional image, the correspondence relationship information, and the designated tomographic image to the display device, the acquisition unit, the tomographic image generation unit, the composite two-dimensional image generation unit, the information generation unit, and the designated tomographic image generation unit; and the display device that includes a reception unit that receives the composite two-dimensional image, the correspondence relationship information, and the designated tomographic image from the image processing apparatus, the display controller, and the acceptance unit.

An image processing system according to a ninth aspect of the present disclosure, in the image processing system according to any one of the first to seventh aspects, further comprises an image processing apparatus that includes a transmission unit that transmits the plurality of projection images, the composite two-dimensional image, and the correspondence relationship information, to the display device, the acquisition unit, the tomographic image generation unit, the composite two-dimensional image generation unit, and the information generation unit; and the display device that includes a reception unit that receives the plurality of projection images, the composite two-dimensional image, and the correspondence relationship information from the image processing apparatus, the designated tomographic image generation unit, the display controller, and the acceptance unit.

According to a further aspect of the present invention, there is provided an image processing method as claimed in claim <NUM>.

According to a further aspect of the present invention, there is provided an image processing program as claimed in claim <NUM>.

Further, an image processing system according to an aspect of the present disclosure is an image processing system having a processor, and the processor acquires a plurality of projection images obtained by tomosynthesis imaging in which radiation is emitted from a radiation source to a breast at different irradiation angles and a projection image is captured at each irradiation angle by a radiation detector; generates a plurality of tomographic images in each of a plurality of tomographic planes of the breast, from the plurality of projection images; generates a composite two-dimensional image from a plurality of images selected from among the plurality of projection images and the plurality of tomographic images; generates correspondence relationship information representing a correspondence relationship between a position in the composite two-dimensional image and a depth of a tomographic plane corresponding to the position; performs control of causing a display device to display the composite two-dimensional image; accepts region information representing a designated region designated with respect to the composite two-dimensional image displayed on the display device; generates, as a designated tomographic image, a tomographic image in a tomographic plane at a depth which corresponds to the designated region in the composite two-dimensional image and is specified on the basis of the correspondence relationship information, in a case where the region information is accepted; performs control of causing the display device to display the generated designated tomographic image, in a case where the designated tomographic image is gerenerated, and further performs control of deleting the plurality of tomographic images from a storage unit after the composite two-dimensional image and the correspondence relationship information are generated.

According to the present disclosure, it is possible to reduce a load associated with the handling of tomographic images.

The embodiments do not limit the present disclosure.

First, an example of the overall configuration of a medical system comprising an image processing system of the embodiment will be described. <FIG> is a configuration diagram illustrating an example of the overall configuration of a medical system <NUM> of the embodiment.

As illustrated in <FIG>, the medical system <NUM> of the embodiment comprises a radiography system <NUM> and an image processing system <NUM>.

The radiography system <NUM> includes a mammography apparatus <NUM> and a console <NUM>.

The console <NUM> of the embodiment has a function of controlling the mammography apparatus <NUM> using, for example, an imaging order and various kinds of information acquired from a radiology information system (RIS) (not illustrated) through a wireless communication local area network (LAN) and an instruction or the like directly performed by a technician or the like. As an example, in the embodiment, a server computer is used as the console <NUM>.

On the other hand, the mammography apparatus <NUM> of the embodiment comprises a radiation source and a radiation detector (which are not illustrated). The mammography apparatus <NUM> irradiates the breast of a subject as an object with radiation (for example, X-rays) using the radiation source to capture a radiographic image of the breast using the radiation detector under the control of the console <NUM>. In addition, the mammography apparatus <NUM> may be an apparatus that images the breast of the subject not only in a state in which the subject is standing (standing position state) but also in a state in which the subject sits on a chair (including a wheelchair) or the like (sitting position state). In the embodiment, a plurality of types of radiographic images, such as a projection image, a tomographic image, and a composite two-dimensional image will be referred to, but in case of collectively referring to the images without distinguishing the types thereof, the plurality of types of radiographic images are simply referred to as a "radiographic image".

In the mammography apparatus <NUM> of the embodiment, a plurality of types of imaging can be performed for capturing a radiographic image. Specifically, the mammography apparatus <NUM> can perform, on the breast, two types of so-called simple imaging of cranio-caudal (CC) imaging where the imaging direction is a craniocaudal direction, and medio-lateral oblique (MLO) imaging where the imaging direction is a mediolateral oblique direction.

Further, the mammography apparatus <NUM> of the embodiment can perform tomosynthesis imaging in which radiation is emitted from the radiation source to the breast at different irradiation angles and a projection image is captured at each irradiation angle by the radiation detector.

Meanwhile, the image processing system <NUM> of the embodiment includes an image preservation system <NUM>, an image processing apparatus <NUM>, and an image interpretation apparatus <NUM>.

First, the configuration of the image preservation system <NUM> will be described.

The image preservation system <NUM> of the embodiment is a system that preserves image data of the radiographic image captured by the radiography system <NUM> and image data of the radiographic image generated by the image processing apparatus <NUM>. The image preservation system <NUM> extracts an image corresponding to a request from, for example, the console <NUM>, the image processing apparatus <NUM>, and the image interpretation apparatus <NUM> from the preserved radiographic images, and transmits the extracted image to the apparatus which is a request source. A specific example of the image preservation system <NUM> is picture archiving and communication systems (PACS).

<FIG> is a block diagram illustrating an example of the image preservation system <NUM> of the embodiment. As illustrated in <FIG>, the image preservation system <NUM> of the embodiment comprises a controller <NUM>, a storage unit <NUM>, and an interface (I/F) unit <NUM>. The controller <NUM>, the storage unit <NUM>, and the I/F unit <NUM> are connected to each other through a bus <NUM>, such as a system bus or a control bus, so as to be able to transmit and receive various kinds of information.

The controller <NUM> of the embodiment controls the overall operation of the image preservation system <NUM>. The controller <NUM> comprises a central processing unit (CPU) 20A, a read only memory (ROM) 20B, and a random access memory (RAM) 20C. Various programs executed by the CPU 20A are stored in the ROM 20B in advance. The RAM 20C temporarily stores various kinds of data.

The storage unit <NUM> is a so-called database that stores the image data of the radiographic image in association with various kinds of information such as an imaging order or information relating to the subject. Specific examples of the storage unit <NUM> include a hard disk drive (HDD) and a solid state drive (SSD). The I/F unit <NUM> has a function of performing communication of various kinds of information with the console <NUM>, the image processing apparatus <NUM>, and the image interpretation apparatus <NUM> using wireless communication or wired communication.

Next, the configuration of the image processing apparatus <NUM> will be described.

The image processing apparatus <NUM> of the embodiment is an apparatus that performs desired image processing on the radiographic image. Specific examples of the image processing apparatus <NUM> include an image processing workstation.

<FIG> is a block diagram illustrating an example of the image processing apparatus <NUM> of the embodiment. As illustrated in <FIG>, the image processing apparatus <NUM> of the embodiment comprises a controller <NUM>, a storage unit <NUM>, a display unit <NUM>, an operation unit <NUM>, and an I/F unit <NUM>. The controller <NUM>, the storage unit <NUM>, the display unit <NUM>, the operation unit <NUM>, and the I/F unit <NUM> are connected to each other through a bus <NUM>, such as a system bus or a control bus, so as to be able to transmit and receive various kinds of information.

The controller <NUM> of the embodiment controls the overall operation of the image processing apparatus <NUM>. The controller <NUM> comprises a CPU 30A, a ROM 30B, and a RAM 30C. Various programs including an image generation processing program <NUM> executed by the CPU 30A are stored in the ROM 30B in advance. The RAM 30C temporarily stores various kinds of data.

The image data of the radiographic image and various other kinds of information are stored in the storage unit <NUM>. Specific examples of the storage unit <NUM> include an HDD, an SSD, and the like. The operation unit <NUM> is used by the user to input an instruction or the like relating to image processing that is performed on the radiographic image. The operation unit <NUM> is not particularly limited, and for example, various switches, a touch panel, a touch pen, a mouse, and the like are exemplified. The display unit <NUM> displays various kinds of information including the radiographic image. In addition, the display unit <NUM> and the operation unit <NUM> may be integrated into a touch panel display. The I/F unit <NUM> performs communication of various kinds of information including the image data of the radiographic image, with the console <NUM>, the image preservation system <NUM>, and the image interpretation apparatus <NUM> using wireless communication or wired communication.

Next, the configuration of the image interpretation apparatus <NUM> will be described.

The image interpretation apparatus <NUM> of the embodiment is an apparatus for the interpreter such as a doctor to interpret the radiographic image captured by the radiography system <NUM>. Specific examples of the image interpretation apparatus <NUM> include a so-called viewer.

<FIG> is a block diagram illustrating an example of the image interpretation apparatus <NUM> of the embodiment. As illustrated in <FIG>, the image interpretation apparatus <NUM> of the embodiment comprises a controller <NUM>, a storage unit <NUM>, a display unit <NUM>, an operation unit <NUM>, and an I/F unit <NUM>. The controller <NUM>, the storage unit <NUM>, the display unit <NUM>, the operation unit <NUM>, and the I/F unit <NUM> are connected to each other through a bus <NUM>, such as a system bus or a control bus, so as to be able to transmit and receive various kinds of information.

The controller <NUM> of the embodiment controls the overall operation of the image interpretation apparatus <NUM>. The controller <NUM> comprises a CPU 40A, a ROM 40B, and a RAM 40C. Various programs including an image interpretation processing program <NUM> executed by the CPU 40A are stored in the ROM 40B in advance. The RAM 40C temporarily stores various kinds of data. The image interpretation processing program <NUM> and the image generation processing program <NUM> of the embodiment are examples of the image processing program of the present disclosure.

The image data of the radiographic image and various other kinds of information are stored in the storage unit <NUM>. Specific examples of the storage unit <NUM> include an HDD, an SSD, and the like. The operation unit <NUM> is used by the interpreter to input an instruction or the like relating to the interpretation of the radiographic image. The operation unit <NUM> is not particularly limited, and for example, various switches, a touch panel, a touch pen, a mouse, and the like are exemplified. The display unit <NUM> displays various kinds of information including the radiographic image. In addition, the display unit <NUM> and the operation unit <NUM> may be integrated into a touch panel display. The I/F unit <NUM> performs communication of various kinds of information including the image data of the radiographic image, with the image preservation system <NUM> and the image processing apparatus <NUM> using wireless communication or wired communication.

<FIG> is a functional block diagram illustrating an example of the configurations of the image processing apparatus <NUM> and the image interpretation apparatus <NUM> of the embodiment. As illustrated in <FIG>, the image processing apparatus <NUM> of the embodiment comprises an acquisition unit <NUM>, a tomographic image generation unit <NUM>, a designated tomographic image generation unit <NUM>, a composite two-dimensional image generation unit <NUM>, an information generation unit <NUM>, a storage controller <NUM>, a transmission unit <NUM>, and a reception unit <NUM>.

For example, in the image processing apparatus <NUM> of the embodiment, the CPU 30A of the controller <NUM> executes the image generation processing program <NUM> stored in the ROM 30B so that the controller <NUM> functions as each of the acquisition unit <NUM>, the tomographic image generation unit <NUM>, the designated tomographic image generation unit <NUM>, the composite two-dimensional image generation unit <NUM>, the information generation unit <NUM>, the storage controller <NUM>, the transmission unit <NUM>, and the reception unit <NUM>.

The acquisition unit <NUM> acquires a plurality of projection images captured by the mammography apparatus <NUM>, and outputs the plurality of acquired projection images to the tomographic image generation unit <NUM>, the designated tomographic image generation unit <NUM>, and the composite two-dimensional image generation unit <NUM>. The acquisition source from which the acquisition unit <NUM> acquires a plurality of projection images is not limited, and for example, the image preservation system <NUM> or the console <NUM> may be the acquisition source, or in a case where images are stored in the storage unit <NUM> in advance, the storage unit <NUM> may be the acquisition source.

The tomographic image generation unit <NUM> reconstructs all or some of the plurality of projection images input from the acquisition unit <NUM> to generate a tomographic image. The tomographic image generation unit <NUM> generates a plurality of tomographic images (hereinafter, referred to as a "tomographic image <NUM>") in each of a plurality of tomographic planes of the breast. The tomographic image <NUM> generated by the tomographic image generation unit <NUM> is temporarily stored in the storage unit <NUM>.

On the other hand, in a case where the reception unit <NUM> has received depth information, the designated tomographic image generation unit <NUM> generates a tomographic image in the tomographic plane of the breast specified by the depth information, as a designated tomographic image. The size or the like of the designated tomographic image generated by the designated tomographic image generation unit <NUM> is optional. For example, the designated tomographic image may be used as a tomographic image of a portion corresponding to a region, which is designated by the interpreter (hereinafter, referred to as a "designated region") in the image interpretation apparatus <NUM>, in the specified tomographic plane. In this case, the designated tomographic image generation unit <NUM> may not generate a tomographic image of a portion corresponding to a region other than the designated region in the specified tomographic plane. In other words, an aspect of generating a tomographic image of only a partial region of the tomographic planes of the entire breast may be adopted.

For example, the designated tomographic image may be used as a tomographic image of a portion corresponding to a region in a predetermined range including the designated region. In this case, in a case where the designated region is a region including a part of a region of interest, a tomographic image of a region including the designated region and the entire region of interest can be used as the designated tomographic image.

For example, the designated tomographic image may be a tomographic image having a size including the image of the entire breast, or may be a tomographic image in the same range (size) as the tomographic image <NUM> generated by the tomographic image generation unit <NUM>.

What kind of tomographic image, for example, which size tomographic image, is used as the designated tomographic image may be determined in advance, or can be set and changed by the interpreter.

In this manner, the tomographic image generation unit <NUM> generates a plurality of tomographic images respectively corresponding to all of a plurality of tomographic planes (depths) of the breast, and the designated tomographic image generation unit <NUM> generates a tomographic image in a specific tomographic plane (depth) as the designated tomographic image.

The method in which each of the tomographic image generation unit <NUM> and the designated tomographic image generation unit <NUM> generates a tomographic image from a plurality of projection images is not particularly limited, and for example, a known method such as a back projection method such as a filtered back projection (FBP) method and a simple back projection method, or a shift addition method can be used.

The composite two-dimensional image generation unit <NUM> generates a composite two-dimensional image from a plurality of radiographic images selected from among a plurality of projection images and a plurality of tomographic images. The composite two-dimensional image generated by the composite two-dimensional image generation unit <NUM> is output to the information generation unit <NUM>. A plurality of radiographic images used in the generation of the composite two-dimensional image by the composite two-dimensional image generation unit <NUM> are not particularly limited, and may be determined in advance or may be selected by the interpreter or the like.

The method in which the composite two-dimensional image generation unit <NUM> generates the composite two-dimensional image is not particularly limited, and a known method can be used. As an example, the composite two-dimensional image generation unit <NUM> of the embodiment uses a method disclosed in <CIT>. <CIT> discloses a technique of generating a composite two-dimensional image in which a lesion or the like detected from a tomographic image is reflected, by blending (composing) a region of interest (ROI) detected from the tomographic image with a two-dimensional image to generate a composite two-dimensional image. The method of detecting a region of interest from a tomographic image is not particularly limited, and for example, a method of extracting a specific structure representing a region of interest from a tomographic image using an algorithm of a known computer aided diagnosis (CAD) (hereinafter, referred to as CAD) is exemplified. In the algorithm by the CAD, it is preferable to derive a probability (for example, likelihood) representing that a pixel in the tomographic image is a region of interest, and to detect the pixel as a pixel constituting an image of the region of interest in a case where the probability is equal to or greater than a predetermined threshold value. Further, a method of extracting a region of interest from a tomographic image by filtering processing using filters for extracting a region of interest or the like may be used.

As the method in which the composite two-dimensional image generation unit <NUM> generates the composite two-dimensional image, a method of generating a composite two-dimensional image by using a minimum intensity projection method or projecting a plurality of tomographic images or at least one of a plurality of tomographic images and at least one of a plurality of projection images in the depth direction in which the tomographic planes of the breast are lined up, disclosed in <CIT> may be used. Further, for example, a method of generating a composite two-dimensional image by reconstructing a plurality of tomographic images or at least one of a plurality of tomographic images and at least one of a plurality of projection images using any method of a filtered back projection method, a maximum likelihood reconstruction method, an iterative reconstruction method, a reconstruction method using the algebraic method, and a 3D reconstruction method, disclosed in <CIT> may be used.

The information generation unit <NUM> generates a depth map <NUM> representing a correspondence relationship between a position in the composite two-dimensional image and the depth of the tomographic plane corresponding to the position. The depth map <NUM> of the embodiment is an example of correspondence relationship information of the present disclosure.

The method in which the information generation unit <NUM> generates the depth map <NUM> is not particularly limited, and a known method can be used. For example, a method disclosed in <CIT> may be used. In the method disclosed in <CIT>, a composite two-dimensional image is divided into a plurality of local regions and a correlation between a plurality of tomographic images and the regions obtained by the division is obtained. For example, as illustrated in <FIG>, a composite two-dimensional image <NUM> is divided into <NUM> (<NUM>□<NUM>) local regions, and a correlation with a plurality of tomographic images is derived for each of the divided regions. Then, the depth map <NUM> is created by associating a depth of a tomographic plane of a tomographic image including a region with the highest correlation from a reference position, with a position of each region in the composite two-dimensional image <NUM>. For example, the reference position may be a contact surface between the breast and a pressing plate (not illustrated) which presses the breast in a case where the breast is imaged by the mammography apparatus <NUM>. Here, the position of the tomographic plane in a case of generating the tomographic image is known. Thus, in the example illustrated in <FIG>, it is possible to specify the position of the tomographic plane corresponding to each of <NUM> local regions in the composite two-dimensional image <NUM> by referring to the depth map <NUM>. In generation of the depth map <NUM>, the number of local regions and the shape of the region are not particularly limited. Both the number of local regions and the shape of the region may be determined in advance according to the size of the composite two-dimensional image <NUM>, a desired accuracy, and the like, and may be set by the interpreter or the like.

The transmission unit <NUM> transmits the composite two-dimensional image generated by the composite two-dimensional image generation unit <NUM>, the depth map <NUM> generated by the information generation unit <NUM>, and the designated tomographic image generated by the designated tomographic image generation unit <NUM> to the image interpretation apparatus <NUM>.

The reception unit <NUM> receives depth information, which will be described in detail below, from the image interpretation apparatus <NUM>, and outputs the depth information to the designated tomographic image generation unit <NUM>.

In a case where the composite two-dimensional image generation unit <NUM> has generated the composite two-dimensional image and the information generation unit <NUM> has generated the depth map <NUM>, the storage controller <NUM> performs control of deleting the tomographic image <NUM> from the storage unit <NUM>. For example, in a case where the transmission unit <NUM> transmits the composite two-dimensional image and the depth map <NUM> to the image interpretation apparatus <NUM>, the storage controller <NUM> of the embodiment deletes the tomographic image <NUM> from the storage unit <NUM>.

On the other hand, as illustrated in <FIG>, the image interpretation apparatus <NUM> of the embodiment comprises a reception unit <NUM>, a display controller <NUM>, an acceptance unit <NUM>, and a transmission unit <NUM>.

For example, in the image interpretation apparatus <NUM> of the embodiment, the CPU 40A of the controller <NUM> executes the image interpretation processing program <NUM> stored in the ROM 40B so that the controller <NUM> functions as the reception unit <NUM>, the display controller <NUM>, the acceptance unit <NUM>, and the transmission unit <NUM>.

The reception unit <NUM> receives the composite two-dimensional image, the depth map <NUM>, and the designated tomographic image which are transmitted from the image processing apparatus <NUM>. The composite two-dimensional image and the designated tomographic image received by the reception unit <NUM> are output to the display controller <NUM>. Further, the depth map <NUM> received by the reception unit <NUM> is stored in the storage unit <NUM>.

The display controller <NUM> performs control of causing the display unit <NUM> to display each of the composite two-dimensional image and the designated tomographic image. The acceptance unit <NUM> accepts region information representing a designated region that is designated with respect to the composite two-dimensional image, which is caused to be displayed on the display unit <NUM> by the display controller <NUM>, by the interpreter using the operation unit <NUM>, and outputs the accepted region information to the transmission unit <NUM>. The transmission unit <NUM> transmits the region information to the image processing apparatus <NUM>.

Next, the operation of the image processing system <NUM> of the embodiment will be described with reference to the drawings.

In the image processing system <NUM> of the embodiment, for example, in a case where the interpreter gives an instruction to perform the interpretation of the radiographic image using the operation unit <NUM> in the image interpretation apparatus <NUM>, the image processing apparatus <NUM> executes the image generation processing and the image interpretation apparatus <NUM> executes image interpretation processing.

First, the image generation processing executed by the image processing apparatus <NUM> will be described. In the image processing apparatus <NUM>, in a case where an instruction to perform image interpretation is given, the CPU 30A of the controller <NUM> executes the image generation processing program <NUM> stored in the ROM 30B to execute the image generation processing of which an example is illustrated in <FIG> is a flowchart illustrating an example of the flow of image generation processing of the image processing apparatus <NUM> of the embodiment.

In the image generation processing illustrated in <FIG>, in step S100, the acquisition unit <NUM> acquires a plurality of projection images corresponding to an image interpretation target according to the user's instruction as described above.

In step S102, the tomographic image generation unit <NUM> reconstructs all or some of the plurality of projection images to generate a plurality of tomographic images <NUM> as described above. In step S104, the composite two-dimensional image generation unit <NUM> generates a composite two-dimensional image from a plurality of radiographic images selected from among the plurality of projection images and the plurality of tomographic images <NUM> as described above. In step S106, the information generation unit <NUM> generates the depth map <NUM> from the composite two-dimensional image and the tomographic image <NUM> as described above.

Is step S108, the transmission unit <NUM> transmits the composite two-dimensional image generated in step S104 and the depth map <NUM> generated in step S106 to the image interpretation apparatus <NUM>. As will be described in detail below, in the image interpretation apparatus <NUM> that has received the composite two-dimensional image and the depth map <NUM>, the display unit <NUM> displays the composite two-dimensional image (refer to <FIG>).

In step S110, the storage controller <NUM> deletes the tomographic image <NUM> from the storage unit <NUM> as described above.

In step S112, the reception unit <NUM> determines whether the depth information is received as described above. The determination of step S112 is negative until the depth information is received from the image interpretation apparatus <NUM>. On the other hand, the determination of step S112 is affirmative in a case where the depth information is received from the image interpretation apparatus <NUM>, and the processing proceeds to step S114.

In step S114, the designated tomographic image generation unit <NUM> generates a tomographic image in a tomographic plane at a depth represented by the depth information, as the designated tomographic image. In step S116, the transmission unit <NUM> outputs the generated designated tomographic image to the image interpretation apparatus <NUM>.

In step S118, the reception unit <NUM> determines whether the present image generation processing is to be ended. For example, in the embodiment, in a case where the display of the radiographic image is ended in the image interpretation apparatus <NUM>, in other words, in a case where the interpreter ends the image interpretation in the image interpretation apparatus <NUM>, the reception unit <NUM> receives an end instruction (refer to step S218 of <FIG>) transmitted from the image interpretation apparatus <NUM>. Therefore, the determination of step S118 is negative until the reception unit <NUM> receives the end instruction, the processing returns to step S112, and the processing of steps S114 to S118 is repeated. On the other hand, in a case where the reception unit <NUM> receives the end instruction, the determination of step S118 is affirmative, and the present image generation processing is ended.

Next, the image interpretation processing executed by the image interpretation apparatus <NUM> will be described. In the image interpretation apparatus <NUM>, in a case where an instruction to perform image interpretation is given, the CPU 40A of the controller <NUM> executes the image interpretation processing program <NUM> stored in the ROM 40B to execute the image interpretation processing of which an example is illustrated in <FIG> is a flowchart illustrating an example of the flow of image interpretation processing of the image interpretation apparatus <NUM> of the embodiment.

In the image interpretation processing illustrated in <FIG>, in step S200, the reception unit <NUM> determines whether the composite two-dimensional image and the depth map <NUM> are received. The determination of step S200 is negative until the reception unit <NUM> receives the composite two-dimensional image and the depth map <NUM> which are transmitted from the image processing apparatus <NUM> by the processing of step S108 in the image generation processing (refer to <FIG>). On the other hand, in a case where the composite two-dimensional image and the depth map <NUM> are received from the image processing apparatus <NUM>, the determination of step S200 is affirmative, and the processing proceeds to step S202. In step S202, the reception unit <NUM> causes the received depth map <NUM> to be stored in the storage unit <NUM>.

In step S204, the display controller <NUM> causes the display unit <NUM> to display the composite two-dimensional image. <FIG> illustrates an example of a display form of a composite two-dimensional image <NUM> displayed on the display unit <NUM>. In the composite two-dimensional image <NUM> illustrated in <FIG>, a breast image WG including a region of interest <NUM> is included.

The interpreter designates a region (above-described designated region), in which a tomographic image is desired to be referred to, with respect to the composite two-dimensional image <NUM> using the operation unit <NUM>. The method in which the interpreter designates the designated region is not particularly limited.

For example, <FIG> illustrates an example of a case in which the region of interest <NUM> included in the breast image WG of the composite two-dimensional image <NUM> illustrated in <FIG> is designated with an arrow icon <NUM> that appears by the interpreter operating the operation unit <NUM>. For example, in the embodiment, in a case where the region of interest <NUM> is designated with the icon <NUM> as illustrated in <FIG>, the acceptance unit <NUM> accepts region information that specifies a rectangular region inscribed in the region of interest <NUM> as the designated region.

As illustrated in <FIG>, in a case where the interpreter moves the icon <NUM> on the composite two-dimensional image <NUM>, the acceptance unit <NUM> accepts region information that specifies a region surrounded by a trajectory <NUM> of the icon <NUM> as the designated region. In this case, it is preferable to add a line image representing the trajectory <NUM> to the composite two-dimensional image <NUM>.

In a case where the designated region is designated by the interpreter in this manner, the acceptance unit <NUM> accepts region information representing the designated region. Therefore, in step S206, the acceptance unit <NUM> determines whether the region information is accepted. In a case where the acceptance unit <NUM> has not accepted the region information yet, the determination of step S206 is negative, and the processing proceeds to step S216. On the other hand, in a case where the acceptance unit <NUM> has accepted the region information, the determination of step S206 is affirmative, and the processing proceeds to step S208.

In step S208, the acceptance unit <NUM> refers to the depth map <NUM> and derives the depth information representing the depth which is associated with the position on the composite two-dimensional image specified by the accepted region information.

In step S210, the transmission unit <NUM> transmits the depth information derived by the acceptance unit <NUM> to the image processing apparatus <NUM>. As described above, the image processing apparatus <NUM> generates the designated tomographic image according to the depth represented by the depth information by the processing of steps S114 to S116 in the image generation processing (refer to <FIG>), and outputs the designated tomographic image to the image interpretation apparatus <NUM>. Therefore, in step S212, the reception unit <NUM> determines whether the designated tomographic image is received. The determination of step S212 is negative until the reception unit <NUM> receives the designated tomographic image. On the other hand, the determination of step S212 is affirmative in a case where the reception unit <NUM> receives the designated tomographic image, and the processing proceeds to step S214.

In step S214, the display controller <NUM> causes the display unit <NUM> to display the designated tomographic image. The display form in which the display controller <NUM> causes the display unit <NUM> to display the designated tomographic image is not particularly limited. For example, as illustrated in <FIG>, a form in which a designated tomographic image <NUM> is displayed on the display unit <NUM> in a state of being superimposed on the designated region of the composite two-dimensional image <NUM> may be used. In this case, a form in which the composite two-dimensional image <NUM> and the designated tomographic image <NUM> are entirely superimposed on each other may be used, or a form in which the composite two-dimensional image <NUM> and the designated tomographic image <NUM> are partially superimposed on each other may be used. In the display form illustrated in <FIG>, by displaying a scale <NUM> representing the position of the tomographic plane of the designated tomographic image <NUM> in the depth direction, and a numerical value <NUM> representing the depth of the tomographic plane, the position of the region of interest <NUM> is easily recognized.

For example, as illustrated in <FIG>, a form in which the composite two-dimensional image <NUM> and the designated tomographic image <NUM> are displayed side by side on the display unit <NUM> may be used. Further, as illustrated in <FIG>, by using a form in which the designated tomographic image <NUM> is displayed to be enlarged compared to the composite two-dimensional image <NUM>, the image interpretation of the region of interest <NUM> can be easily performed.

In step S216, the transmission unit <NUM> determines whether the image interpretation is to be ended. For example, in the embodiment, since the interpreter gives an instruction to end the processing using the operation unit <NUM> in case of ending the image interpretation, the transmission unit <NUM> determines whether the processing is to be ended by determining whether an instruction to end the image interpretation is given. In a case where the image interpretation is not to be ended, in other words, in a case where an instruction to end the image interpretation is not given, the determination of step S216 is negative, and the processing returns to step S206, and the processing of steps S208 to S214 is repeated. On the other hand, in a case where the image interpretation is to be ended, in other words, in a case where an instruction to end the image interpretation is given, the determination of step S216 is affirmative, the processing proceeds to step S218.

In step S218, the transmission unit <NUM> transmits the end instruction representing that the image interpretation is to be ended to the image processing apparatus <NUM>. In step S220, the display controller <NUM> ends the display of the composite two-dimensional image <NUM> and the designated tomographic image <NUM> on the display unit <NUM>, and then the present image interpretation processing is ended.

By storing the generated composite two-dimensional image and depth map <NUM> in an association manner in the image preservation system <NUM> or the like, from the next time, in a case where interpretation of the composite two-dimensional image is performed in the image interpretation apparatus <NUM>, it is possible to skip the processing of steps S100 to S110 in the image generation processing. In this case, in step S200 in the image interpretation processing, the reception unit <NUM> of the image interpretation apparatus <NUM> may receive the composite two-dimensional image and the depth map <NUM> from the image preservation system <NUM> or the like.

An aspect in which the image processing apparatus <NUM> generates the designated tomographic image has been described in the first embodiment, but in the second embodiment, an aspect in which the image interpretation apparatus <NUM> generates the designated tomographic image in the image processing system <NUM> will be described.

<FIG> is a functional block diagram illustrating an example of the configurations of the image processing apparatus <NUM> and the image interpretation apparatus <NUM> of the embodiment. As illustrated in <FIG>, the image processing apparatus <NUM> of the second embodiment is different from the image processing apparatus <NUM> of the first embodiment (refer to <FIG>) in that the reception unit <NUM> and the designated tomographic image generation unit <NUM> are not provided.

Further, the image processing apparatus <NUM> of the second embodiment is different from the image processing apparatus <NUM> of the first embodiment in that the transmission unit <NUM> transmits a plurality of projection images acquired by the acquisition unit <NUM> to the image interpretation apparatus <NUM>.

On the other hand, as illustrated in <FIG>, the image interpretation apparatus <NUM> of the second embodiment is different from the image interpretation apparatus <NUM> of the first embodiment (refer to <FIG>) in that the designated tomographic image generation unit <NUM> is provided and the transmission unit <NUM> is not provided.

Further, the reception unit <NUM> of the image interpretation apparatus <NUM> of the second embodiment is different from the reception unit <NUM> of the first embodiment in that the reception unit <NUM> receives a plurality of projection images <NUM> from the image processing apparatus <NUM> and causes the projection images <NUM> to be stored in the storage unit <NUM>.

Each unit in the image processing apparatus <NUM> and the image interpretation apparatus <NUM> other than the transmission unit <NUM> of the image processing apparatus <NUM> and the reception unit <NUM> of the image interpretation apparatus <NUM> has the same function as that of the first embodiment even in a case where the apparatus in which the unit is provided is different.

Next, the operation of the image processing system <NUM> of the embodiment will be described.

<FIG> is a flowchart illustrating an example of the flow of image generation processing of the image processing apparatus <NUM> of the embodiment. The image generation processing illustrated in <FIG> is different from the image generation processing of the first embodiment (refer to <FIG>) in that processing of step S109 is provided instead of the processing of step S108 and the processing of steps S112 to S118 is not provided.

As illustrated in <FIG>, in step S109, the transmission unit <NUM> transmits the composite two-dimensional image, the depth map <NUM>, and the projection image <NUM> to the image interpretation apparatus <NUM>. As described above, the transmission unit <NUM> of the embodiment transmits a plurality of projection images required for generating the designated tomographic image to the image interpretation apparatus <NUM>.

As illustrated in <FIG>, in the image generation processing of the embodiment, in a case where the storage controller <NUM> deletes the tomographic image <NUM> from the storage unit <NUM> in step S110, the present image generation processing is ended.

On the other hand, <FIG> is a flowchart illustrating an example of the flow of image interpretation processing of the image interpretation apparatus <NUM> of the embodiment. The image interpretation processing illustrated in <FIG> is different from the image interpretation processing of the first embodiment (refer to <FIG>) in that processing of step S203 is provided instead of the processing of step S202, processing of step S211 is provided instead of the processing of step S210 and step S212, and the processing of step S218 is not provided.

As illustrated in <FIG>, in step S203, the reception unit <NUM> causes the depth map <NUM> and the projection image <NUM> to be stored in the storage unit <NUM>.

As illustrated in <FIG>, in a case where the depth information is derived in step S208, in step S211, the tomographic image generation unit <NUM> generates a tomographic image in a tomographic plane at a depth represented by the depth information, as the designated tomographic image, similarly to step S114 of the image generation processing of the first embodiment (refer to <FIG>), and the processing proceeds to step S214.

Further, as illustrated in <FIG>, in a case where an instruction to end the image interpretation is given so that the determination of step S216 is affirmative, the processing proceeds to step S220.

In the image processing system <NUM> of the embodiment, since the plurality of projection images <NUM>, which are used for the image interpretation apparatus <NUM> to generate the designated tomographic image, are received and stored, the image interpretation apparatus <NUM> can generate the designated tomographic image, and therefore, the image processing apparatus <NUM> may not generate the designated tomographic image. In a case where the image interpretation apparatus <NUM> of the embodiment stores the plurality of projection images <NUM> in advance although the processing amount is increased with the generation of the designated tomographic image, the transmission/reception of any information with respect to the image processing apparatus <NUM> is not required after the image interpretation apparatus <NUM> has received the composite two-dimensional image, the depth map <NUM>, and the plurality of projection images <NUM> from the image processing apparatus <NUM>. Therefore, a load on communication can be reduced. In particular, in the image processing system <NUM> of the embodiment, high effects are obtained in a case where the interpreter repeats the generation of the designated tomographic image many times.

As described above, the image processing system <NUM> of the above-described embodiments comprises the acquisition unit <NUM>, the tomographic image generation unit <NUM>, the designated tomographic image generation unit <NUM>, the composite two-dimensional image generation unit <NUM>, the information generation unit <NUM>, the display controller <NUM>, and the acceptance unit <NUM>. The acquisition unit <NUM> acquires a plurality of projection images obtained by tomosynthesis imaging in which radiation is emitted from the radiation source to the breast at different irradiation angles and a projection image is captured at each irradiation angle by the radiation detector. The tomographic image generation unit <NUM> generates a plurality of tomographic images in each of a plurality of tomographic planes of the breast, from the plurality of projection images. The composite two-dimensional image generation unit <NUM> generates a composite two-dimensional image from a plurality of images selected from among the plurality of projection images and the plurality of tomographic images. The information generation unit <NUM> generates the depth map <NUM> representing a correspondence relationship between a position in the composite two-dimensional image and the depth of the tomographic plane corresponding to the position. The display controller <NUM> performs control of causing the display unit <NUM> to display the composite two-dimensional image. The acceptance unit <NUM> accepts the region information representing the designated region designated with respect to the composite two-dimensional image displayed on the display unit <NUM>. In a case where the acceptance unit <NUM> has accepted the region information, the designated tomographic image generation unit <NUM> generates a tomographic image in a tomographic plane at a specified depth on the basis of the depth map <NUM>, which corresponds to the designated region in the composite two-dimensional image, as the designated tomographic image. Further, in a case where the designated tomographic image is generated, the display controller <NUM> performs control of causing the display unit <NUM> to display the generated designated tomographic image.

With the above-described configuration, in the image processing system <NUM> of the above-described embodiments, in a case where the designated region is designated from the composite two-dimensional image, a designated tomographic image in a tomographic plane at a specified depth can be generated on the basis of the depth map <NUM> and displayed. Thus, after the composite two-dimensional image and the depth map <NUM> are generated, the plurality of tomographic images <NUM> generated by the tomographic image generation unit <NUM> may not be stored.

Thus, with the above-described configuration, in the image processing system <NUM> of the above-described embodiments, it is possible to reduce a load associated with the handling of tomographic images.

In the above-described embodiments, an aspect in which the designated tomographic image generation unit <NUM> generates one designated tomographic image corresponding to a tomographic plane at a depth according to the depth information has been described, but the designated tomographic image generated by the designated tomographic image generation unit <NUM> is not limited to this aspect. For example, an aspect in which tomographic images corresponding to tomographic planes at depths before and after a depth according to the depth information are also generated by the designated tomographic image generation unit <NUM> and are displayed on the display unit <NUM> may be used. <FIG> illustrates an example of a display form in which a designated tomographic image <NUM> at a depth according to the depth information, tomographic images <NUM> and <NUM> corresponding to tomographic planes at depths before and after the depth according to the depth information are displayed on the display unit <NUM>. In this case, since a plurality of tomographic images are displayed, it is preferable to display numerical values <NUM> to <NUM> respectively representing the depths of the tomographic planes of the designated tomographic image <NUM> and the tomographic images <NUM> and <NUM>.

In the above-described embodiments, regarding the designated region, an aspect in which the acceptance unit <NUM> specifies the depth from the depth map <NUM> has been described, but an aspect in which the designated tomographic image generation unit <NUM> specifies the depth from the depth map <NUM> may be used.

Further, an aspect in which the image processing system <NUM> of the above-described embodiments comprises the tomographic image generation unit <NUM> and the designated tomographic image generation unit <NUM> as generation units has been described, but the present disclosure is not limited to the aspect. An aspect in which the image processing system <NUM> includes one generation unit having the functions of the tomographic image generation unit <NUM> and the designated tomographic image generation unit <NUM> may be used.

In addition, in the above-described embodiments, an aspect in which the image processing system <NUM> comprises the image preservation system <NUM>, the image processing apparatus <NUM>, and the image interpretation apparatus <NUM> has been described, but the present disclosure is not limited to the aspect, and for example, an aspect in which the image preservation system <NUM> and the image processing apparatus <NUM> are integrated may be used.

In the above-described embodiments, for example, the following various processors can be used as the hardware structure of processing units executing various kinds of processing, such as the tomographic image generation unit <NUM>, the designated tomographic image generation unit <NUM>, the composite two-dimensional image generation unit <NUM>, the information generation unit <NUM>, the storage controller <NUM>, the transmission unit <NUM>, the reception unit <NUM>, the reception unit <NUM>, the display controller <NUM>, the acceptance unit <NUM>, and the transmission unit <NUM>. The various processors include, for example, a programmable logic device (PLD) that is a processor of which the circuit configuration can be changed after manufacture, such as a field-programmable gate array (FPGA), and a dedicated electric circuit that is a processor having a dedicated circuit configuration designed to execute a specific process, such as an application specific integrated circuit (ASIC), in addition to the CPU that is a general-purpose processor which executes software (program) to function as various processing units as described above.

One processing unit may be configured by one of the various processors or a combination of the same or different kinds of two or more processors (for example, a combination of a plurality of FPGAs or a combination of a CPU and an FPGA). In addition, a plurality of processing units may be configured by one processor.

As an example where a plurality of processing units are configured by one processor, first, there is an aspect where one processor is configured by a combination of one or more CPUs and software as typified by a computer, such as a client or a server, and this processor functions as a plurality of processing units. Second, there is an aspect where a processor fulfilling the functions of the entire system including a plurality of processing units by one integrated circuit (IC) chip as typified by a system on chip (SoC) or the like is used. In this manner, various processing units are configured by using one or more of the above-described various processors as hardware structures.

In addition, specifically, an electric circuit (circuitry) obtained by combining circuit elements, such as semiconductor elements, can be used as the hardware structure of the various processors.

In the above-described embodiments, an aspect in which the image generation processing program <NUM> is stored (installed) in the ROM 30B in advance and the image interpretation processing program <NUM> is stored (installed) in the ROM 40B in advance has been described, but the present disclosure is not limited thereto. Each of the image generation processing program <NUM> and the image interpretation processing program <NUM> may be provided by being recorded in a recording medium such as a compact disk read only memory (CD-ROM), a digital versatile disk read only memory (DVD-ROM), and a Universal Serial Bus (USB) memory. In addition, each of the image generation processing program <NUM> and the image interpretation processing program <NUM> may be downloaded from an external device via a network.

For example, the configurations and operations of the medical system <NUM>, the radiography system <NUM>, and the image processing system <NUM> described in the above-described embodiments are illustrative and may be modified in accordance with situations in a range without departing from the scope of the invention.

Claim 1:
An image processing system (<NUM>) comprising:
an acquisition unit (<NUM>) that acquires a plurality of projection images (<NUM>) obtained by tomosynthesis imaging in which radiation is emitted from a radiation source to a breast at different irradiation angles and a projection image (<NUM>) is captured at each irradiation angle by a radiation detector;
a tomographic image generation unit (<NUM>) that generates a plurality of tomographic images in each of a plurality of tomographic planes of the breast, from the plurality of projection images;
a composite two-dimensional image generation unit (<NUM>) that generates a composite two-dimensional image from a plurality of images selected from among the plurality of projection images and the plurality of tomographic images;
an information generation unit (<NUM>) that generates, correspondence relationship information representing a correspondence relationship between a position in the composite two-dimensional image and a depth of a tomographic plane corresponding to the position;
a display controller (<NUM>) that performs control of causing a display device to display the composite two-dimensional image;
an acceptance unit (<NUM>) that accepts region information representing a designated region designated with respect to the composite two-dimensional image displayed on the display device; and
a designated tomographic image generation unit (<NUM>) that generates, as a designated tomographic image, a tomographic image in a tomographic plane at a depth which corresponds to the designated region in the composite two-dimensional image and is specified on the basis of the correspondence relationship information, in a case where the acceptance unit accepts the region information;
wherein, in a case where the designated tomographic image is generated, the display controller (<NUM>) further performs control of causing the display device to display the generated designated tomographic image; the system characterised by:
a storage controller (<NUM>) that performs control of deleting the plurality of tomographic images from a storage unit, in which the plurality of tomographic images are stored, after the composite two-dimensional image and the correspondence relationship information are generated.