Medical information processing system, non-transitory computer readable storage medium, and ultrasound diagnosis apparatus

A medical information processing system according to an embodiment includes a storage, and processing circuitry. The storage stores therein a mammography image of a breast of a patient and information indicating an image taking direction of the mammography image. The processing circuitry sets a region of interest in the mammography image. The processing circuitry specifies position information of the region of interest in a schematic drawing that schematically expresses the breast, on the basis of position information of the region of interest in the mammography image and the information indicating the image taking direction. The processing circuitry outputs the position information of the region of interest in the schematic drawing.

FIELD

Embodiments described herein relate generally to a medical information processing system, a non-transitory computer readable storage medium, and an ultrasound diagnosis apparatus.

BACKGROUND

Conventionally, it has been common that mammary gland image diagnosis processes for breast cancer examinations or the like are performed by using mammography images taken by mammography apparatuses. In contrast, in recent years, a project called “Japan Strategic Anti-cancer Randomized Trial (J-START)” was started, so as to start performing a mammary gland image diagnosis process by using both mammography images and ultrasound images together for breast cancer examinations.

DETAILED DESCRIPTION

A medical information processing system according to an embodiment includes a storage, and processing circuitry. The storage stores therein a mammography image of a breast of a patient and information indicating an image taking direction of the mammography image. The processing circuitry sets a region of interest in the mammography image. The processing circuitry specifies position information of the region of interest in a schematic drawing that schematically expresses the breast, on the basis of position information of the region of interest in the mammography image and the information indicating the image taking direction. The processing circuitry outputs the position information of the region of interest in the schematic drawing.

Exemplary embodiments of a medical information processing system, a medical information processing computer program (hereinafter, “medical information processing program”), and an ultrasound diagnosis apparatus will be explained below, with reference to the accompanying drawings.

First Embodiment

FIG. 1is a diagram of an exemplary configuration of a medical information processing system according to a first embodiment. The medical information processing system according to the first embodiment is installed in a hospital where a breast cancer examination is carried out and is used for a mammary gland diagnosis process in which both mammography images and ultrasound images are used together. For example, as illustrated inFIG. 1, a medical information processing system100according to the first embodiment includes a mammography apparatus10, an ultrasound diagnosis apparatus20, an image processing apparatus30, and an image display apparatus40. These apparatuses are connected to one another via a network50so as to transmit and receive images taken by the mammography apparatus10and the ultrasound diagnosis apparatus20and the like, to and from one another.

The mammography apparatus10is configured to radiate X-rays onto a breast of an examined subject (hereinafter, “patient”) to detect X-rays that have passed through the breast, and to generate a mammography image.

FIGS. 2 and 3are diagrams of an exemplary configuration of the mammography apparatus10according to the first embodiment. For example, as illustrated inFIG. 2, the mammography apparatus10includes a pedestal11and a stand12. The stand12is provided so as to stand on the pedestal11and is configured to support an imaging stage13, a pressing plate14, an X-ray output unit15, and an X-ray detecting unit16. In this situation, the imaging stage13, the pressing plate14, and the X-ray detecting unit16are supported so as to be movable in up-and-down directions.

The imaging stage13is a stage that supports a breast B of the patient and has a supporting surface13aon which the breast B is placed. The pressing plate14is positioned above the imaging stage13and is provided so as to be movable in directions to approach and to get away from the imaging stage13, while opposing the imaging stage13in parallel. In this situation, the pressing plate14is configured to press the breast B supported by the imaging stage13, when the pressing plate14has moved in the direction to approach the imaging stage13. The breast B pressed by the pressing plate14is flattened and spread so that overlapping of mammary glands in the breast B is reduced.

Further, as illustrated inFIG. 3, the mammography apparatus10includes an operating unit17a, a raising and lowering driver17b, a high-voltage generator17c, an image processor17d, an image storage17e, a display17f, a communication controller17g, and a system controller17h. The operating unit17ais configured to receive input operations of various types of commands and the like from an operator. The raising and lowering driver17bis connected to the imaging stage13and is configured to raise and lower the imaging stage13in up-and-down directions. Further, the raising and lowering driver17bis connected to the pressing plate14and is configured to raise and lower the pressing plate14in up-and-down directions (the directions to approach and to get away from the imaging stage13).

The X-ray output unit15includes an X-ray tube15aand an X-ray converging device15b. The X-ray tube15ais configured to generate the X-rays. The X-ray converging device15bis positioned between the X-ray tube15aand the pressing plate14and is configured to control the radiation range of the X-rays generated from the X-ray tube15a. The high-voltage generator17cis connected to the X-ray tube15aand is configured to supply a high voltage used by the X-ray tube15ato generate the X-rays.

The X-ray detecting unit16includes an X-ray detector16aand a signal processor16b. The X-ray detector16ais configured to detect X-rays that have passed through the breast B and the imaging stage13and convert the detected X-rays into an electrical signal (passed X-ray data). The signal processor16bis configured to generate X-ray projection data from the electrical signal resulting from the conversion performed by the X-ray detector16a.

The image processor17dis connected to the signal processor16band the image storage17eand is configured to generate the mammography image on the basis of the X-ray projection data generated by the signal processor16band to store the generated mammography image into the image storage17e. Further, the image processor17dis connected to the display17fand is configured to cause the display17fto display the generated mammography image. In this situation, the image processor17dis capable of switching between different types of mammography images to be generated, on the basis of an input operation from the operating unit17a.

The communication controller17gis configured to control communication performed with another apparatus via the network50. For example, the communication controller17gis configured to transfer the mammography image generated by the image processor17dto another apparatus via the network50. The apparatus at the transfer destination is able to perform an image display process or an image processing process on the mammography image transferred thereto via the network50.

The system controller17his connected to the operating unit17a, the raising and lowering driver17b, the high-voltage generator17c, the X-ray converging device15b, the image processor17d, and the communication controller17gand is configured to comprehensively control the entirety of the mammography apparatus10.

Returning to the description ofFIG. 1, the ultrasound diagnosis apparatus20is configured to generate an ultrasound image on the basis of reflected-wave data acquired by scanning the patient by using an ultrasound probe that transmits and receives an ultrasound wave.

FIG. 4is a diagram of an exemplary configuration of the ultrasound diagnosis apparatus20according to the first embodiment. As illustrated inFIG. 4, the ultrasound diagnosis apparatus20according to the first embodiment includes an ultrasound probe21, a display23, an input unit22, and an apparatus main body24.

The ultrasound probe21includes a plurality of piezoelectric transducer elements. The plurality of piezoelectric transducer elements are configured to generate an ultrasound pulse on the basis of a drive signal supplied thereto from a transmitter/receiver241included in the apparatus main body24(explained later), to receive a reflected wave from a patient P, and to convert the received reflected wave into an electric signal. Further, the ultrasound probe21includes matching layers provided for the piezoelectric transducer elements, as well as a backing member that prevents ultrasound waves from propagating rearward from the piezoelectric transducer elements.

When the ultrasound pulse is transmitted from the ultrasound probe21to the patient P, the transmitted ultrasound pulse is repeatedly reflected on a surface of discontinuity of acoustic impedances at a tissue in the body of the patient P and is received as an echo signal by the plurality of piezoelectric transducer elements included in the ultrasound probe21. The amplitude of the received echo signal is dependent on the difference between the acoustic impedances on the surface of discontinuity on which the ultrasound pulse is reflected. When the transmitted ultrasound pulse is reflected on the surface of a moving blood flow, a cardiac wall, or the like, the echo signal is, due to the Doppler effect, subject to a frequency shift, depending on a velocity component of the moving members with respect to the ultrasound wave transmission direction.

The display23may be a monitor or the like and is configured to display a Graphical User Interface (GUI) used by the operator of the ultrasound diagnosis apparatus20to input various types of instructions and setting requests through the input unit22and to display an ultrasound image and an analysis result generated by the apparatus main body24.

The input unit22is configured by using a mouse, a keyboard, a button, a panel switch, a touch command screen, a foot switch, and/or a trackball and is connected to the apparatus main body24. Further, the input unit22is configured to receive the various types of instructions and setting requests from the operator of the ultrasound diagnosis apparatus20and to transfer the received various types of instructions and setting requests to the apparatus main body24.

The apparatus main body24is configured to generate the ultrasound image on the basis of the reflected waves received by the ultrasound probe21. As illustrated inFIG. 4, the apparatus main body24includes the transmitter/receiver241, a B-mode processor242, a Doppler processor243, an image processor244, an image memory245, an internal storage246, a communication controller247, and a controller248.

The transmitter/receiver241includes a trigger generating circuit, a transmission delaying circuit, a pulser circuit, and the like and is configured to supply the drive signal to the ultrasound probe21. The pulse circuit is configured to repeatedly generate a rate pulse for forming an ultrasound pulse having a predetermined Pulse Repetition Frequency (PRF). The PRF may be referred to as a rate frequency. Further, the transmission delaying circuit applies a transmission delay period that is required to converge the ultrasound pulse generated by the ultrasound probe21into the form of a beam and to determine transmission directionality and that corresponds to each of the piezoelectric transducer elements, to each of the rate pulses generated by the pulser circuit. Further, the trigger generating circuit applies the drive signal (a drive pulse) to the ultrasound probe21with timing based on the rate pulses. In other words, the transmission delaying circuit arbitrarily adjusts the transmission directions from the piezoelectric transducer element surfaces, by varying the transmission delay periods applied to the rate pulses.

Further, the transmitter/receiver241has a function to be able to instantly change the transmission frequency, the transmission drive voltage, and the like, for the purpose of executing a predetermined scanning sequence on the basis of an instruction from the controller248(explained later). In particular, the configuration to change the transmission drive voltage is realized by using a linear-amplifier-type transmitting circuit of which the value can be instantly switched or by using a mechanism configured to electrically switch between a plurality of power source units.

Further, the transmitter/receiver241includes an amplifying circuit, an Analog/Digital (A/D) converter, a reception delaying circuit, an adder, a quadrature detection circuit, and the like and is configured to generate reflected-wave data by performing various types of processes on the reflected-wave signal received by the ultrasound probe21. The amplifying circuit performs a gain correction process by amplifying the reflected-wave signal for each of channels. The A/D converter applies an A/D conversion to the gain-corrected reflected-wave signals. The reception delaying circuit applies a reception delay period required to determine reception directionality to the digital data. The adder performs an adding process on the reflected-wave signals to which the reception delay period has been applied by the reception delaying circuit. As a result of the adding process performed by the adder, reflected components from the direction corresponding to the reception directionality of the reflected-wave signals are emphasized.

The B-mode processor242is configured to receive the reflected-wave data from the transmitter/receiver241and to generate data (B-mode data) in which the strength of each signal is expressed by a degree of brightness, by performing a logarithmic amplification, an envelope detection process, and the like on the received reflected-wave data. Further, the B-mode processor242is configured to generate M-mode data (explained later).

The Doppler processor243is configured to obtain velocity information from the reflected-wave data received from the transmitter/receiver241by performing a frequency analysis, to extract bloodstream, tissues, and contrast-agent echo components under the influence of the Doppler effect, and to further generate data (Doppler data) obtained by extracting moving member information such as an average velocity, a dispersion, a power, and the like, for a plurality of points.

The image processor244is configured to generate an ultrasound image from the B-mode data and the M-mode data generated by the B-mode processor242and the Doppler data generated by the Doppler processor243. More specifically, the image processor244generates a B-mode image from the B-mode data, generates an M-mode image from the M-mode data, and generates a Doppler image from the Doppler data. Further, by performing a coordinate transformation process, a data interpolation process, or the like, the image processor244converts (by performing a scan convert process) a scanning line signal sequence from an ultrasound scan into a scanning line signal sequence in a video format used by, for example, television and generates the ultrasound image serving as a displayed image (the B-mode image, the M-mode image, or the Doppler image).

The image memory245is a memory configured to store therein the ultrasound image generated by the image processor244and an image generated by performing an image processing process on the ultrasound image. For example, after a diagnosis process, the operator is able to invoke any of the images recorded during a medical examination and is able to play back the images as still images or as a moving picture realized with multiple images. Further, the image memory245may store therein an image brightness-level signal that has passed through the transmitter/receiver241, other raw data, image data obtained via the network50, and the like.

The internal storage246is configured to store therein an apparatus control computer program for executing the ultrasound wave transmissions and receptions, image processing processes, and display processes, as well as various types of data such as diagnosis information (e.g., patients' IDs, observations of medical doctors), diagnosis protocols, and/or various types of setting information. Further, the internal storage246may also be used for storing any of the images stored in the image memory245.

The communication controller247is configured to control communication performed with another apparatus via the network50. For example, the communication controller247transfers the ultrasound image generated by the image processor244to another apparatus via the network50. The apparatus at the transfer destination is able to perform an image display process or an image processing process on the ultrasound image transferred thereto via the network50.

The controller248is configured to control the entirety of processes performed by the ultrasound diagnosis apparatus20. The controller248includes processing circuitry such as a Central Processing unit (CPU) and a memory and is configured to control operations of the ultrasound diagnosis apparatus20by employing the CPU and the memory to execute various types of computer programs. More specifically, on the basis of various types of instructions and setting requests that are input thereto by the operator via the input unit22and various types of computer programs and various types of setting information that are read from the internal storage246, the controller248controls processes performed by the transmitter/receiver241, the B-mode processor242, the Doppler processor243, and the image processor244, and also exercises control so that the display23displays the ultrasound image and the like stored in the image memory245.

Returning to the description ofFIG. 1, the image processing apparatus30is configured to process the mammography image generated by the mammography apparatus10, the ultrasound image generated by the ultrasound diagnosis apparatus20, and the like. The image processing apparatus30is primarily used when a mammography examination is performed by a mammography examination technician. Further, the image processing apparatus30is configured to receive an input of an observation related to the mammography image from the mammography examination technician and to store therein information indicating the received observation as observation information. For example, the image processing apparatus30may be configured by using an image storing server, a workstation, or the like.

The image display apparatus40is configured to obtain the mammography image, the ultrasound image, and the observation information related to the mammography image from the image processing apparatus30and to display the obtained images and information. The image display apparatus40is primarily used when an ultrasound examination is performed by an ultrasound examination technician. For example, the image display apparatus40may be a tablet terminal that can be carried around by the operator and that is connectable to the network50via a wireless Local Area Network (LAN). Alternatively, the image display apparatus40may be a notebook personal computer, for example.

Conventionally, it has been common that mammary gland image diagnosis processes for breast cancer examinations or the like are performed by using mammography images taken by mammography apparatuses. In contrast, in recent years, the project called “Japan Strategic Anti-Cancer Randomized Trial (J-START)” was started so as to start performing a mammary gland image diagnosis process by using both mammography images and ultrasound images together for breast cancer examinations.

FIG. 5is a drawing for explaining a mammary gland image diagnosis process in which both mammography images and ultrasound images are used together. For example, as illustrated on the left side ofFIG. 5, to perform a mammary gland image diagnosis process by using both mammography images and ultrasound images together, mammography images in a Mediolateral-Oblique (MLO) direction and a Cranio-Caudal (CC) direction are taken, at first, of each of the left and the right breasts of the patient, by using a mammography apparatus. After that, as illustrated on the right side ofFIG. 5, an ultrasound image of each of the left and the right of the same patient is taken by using an ultrasound diagnosis apparatus, while referring to the mammography images and observations thereon, on either the same day or a different day.

During such a mammary gland image diagnosis process, generally speaking, in many situations, the technician who takes and interprets the mammography images is a different person from the technician who takes and interprets the ultrasound images. For this reason, conventionally, technicians who are in charge of ultrasound examinations are required to have sufficient knowledge and understanding of interpretations and observations of mammography images, in order to perform the ultrasound examinations while referring to the mammography images and the observations thereon. More specifically, as overall evaluation criteria, the breast cancer examination study committee of the Japan Association of Breast and Thyroid Sonology has already presented the following: “an ultrasound examination should be carefully performed on high-density areas in a mammography image”, “When a lump of which the border is clear and smooth is observed in a mammography image, if an ultrasound examination confirms that the lump is evidently a benign lesion (e.g., a simple cyst), no further detailed examination will be required”, and “When a focal asymmetric density (FAD) is observed in a mammography image, if an ultrasound examination confirms that the mammary gland is normal, no further detailed examination will be required”. Technicians who are in charge of ultrasound examinations are thus required to make these judgements appropriately.

To cope with this situation, according to the first embodiment, the image processing apparatus30sets a region of interest in at least one of the mammography images of the breast of the patient and specifies and outputs position information of a region of interest in a schematic drawing that schematically expresses the breast, on the basis of position information of the region of interest in the mammography image and pieces of information indicating image taking directions of the mammography images. As a result, the ultrasound examination technician is able to easily understand the position in the schematic drawing corresponding to the region of interest set in the mammography image. It is therefore possible to improve the level of precision of the mammary gland image diagnosis process. In the following sections, the image processing apparatus30according to the first embodiment will be explained in detail.

FIG. 6is a diagram of an exemplary configuration of the image processing apparatus30according to the first embodiment. As illustrated inFIG. 6, the image processing apparatus30includes an input unit31, a display32, a communication controller33, a storage34, and a controller35.

The input unit31is configured to receive inputs of various types of operations and various types of information from an operator. For example, the input unit31may be configured by using a keyboard, a mouse, a button, a trackball, and/or a touch panel.

The display32is configured to display a GUI used for receiving the various types of operations from the operator and various types of images. For example, the display32may be configured by using a liquid crystal display device, a Cathode Ray Tube (CRT) display device, or a touch panel.

The communication controller33is configured to control communication performed with another apparatus via the network50. For example, the communication controller33may be configured by using a network card or a network adaptor and may perform the communication with the other apparatus by connecting to the network50via an Ethernet (registered trademark) LAN. Alternatively, for example, the communication controller33may perform wireless communication with the other apparatus by connecting to the network50via a wireless LAN.

The storage34is a storage device such as a hard disk, a semiconductor memory, or the like and is configured to store various types of information therein. More specifically, the storage34includes an image data storage34aand an observation information storage34b.

The image data storage34ais configured to store therein mammography images obtained by imaging the breast (hereinafter, mammography images of the breast) of the patient and information indicating the image taking directions of the mammography images. More specifically, the image data storage34astores therein the mammography images and pieces of information each of which indicates an image taking direction and is kept in association with a corresponding one of the images. An image data obtaining unit35a(explained later) stores the mammography images and the pieces of information each indicating the image taking direction into the image data storage34a.

For example, the image data storage34astores therein a mammography image in an MLO direction (an MLO image) and a mammography image in a CC direction (a CC image). Further, the pieces of information each indicating the image taking direction in this situation are, for example, pieces of position information each of which is expressed in an apparatus coordinate system of the mammography apparatus. When the mammography apparatus has generated each mammography image, the piece of information indicating the image taking direction thereof is appended to the image as additional information.

The observation information storage34bis configured to store therein the observation information related to the mammography images of the patient. An observation information generating unit35b(explained later) stores the observation information into the observation information storage34b.

The controller35includes processing circuitry such as a Central Processing unit (CPU) and a memory and is configured to control operations of the image processing apparatus30by employing the CPU and the memory to execute various types of computer programs. More specifically, the controller35includes the image data obtaining unit35a, the observation information generating unit35b, a display controlling unit35c, a region setting unit35d, a position specifying unit35e, and a transmitting unit35f.

The image data obtaining unit35ais configured to obtain the mammography images of the breast of the patient and the pieces of information indicating the image taking directions of the mammography images. In this situation, the image data obtaining unit35aobtains an MLO image and a CC image for each of the left and the right breasts of the patient. More specifically, the image data obtaining unit35aobtains the mammography images related to the patient serving as a diagnosis target and the pieces of information indicating the image taking directions of the mammography images, by communicating with the mammography apparatus10via the communication controller33, and further stores the mammography images and the pieces of information indicating the image taking directions that were obtained into the image data storage34a.

The observation information generating unit35bis configured to generate the observation information related to the mammography images of the patient, on the basis of an observation input by the operator. More specifically, the observation information generating unit35breceives an input of the observation related to the mammography images from a mammography examination technician via the input unit31. Further, the observation information generating unit35bgenerates the observation information indicating the received observation and stores the generated observation information into the observation information storage34b.

The display controlling unit35cis configured to cause the display32to display a reference screen used for referencing the mammography images. More specifically, when having received a display request from the operator via the input unit31, the display controlling unit35creads the mammography images related to the patient serving as the diagnosis target from the image data storage34aand reads the observation information related to the patient serving as the diagnosis target from the observation information storage34b. Further, the display controlling unit35ccauses the display32to display the reference screen on which the mammography images and the observation information that were read are arranged.

The region setting unit35dis configured to set a region of interest in at least one of the mammography images. For example, the region setting unit35dsets a region of interest in each of the MLO and the CC images for each of the left and the right breasts of the patient. More specifically, via the input unit31, the region setting unit35dreceives, from the operator, an operation to designate an area of an arbitrary size in an arbitrary position of each of the mammography images arranged on the reference screen displayed by the display controlling unit35c. After that, the region setting unit35dsets the areas designated by the operator as the regions of interest.

In this situation, for example, the region setting unit35dmay automatically detect a candidate region for a lesion from at least one of the mammography images by using a Computer Aided Diagnosis (CAD) function and may set the detected region as the region of interest. Further, for example, the region setting unit35dmay receive an operation from the operator to make an adjustment between the MLO image and the CC image with respect to the region detected by the CAD function and may set the region after the adjustment as the region of interest.

The position specifying unit35eis configured to specify position information of each of the regions of interest in the schematic drawing that schematically expresses the breast, on the basis of pieces of position information of the regions of interest in the mammography images and the pieces of information indicating the image taking directions. More specifically, the position specifying unit35ereads the mammography images of the patient serving as the examination target and the pieces of information indicating the image taking directions of the mammography images from the image data storage34aand specifies the position of the region of interest in the schematic drawing on the basis of the mammography images and the pieces of information indicating the image taking directions that were read. In this situation, the schematic drawing (which may be referred to as a schema) may be a drawing of any type as long as the drawing is able to indicate positional relationships in the breast.

FIG. 7is a drawing of an example of the schematic drawing used by the position specifying unit35eaccording to the first embodiment. The example inFIG. 7illustrates a schematic drawing of a mammary gland region, as an example of the schematic drawing that schematically expresses the breast. For example, as illustrated inFIG. 7, the schematic drawing of the mammary gland region includes, for each of the left and the right breasts, a circular region indicating the region of the breast (hereinafter, “breast region”) and a substantially triangular region indicating the region of the axilla (hereinafter, “axilla region”).

In this situation, each of the circular regions indicating the breast regions is divided into four regions “A” to “D”, as a result of dividing in up-and-down directions and left-and-right directions. For example, the region called “A” (hereinafter, “the region A”) indicates the region in the inner upper position of the breast, whereas the region called “B” (hereinafter, “the region B”) indicates the region in the inner lower position of the breast. Further, for example, the region called “C” (hereinafter, “the region C”) indicates the region in the outer upper position of the breast, whereas the region called “D” (hereinafter, “the region D”) indicates the region in the outer lower position of the breast. Further, the substantially triangular region “C′” indicating the axilla region (hereinafter, “the region C′”) extends upward diagonally from the region C and is shaped so as to become narrower as the distance from the region C increases.

Returning to the description ofFIG. 6, for example, the position specifying unit35eis configured to specify position information of each of first and second regions of interest in the schematic drawing, on the basis of position information of the first region of interest that is the region of interest in the MLO image, position information of the second region of interest that is the region of interest in the CC image, the information indicating the image taking direction of the MLO image, and the information indicating the image taking direction of the CC image. In this situation, for example, the position specifying unit35especifies the positions of the regions of interest in the schematic drawing, on the basis of the pieces of position information expressed in the apparatus coordinate system of the mammography apparatus that took the mammography images.

FIG. 8is a drawing for explaining a process of specifying the pieces of position information of the regions of interest in the schematic drawing performed by the position specifying unit35eaccording to the first embodiment.FIG. 8illustrates an example of a method for transforming pieces of position information in the mammography images into pieces of position information in a schematic drawing, on the basis of the image taking directions expressed in the apparatus coordinates system of the mammography apparatus. In the following sections, an example will be explained in which the schematic drawing of the mammary gland region illustrated inFIG. 7is used.

First, on the basis of the apparatus coordinate system of the mammography apparatus, the position specifying unit35especifies the pieces of position information of the regions corresponding to the breast region (the regions A to D) in the schematic drawing of the mammary gland region and the pieces of position information of the regions of interest set in the mammography images. For example, by using the MLO and the CC images, the position specifying unit35especifies the pieces of position information of the regions corresponding to the breast region and the pieces of position information of the regions of interest each of which is set in a different one of the MLO and the CC images.

First, the position specifying unit35esets an X—CCaxis, a Y—CCaxis, and the origin of the detector in the CC image. For example, of the two vertices positioned on the patient side among the four vertices of a rectangular detection surface of the detector, the position specifying unit35esets the vertex corresponding to the upper side of the breast as the origin. Further, the position specifying unit35edetects a skin surface S—CCrendered in the CC image. In this situation, as the method for detecting the skin surface S—CC, any of various types of generally-known image detecting methods may be used.

Further, of the two points at which the detected S cc and X—CCaxes are in contact with each other, the position specifying unit35edetermines the point positioned closer to the origin to be CC—1and the point positioned farther from the origin to be CC—2. In this situation, the position specifying unit35emay receive an operation from the operator to designate the positions of CC—1and CC—2out of the CC image and may set the points on the basis of the received operation. After that, the position specifying unit35egenerates a straight line L—CC1with a projection from the point CC—1toward the tube bulb and generates a straight line L—CC2with a projection from the point CC—2toward the tube bulb.

Further, the position specifying unit35esets an X—MLOaxis, a Y—MLOaxis, and the origin of the detector in the MLO image. For example, of the two vertices positioned on the patient side among the four vertices of a rectangular detection surface of the detector, the position specifying unit35esets the vertex corresponding to the outer side of the breast as the origin. Further, the position specifying unit35edetects a skin surface S—MLOrendered in the MLO image. In this situation, as the method for detecting the skin surface S—MLO, any of various types of generally-known image detecting methods may be used, similarly to the method for detecting the skin surface S—CC. After that, the position specifying unit35edetects a breast region in the MLO image.

FIG. 9is a drawing for explaining a process of detecting the breast region in the MLO image performed by the position specifying unit35eaccording to the first embodiment. As illustrated inFIG. 9, the position specifying unit35edefines a straight line L—S1expressed with a linear function Y—MLO=aX—MLO+b (where a is an arbitrary positive number) and detects the point at which the straight line first intersects the skin surface S—MLOby gradually increasing the value of b starting with a sufficiently small negative value. After that, the position specifying unit35esets the point obtained by perpendicularly projecting the detected point onto the X—MLOaxis as MLO—upper.

Further, the position specifying unit35edefines a straight line LS2expressed with a linear function Y—MLO=−aX—MLO+b (where a is an arbitrary positive number) and detects, in a similar manner, the point at which the straight line first intersects the skin surface S—MLOby gradually increasing the value of b starting with a sufficiently small negative value. After that, the position specifying unit35esets the point obtained by perpendicularly projecting the detected point onto the X—MLOaxis as MLO—lower.

The values of a and b in the linear functions used above may be set to predetermined values in advance so as to be stored in the storage34or the like and so as to be changed in response to an instruction from the operator. Further, for example, the position specifying unit35emay detect the position of the nipple rendered in the MLO image and may use the slope of the straight line passing through the detected nipple position and the origin of the detector as the value of a.

The position specifying unit35edetects the region between the point MLO—upperand the point MLO—lowerthat were set in this manner as the breast region. Alternatively, the position specifying unit35emay receive an operation from the operator to designate the positions of the point MLO—upperand the point MLO—lowerout of the MLO image and may set the points on the basis of the received operation.

Returning to the description ofFIG. 8, the position specifying unit35efurther generates a straight line L—MLO—uwith a projection from the point MLO—uppertoward the tube bulb and generates a straight line L—MLO—1with a projection from the point MLO—lowertoward the tube bulb. After that, the position specifying unit35ecalculates an ellipse E inscribed in the region enclosed by the straight line L—CC1, the straight line L—CC2, the straight line L—MLO—u, and the straight line L—MLO—1. It should be noted that the ellipse E may be a perfect circle.

Subsequently, the position specifying unit35edetermines the point obtained by a perpendicular projection from the region of interest (ROI) set in the MLO image onto the X—MLOaxis to be ROI—MLOand determines the point obtained by a perpendicular projection from the ROI set in the CC image onto the X—MLOaxis to be ROI—CC. Further, the position specifying unit35edetermines the region defined by causing the straight line projecting from the point ROI—MLOtoward the tube bulb to intersect the straight line projecting from the point ROI—CCtoward the tube bulb as a region M. In this situation, when the region of interest set in each of the MLO and the CC images is a point, the region M is also a point. When at least one of the regions of interest set in the MLO and the CC images is an area, the region M is a region having an area.

Further, the position specifying unit35esets the center position of the ellipse E as a nipple position N. Alternatively, the position specifying unit35emay extract a nipple position N—MLOfrom the MLO image and extract a nipple position N—CCfrom the CC image, so as to set the intersection point between a straight line projecting from the nipple position N—MLOtoward the tube bulb and a straight line projecting from the nipple position N—CCtoward the tube bulb to be a nipple position of the ellipse E. In this situation, as the method for detecting the nipple position N, any of various types of generally-known image detecting methods may be used.

Subsequently, the position specifying unit35especifies position information of the region corresponding to the region C′ in the schematic drawing of the mammary gland region, on the basis of the apparatus coordinate system of the mammography apparatus. For example, of the two intersection points at which a straight line passing through the tube bulb position used when the CC image was taken and the nipple position N intersects the ellipse E, the position specifying unit35edetermines the point positioned closer to the detector as a point A. Of the two intersection points at which a straight line passing through the tube bulb position used when the MLO image was taken and the nipple position N intersects the ellipse E, the position specifying unit35edetermines the point positioned closer to the detector as a point B.

Further, the position specifying unit35edetermines a point at which a straight line projecting from the origin of the X—MLOaxis, which is the utmost endpoint of the detector when the MLO image was taken, toward the tube bulb intersects another straight line that extends parallel to the X—MLOaxis and that passes through the nipple position N of the ellipse N to be a point C. After that, the position specifying unit35especifies the region A-B-C enclosed by the line segment connecting the point A to the point C, the line segment connecting the point B to the point C, and the ellipse E as the region corresponding to the region C′.

After that, the position specifying unit35especifies the position information of the region of interest in the schematic drawing by transforming the specified position information of the region of the ellipse E, the position information of the region corresponding to the region C′, and the position information of the region M indicating the region of interest into pieces of position information in the schematic drawing.

FIG. 10is a drawing for explaining the process of specifying the position information of the region of interest in the schematic drawing performed by the position specifying unit35eaccording to the first embodiment. For example, as illustrated in the top left and middle left sections ofFIG. 10, the position specifying unit35etransforms the pieces of position information of the regions into the pieces of position information in the schematic drawing by either enlarging or reducing the region of the ellipse E, the region corresponding to the region C′, and the region M indicating the region of interest, in such a manner that each of the regions fits the shape of the schematic drawing of the mammary gland region.

First, the position specifying unit35etransforms the position information of the region of the ellipse E into position information of the circular region indicating the breast region in the schematic drawing of the mammary gland region. In this situation, as the method for transforming the position information of the region of the ellipse E into the position information of the circular region, any of various types of generally-known coordinate transforming methods may be used. For example, the position specifying unit35emay use a method by which position coordinates of an ellipse are transformed into position coordinates of a circle by using a predetermined transformation matrix.

FIG. 11is a drawing for explaining an example of the region transformation process performed by the position specifying unit35eaccording to the first embodiment. For example, as illustrated inFIG. 11, when the position coordinates of the ellipse before the transformation is expressed as (X,Y)=(X1,X2), whereas the position coordinates of the circle after the transformation is expressed as (X′,Y′)=(X1′,X2′), the position coordinates of the ellipse can be expressed by using Expression (1) below that uses a transformation matrix Aks.

In this situation, when a symmetric matrix Fksis expressed by using b, c, and d as presented in Expression (2) below, the transformation matrix Aksin Expression (1) can be expressed as presented in Expression (3) below.

In other words, in the present example, it is possible to transform the ellipse into the circle of a desired size, by calculating the transformation matrix Akswhile setting the values of b, c, and d appropriately in accordance with the size of the circle after the transformation. Accordingly, by using this method, the position specifying unit35eis able to transform the region of the ellipse E into the circular region corresponding to the breast region (the regions A to D) in the schematic drawing of the mammary gland region, by setting the values of b, c, and d appropriately in accordance with the size of the schematic drawing of the mammary gland region.

After that, the position specifying unit35etransforms the pieces of position information of the region C′ and the region M into pieces of information in the schematic drawing of the mammary gland region, by also transforming the region C′ and the region M while using the same transformation matrix as the transformation matrix Aksused for the transformation of the ellipse E.

In this situation, for example, if no region of interest is set in the CC image, the position specifying unit35especifies a straight line region obtained by projecting the region of interest set in the MLO image onto the ellipse E, as a region of interest in the schematic drawing. Similarly, if no region of interest is set in the MLO image, the position specifying unit35especifies a straight line region obtained by projecting the region of interest set in the CC image onto the ellipse E as a region of interest in the schematic drawing.

Returning to the description ofFIG. 10, after transforming the pieces of position information of the region of the ellipse E, the region corresponding to the region C′, and the region M indicating the region of interest into the piece of position information in the schematic drawing, the position specifying unit35egenerates display information in which the region M indicating the region of interest is arranged in a template of the schematic drawing of the mammary gland region, on the basis of the pieces of position information of the regions in the schematic drawing, as illustrated in, for example, the bottom left and bottom right sections ofFIG. 10. In this situation, the display information may be generated as image data in a format such as, for example, Joint Photographic Experts Group (JPEG), Graphics Interchange Format (GIF), or a bitmap.

After that, the position specifying unit35estores the generated display information into the storage34so as to be kept in association with the corresponding mammography image. For example, the position specifying unit35eappends the generated display information to the mammography image as additional information and stores the display information into the image data storage34atogether with the image data of the mammography image. Alternatively, the position specifying unit35emay store the generated display information into the image data storage34aso as to be kept in correspondence with the patient ID assigned to the targeted patient.

For example, when the position specifying unit35egenerates the display information by using the template of the schematic drawing, the position specifying unit35emay not only indicate the position information of the region of interest in the schematic drawing, but also display, in the schematic drawing, a region in which the mammary gland density is higher than a predetermined value, a calcified region, a tumor region, and/or the like, by extracting these regions from the mammography image. For example, the position specifying unit35emay display the parts corresponding to these regions in the schematic drawing in colors that are different from the color of the schematic drawing and are varied for the different types of regions or may display marks that are determined in advance for the different types of regions.

The display information generated by the position specifying unit35ein this manner is transmitted to either the image display apparatus40or the ultrasound diagnosis apparatus20by, for example, the transmitting unit35f(explained later). After that, the image display apparatus40or the ultrasound diagnosis apparatus20that received the display information outputs the position information of the region of interest in the schematic drawing to the display32in a predetermined display format thereof, on the basis of the received display information.

For example, the image display apparatus40or the ultrasound diagnosis apparatus20outputs the position information of the region of interest in the schematic drawing, as reference information used during an ultrasound diagnosis process. Further, for example, the image display apparatus40or the ultrasound diagnosis apparatus20outputs the position information in the schematic drawing that corresponds to the patient ID of a patient who is currently undergoing an ultrasound diagnosis process.

FIG. 12is a drawing of an example of the display information displayed by the image display apparatus40according to the first embodiment. For example, as illustrated inFIG. 12, the image display apparatus40causes a display42to display a reference screen on which mammography images61to64, mammary gland region schematic drawings65and66, a patient ID region67, and an observation display region68are arranged.

In this situation, for example, the mammography image61is a CC image related to the left breast of the patient. The mammography image62is an MLO image related to the left breast of the patient. The mammography image63is an MLO image related to the right breast of the patient. The mammography image64is a CC image related to the right breast of the patient. Further, the patient ID region67is a region used for displaying the patient ID. The observation display region68is a region used for displaying observation information related to the mammography images of the patient.

Further, the schematic drawing65indicates a mammary gland region of the left breast of the patient. The schematic drawing66indicates a mammary gland region of the right breast of the patient. In this situation, each of the schematic drawings65and66is a schematic drawing of a mammary gland region displayed on the basis of the display information transmitted from the image display apparatus40and, for example, a rectangular mark R indicating the region of interest is displayed in one of the schematic drawings.

Returning to the description ofFIG. 6, the transmitting unit35fis configured to transmit the display information generated by the position specifying unit35eeither to the image display apparatus40or the ultrasound diagnosis apparatus20, in response to an instruction from the operator. More specifically, the transmitting unit35freceives a display information transmission instruction from the operator of either the image display apparatus40or the ultrasound diagnosis apparatus20via the input unit31. Further, when having received the display information transmission instruction, the transmitting unit35freads the display information designated by the operator from the storage34and transmits the read display information to either the image display apparatus40or the ultrasound diagnosis apparatus20.

In this situation, for example, in response to a request from either the image display apparatus40or the ultrasound diagnosis apparatus20, the transmitting unit35fmay transmit the display information generated by the position specifying unit35eto either the image display apparatus40or the ultrasound diagnosis apparatus20. In that situation, for example, the transmitting unit35freceives the request for the display information from either the image display apparatus40or the ultrasound diagnosis apparatus20via the communication controller33. After that, when having received the request for the display information, the transmitting unit35freads the requested display information from the storage34. For example, when the display information is stored in the storage34while being kept in correspondence with the patient ID, the transmitting unit35freceives a patient ID as the request from the image display apparatus40or the ultrasound diagnosis apparatus20, so as to read the display information kept in correspondence with the received patient ID. After that, the transmitting unit35ftransmits the read display information to either the image display apparatus40or the ultrasound diagnosis apparatus20that transmitted the request.

FIG. 13is a flowchart of a processing procedure of a process performed by the image processing apparatus30according to the first embodiment. As illustrated inFIG. 13, when the image processing apparatus30has received an instruction to display mammography images and observation information from the operator (step S101: Yes), the image data obtaining unit35aobtains a mammography image in an MLO direction and a mammography image in a CC direction for each of the left and the right breasts of the patient (steps S102and S103). Further, the observation information generating unit35bgenerates observation information related to the breast of the patient (step S104).

Subsequently, the display controlling unit35ccauses the display32to display a reference screen used for referencing the mammography images (step S105). After that, via the input unit31, the region setting unit35dreceives, from the operator, an operation to designate a desired area in at least one of the MLO-direction mammography images arranged on the reference screen and sets the designated area as a region of interest (step S106). Further, via the input unit31, the region setting unit35dreceives, from the operator, an operation to designate a desired area in at least one of the CC-direction mammography images arranged on the reference screen and sets the designated area as a region of interest (step S107).

Further, when the region of interest has been set in each of the MLO-direction and CC-direction mammography images (step S108: Yes), the position specifying unit35especifies position information of the region of interest in the schematic drawing, on the basis of the pieces of information of the regions of interest that were set in the images and the pieces of information indicating the image taking directions of the images (step S109). After that, the position specifying unit35eoutputs the specified position information of the region of interest in the schematic drawing to, for example, the image display apparatus40, the ultrasound diagnosis apparatus20, or the like (step S110).

Modification Examples of First Embodiment

In the first embodiment described above, the example is explained in which the position specifying unit35especifies the position of the region of interest in the schematic drawing, on the basis of the position information expressed in the apparatus coordinate system of the mammography apparatus that took the mammography images. However, possible embodiments are not limited to this example. For instance, the position specifying unit35emay specify the position information of the region of interest in the schematic drawing by performing a predetermined image processing process on the mammography images.

FIG. 14is a drawing for explaining a process of specifying position information of a region of interest in a schematic drawing performed by the position specifying unit35eaccording to a modification example of the first embodiment. For example, as illustrated inFIG. 14, the position specifying unit35edivides an MLO image into a region on the abdomen side and a region on the head side by using a straight line L—MLOpassing through the nipple. Further, the position specifying unit35esets a region that is positioned parallel to the straight line L—MLOand that has a width to include the region of interest (the circular region indicated with a broken line inFIG. 14) set in the MLO image, as a region R—MLO.

Further, the position specifying unit35edivides a CC image into an inner region (the region corresponding to the region AB) and an outer region (the region corresponding to the region CD) by using a straight line L—CCpassing through the nipple. Further, of the two regions resulting from the division, the position specifying unit35esets a region including the region of interest (the circular region indicated with a broken line inFIG. 14) set in the CC image as a region R—CC.

After that, on the basis of pieces of information of the region R—MLOand the region R—CC, the position specifying unit35especifies position information of the overlapping section between the region R—MLOand the region R—CCin the schematic drawing. Further, the position specifying unit35egenerates display information in which the overlapping section between the region R—MLOand the region R—CCis arranged in a template of the schematic drawing of the mammary gland region, on the basis of the specified position information.

In this situation, when generating the display information by using the template of the schematic drawing, for example, the position specifying unit35emay further display depth information of the region of interest, as position information of the region of interest in the schematic drawing. For example, the position specifying unit35edivides a CC image equally in the Y direction into N sections (where N is an arbitrary integer) and displays the position of the region of interest by using an N-quantile method. In one example, for instance, the position specifying unit35edivides the CC image equally in the Y direction into four sections and, if the region of interest is set in the second section from the opposite side of the chest wall, the position specifying unit35edisplays the position as “two quartiles”.

As explained above, in the first embodiment, the image processing apparatus30sets the regions of interest in the mammography images of the breast of the patient and specifies and outputs the position information of the region of interest in the schematic drawing that schematically expresses the breast, on the basis of the pieces of position information of the regions of interest in the mammography images and the pieces of information indicating the image taking directions of the mammography images. This arrangement makes it possible, for example, for the mammography examination technician to set the regions of interest in the mammography images by using the image processing apparatus30and for the ultrasound examination technician to easily understand, by using either the image display apparatus40or the ultrasound diagnosis apparatus20, the position in the schematic drawing corresponding to the regions of interest set by the mammography examination technician. It is therefore possible to improve the level of precision of the mammary gland image diagnosis process.

FIG. 15is a drawing for explaining a relationship between mammography images and a schematic drawing according to the first embodiment. For example, as illustrated inFIG. 15, there may be some situations where it is difficult to understand from only looking at the mammography images, to which position the regions of interest (the circular regions indicated with broken lines inFIG. 15) set in the MLO and the CC images correspond, when an ultrasound imaging process is performed on the breast. Even in those situations, according to the first embodiment, the mark R indicating the region of interest is displayed in the schematic drawing that schematically expresses the breast, the ultrasound examination technician is able to easily understand the position in the breast that needs to be examined during the ultrasound diagnosis process.

In the first embodiment above, the example is explained in which the region of interest is displayed in the schematic drawing, on the basis of the region of interest set in each of the MLO and the CC images; however, possible embodiments are not limited to this example. For instance, it is known that during an image taking process performed by a mammography apparatus, a blind area may occur due to the curve of the chest wall and a relationship between fixed tissues and movable tissues. When a lesion is present in such a blind area, there may be some situations where no region of interest is set in a mammography image.

FIG. 16is a drawing illustrating a blind area in an MLO image taking process.FIG. 17is a drawing illustrating a blind area in a CC image taking process. For example, as illustrated inFIG. 16, during MLO image taking processes, a blind area301can easily occur in an upper inner part (see the left section ofFIG. 16) and a lower part (see the right section ofFIG. 16) of the breast. Further, for example, as illustrated inFIG. 17, during CC image taking processes, a blind area302can easily occur in an upper part of the breast (see the right section ofFIG. 17) and an outer part positioned closer to the axilla (see the left section ofFIG. 17).

In this situation, for example, if a lesion is present in a blind area occurring during an MLO or CC image taking process, there may be some situations where no region of interest is set in at least one of the MLO and the CC images. For example, as illustrated in the left section ofFIG. 16, if a lesion303is present in the blind area in the upper inner part of the breast, there may be some situations where the lesion303is not rendered in the MLO image and, as a result, no region of interest is set in the MLO image. As another example, as illustrated in the right section ofFIG. 17, if a lesion304is present in the blind area in the upper part of the breast, there may be some situations where the lesion304is not rendered in the CC image, and as a result, no region of interest is set in the CC image.

To cope with this situation, for example, when no region of interest is set in at least one of the MLO and the CC images, it is also acceptable to further cause information indicating the blind area to be displayed in the schematic drawing of the breast. In that situation, for example, the image processing apparatus30is configured so that the position specifying unit35efurther specifies whether a blind area is occurring or not, on the basis of the mammography images. Further, on the basis of the specified result, the position specifying unit35egenerates display information indicating position information of the blind area in the schematic drawing, in addition to the position information of the region of interest in the schematic drawing. Further, for example, on the basis of the display information generated by the position specifying unit35e, either the image display apparatus40or the ultrasound diagnosis apparatus20further outputs the position information of the blind area in the schematic drawing, together with the position information of the region of interest in the schematic drawing.

For example, when specifying the region of interest by using the method indicated inFIGS. 8 to 11, the position specifying unit35eat first detects whether or not a region of interest is set in each of the MLO and the CC images. After that, if a region of interest is set in the CC image, but no region of interest is set in the MLO image, the position specifying unit35especifies that a blind area is occurring in the MLO image. On the contrary, if a region of interest is set in the MLO image, but no region of interest is set in the CC image, the position specifying unit35especifies that a blind area is occurring in the CC image.

After that, on the basis of the specified result, the position specifying unit35egenerates display information in which the position information of the blind area is displayed together with the position information of the region of interest, in the template of the schematic drawing of the mammary gland region.

FIGS. 18 and 19are drawings for explaining a display of a blind area realized by the position specifying unit35eaccording to another modification example of the first embodiment. For example, when having specified that a blind area is occurring in an MLO image, the position specifying unit35edisplays the region A in the schematic drawing illustrated inFIG. 7by using a display mode different from the display mode used for the other regions, as illustrated inFIG. 18. In that situation, for example, the position specifying unit35emay display the region A in a color different from the color of the other regions or with a pattern different from the pattern of the other regions. AlthoughFIG. 18illustrates the example in which the display mode used for the entirety of the region A is arranged to be different, it is also acceptable to, for example, arrange the display mode of only a part of the region A to be different. For example, it is acceptable to arrange the display mode of a part positioned close to an outer circumferential part of the region A to be different.

In this situation, as mentioned above, in the first embodiment, if no region of interest is set in the MLO image, the position specifying unit35especifies a straight line region obtained by projecting the region of interest set in the CC image, as a region of interest in the schematic drawing. As a result, for example, as illustrated inFIG. 18, the straight-line-shaped region of interest R is displayed in the schematic drawing. This type of display, for example, makes it possible for an ultrasound examination technician to determine that it is necessary to perform an ultrasound examination especially carefully on the part corresponding to the region A, within the straight-line-shaped region of interest R.

Further, for example, when having specified that a blind area is occurring in a CC image, the position specifying unit35euses, as illustrated inFIG. 19, a template305of a schematic drawing shaped as a lateral view of the breast, in place of the schematic drawing illustrated inFIG. 7. In this situation, for example, the position specifying unit35euses a schematic drawing of the right breast if the blind area is occurring in the CC image of the right breast and uses a schematic drawing of the left breast if the blind area is occurring in the CC image of the left breast. In that situation, for example, the schematic drawing of the right breast and the schematic drawing of the left breast are defined to have shapes that are symmetrical to each other in the left-and-right direction.

Further, in the template305of the schematic drawing, the position specifying unit35egenerates display information indicating a region expressing a general shape of a blind area expected during CC image taking processes, together with the position information of the region of interest. For example, in the schematic drawing, the position specifying unit35edisplays a region306indicating a blind area occurring in an upper part of the breast by using a display mode different from the display mode used for the other region. In that situation, for example, the position specifying unit35edisplays the region306in a color different from the color of the other region or with a pattern different from the pattern of the other region.

In this situation, as mentioned above, in the first embodiment, if no region of interest is set in the CC image, the position specifying unit35especifies a straight line region obtained by projecting the region of interest set in the MLO image, as a region of interest in the schematic drawing. As a result, for example, as illustrated inFIG. 19, the straight-line-shaped region of interest R is displayed in the schematic drawing. This type of display, for example, makes it possible for an ultrasound examination technician to determine that it is necessary to perform an ultrasound examination especially carefully on the side positioned closer to the upper part of the breast, within the straight-line-shaped region of interest R.

In the description above, the example is explained in which, when no region of interest is set in one of the MLO and the CC images, the position specifying unit35eautomatically displays the position information of the blind area in the schematic drawing. However, possible embodiments are not limited to this example. For instance, the position specifying unit35emay display the region to be displayed as a blind area, on the basis of an instruction from the operator. In that situation, for example, when no region of interest is set in one of the MLO and the CC images, the position specifying unit35ereceives, from the operator, an operation to designate the area of the region to be displayed as the blind area, in the mammography image in which no region of interest is set. After that, on the basis of the area designated by the operator, the position specifying unit35edisplays the blind area in the template of the schematic drawing.

In the description above, the example is explained in which the blind area is displayed in the schematic drawing of the breast when no region of interest is set in at least one of the MLO and the CC images; however, possible embodiments are not limited to this example.

For instance, the position specifying unit35emay specify the position information of the region of interest in the schematic drawing, on the basis of a region of interest set only in one of the MLO and the CC images. In that situation, for example, when specifying the region of interest by using the method illustrated inFIG. 14, the position specifying unit35eat first detects whether or not a region of interest is set in each of the MLO and the CC images. After that, in accordance with the detection result of the region of interest, the position specifying unit35egenerates display information in which the one or more regions of interest are arranged in the schematic drawing.

FIGS. 20 and 21are drawings for explaining a process of specifying position information of a region of interest in a schematic drawing according to yet another modification example of the first embodiment. For example, when a region of interest (the circular region indicated with a broken line inFIG. 20) is set in a CC image, but no region of interest is set in an MLO image, the position specifying unit35edivides the CC image, as illustrated inFIG. 20, into an inner region (the region corresponding to the region AB) and an outer region (the region corresponding to the region CD) by using a straight line L—CCpassing through the nipple and further sets, of the two regions resulting from the division, the region including the region of interest set in the CC image, as a region R—CC. Further, the position specifying unit35especifies position information of the region R—CCin the schematic drawing, on the basis of position information of the region R—CCexpressed in the apparatus coordinate system of the mammography apparatus. After that, the position specifying unit35egenerates display information in which the region R—CCis arranged in a template of the schematic drawing of the mammary gland region, on the basis of the specified position information. For example, when the region of interest is set in the inner region of the CC image, the position specifying unit35edisplays a frame indicating the region R—CCserving as the region of interest, in the region AB of the schematic drawing. As another example, when the region of interest is set in the outer region of the CC image, the position specifying unit35edisplays a frame indicating the region R—CCserving as the region of interest, in the region CD of the schematic drawing.

In another example, when a region of interest (the circular region indicated with a broken line inFIG. 21) is set in an MLO image, but no region of interest is set in a CC image, the position specifying unit35edivides the MLO image, as illustrated inFIG. 21, into an abdomen-side region and a head-side region by using a straight line L—MLOpassing through the nipple. Further, the position specifying unit35esets a region that is positioned parallel to the straight line L—MLOand that has a width to include the region of interest (the circular region indicated with a broken line inFIG. 21) set in the MLO image, as a region R—MLO. Further, the position specifying unit35especifies position information of the region R—MLOin the schematic drawing, on the basis of position information of the region R—MLO. After that, the position specifying unit35egenerates display information in which the region R—MLOis arranged in a template of the schematic drawing of the mammary gland region, on the basis of the specified position information. Alternatively, the position specifying unit35emay set the entirety of a region, either the abdomen-side region or the head-side region, that includes the region of interest set in the MLO image, as the region R—MLO. For example, when the region of interest is set in the abdomen-side region of the MLO image, the position specifying unit35edisplays a frame indicating the region R—MLOserving as the region of interest, in the region DB of the schematic drawing. As another example, when the region of interest is set in the head-side region of the MLO image, the position specifying unit35edisplays a frame indicating the region R—MLOserving as the region of interest, in the region AC of the schematic drawing.

As explained above, even when no region of interest is set in one of the MLO and the CC images, the position specifying unit35especifies the position information of the region of interest in the schematic drawing, on the basis of the one of the mammography images in which the region of interest is set. With this arrangement, even if a blind area is occurring in either the MLO image or the CC image, because the position of the region of interest set in at least one of the mammography images is indicated in the schematic drawing, it is possible to assist the ultrasound examination technician.

Second Embodiment

In the first embodiment above, the example is explained in which the image processing apparatus30sets the one or more regions of interest in the mammography images and specifies the position information of the region of interest in the schematic drawing. In contrast, in a second embodiment, an example will be explained in which the image display apparatus40sets one or more regions of interest in mammography images and specifies the position information of the region of interest in a schematic drawing. The configurations of the apparatuses included in a medical information processing system according to the second embodiment are the same as those illustrated inFIG. 1.

FIG. 22is a diagram of an exemplary configuration of the image display apparatus40according to the second embodiment. As illustrated inFIG. 22, the image display apparatus40includes an input unit41, a display42, a communication controller43, a storage44, and a controller45.

The input unit41is configured to receive inputs of various types of operations and various types of information from an operator. For example, the input unit41may be configured by using a button, a touch panel, and/or a pen tablet.

The display42is configured to display a GUI used for receiving the various types of operations from the operator and various types of images. For example, the display42may be configured by using a liquid crystal display device or a touch panel.

The communication controller43is configured to control communication performed with another apparatus via the network50. For example, the communication controller43may perform wireless communication with the other apparatus by connecting to the network50via a wireless LAN.

The storage44is a storage device such as a hard disk, a semiconductor memory, or the like and is configured to store various types of information therein. More specifically, the storage44includes an image data storage44aand an observation information storage44b.

The image data storage44ais configured to store therein mammography images of the breast of the patient and information indicating the image taking directions of the mammography images. More specifically, similarly to the image data storage34adescribed in the first embodiment, the image data storage44astores therein the mammography images and the pieces of information each of which indicates an image taking direction and is kept in association with a corresponding one of the images. An image data obtaining unit45a(explained later) stores the mammography images into the image data storage44a.

The observation information storage44bis configured to store therein the observation information related to the mammography images of the patient. An observation information obtaining unit45b(explained later) stores the observation information into the observation information storage44b.

The controller45includes processing circuitry such as a CPU and a memory and is configured to control operations of the image display apparatus40by employing the CPU and the memory to execute various types of computer programs. More specifically, the controller45includes the image data obtaining unit45a, the observation information obtaining unit45b, a display controlling unit45c, a region setting unit45d, and a position specifying unit45e.

The image data obtaining unit45ais configured to obtain the mammography images of the breast of the patient and the pieces of information indicating the image taking directions of the mammography images. In this situation, the image data obtaining unit45aobtains a mammography image in an MLO direction and a mammography image in a CC direction for each of the left and the right breasts of the patient.

More specifically, the image data obtaining unit45aobtains the mammography images related to the patient serving as a diagnosis target and the pieces of information indicating the image taking directions of the mammography images, by communicating with the image processing apparatus30via the communication controller43, and further stores the mammography images and the pieces of information indicating the image taking directions that were obtained into the image data storage44a. Alternatively, the image data obtaining unit45amay obtain the mammography images related to the patient serving as the diagnosis target and the pieces of information indicating the image taking directions of the mammography images, by communicating with the mammography apparatus10via the communication controller43.

The observation information obtaining unit45bis configured to obtain the observation information related to the breast of the patient. More specifically, the observation information obtaining unit45bobtains the observation information of the mammography images related to the patient serving as the diagnosis target by communicating with the image processing apparatus30via the communication controller43. Further, the observation information obtaining unit45bstores the obtained observation information into the observation information storage44b.

The display controlling unit45cis configured to cause the display42to display a reference screen used for referencing the mammography images. More specifically, when having received a display request from the operator via the input unit41, the display controlling unit45creads the mammography images related to the patient serving as the diagnosis target from the image data storage44aand reads the observation information related to the patient serving as the diagnosis target from the observation information storage44b. Further, the display controlling unit45ccauses the display42to display the reference screen on which the mammography images and the observation information that were read are arranged.

The region setting unit45dis configured to set a region of interest in at least one of the mammography images. More specifically, by using the same method as the one used by the region setting unit35ddescribed in the first embodiment, the region setting unit45dsets the region of interest in at least one of the mammography images. For example, the region setting unit45dsets a region of interest in each of the MLO and the CC images for each of the left and the right breasts of the patient.

The position specifying unit45eis configured to specify position information of each of the regions of interest in the schematic drawing that schematically expresses the breast, on the basis of pieces of position information of the regions of interest in the mammography images and the pieces of information indicating the image taking directions. More specifically, the position specifying unit45especifies the position information of each of the regions of interest in the schematic drawing by using the same method as the one used by the position specifying unit35edescribed in the first embodiment.

After that, similarly to the position specifying unit35edescribed in the first embodiment, the position specifying unit45egenerates display information in which a region indicating the region of interest is arranged in a template of the schematic drawing of the mammary gland region. After that, for example, the position specifying unit45ecauses the display42to display the reference screen illustrated inFIG. 12, on the basis of the generated display information.

As explained above, in the second embodiment, the image display apparatus40sets the regions of interest in the mammography images of the breast of the patient and specifies and outputs the position of the region of interest in the schematic drawing that schematically expresses the breast, on the basis of the pieces of position information of the regions of interest in the mammography images and the pieces of information indicating the image taking directions of the mammography images. This arrangement makes it possible, for example, for the ultrasound examination technician to easily understand, by using the image display apparatus40, the position in the schematic drawing corresponding to the regions of interest set in the mammography images. It is therefore possible to improve the level of precision of the mammary gland image diagnosis.

Third Embodiment

In the first and the second embodiments described above, the example is explained in which either the image processing apparatus30or the image display apparatus40sets the regions of interest in the mammography images and specifies the position information of the region of interest in the schematic drawing. In contrast, in a third embodiment, an example will be explained in which the ultrasound diagnosis apparatus20sets regions of interest in mammography images and specifies position information of the region of interest in a schematic drawing. The configurations of the apparatuses included in a medical information processing system according to the third embodiment are the same as those illustrated inFIG. 1.

FIG. 23is a diagram of a detailed exemplary configuration of the ultrasound diagnosis apparatus20according to the third embodiment. From among the elements included in the ultrasound diagnosis apparatus20illustrated inFIG. 4,FIG. 23illustrates the input unit22, the display23, the apparatus main body24, the image memory245, the internal storage246, the communication controller247, and the controller248.

As illustrated inFIG. 23, for example, the image memory245includes an image data storage245a. The internal storage246includes an observation information storage246a. Alternatively, the image data storage245amay be included in the internal storage246. Similarly, the observation information storage246amay be included in the image memory245.

The image data storage245ais configured to store therein the mammography images of the breast of the patient. An image data obtaining unit248a(explained later) stores the mammography images into the image data storage245a.

The observation information storage246ais configured to store therein observation information related to the mammography images of the patient. An observation information obtaining unit248b(explained later) stores the observation information into the observation information storage246a.

Further, as illustrated inFIG. 23, for example, the controller248includes the image data obtaining unit248a, the observation information obtaining unit248b, a display controlling unit248c, a region setting unit248d, and a position specifying unit248e.

The image data obtaining unit248ais configured to obtain the mammography images of the breast of the patient. In this situation, the image data obtaining unit248aobtains a mammography image in an MLO direction and a mammography image in a CC direction for each of the left and the right breasts of the patient.

More specifically, the image data obtaining unit248aobtains the mammography images related to the patient serving as a diagnosis target, by communicating with the image processing apparatus30via the communication controller247and stores the obtained mammography images into the image data storage245a. Alternatively, the image data obtaining unit248amay obtain the mammography images related to the patient serving as the diagnosis target, by communicating with the mammography apparatus10via the communication controller247.

The observation information obtaining unit248bis configured to obtain the observation information related to the breast of the patient. More specifically, the observation information obtaining unit248bobtains the observation information of the mammography images related to the patient serving as the diagnosis target, by communicating with the image processing apparatus30via the communication controller247. After that, the observation information obtaining unit248bstores the obtained observation information into the observation information storage246a.

The display controlling unit248cis configured to cause the display23to display a reference screen used for referencing the mammography images. More specifically, when having received a display request from the operator via the input unit22, the display controlling unit248ccauses the display23to display the reference screen on which the mammography images and the observation information are arranged, similarly to the display controlling unit35cdescribed in the first embodiment. In that situation, the display controlling unit248cuses the mammography images of the patient stored in the image data storage245aand the observation information of the patient stored in the observation information storage246a.

The region setting unit248dis configured to set a region of interest in at least one of the mammography images. More specifically, the region setting unit248dsets the region of interest in at least one of the mammography images, by using the same method as the one used by the region setting unit35ddescribed in the first embodiment. For example, the region setting unit248dsets a region of interest in each of the MLO and the CC images for each of the left and the right breasts of the patient.

The position specifying unit248eis configured to specify the position information of each of the regions of interest in the schematic drawing that schematically expresses the breast, on the basis of pieces of position information of the regions of interest in the mammography images and the pieces of information indicating the image taking directions. More specifically, the position specifying unit248especifies the position information of each of the regions of interest in the schematic drawing, by using the same method as the one used by the position specifying unit35edescribed in the first embodiment.

After that, similarly to the position specifying unit35edescribed in the first embodiment, the position specifying unit248egenerates display information in which a region indicating the region of interest is arranged in a template of the schematic drawing of the mammary gland region. After that, for example, the position specifying unit248ecauses the display23to display the same reference screen as the one illustrated inFIG. 12, on the basis of the generated display information.

As explained above, in the third embodiment, the ultrasound diagnosis apparatus20sets the regions of interest in the mammography images of the breast of the patient and specifies and outputs the position information of the region of interest in the schematic drawing that schematically expresses the breast, on the basis of the pieces of position information of the regions of interest in the mammography images and the pieces of information indicating the image taking directions of the mammography images. This arrangement makes it possible, for example, for the ultrasound examination technician to easily understand, by using the ultrasound diagnosis apparatus20, the position in the schematic drawing corresponding to the regions of interest set in the mammography images. It is therefore possible to improve the level of precision of the mammary gland image diagnosis process.

Fourth Embodiment

In the first to the third embodiments above, the example is explained in which the regions of interest are set on the basis of the operation by the operator; however, possible embodiments are not limited to this example. For instance, another arrangement is acceptable in which a mammary gland parenchyma region is extracted from a mammography image, so that a region of interest in the breast is automatically set on the basis of the extracted mammary gland parenchyma region.

Accordingly, in a fourth embodiment, an example will be explained in which an image display apparatus140is configured to extract a mammary gland parenchyma region from a mammography image of the breast of a patient and to specify and display a region of interest in the breast, on the basis of the extracted mammary gland parenchyma region. This arrangement makes it possible for an ultrasound examination technician to perform an ultrasound examination especially carefully on the region in which the density of the mammary gland parenchyma is high within the mammography image. It is therefore possible to improve the level of precision of the mammary gland diagnosis process. In the following sections, the image display apparatus140according to the fourth embodiment will be explained in detail.

FIG. 24is a diagram of an exemplary configuration of the image display apparatus140according to the fourth embodiment. As illustrated inFIG. 24, the image display apparatus140includes an input unit141, a display142, a communication controller143, a storage144, and a controller145.

The input unit141is configured to receive inputs of various types of operations and various types of information from an operator. For example, the input unit141may be configured by using a button, a touch panel, and/or a pen tablet.

The display142is configured to display a GUI used for receiving the various types of operations from the operator and various types of images. For example, the display142may be configured by using a liquid crystal display device or a touch panel.

The communication controller143is configured to control communication performed with another apparatus via the network50. For example, the communication controller143may perform wireless communication with the other apparatus by connecting to the network50via a wireless LAN.

The storage144is a storage device such as a hard disk, a semiconductor memory, or the like and is configured to store various types of information therein. More specifically, the storage144includes an image data storage144aand an observation information storage144b.

The image data storage144ais configured to store therein mammography images of the breast of the patient. An image data obtaining unit145a(explained later) stores the mammography images into the image data storage144a.

The observation information storage144bis configured to store therein the observation information related to the mammography images of the patient. An observation information obtaining unit145b(explained later) stores the observation information into the observation information storage144b.

The controller145includes processing circuitry such as a CPU and a memory and is configured to control operations of the image display apparatus140by employing the CPU and the memory to execute various types of computer programs. More specifically, the controller145includes the image data obtaining unit145a, the observation information obtaining unit145b, a display controlling unit145c, an extracting unit145d, and a specifying unit145e.

The image data obtaining unit145ais configured to obtain the mammography images of the breast of the patient. In this situation, the image data obtaining unit145aobtains a mammography image in an MLO direction and a mammography image in a CC direction for each of the left and the right breasts of the patient.

More specifically, the image data obtaining unit145aobtains the mammography images related to the patient serving as a diagnosis target by communicating with the image processing apparatus30via the communication controller143and stores the obtained mammography images into the image data storage144a. Alternatively, the image data obtaining unit145amay obtain the mammography images related to the patient serving as the diagnosis target, by communicating with the mammography apparatus10via the communication controller143.

The observation information obtaining unit145bis configured to obtain the observation information related to the breast of the patient. More specifically, the observation information obtaining unit145bobtains the observation information of the mammography images related to the patient serving as the diagnosis target by communicating with the image processing apparatus30via the communication controller143. Further, the observation information obtaining unit145bstores the obtained observation information into the observation information storage144b.

The display controlling unit145cis configured to cause the display142to display a reference screen used for referencing the mammography images. More specifically, when having received a display request from the operator via the input unit141, the display controlling unit145creads the mammography images related to the patient serving as the diagnosis target from the image data storage144aand reads the observation information related to the patient serving as the diagnosis target from the observation information storage144b. Further, the display controlling unit145ccauses the display142to display the reference screen on which the mammography images and the observation information that were read are arranged.

FIG. 25is a drawing of an example of the reference screen displayed by the display controlling unit145caccording to the fourth embodiment. For example, as illustrated inFIG. 25, the display controlling unit145ccauses the display142to display a reference screen on which mammography images161to164, mammary gland region schematic drawings165and166, a processing start button167, and an observation display region168are arranged.

In this situation, the mammography image161is a mammography image in a CC direction related to the left breast of the patient. The mammography image162is a mammography image in an MLO direction related to the left breast of the patient. The mammography image163is a mammography image in an MLO direction related to the right breast of the patient. The mammography image164is a mammography image in a CC direction related to the right breast of the patient. Further, the schematic drawing165indicates a mammary gland region of the left breast of the patient. The schematic drawing166indicates a mammary gland region of the right breast of the patient. The processing start button167is a button used for receiving an instruction to start a mammary gland parenchyma region extracting process, from the operator. The observation display region168is a region used for displaying the observation information related to the mammography images of the patient.

Returning to the description ofFIG. 24, the extracting unit145dis configured to extract mammary gland parenchyma regions from the mammography images of the breast of the patient. Further, the extracting unit145dis configured to display the extracted mammary gland parenchyma regions so as to be superimposed on the mammography images displayed on the reference screen by the display controlling unit145c. In this situation, the extracting unit145dextracts a mammary gland parenchyma region from each of the MLO-direction and CC-direction mammography images for each of the left and the right breasts of the patient.

More specifically, when the operator of the image display apparatus140has pressed the processing start button167on the reference screen, the extracting unit145dreads the mammography images related to the patient serving as the diagnosis target from the image data storage144aand extracts the mammary gland parenchyma regions from the read mammography images. In this situation, the extracting unit145dmay extract, in advance, the mammary gland parenchyma regions from the mammography images at the point in time when the image data obtaining unit145ahas obtained the mammography images of the patient.

For example, the extracting unit145dextracts the mammary gland parenchyma regions by performing a threshold process on the basis of a distribution of brightness values in the mammography image.

FIGS. 26 and 27are drawings for explaining a process of extracting a mammary gland parenchyma region performed by the extracting unit145daccording to the fourth embodiment. Generally speaking, it is known that mammary gland parenchyma and the pectoralis major muscle are rendered in white, whereas fat is rendered in black in mammography images, because mammary gland parenchyma, the pectoralis major muscle, and fat have mutually-different X-ray transmission coefficients. For this reason, by analyzing the characteristics of a distribution of brightness values in a mammography image by using a histogram of the brightness values, it is possible to extract a region that is presumed to be mammary gland parenchyma.

For example, as illustrated inFIG. 26, the extracting unit145ddivides the brightness values of the pixels included in the mammography image into three ranges, by using a threshold value A for separating brightness values of fat from brightness values of mammary gland parenchyma and a threshold value B for separating brightness values of mammary gland parenchyma and brightness values of the chest wall. In this manner, the extracting unit145dextracts a region in which the density of mammary gland parenchyma is high as the mammary gland parenchyma region, by using the threshold value A indicating a lower limit and the threshold value B indicating an upper limit with respect to the brightness values corresponding to the mammary gland parenchyma.

Further, for example, the extracting unit145dassigns a pixel value “0” to each of the pixels of which the brightness values are equal to or smaller than the threshold value A, assigns a pixel value “1” to each of the pixels of which the brightness values are larger than the threshold value A but are equal to or smaller than the threshold value B, and assigns a pixel value “2” to each of the pixels of which the brightness values are larger than the threshold value B. As a result, as illustrated in the left and the middle sections ofFIG. 27, the mammography image serving as a processing target is three-valued and is separated into a region171corresponding to the pixel value “O”, a region172corresponding to the pixel value “1”, and a region173corresponding to the pixel value “2”. In this situation, the region171is a region representing the fat, whereas the region172is a region representing the mammary gland parenchyma, and the region173is a region representing the chest wall. After that, the extracting unit145dextracts the mammary gland parenchyma region, by extracting the region172corresponding to the pixel value “1”. In this situation, the extracting unit145dmay extract the outline (the boundary) of the region corresponding to the pixel value “1”, as the mammary gland parenchyma region. After that, as illustrated in the right section ofFIG. 27, the extracting unit145ddisplays the extracted mammary gland parenchyma region so as to be superimposed on the mammography image displayed on the reference screen by the display controlling unit145c.

Alternatively, the extracting unit145dmay extract the mammary gland parenchyma region by using any other edge detecting method. For example, the extracting unit145dmay extract the mammary gland parenchyma region by using a derivative edge detecting method, on the basis of the distribution of the brightness values in the mammography image. The derivative edge detecting method is a method by which the zero crossing of a second derivative in the gradient direction of a gradient in brightness values is detected with respect to a change in the brightness values. By using this method, it is possible to calculate the position of an edge with an accuracy of subpixels.

Alternatively, the extracting unit145dmay extract the mammary gland parenchyma region by using a Sobel operator on the basis of the distribution of the brightness values in the mammography image. The Sobel operator is used in a method for calculating the intensity (the derivative) of a gradient in brightness values by performing a local multiply-accumulation operation with respect to a change in the brightness values. For example, the Sobel operator uses coefficient matrices fxand fypresented in Expression (4) below, where fxis a coefficient matrix for detecting an edge in the column direction, whereas fyis a coefficient matrix for detecting an edge in the row direction.

Further, with the use of Expression (5) presented below, the intensity |Λf| of the gradient in the brightness values is calculated.
|Λf|=√{square root over (fx2+fy2)}  (5)

Furthermore, with the use of Expression (6) presented below, the direction θ of the edge is calculated.
θ=tan−1(fx/fy)  (6)

Returning to the description ofFIG. 24, the specifying unit145eis configured to specify a region of interest in the breast of the patient, on the basis of the mammary gland parenchyma regions extracted by the extracting unit145d. Further, the specifying unit145eis configured to display the specified region of interest in a schematic drawing of the mammary gland region displayed on the reference screen by the display controlling unit145c. In this situation, the specifying unit145especifies the region of interest in the breast, on the basis of the mammary gland parenchyma region extracted by the extracting unit145dfrom each of the MLO-direction and CC-direction mammography images, for each of the left and the right breasts of the patient.

For example, the specifying unit145especifies an overlapping section between the region set by the operator on the basis of the mammary gland parenchyma region extracted from the MLO-direction mammography image and the region set by the operator on the basis of the mammary gland parenchyma region extracted from the CC-direction mammography image, as the region of interest. By specifying the overlapping section between the regions set in the mammography images in the mutually-different directions as the region of interest in this manner, it is possible to more accurately indicate the region on which the ultrasound examination technician should perform an ultrasound examination especially carefully.

FIG. 28is a drawing for explaining a process of specifying the region of interest performed by the specifying unit145eaccording to the fourth embodiment. Although an example will be explained below in which a region of interest in the right breast of the patient is specified, it is also possible to specify a region of interest in the left breast by using the same method.

As illustrated in the top left section ofFIG. 28, on the reference screen, a mammary gland parenchyma region181extracted by the extracting unit145dfrom an MLO-direction mammography image163is displayed in the MLO-direction mammography image. Further, as illustrated in the bottom left section ofFIG. 28, on the reference screen, a mammary gland parenchyma region182extracted by the extracting unit145dfrom a CC-direction mammography image164is displayed in the CC-direction mammography image.

When the mammary gland parenchyma regions are displayed by the extracting unit145d, the specifying unit145ereceives, from the operator, an operation to set a region in a schematic drawing166of the mammary gland region via the input unit141. More specifically, as illustrated in the top middle section ofFIG. 28, the specifying unit145ereceives the operation performed by the operator to set a region183in the schematic drawing166of the mammary gland region, on the basis of the mammary gland parenchyma region181displayed in the MLO-direction mammography image163. Further, as illustrated in the bottom middle section ofFIG. 28, the specifying unit145ereceives the operation performed by the operator to set a region184in the schematic drawing166of the mammary gland region, on the basis of the mammary gland parenchyma region182displayed in the CC-direction mammography image164. After that, when the two regions183and184have been set by the operator, the specifying unit145especifies the overlapping section of the regions, as a region of interest185, as illustrated in the right section ofFIG. 28.

In this situation, for example, the specifying unit145emay automatically set a region of interest, instead of receiving the setting of a region of interest from the operator. For example, the specifying unit145emay specify the region of interest by setting the length of the region of interest in one direction on the basis of the size of the mammary gland parenchyma region extracted from the MLO-direction mammography image and setting the length of the region of interest in the other direction on the basis of the size of the mammary gland parenchyma region extracted from the CC-direction mammography image.

FIG. 29is a drawing for explaining the other example of the process of specifying a region of interest performed by the specifying unit145eaccording to the fourth embodiment. For example, as illustrated in the upper section ofFIG. 29, the specifying unit145esets correspondence relationships between the positions of the regions in the mammography image and the positions of the regions in the schematic drawing of the mammary gland region, in advance. Further, as illustrated in the lower section ofFIG. 29, the specifying unit145ecalculates the length x of the mammary gland parenchyma region in the inner-outer direction in the CC-direction mammography image and sets the length x′ of the region of interest in the inner-outer direction, in accordance with the calculated length x. In this situation, the specifying unit145esets the length y′ of the region of interest in the upper-lower direction, in accordance with the length of the mammary gland parenchyma region in the upper-lower direction in the MLO-direction mammography image.

FIG. 30is a flowchart of a processing procedure of a process performed by the image display apparatus140according to the fourth embodiment. As illustrate inFIG. 30, when the image display apparatus140has received an instruction to display mammography images and observation information from the operator (step S201: Yes), the image data obtaining unit145aobtains a mammography image in an MLO direction and a mammography image in a CC direction for each of the left and the right breasts of the patient (steps S202and S203). Further, the observation information obtaining unit145bobtains observation information related to the breast of the patient (step S204).

Subsequently, the display controlling unit145ccauses the display142to display a reference screen used for referencing the mammography images (step S205). After that, when the operator has pressed the processing start button167on the reference screen (step S206: Yes), the extracting unit145dextracts a mammary gland parenchyma region from each of the MLO-direction and CC-direction mammography images of the patient (steps S207and S208).

Subsequently, on the basis of the mammary gland parenchyma region extracted by the extracting unit145dfrom each of the MLO-direction and CC-direction mammography images, the specifying unit145especifies a region of interest in the breast (step S209). Further, the specifying unit145ecauses the specified region of interest to be displayed on the reference screen displayed by the display controlling unit145c(step S210).

As explained above, in the fourth embodiment, the image display apparatus140extracts the mammary gland parenchyma regions from the mammography images of the breast of the patient and specifies and displays the region of interest in the breast, on the basis of the extracted mammary gland parenchyma regions. This arrangement makes it possible for the ultrasound examination technician to perform an ultrasound examination especially carefully on the region in which the density of the mammary gland parenchyma is high within the mammography image. It is therefore possible to improve the level of precision of the mammary gland image diagnosis process.

Fifth Embodiment

In the fourth embodiment above, the example is explained in which the image display apparatus140specifies the region of interest by extracting the mammary gland parenchyma regions from the mammography images of the patient and displays the specified region of interest. In contrast, in a fifth embodiment, an example will be explained in which the image processing apparatus130specifies a region of interest by extracting mammary gland parenchyma regions from mammography images of a patient, whereas the image display apparatus40displays the region of interest specified by the image processing apparatus130. The configurations of the apparatuses included in a medical information processing system according to the fifth embodiment are the same as those illustrated inFIG. 1.

FIG. 31is a diagram of an exemplary configuration of the image processing apparatus130according to the fifth embodiment. As illustrated inFIG. 31, the image processing apparatus130includes an input unit131, a display132, a communication controller133, a storage134, and a controller135.

The input unit131is configured to receive inputs of various types of operations and various types of information from the operator. For example, the input unit131may be configured by using a keyboard, a mouse, a button, a trackball, and/or a touch panel.

The display132is configured to display a GUI used for receiving the various types of operations from the operator and various types of images. For example, the display132may be configured by using a liquid crystal display device, a Cathode Ray Tube (CRT) display device, or a touch panel.

The communication controller133is configured to control communication performed with another apparatus via the network50. For example, the communication controller133may be configured by using a network card or a network adaptor and may perform the communication with the other apparatus by connecting to the network50via an Ethernet (registered trademark) LAN. Further, for example, the communication controller133may perform wireless communication with the other apparatus by connecting to the network50via a wireless LAN.

The storage134is a storage device such as a hard disk, a semiconductor memory, or the like and is configured to store various types of information therein. More specifically, the storage134includes an image data storage134aand an observation information storage134b.

The image data storage134ais configured to store therein mammography images of the breast of the patient. An image data obtaining unit135a(explained later) stores the mammography images into the image data storage134a.

The observation information storage134bis configured to store therein the observation information related to the mammography images of the patient. An observation information generating unit135b(explained later) stores the observation information into the observation information storage134b.

The controller135includes processing circuitry such as a CPU and a memory and is configured to control operations of the image processing apparatus130by employing the CPU and the memory to execute various types of computer programs. More specifically, the controller135includes the image data obtaining unit135a, the observation information generating unit135b, a display controlling unit135c, an extracting unit135d, a specifying unit135e, and a transmitting unit135f.

The image data obtaining unit135ais configured to obtain the mammography images of the breast of the patient. In this situation, the image data obtaining unit135aobtains a mammography image in an MLO direction and a mammography image in a CC direction for each of the left and the right breasts of the patient. More specifically, the image data obtaining unit135aobtains the mammography images related to the patient serving as a diagnosis target by communicating with the mammography apparatus10via the communication controller133.

The observation information generating unit135bis configured to generate the observation information related to the mammography images of the patient, on the basis of an observation input by the operator. More specifically, the observation information generating unit135breceives an input of the observation related to the mammography images from a mammography examination technician via the input unit131. Further, the observation information generating unit135bgenerates the observation information indicating the received observation and stores the generated observation information into the observation information storage134b.

The display controlling unit135cis configured to cause the display132to display a reference screen used for referencing the mammography images. More specifically, when having received a display request from the operator via the input unit131, the display controlling unit135ccauses the display132to display the reference screen on which the mammography images and the observation information are arranged, similarly to the display controlling unit145cdescribed in the fourth embodiment. In that situation, the display controlling unit135cuses the mammography images of the patient stored in the image data storage134aand the observation information of the patient stored in the observation information storage134b.

The extracting unit135dis configured to extract mammary gland parenchyma regions from the mammography images of the breast of the patient. Further, the extracting unit135dis configured to display the extracted mammary gland parenchyma regions so as to be superimposed on the mammography images displayed on the reference screen by the display controlling unit135c. In this situation, the extracting unit135dextracts a mammary gland parenchyma region from each of the MLO-direction and CC-direction mammography images for each of the left and the right breasts of the patient.

More specifically, when the operator of the image processing apparatus130has pressed the processing start button on the reference screen, the extracting unit135dreads the mammography images related to the patient serving as the diagnosis target from the image data storage134aand extracts the mammary gland parenchyma regions from the read mammography images, by using the same method as the one used by the extracting unit145ddescribed in the fourth embodiment. In this situation, the extracting unit135dmay extract, in advance, the mammary gland parenchyma regions from the mammography images at the point in time when the image data obtaining unit135ahas obtained the mammography images of the patient.

The specifying unit135eis configured to specify a region of interest in the breast of the patient, on the basis of the mammary gland parenchyma regions extracted by the extracting unit135d. Further, the specifying unit135eis configured to display the specified region of interest in a schematic drawing of the mammary gland region displayed on the reference screen by the display controlling unit135c. In this situation, the specifying unit135especifies the region of interest in the breast, on the basis of the mammary gland parenchyma region extracted by the extracting unit135dfrom each of the MLO-direction and CC-direction mammography images, for each of the left and the right breasts of the patient.

More specifically, by using the same method as the one used by the specifying unit145edescribed in the fourth embodiment, the specifying unit135especifies the region of interest in the breast of the patient on the basis of the mammary gland parenchyma regions extracted by the extracting unit135dand displays the specified region of interest on the reference screen.

In response to an instruction from the operator, the transmitting unit135fis configured to transmit information indicating the region of interest specified by the specifying unit135e, to the image display apparatus40. More specifically, the transmitting unit135freceives an instruction to transmit the display information, from the operator of the image processing apparatus130via the input unit131. When having received the instruction to transmit the display information, the transmitting unit135fgenerates the display information for displaying a screen including the same information as that on the reference screen after the region of interest has been specified by the specifying unit135eand transmits the generated display information to the image display apparatus40. For example, the transmitting unit135fgenerates the display information for displaying, on the reference screen displayed by the display controlling unit135c, a screen on which the mammary gland parenchyma regions extracted by the extracting unit135dare superimposed on the mammography images and on which the region of interest specified by the specifying unit135eis indicated in the schematic drawing of the mammary gland region, and transmits the generated display information to the image display apparatus40.

In this situation, for example, in response to a request from the image display apparatus40, the transmitting unit135fmay transmit information indicating the region of interest specified by the specifying unit135eto the image display apparatus40. In that situation, for example, the transmitting unit135freceives a request for the display information from the image display apparatus40via the communication controller133. Further, when having received the request for the display information, the transmitting unit135fgenerates the display information described above and transmits the display information to the image display apparatus40that transmitted the request.

As explained above, in the fifth embodiment, the image processing apparatus130extracts the mammary gland parenchyma regions from the mammography images of the breast of the patient and specifies the region of interest in the breast on the basis of the extracted mammary gland parenchyma regions. Further, the image display apparatus40displays the information indicating the region of interest specified by the image processing apparatus130. For example, this arrangement makes it possible for the mammography examination technician to specify the region of interest by using the image processing apparatus130and for the ultrasound examination technician to perform an ultrasound examination while using the image display apparatus40, especially carefully on the region of interest specified by the mammography examination technician. As a result of the mammography examination technician specifying the region of interest in this manner, it is possible to further improve the level of precision of the mammary gland image diagnosis.

In the fifth embodiment, the example is explained in which the extracting unit135dextracts the mammary gland parenchyma regions from the mammography images, when the operator of the image processing apparatus130has pressed the processing start button on the reference screen. However, for example, another arrangement is acceptable in which the extracting unit135dextracts the mammary gland parenchyma regions from the mammography images, when having received a request to start the processing from the image display apparatus40. In that situation, for example, in response to a request from the image display apparatus40, the display controlling unit135ctransmits, in advance, the information for displaying the reference screen on which the mammography images and the observation information are arranged, to the image display apparatus40. Further, when the operator of the image display apparatus40has instructed to start the processing via the reference screen, the extracting unit135dreceives the request to start the processing from the image display apparatus40via the network50.

Sixth Embodiment

In the first and the fifth embodiments above, the example is explained in which the image display apparatus40displays the region of interest in the breast of the patient. In contrast, in a sixth embodiment, an example will be explained in which the ultrasound diagnosis apparatus20extracts mammary gland parenchyma regions from the mammography images of the patient, specifies a region of interest, and displays the specified region of interest. The configurations of the apparatuses included in a medical information processing system according to the sixth embodiment are the same as those illustrated inFIG. 1.

FIG. 32is a diagram of a detailed exemplary configuration of the ultrasound diagnosis apparatus20according to the sixth embodiment. From among the elements included in the ultrasound diagnosis apparatus20illustrated inFIG. 4,FIG. 32illustrates an input unit122, a display123, an apparatus main body124, an image memory1245, an internal storage1246, a communication controller1247, and a controller1248.

As illustrated inFIG. 32, for example, the image memory1245includes an image data storage1245a, whereas the internal storage1246includes an observation information storage1246a. Alternatively, the image data storage1245amay be included in the internal storage1246. Similarly, the observation information storage1246amay be included in the image memory1245.

The image data storage1245ais configured to store therein the mammography images of the breast of the patient. An image data obtaining unit1248a(explained later) stores the mammography images into the image data storage1245a.

The observation information storage1246ais configured to store therein observation information related to the mammography images of the patient. An observation information obtaining unit1248b(explained later) stores the observation information into the observation information storage1246a.

Further, as illustrated inFIG. 32, for example, the controller1248includes the image data obtaining unit1248a, the observation information obtaining unit1248b, a display controlling unit1248c, an extracting unit1248d, and a specifying unit1248e.

The image data obtaining unit1248ais configured to obtain the mammography images of the breast of the patient. In this situation, the image data obtaining unit1248aobtains a mammography image in an MLO direction and a mammography image in a CC direction for each of the left and the right breasts of the patient.

More specifically, the image data obtaining unit1248aobtains the mammography images related to the patient serving as a diagnosis target, by communicating with the image processing apparatus30via the communication controller1247and stores the obtained mammography images into the image data storage1245a. Alternatively, the image data obtaining unit1248amay obtain the mammography images related to the patient serving as the diagnosis target, by communicating with the mammography apparatus10via the communication controller1247.

The observation information obtaining unit1248bis configured to obtain the observation information related to the breast of the patient. More specifically, the observation information obtaining unit1248bobtains the observation information of the mammography images related to the patient serving as the diagnosis target, by communicating with the image processing apparatus30via the communication controller1247. After that, the observation information obtaining unit1248bstores the obtained observation information into the observation information storage1246a.

The display controlling unit1248cis configured to cause the display123to display a reference screen used for referencing the mammography images. More specifically, when having received a display request from the operator via the input unit122, the display controlling unit1248ccauses the display123to display the reference screen on which the mammography images and the observation information are arranged, similarly to the display controlling unit145cdescribed in the fourth embodiment. In that situation, the display controlling unit1248cuses the mammography images of the patient stored in the image data storage1245aand the observation information of the patient stored in the observation information storage1246a.

The extracting unit1248dis configured to extract mammary gland parenchyma regions from the mammography images of the breast of the patient. Further, the extracting unit1248dis configured to display the extracted mammary gland parenchyma regions so as to be superimposed on the mammography images displayed on the reference screen by the display controlling unit1248c. In this situation, the extracting unit1248dextracts a mammary gland parenchyma region from each of the MLO-direction and CC-direction mammography images for each of the left and the right breasts of the patient.

More specifically, when the operator has pressed the processing start button on the reference screen, the extracting unit1248dreads the mammography images related to the patient serving as the diagnosis target from the image data storage1245aand extracts the mammary gland parenchyma regions from the read mammography images, by using the same method as the one used by the extracting unit145ddescribed in the fourth embodiment. In this situation, the extracting unit1248dmay extract, in advance, the mammary gland parenchyma regions from the mammography images at the point in time when the image data obtaining unit1248ahas obtained the mammography images of the patient.

The specifying unit1248eis configured to specify a region of interest in the breast of the patient, on the basis of the mammary gland parenchyma regions extracted by the extracting unit1248d. Further, the specifying unit1248eis configured to display the specified region of interest in a schematic drawing of the mammary gland region displayed on the reference screen by the display controlling unit1248c. In this situation, the specifying unit1248especifies the region of interest in the breast, on the basis of the mammary gland parenchyma region extracted by the extracting unit1248dfrom each of the MLO-direction and CC-direction mammography images, for each of the left and the right breasts of the patient.

More specifically, by using the same method as the one used by the specifying unit145edescribed in the fourth embodiment, the specifying unit1248especifies the region of interest in the breast of the patient, on the basis of the mammary gland parenchyma regions extracted by the extracting unit1248dand displays the specified region of interest on the reference screen.

As explained above, in the sixth embodiment, the ultrasound diagnosis apparatus20extracts the mammary gland parenchyma regions from the mammography images of the breast of the patient and specifies and displays the region of interest in the breast on the basis of the extracted mammary gland parenchyma regions. With this arrangement, without using the image display apparatus40, it is possible to indicate, to an ultrasound examination technician, the region on which the examination should be performed especially carefully, via the ultrasound diagnosis apparatus20.

In the sixth embodiment, the example is explained in which the ultrasound diagnosis apparatus20extracts the mammary gland parenchyma regions from the mammography images, specifies the region of interest, and displays the specified region of interest. However, for example, the image processing apparatus30may extract the mammary gland parenchyma regions from the mammography images and specify the region of interest, as described in the fifth embodiment.

In that situation, for example, in the ultrasound diagnosis apparatus20, the communication controller1247receives, from the image processing apparatus30, the mammography images of the breast of the patient and information of the regions of interest in the mammography images via the network50. Further, the image processor244generates ultrasound images on the basis of the data acquired by the ultrasound probe21. Further, the display controlling unit1248ccauses the display123to display the mammography images received by the communication controller1247, a region of interest, and the ultrasound images generated by the image processor244.

In the first to the sixth embodiments above, the example is explained in which the mammography images taken by the mammography apparatus10are used; however, possible embodiments are not limited to this example. For instance, images taken by a screen film system may be used in place of the mammography images.

Further, in the first to the sixth embodiments above, the example is explained in which the MLO-direction and CC-direction mammography images are used; however, possible embodiments are not limited to this example. For instance, it is acceptable to use mammography images taken in directions other than the MLO and the CC directions.

It is also possible to realize the functions of the controller35included in the image processing apparatus30, the functions of the controller45included in the image display apparatus40, and the functions of the controller248included in the ultrasound diagnosis apparatus20described in the embodiments above, by using software. For example, the functions of the controlling units may be realized by causing a computer to execute a medical information processing program that defines the procedures of the processes explained as being performed by the controlling units. In that situation, the medical information processing program is, for example, stored in a hard disk, a semiconductor memory device, or the like, so as to be read and executed by processing circuitry such as a processor such as a CPU, an MPU, or the like. Further, the medical information processing program may be distributed as being recorded on a computer-readable recording medium such as a Compact Disk Read-Only Memory (CD-ROM), a Magnetic Optical (MO) disk, or a Digital Versatile Disk (DVD).

According to at least one aspect of the embodiments described above, it is possible to improve the level of precision of the mammary gland image diagnosis process.