Bone suppression for chest radiographs using deep learning

A system and method for generating a rib suppressed radiographic image using deep learning computation. The method includes using a convolutional neural network module trained with pairs of a chest x-ray image and its counterpart bone suppressed image. The bone suppressed image is obtained using a bone suppression algorithm applied to the chest x-ray image. The convolutional neural network module is then applied to a chest x-ray image or the bone suppressed image to generate an enhanced bone suppressed image.

This application is related in certain respects to U.S. Pat. No. 8,913,817, issued Dec. 16, 2014, in the name of Huo et al., entitled RIB SUPPRESSION IN RADIOGRAPHIC IMAGES; U.S. Pat. No. 9,269,139, issued Feb. 23, 2016, in the name of Huo et al., entitled RIB SUPPRESSION IN RADIOGRAPHIC IMAGES; U.S. Pat. No. 9,659,390, issued May 23, 2017, in the name of Huo et al., entitled TOMOSYNTHESIS RECONSTRUCTION WITH RIB SUPPRESSION; U.S. Pat. No. 9,668,712, issued Jun. 6, 2017, in the name of Foos et al., entitled METHOD AND SYSTEM FOR QUANTITATIVE IMAGING; and U.S. Pat. No. 8,961,011, issued Feb. 24, 2015, in the name of Lalena, entitled MOBILE RADIOGRAPHY UNIT HAVING MULTIPLE MONITORS; all of which are hereby incorporated by reference as if fully set forth herein in their entirety.

BACKGROUND OF THE INVENTION

The disclosure relates generally to the field of medical imaging, and in particular to bone suppression for chest radiographs. More specifically, the disclosure relates to a method of bone suppression for chest radiographs using deep learning convolutional networks, and program modules for executing such deep learning convolutional algorithms.

BRIEF DESCRIPTION OF THE INVENTION

A system and method for generating a rib suppressed radiographic image using deep learning computation. The method includes using a convolutional neural network module trained with pairs of a chest x-ray image and its counterpart bone suppressed image, or, in other words, pairs of radiographic images including a starting image and a target image. The bone suppressed image is obtained using a known bone suppression algorithm applied to the chest x-ray image. A known convolutional neural network module trained to suppress rib content is then applied to a chest x-ray image or the bone suppressed image to generate an enhanced bone suppressed chest x-ray image.

In one embodiment, a system includes an x-ray imaging system to capture a radiographic image. A processor is configured to apply a bone suppression algorithm to the captured radiographic image to generate a bone suppressed version of the captured radiographic image. A trained deep learning module is trained by both the captured radiographic image and the bone suppressed version of the captured radiographic image to generate an enhanced bone suppressed image of the radiographic image.

In one embodiment, a system includes electronic memory for storing a radiographic image. A processor is configured to apply a bone suppression algorithm to the stored radiographic image to generate a bone suppressed version of the stored radiographic image. A deep learning module receives the stored radiographic image and the bone suppressed version of the stored radiographic image. The deep learning module is configured to be trained on the images to generate an enhanced bone suppressed image using the stored radiographic image.

In one embodiment, a system includes an x-ray imaging system to capture a current radiographic image. A deep learning module trained on a plurality of previously captured radiographic images and a corresponding plurality of bone-suppressed radiographic images generated from the plurality of previously captured radiographic images. A processor of the system is configured to apply the trained deep learning module to the captured current radiographic image to generate an enhanced bone-suppressed radiographic image.

In one embodiment, a method includes receiving a digital radiographic image and applying a bone suppression algorithm to the received digital radiographic image to generate a digital bone suppressed radiographic image. A deep learning neural network module trained for suppressing bone regions of digital radiographic images is accessed to generate an enhanced digital bone suppressed radiographic image using the received digital radiographic image.

In one embodiment, a method includes receiving a digital radiographic image and applying a bone suppression algorithm to the received digital radiographic image to generate a digital bone suppressed radiographic image. A deep learning neural network module trained for suppressing bone regions of digital radiographic images is accessed to generate an enhanced digital bone suppressed radiographic image using the generated digital bone suppressed radiographic image.

In one embodiment, a computer implemented method includes acquiring a digital radiographic image, applying a bone suppression algorithm to the acquired radiographic image to generate a bone suppressed radiographic image. A convolutional neural network module trained using a plurality of radiographic training images is applied to the bone suppressed radiographic image or the acquired digital radiographic image to generate an enhanced bone suppressed radiographic image.

In at least one arrangement, there is provided an x-ray imaging system to capture a medical image, a deep learning module, and a processor applying a bone suppression algorithm to the captured medical image, and applying the deep learning module to the bone suppressed captured medical image to generate an enhanced bone suppressed image.

In at least one arrangement, there is provided an x-ray imaging system to capture a medical image, a deep learning module trained using a plurality of medical images and a plurality of bone-suppressed images. A processor applies the trained deep learning module to the captured medical image to generate an enhanced medical image.

In at least one arrangement, there is provided an x-ray imaging system to capture a medical image, a deep learning module trained using a plurality of medical images and a plurality of bone-suppressed images derived from at least some of the plurality of medical images, and a processor to apply the trained deep learning module to the captured medical image to generate an enhanced medical image.

In at least one arrangement, there is provided a method including the steps of acquiring a digital medical image using an x-ray projection imaging system, applying a bone suppression algorithm to the medical image to generate a bone suppressed image, providing a deep learning convolutional neural network module trained using a plurality of medical images, applying the neural network module to the bone suppressed image to generate an enhanced bone suppressed image, and displaying, storing, or transmitting the enhanced bone suppressed image.

In at least one arrangement, there is provided a method including the steps of acquiring a digital medical image using an x-ray imaging system, applying a bone suppression algorithm to the medical image to generate a bone suppressed image, providing a plurality of training images, providing a convolutional neural network module trained using at least some of the plurality of training images, applying the convolutional neural network module to a bone suppressed image to generate an enhanced bone suppressed image, and displaying, storing, or transmitting the enhanced bone suppressed image.

In at least one method the steps include providing a plurality of training images including providing a plurality of digital medical images and providing a plurality of bone-suppressed images derived from at least some of the plurality of digital medical images.

In at least one method, providing the plurality of training images includes providing a plurality of chest x-ray images and a plurality of rib suppressed chest x-ray images.

In at least one method, providing the plurality of training images includes providing (i) a plurality of chest x-ray images and (ii) a plurality of bone suppressed chest x-ray images derived from the plurality of chest x-ray images.

DETAILED DESCRIPTION OF THE EMBODIMENTS

FIG. 1is a schematic diagram of an exemplary digital radiographic (DR) imaging system100that may be used to practice embodiments of the present invention disclosed herein. The DR imaging system100includes an x-ray source102configured to generate radiographic energy. The x-ray source102may include a single x-ray source or it may include multiple x-ray sources. The imaging system100further includes a generally planar DR detector104, although a curved detector may also be used. A computer system106having a processor, electronic memory, and digital monitor configured to display images captured by the DR detector104is electrically connected to the x-ray source102and DR detector104. The DR detector104may include a two dimensional array of photosensor cells arranged in electronically addressable rows and columns. The DR detector104may be positioned to receive x-rays108passing through an object110, such as a human patient, during radiographic energy exposures, or radiographic firing of energy pulses, as emitted by the x-ray source102during an imaging procedure. As shown inFIG. 1, the radiographic imaging system100may use an x-ray source102that emits collimated x-rays108, e.g. a directed x-ray beam109such as a cone beam having a field of view, selectively aimed at and passing through a preselected region111of the object110. The x-ray beam109may be attenuated by varying degrees along its plurality of rays according to the internal 3D structure of the object110, whereby the attenuated rays are detected by the array of photosensitive cells in DR detector104. The DR detector104is positioned, as much as possible, in a perpendicular relation to a substantially central path112of the plurality of rays108emitted by the x-ray source102. The array of photosensitive cells (pixels) of DR detector104may be electronically read out (scanned) by their position according to column and row. As used herein, the terms “column” and “row” refer to the vertical and horizontal arrangement of the photosensor cells and, for clarity of description, it will be assumed that the rows extend horizontally and the columns extend vertically. However, the orientation of the columns and rows is arbitrary and does not limit the scope of any embodiments disclosed herein. Furthermore, the term “object” may be illustrated as a human patient in the description ofFIG. 1, however, an object of a DR imaging system100or100, as the term is used herein, may be a human or an animal.

In one exemplary embodiment, the photosensitive cells of DR detector104may be read out to capture one or a plurality of projection images under control of a detector control circuit107so that the exposure data (digital images) from the array of detector104may be transmitted to the computer system106. Each photosensitive cell in the DR detector104may independently detect and store an attenuation value which is generated by a charge level generated in proportion to an intensity, or energy level, of the attenuated radiographic radiation, or x-rays, received and absorbed in the photosensitive cells. Thus, each photosensitive cell, when read-out, provides information, or an attenuation value, defining a pixel of a radiographic image that may be digitally decoded by image processing electronics in the computer system106and displayed by the monitor for viewing by a user. Image processing electronics may be included within the DR detector104housing, whereby the radiographic images may be relayed to a computer system106by cable or wirelessly via electromagnetic wave transmission. As shown inFIG. 1, the source102and DR detector104may be affixed to an exemplary stationary C-arm101, or a rotating mechanism controlling such C-arm101and configured to revolve the source102and detector104in either of the angular directions indicated by the arrow103about an imaging axis z that coincides with the object110while the DR detector captures a plurality of radiographic projection images of the object110at a number of angular imaging positions as the C-arm rotates about the object110.

The computer system106includes a processor and electronic memory and may communicate with a detector control circuit107and x-ray generator104over a connected cable (wired) or, as described herein, the DR detector104may be equipped with a wireless transmitter to transmit radiographic image data wirelessly to the computer system106for image processing therein. The detector control107may also include a processor and electronic memory (not shown) to control operations of the DR detector104as described herein, or such control operations may be implemented using the computer system106by use of programmed instructions. Programmed instructions stored in memory accessible to computer system106may be executed to perform the reconstruction algorithms described herein. The computer system may also be used to control activation of the x-ray generator105and the x-ray source102during a radiographic exposure, or scan, controlling an x-ray tube electric current magnitude, and thus the fluence of x-rays in x-ray beam109, and/or the x-ray tube voltage, and thus the energy level of the x-rays in x-ray beam109.

The DR detector104may transmit image (pixel) data to the monitor computer system106based on the radiographic exposure data received from its array of photosensitive cells. Alternatively, the DR detector may be equipped to process the image data and store it, or it may store raw unprocessed image data, in local or remotely accessible memory. The photosensitive cells in DR detector104may each include a sensing element sensitive to x-rays, i.e. it directly absorbs x-rays and generates an amount of charge carriers in proportion to a magnitude of the absorbed x-ray energy. A switching element may be configured to be selectively activated to read out the charge level of a corresponding x-ray sensing element. Alternatively, photosensitive cells of the indirect type may each include a sensing element sensitive to light rays in the visible spectrum, i.e. it absorbs light rays and generates an amount of charge carriers in proportion to a magnitude of the absorbed light energy, and a switching element that is selectively activated to read the charge level of the corresponding sensing element. A scintillator, or wavelength converter, is disposed over the light sensitive sensing elements to convert incident x-ray radiographic energy to visible light energy. Thus, it should be noted that the DR detector104, in the embodiments disclosed herein, may include an indirect or direct type of DR detector.

In one embodiment, the photosensitive cell array may be read out by sequentially switching rows of the photosensitive array to a conducting (on) state by means of read-out circuits. This digital image information may be subsequently processed by computer system106to yield a digital projection image which may then be digitally stored and immediately displayed on a monitor, or it may be displayed at a later time by accessing the digital electronic memory containing the stored image. A projection images captured and transmitted by the detector104may be accessed by computer system106to generate a bone suppressed image using algorithms as described herein. The flat panel DR detector104having an imaging array as described herein is capable of both single-shot (e.g., static, projection) and continuous image acquisition.

One embodiment of the computer system106further includes various software modules and hardware components to be implemented for generating a bone suppressed radiographic image using the methods described herein. According to one aspect of the current invention, bone suppressed images are generated using cone beam radiographic image data.

Referring toFIG. 2, there is shown a flowchart in accordance with one embodiment of the present disclosure. As illustrated, the method includes step152wherein a medical radiographic image such as a digital chest X-ray image (FIG. 6A) is accessed, received, as by digital transmission, or captured using an x-ray system100as described herein. At step154, a rib or bone suppression algorithm is applied to the radiographic image so as to suppress ribs or bone structures in the image, as illustrated herein below. The resulting radiographic image is a rib or bone suppressed radiographic image (FIG. 6B) generated at step156. At step158, the original captured chest x-ray image, e.g. starting image, and its counterpart bone-suppressed image, e.g., target image, are input to a deep learning convolution neural network module, which are used by the network module for training purposes at step160. The training steps152-160may be repeated a finite number of times.

At step162, a medical radiographic image such as a digital chest X-ray image is provided to the trained deep learning convolution neural network which, at step164, applies its learned algorithm, via the training procedure, to suppress bony structures in the provided medical radiographic image to generate, at step166, an enhanced bone-suppressed radiographic image derived from the provided medical radiographic image.

Preferably, the training of the deep learning module involves many pairs of chest x-ray images (starting images) and their counterpart bone suppressed x-ray images (target images) using known bone suppression algorithms such as those in the prior art incorporated herein by reference. The module, or model, is trained to generate an output radiographic image that is rib free, similar to the target bone suppressed image generated from each input chest x-ray image. Each of the bone suppressed radiographic images are preferably derived directly from its associated original radiographic projection chest image using the known bone suppression algorithms. In a preferred embodiment, the bone suppressed images are not derived using previous approaches whereby a radiographic subtraction procedure is applied or using other manipulations of two images. For example, in other previous systems, a pair of x-ray images may be obtained using two exposures of a patient at two different x-ray source energies in a dual energy x-ray imaging system, wherein one of the two energies is less sensitive to bony structures. Thus, the present invention does not require multiple exposures at any stage of its generation of bone suppressed images, including the training stage as illustrated inFIG. 2. That is, no intermediate radiographic image is generated—the bone suppressed image is generated directly from an original projection chest x-ray image. As such, Applicants have recognized that for the methods described herein, the bone suppressed images are preferably derived directly from radiographic chest images via the bone suppression algorithm, radiographic image manipulation or radiographic image filtering. Examples of such known algorithms, image manipulation or image filtering are described in the three prior art references to Huo incorporated herein. Applicants refer herein to well known bone suppression algorithms and do not further describe details of such algorithms.

In one embodiment, another known technology using convolutional neural networks (CNN) or Generative Adversarial Networks (GANs) as deep learning modules, or models, may be trained to suppress bony structures in radiographic images. The deep learning CNN module is comprised of a library of training images, particularly a plurality of image pairs, for example: a chest x-ray image (starting image) and its corresponding counterpart bone suppressed image (target). In a preferred arrangement, the deep learning module is trained using a plurality of radiographic images and a plurality of bone-suppressed radiographic images derived from at least some of the plurality of starting radiographic images.

In machine learning, the CNN is a class of deep, feed-forward artificial neural networks, often applied to analyzing visual imagery. CNNs use a variation of multilayer perceptrons, or processing elements, designed to require minimal preprocessing. Convolutional networks were inspired by biological processes in that the connectivity pattern between neurons resembles the organization of an animal visual cortex. In one embodiment, the present disclosure describes herein below using GANs to suppress ribs in radiographic images inFIG. 3.

Referring now to the flow charts ofFIGS. 2A-2D, there are shown alternative methods for the method ofFIG. 2.FIG. 2Aillustrates an exemplary x-ray imaging system200A, which may resemble the x-ray imaging system ofFIG. 1in certain respects, wherein a library of training images comprising pairs of original captured radiographic images and corresponding bone suppressed radiographic images, derived directly from the original captured radiographic images, are stored in digital electronic memory202. The x-ray imaging system200A also includes a neural network module stored in the digital electronic memory202which is available for applying a learned bone suppression algorithm to radiographic images. The learned bone suppression algorithm may be applied to radiographic images captured by the x-ray imaging system200A or to radiographic images captured elsewhere and provided to the x-ray imaging system200A such as by providing access to electronic memory containing the radiographic images to be bone suppressed or by transmitting the radiographic images to the x-ray imaging system200A. The neural network module may be trained using the library of training images. The x-ray imaging system200A includes a processing system204having a processor, digital memory, and a display for displaying radiographic images. The processor may be programmable to apply a conventional bone suppression algorithm, as disclosed in the patents incorporated herein by reference, to radiographic images. The processor may be programmed to access a non-bone suppressed radiographic image206and to apply the conventional bone suppression algorithm208thereto. The processor may be programmed to apply the bone suppression algorithm learned by the neural network to the conventionally bone suppressed radiographic image208to generate an enhanced bone suppressed radiographic image210which is then capable of being output to the display212.

FIG. 2Billustrates an exemplary x-ray imaging system200B, which may resemble the x-ray imaging system ofFIG. 1in certain respects, wherein a library of training images including pairs of original captured radiographic images and corresponding bone suppressed radiographic images, derived directly from the original captured radiographic images, are stored in digital electronic memory220. The x-ray imaging system200B also includes a neural network module stored in the digital electronic memory220which is available for training and for applying a learned bone suppression algorithm to radiographic images. The learned bone suppression algorithm may be applied to radiographic images captured by the x-ray imaging system200B or to radiographic images captured elsewhere and provided to the x-ray imaging system200B such as by providing access to electronic memory containing the radiographic images to be bone suppressed or by transmitting the radiographic images to the x-ray imaging system200B. The neural network module may be trained using the library of training images. The x-ray imaging system200B includes a processing system222having a processor, digital memory, and a display for displaying radiographic images. The processor may be programmable to store in the digital electronic memory radiographic images224captured by the x-ray imaging system200B and to apply a conventional bone suppression algorithm226, as disclosed in the patents incorporated herein by reference, to the stored radiographic images224to generate bone suppressed radiographic images228. The processor may be programmed to store pairs of the captured radiographic images224and corresponding bone suppressed radiographic images228in the library of training images to be used by the neural network module for training.

FIG. 2Cillustrates an exemplary x-ray imaging system200C, which may resemble the x-ray imaging system ofFIG. 1in certain respects, wherein a library of training images comprising pairs of original captured radiographic images and corresponding bone suppressed radiographic images, derived directly from the original captured radiographic images, are stored in digital electronic memory230. The x-ray imaging system200C also includes a neural network module stored in the digital electronic memory230which is available for being trained using the library of training images and for applying a learned bone suppression algorithm to radiographic images. The learned bone suppression algorithm may be applied to radiographic images captured by the x-ray imaging system200C or to radiographic images captured elsewhere and provided to the x-ray imaging system200C such as by providing access to electronic memory containing the radiographic images to be bone suppressed or by transmitting the radiographic images to the x-ray imaging system200C. The neural network module may be trained using the library of training images. The x-ray imaging system200C includes a processing system232having a processor, digital memory, and a display for displaying radiographic images. The processor may be programmable to provide a radiographic image234to the neural network after it's trained236so that the neural network generates a bone suppressed image by applying its trained bone suppression programming to the radiographic image to generate a neural network bone suppressed radiographic image238.

FIG. 2Dillustrates an exemplary x-ray imaging system200D, which may resemble the x-ray imaging system ofFIG. 1in certain respects, and may be operated according to the methods described with reference to bothFIGS. 2B and 2C. The imaging system200D includes a library of training images comprising pairs of original captured radiographic images and corresponding bone suppressed radiographic images, derived directly from the original captured radiographic images, stored in digital electronic memory240. The x-ray imaging system200D also includes a neural network module stored in the digital electronic memory240which is available for training and for applying a learned bone suppression algorithm to radiographic images. The learned bone suppression algorithm may be applied to radiographic images captured by the x-ray imaging system200D or to radiographic images captured elsewhere and provided to the x-ray imaging system200D such as by providing access to electronic memory containing the radiographic images to be bone suppressed or by transmitting the radiographic images to the x-ray imaging system200D. The neural network module may be trained using the library of training images. The x-ray imaging system200D includes a processing system242having a processor, digital memory, and a display for displaying radiographic images. The processor may be programmable to store in the digital electronic memory240radiographic images243captured by the x-ray imaging system200D and to apply a conventional bone suppression algorithm245, as disclosed in the patents incorporated herein by reference, to the stored radiographic images243to generate bone suppressed radiographic images247. The processor may be programmed to store pairs of the captured radiographic images243and corresponding bone suppressed radiographic images247in the library of training images to be used by the neural network module for training. Thus, the library of training images is supplied by use of the x-ray imaging system200D. The processor may further be programmable to provide the radiographic image243to the neural network after it's trained246so that the neural network generates a bone suppressed image by applying its trained bone suppression programming to the radiographic image provided by the processor to generate a neural network bone suppressed radiographic image248which may then further be conditioned as necessary for display250.

FIGS. 3-5illustrate a known GANs type neural network module306that may be trained to generate enhanced bone suppressed radiographic images using non-bone suppressed radiographic images as inputs. The neural network module306includes a generator306aand discriminator306billustrated inFIGS. 4 and 5, respectively. In a training scheme, the GANs type neural network module306may be said to iteratively execute an optimization process by receiving a chest x-ray image302at the generator306aand repeatedly attempting to generate a rib-suppressed x-ray image as an output to a discriminator306b. The discriminator306breceives the generator306agenerated image and a target bone suppressed image304which was generated from the chest x-ray image302using a known bone suppression algorithm. The discriminator compares the generator306agenerated x-ray image with the target x-ray image and scores the generator306agenerated image. The generator306auses the scoring data, in a known back propagation method, to repeatedly adjust a weighting scheme for image generation in the generator306auntil the generator's bone suppressed image output to the discriminator306bis determined by the discriminator306bto be acceptable, such as image308, by satisfying a scoring threshold.FIG. 4is a schematic diagram of a known mapping function carried out by the generator306a, wherein the generator306areceives a chest x-ray image xsand generates, via a network of encoding and decoding layers, a bone suppressed image G(xs).FIG. 5is a schematic diagram of a known discriminator306bthat scores a comparison between the bone suppressed image from the generator306aG(xs) and the target bone suppressed image xd. The generator306amay continue to adjust the generated bone suppressed images until a threshold acceptable score from the discriminator306bis achieved, whereby the neural network may be considered to have completed training. The generator/discriminator may cooperate using two dimensional segments of the generated bone suppressed image G(xs) rather than on an entire frame of data. In one embodiment, the discriminator306bmay be programmed to estimate whether bone suppressed image data G(xs) provide by generator306awas generated by the generator306aor whether it originated from the target image xdas part of the discriminator306bscoring scheme. This type of scoring may be said to comprise a contest between the generator306aand discriminator306b.

FIGS. 6A-6Care exemplary radiographic images as described herein above.FIG. 6Ais an exemplary original projection image (starting image) captured by a radiographic imaging system.FIG. 6Bis an exemplary bone suppressed radiographic image version ofFIG. 6Ausing the known bone suppression algorithms disclosed in the prior art patents incorporated herein by reference.FIG. 6Cis an exemplary enhanced bone suppressed radiographic image generated by a trained neural network.

Program code and/or executable instructions embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

These computer program instructions may also be stored in a computer readable medium that can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions which implement the function/act specified in the flowchart and/or block diagram block or blocks. Applicants have described a computer storage product having at least one computer storage medium having instructions stored therein causing one or more computers to perform the method described above. Applicants have described a computer storage medium having instructions stored therein for causing a computer to perform the method described above. Applicants have described a computer product embodied in a computer readable medium for performing the steps of the method described above.