Patent Description:
Medical images are usually stored and transferred through picture archiving and communication systems (PACS) according to digital imaging and communications in medicine (DICOM) standards. Under the DICOM standards, besides image data, a medical image also includes metadata. Metadata is data associated with the image such as information relating to the patient, image acquisition, and the imaging device. For example, metadata includes the laterality of the patient indicating which side of the patient is depicted on the left side of the image. Metadata is typically generated based on the imaging protocol used to acquire the medical image. DICOM standards allow communication and management of medical images and integration of medical devices such as scanners, workstations, and PACS viewers across different manufacturers.

Because of its mobility and relatively-small size, portable x-ray imaging has become one of the most prevalent imaging modalities in the field of medical imaging. The laterality of images acquired by portable x-ray imaging may be incorrect due to human error. Medical images having a proper laterality are displayed as the right side of the image depicting the left side of the patient. However, an operator of the device may mistakenly place the detector in a wrong way such as placing the detector in front of the patient when taking a chest x-ray image of an anterior-posterior (AP) view, or enters a wrong laterality for the image in the user interface. As a result, a flipped image is generated. While a physician has medical skills to determine that the generated image reflects laterality different from that stored in the image metadata, an incorrect laterality may still result in the physician spending additional time to determine the correct laterality of the image before performing diagnosis. In addition, having wrong laterality information in the metadata may cause problems with hanging protocols for displaying images in a way that the user finds more useful when the images are sent to the PACS. Manually rotating the images and storing the correct laterality in the metadata before sending them to PACS is an inefficient use of technologists' time. Further, images with wrong laterality may degrade the performance of a computer-aided diagnostic system or an artificial-intelligence diagnostic system if they are used as input.

<NPL>, discloses classifying laterality of posteroanterior radiographs of a hand using a LeNet convolutional neural network (CNN) in order to correct the laterality.

<NPL>, discloses using deep learning model to classify radiographs by laterality, including an unknown category where markers are missing or uninterpretable.

<NPL>, discloses detecting and correcting laterality errors in radiology reports.

<CIT> discloses assigning mammographic view and laterality to individual images in groups of digitized mammograms.

<NPL>, discloses classifying laterality of fundus images using a convolutional neural network model.

<NPL>, discloses a neural network classification scheme that enables a picture archiving and communications system workstation to determine the correct orientation of posteroanterior or anteroposterior chest radiographs orientation.

The disclosure includes systems and methods for detecting laterality of images using a neural network model. Laterality used herein is the status of whether a left side of a patient is properly denoted in a medical image or the medical image is displayed as if flipped. Being flipped may be horizontally flipped, vertically flipped, or being flipped along an oblique axis. Chest x-ray images in <FIG> and <FIG> are used as exemplary images to illustrate the systems and methods as described herein. The applicability of the systems and methods, however, is not limited to chest x-ray images. The systems and methods are also applicable to x-ray images of other anatomies or mammography images, and applicable to other types of medical images, such as magnetic resonance imaging (MRI) images, computed tomography (CT) images, positron emission tomography (PET) images, and ultrasound images. Method aspects will be in part apparent and in part explicitly discussed in the following description.

Chest exams performed with a portable x-ray system are one of the most frequently performed procedures. The acquired images are often in an anterior-posterior render view, where the images are taken as if a camera were aiming at the patient from the front of the patient toward the back of the patient.

<FIG> illustrate exemplary laterality classes. To take a chest x-ray with a protocol for an AP view, a detector is placed behind a patient and an x-ray source <NUM> in front of the patient (as shown in <FIG> shows an acquired x-ray image <NUM> with this positioning of the detector. The metadata of the x-ray image <NUM> indicates the right side of the image depicts the left side of the patient. The hanging protocols in PACS are typically based on DICOM metadata. As such, a heart <NUM> of the patient is correctly depicted in x-ray image <NUM> as positioned on the left side of the patient (as shown in <FIG>). X-ray image <NUM> has a laterality class of being proper. In comparison, when a technologist places the detector in front of the patient with the x-ray source <NUM> behind the patient, the intended view is posterior-anterior (PA) (as shown in <FIG>). Instead, the technologist mistakenly chooses a protocol for AP. <FIG> shows another acquired x-ray image <NUM> with this positioning of the detector. Because the metadata for laterality of x-ray image <NUM> indicates that the right side of the image depicts the left side of the patient, the left side of the patient is depicted on the left side of x-ray image <NUM> such that heart <NUM> of the patient is incorrectly displayed as on the right side of the patient (as shown in <FIG>). That is, x-ray image <NUM> is displayed as horizontally flipped or mirrored. The x-ray image <NUM> has a laterality class of being flipped. Two laterality classes are used only as examples. The implementation of the systems and methods disclosed herein are not limited to two laterality classes. For example, the laterality classes may include more than two classes of being proper and being flipped, and may further include being horizontally flipped, being vertically flipped, being flipped along an oblique axis, or any combination thereof.

Traditionally, a lead marker <NUM> is placed beside a patient's anatomy, indicating the left or right side of the patient. Lead markers need additional workflow steps and are prone to human errors, such as using a wrong letter or neglecting to include one. In x-ray image <NUM> (as shown in <FIG>), lead marker <NUM> of letter L is correctly depicted as on the left side of the patient. In x-ray image <NUM> (as shown in <FIG>), however, lead marker <NUM> is flipped and incorrectly depicted as on the right side of the patient.

To evaluate the prevalence of mirrored chest x-ray images, an analysis of <NUM>,<NUM> clinical images is conducted. The analysis shows <NUM>% of the images were mirrored. In many systems, before sending an image to a PACS, a technologist needs to manually rotate the image to correct the mirrored images. If a technologist takes two clicks to correct the mirrored images, the technologist may spend <NUM> hours/year in conducting the manual correction. With an artificial intelligence (AI) algorithm in the systems and methods described herein being <NUM>% accurate, the technologist may spend <NUM> minutes/year for the manual correction. Further, a lead marker may be completely eliminated thereby reducing human errors and expediting workflow.

<FIG> is a schematic diagram of an exemplary system <NUM> for laterality detection of an image. The image may be a medical image. System <NUM> includes a laterality detection computing device <NUM> configured to detect and correct the laterality of an input image. Computing device <NUM> further includes a neural network model <NUM>.

In the exemplary embodiment, system <NUM> further includes a metadata editor <NUM> configured to update the metadata of the image. System <NUM> may further include a user interface manager <NUM> configured to receive user inputs on choices in detecting and correcting the laterality of an input image based on these inputs. For example, a user may turn on or off the correction of the laterality of the input image. Due to rare medical conditions such as dextrocardia and situs inversus, in which a person's heart may reside on the right side of the chest, a user may not want an image automatically flipped when the detected laterality would be incorrect for normal anatomies.

System <NUM> may further include a post-processor <NUM> for post-processing the image after laterality of the image has been classified and/or the image has been corrected. Post-processing may include but be not limited to applying mapping of intensity levels for enhancement of the image to match a preference of the radiologist. In some embodiments, system <NUM> is configured to detect whether the lead marker is flipped, and compare the result with the detected laterality of the input image to determine whether the lead marker was placed flipped while the laterality of the image being proper or the lead marker is shown flipped as a result of the laterality of the image being flipped. Post-processing may also include post-processes associated with a lead marker and/or a digital marker, such as blocking out a wrongly-placed or flipped lead marker, replacing a flipped marker, replacing a wrong digital marker, or generating and displaying a digital marker.

<FIG> shows an exemplary sequence diagram <NUM> of data flowing through system <NUM> without user interface manager <NUM> to take user inputs. In sequence <NUM>, an unclassified image <NUM> is inputted into laterality detection computing device <NUM>. Output from computing device <NUM> is inputted into post-processor <NUM> and metadata editor <NUM>. The output from post-processor <NUM> and metadata editor <NUM> are then inputted into a user interface <NUM> to display an output image from post-processor <NUM> with updated metadata <NUM> of the image. The output image may be a corrected image of unclassified image <NUM>. Alternatively, the output image may be a post-processed image of unclassified image <NUM> without correction if a user has turned off correction of the laterality. An alert indicating the laterality of the displayed image may be provided on image display <NUM> or on user interface <NUM>. The alert may indicate whether the laterality is proper or flipped. The alert may also indicate a proper placement of a marker. In an example, the alert may indicate whether the detected laterality matches the protocol used, the lead marker placed, the digital marker placed, or any combination thereof. In another example, the alert may indicate that the laterality of unclassified image <NUM> is flipped and that the laterality of the displayed image has been adjusted or corrected. In another example, the alert may indicate whether the lead marker or the digital marker was properly placed. The output from post-processor <NUM> and metadata editor <NUM> may also be sent to PACS for archiving and transmitting to other systems for further processing.

<FIG> shows another exemplary sequence diagram <NUM> of data flowing through system <NUM> with user inputs being provided from user interface manager <NUM>. User interface manager <NUM> receives a user policy that includes inputs from a user. A user may choose to detect the laterality class of an input image without correcting its laterality. The user may also choose to both detect the laterality class and correct the laterality of the input image. In some embodiments, a user may choose an occasion for carrying out the function of laterality detection computing device <NUM>. For example, a user may choose not to correct the laterality when a chest x-ray is to be taken.

In the exemplary embodiment, user interface manager <NUM> communicates with computing device <NUM>, metadata editor <NUM>, and post-processor <NUM> to transmit the user inputs and update display on the user interface. In sequence <NUM>, an unclassified image <NUM> is provided to laterality detection computing device <NUM>. The output of computing device <NUM> is provided to post-processor <NUM> and user interface manager <NUM>. An output image <NUM> is output by post-processor <NUM>. Input image metadata <NUM> is provided to metadata editor <NUM>. Output image metadata <NUM> is output from metadata editor <NUM>. Compared to sequence <NUM> where a user policy on the process of detecting and correcting laterality is predefined, in sequence <NUM>, a user policy is provided by user interface manager <NUM>.

<FIG> illustrates an exemplary sequence diagram <NUM> of data flow in system <NUM> (shown in <FIG>). An unclassified image <NUM> having not been processed to be classified for its laterality class is inputted into neural network model <NUM>. Unclassified image <NUM> may have been preprocessed such as being resized to derive a resized image. In some embodiments, unclassified image <NUM> may have not been preprocessed. Neural network model <NUM> is configured to output the laterality class of unclassified image <NUM>. The neural network model <NUM> provides one or more outputs <NUM> that include the laterality class of unclassified image <NUM>. Computing device <NUM> may further include a corrector <NUM> that is configured to correct the laterality of unclassified image <NUM>. If laterality correction is turned on, based on the laterality class output by neural network model <NUM>, unclassified image <NUM> is adjusted to have laterality of target laterality, such as being proper where the right side of the image depicts the left side of the patient. As a result, an output image <NUM> having the target laterality is output from laterality detection computing device <NUM>. In some embodiments, when laterality correction is turned off, the laterality of output image <NUM> is not adjusted even if the laterality class detected by neural network model <NUM> is not the target laterality.

To train neural network model <NUM>, training x-ray images are provided as inputs to neural network model and observed laterality classes associated with the training x-ray images are provided as outputs of neural network model <NUM>. Observed laterality classes may include being proper (see <FIG>) and being flipped (see <FIG>).

The training x-ray images may be preprocessed before being provided to the neural network model. Exemplary preprocessing algorithms include, but are not limited to, look-up table mapping, histogram equalization, normalization, intensity transformation, gradient computation, edge detection, or a combination thereof. Training x-ray images may be down-sized before being provided to the neural network model to ease the computation burden of the neural network model on a computing device. The training x-ray images may be down-sized by reducing the image resolution of the training x-ray images to generate downsized training x-ray images. In one example, unclassified image <NUM> may have been applied with these preprocessing before being input into laterality detection computing device <NUM>.

In some embodiments, the features of the training x-ray images may be extracted before being provided to the neural network model. The image features are generally derived from the distribution of intensity values of image pixels. For example, histograms of oriented gradients (HOG) features are derived by analyzing gradient laterality in localized regions of an image. The image is divided in small regions (called cells) of varying sizes. Neighboring cells may be combined in a larger region called a block. HOG features are not invariant to laterality. Features may be indicative of edges in the training x-ray images or landmarks such as certain anatomy in a patient. The extracted features from the training x-ray images are then provided to the neural network model. The features may be used in a supervised learning algorithm of the neural network model.

<FIG> illustrates another exemplary sequence diagram <NUM> of data flow in system <NUM>. Different from the neural network model depicted in <FIG>, neural network model <NUM> is configured to detect and correct laterality of an input image. Unclassified image <NUM> is input into neural network model <NUM>. Neural network model <NUM> transforms the unclassified image <NUM> directly to corrected image <NUM> that has the target laterality if laterality correction is turned on.

<FIG> illustrates an exemplary sequence diagram <NUM> of data flow in system <NUM> with computing device <NUM> implemented with neural network model <NUM> that is configured to detect and correct the laterality of unclassified image <NUM>. The unclassified image <NUM> is input into neural network model <NUM>. Neural network model <NUM> transforms the unclassified image <NUM> directly to corrected image <NUM> that has the target laterality such as being proper. Specifically, in neural network model <NUM>, unclassified image <NUM> is flipped to derive a flipped image <NUM>. A plurality of images <NUM> including flipped image <NUM> and unclassified image <NUM> are input into convolutional neural network <NUM> to predict which one of the images has target laterality. Convolutional neural network <NUM> predicts the index associated with the image that has the target laterality. A selector layer or operation <NUM> in neural network model <NUM> selects and outputs the image <NUM> associated with the predicted index.

In the exemplary embodiment, neural network model <NUM> is trained by inputting an image with known (ground truth) laterality to neural network model <NUM>. Inside neural network model <NUM>, a plurality of images including the input image and the flipped image of the input image are generated. Flipping is an operation and does not have trainable parameters. A ground truth vector is generated based on the ground truth of which image between the input image and the flipped images has the target laterality. For example, a ground truth vector of [<NUM><NUM>] indicates the input image has the target laterality, a ground truth vector of [<NUM><NUM>] indicates the flipped image has the target laterality. The flipped image and the input image are given to convolutional neural network <NUM> along with the ground truth vector to train a "selector" model, i.e., convolutional neural network <NUM>. Convolutional neural network <NUM> predicts the index associated with the image having the target laterality. Weights of convolutional neural network <NUM> are updated based on the predicted index in comparison with the ground truth vector. A selector layer or operation selects and outputs the image associated with the predicted index.

Although two classes of being proper and being flipped are illustrated in <FIG>, the laterality classes may further include being horizontally flipped, being vertically flipped, being flipped along an oblique axis, or any combination thereof. Images <NUM> inputted into neural network model <NUM> may include a plurality of flipped images <NUM> that are flipped along various axes.

<FIG> depicts an exemplary artificial neural network model <NUM>. The example neural network model <NUM> includes layers of neurons <NUM>, <NUM>-<NUM> to <NUM>-n, and <NUM>, including input layer <NUM>, one or more hidden layers <NUM>-<NUM> through <NUM>-n, and output layer <NUM>. Each layer may include any number of neurons, i.e., q, r, and n in <FIG> may be any positive integers. It should be understood that neural networks of a different structure and configuration from that depicted in <FIG> may be used to achieve the methods and systems described herein.

In the exemplary embodiment, input layer <NUM> may receive different input data. For example, input layer <NUM> includes a first input a<NUM> representing training x-ray images, a second input a<NUM> representing patterns identified in the training x-ray images, a third input a<NUM> representing edges of the training x-ray images, and so on. Input layer <NUM> may include thousands or more inputs. In some embodiments, the number of elements used by neural network model <NUM> changes during the training process, and some neurons are bypassed or ignored if, for example, during execution of the neural network, they are determined to be of less relevance.

In the example embodiment, each neuron in hidden layer(s) <NUM>-<NUM> through <NUM>-n processes one or more inputs from input layer <NUM>, and/or one or more outputs from neurons in one of the previous hidden layers, to generate a decision or output. Output layer <NUM> includes one or more outputs each indicating a label, confidence factor, weight describing the inputs, and/or an output image. The confidence factor and/or weight are reflective of how strongly an output laterality class indicates laterality of an image. In some embodiments, however, outputs of neural network model <NUM> are obtained from a hidden layer <NUM>-<NUM> through <NUM>-n in addition to, or in place of, output(s) from output layer(s) <NUM>.

In some embodiments, each layer has a discrete, recognizable, function with respect to input data. For example, if n=<NUM>, a first layer analyzes the first dimension of the inputs, a second layer the second dimension, and the final layer the third dimension of the inputs. Dimensions may correspond to aspects considered strongly determinative, then those considered of intermediate importance, and finally those of less relevance.

In other embodiments, the layers are not clearly delineated in terms of the functionality they perform. For example, two or more of hidden layers <NUM>-<NUM> through <NUM>-n may share decisions relating to labeling, with no single layer making an independent decision as to labeling.

<FIG> depicts an example neuron <NUM> that corresponds to the neuron labeled as "<NUM>,<NUM>" in hidden layer <NUM>-<NUM> of <FIG>, according to one embodiment. Each of the inputs to neuron <NUM> (e.g., the inputs in the input layer <NUM> in <FIG>) is weighted such that input a<NUM> through ap corresponds to weights w<NUM> through wp as determined during the training process of neural network model <NUM>.

In some embodiments, some inputs lack an explicit weight, or have a weight below a threshold. The weights are applied to a function α (labeled by reference numeral <NUM>), which may be a summation and may produce a value z<NUM> which is input to a function <NUM>, labeled as f<NUM>,<NUM>(z<NUM>). The function <NUM> is any suitable linear or non-linear function. As depicted in <FIG>, the function <NUM> produces multiple outputs, which may be provided to neuron(s) of a subsequent layer, or used as an output of neural network model <NUM>. For example, the outputs may correspond to index values of a list of labels, or may be calculated values used as inputs to subsequent functions.

It should be appreciated that the structure and function of the neural network model <NUM> and neuron <NUM> depicted are for illustration purposes only, and that other suitable configurations exist. For example, the output of any given neuron may depend not only on values determined by past neurons, but also on future neurons.

Neural network model <NUM> may include a convolutional neural network, a deep learning neural network, a reinforced or reinforcement learning module or program, or a combined learning module or program that learns in two or more fields or areas of interest. Deep learning networks have shown superior performance in terms of accuracy, compared to non-deep learning networks. Neural network model <NUM> may be trained using supervised or unsupervised machine learning programs. Machine learning may involve identifying and recognizing patterns in existing data in order to facilitate making predictions for subsequent data. Models may be created based upon example inputs in order to make valid and reliable predictions for novel inputs.

Additionally or alternatively, the machine learning programs may be trained by inputting sample data sets or certain data into the programs, such as images, and object statistics and information. The machine learning programs may utilize deep learning algorithms that may be primarily focused on pattern recognition, and may be trained after processing multiple examples. The machine learning programs may include Bayesian Program Learning (BPL), voice recognition and synthesis, image or object recognition, optical character recognition, and/or natural language processing - either individually or in combination. The machine learning programs may also include natural language processing, semantic analysis, automatic reasoning, and/or machine learning.

Supervised and unsupervised machine learning techniques may be used. In supervised machine learning, a processing element may be provided with example inputs and their associated outputs, and may seek to discover a general rule that maps inputs to outputs, so that when subsequent novel inputs are provided the processing element may, based upon the discovered rule, accurately predict the correct output. In unsupervised machine learning, the processing element may be required to find its own structure in unlabeled example inputs.

Based upon these analyses, the neural network model <NUM> may learn how to identify characteristics and patterns that may then be applied to analyzing image data, model data, and/or other data. For example, model <NUM> may learn to identify laterality of an input image.

<FIG> shows an exemplary convolutional neural network <NUM> according to one aspect of the disclosure. Neural network model <NUM> includes convolutional neural network <NUM> such as convolutional neural network <NUM> (shown in <FIG>). Convolutional neural network <NUM> includes a convolutional layer <NUM>. In a convolutional layer <NUM>, convolution is used in place of general matrix multiplication in a neural network model. Neural network <NUM> includes one or more convolutional layer blocks <NUM>, a fully-connected layer <NUM> where the neurons in this layer is connected with every neuron in the prior layer, and an output layer <NUM> that provides outputs.

In the exemplary embodiment, convolutional layer block <NUM> includes convolutional layer <NUM> and a pooling layer <NUM>. Each convolutional layer <NUM> is flexible in terms of its depth such as the number of convolutional filters and sizes of convolutional filters. Pooling layer <NUM> is used to streamline the underlying computation and reduce the dimensions of the data by combining outputs of neuron clusters at the prior layer into a single neuron in pooling layer <NUM>. Convolutional layer block <NUM> may further include a normalization layer <NUM> between convolutional layer <NUM> and pooling layer <NUM>. Normalization layer <NUM> is used to normalize the distribution within a batch of training images and update the weights in the layer after the normalization. The number of convolutional layer block <NUM> in neural network <NUM> may depend on the image quality of training x-ray images and levels of details in extracted features.

In operation, in training, training x-ray images and other data such as extracted features of the training x-ray images are inputted into one or more convolutional layer blocks <NUM>. Observed laterality classes and/or corrected training x-ray images are provided as outputs of output layer <NUM>. Neural network <NUM> is adjusted during the training. Once neural network <NUM> is trained, an input x-ray image is provided to the one or more convolutional layer blocks <NUM> and output layer <NUM> provides outputs that include laterality classes and may also include corrected x-ray image of the input x-ray image.

Convolutional neural network <NUM> may be implemented as convolutional neural network <NUM> (shown in <FIG>), <NUM> (shown in <FIG>), <NUM> (shown in <FIG>). <FIG> shows exemplary convolutional neural network <NUM> that includes a plurality of convolutional layers <NUM>. Neural network <NUM> includes layers <NUM>-<NUM>. Each of layers <NUM>-<NUM> includes a various combination of convolutional layer <NUM> with normalization layer <NUM> and pooling layer <NUM>. For example, layers <NUM> include convolutional layer <NUM> and pooling layer <NUM>. Layers <NUM>, <NUM>, <NUM>, <NUM> include normalization layer <NUM> in addition to convolutional layer <NUM> and pooling layer <NUM>. Layers <NUM>, <NUM> include convolutional layer <NUM> and normalization layer <NUM>. In one example, layers <NUM>, <NUM> include convolutional layer <NUM>. Layer <NUM> is a global average pooling layer <NUM>. Global average pooling layer <NUM> reduces overfitting by reducing the number of parameters. Global average pooling layer <NUM> may also make the extracted features spatially dependent. Output of global average pooling layer <NUM> are provided to layer <NUM>. Layers <NUM>, <NUM> are fully-connected layer <NUM>. The last layer <NUM> of neural network <NUM> is an output layer <NUM> that determines whether the input <NUM> is a flipped image or a proper image, or has a laterality class of being flipped or being proper.

<FIG> shows one more exemplary convolutional neural network <NUM>. Convolutional neural network <NUM> includes one or more pairs <NUM> of a convolutional block <NUM> and a residual block <NUM>. Convolutional block <NUM> includes a convolutional layer <NUM>. In residual block, a layer may provide its output to a preceding layer that is not immediately preceding. Convolutional neural network <NUM> further includes a convolutional layer <NUM> following pairs <NUM>.

<FIG> shows one more exemplary convolutional neural network <NUM>. Convolutional neural network <NUM> includes a plurality of convolutional layer <NUM>, normalization layer <NUM>, pooling layer <NUM>, convolutional layer <NUM> again, and then global average pooling layer <NUM>. In convolutional neural network <NUM>, <NUM>, <NUM>, the parameters of convolutional layer <NUM>, normalization layer <NUM>, pooling layer <NUM>, and global average pooling layer <NUM> may be different, or be kept the same, across different layers.

<FIG> illustrates a flow chart of an exemplary method <NUM> of detecting laterality of an x-ray image. Method <NUM> includes executing <NUM> a neural network model. Method <NUM> further includes receiving <NUM> training x-ray images and observed laterality classes associated with the training x-ray images. The observed laterality classes are laterality classes of the training x-ray images such as being proper or being flipped. The training x-ray images and the observed laterality classes are input into the neural network with the training x-ray images as inputs and the observed laterality classes as outputs.

In the exemplary embodiment, method <NUM> also includes analyzing <NUM> the training x-ray images. Further, method <NUM> includes calculating <NUM> predicted laterality classes for the training x-ray images using the neural network model. Moreover, method <NUM> includes comparing <NUM> the laterality of the corrected training x-ray images with the target laterality. Method <NUM> also includes adjusting <NUM> the neural network model based on the comparison. For example, the parameters and the number of layers and neurons of the neural network model are adjusted based on the comparison.

<FIG> illustrates a flow chart of method <NUM> of detecting laterality of an input image. Method <NUM> includes executing <NUM> a neural network model. The neural network model is trained with training x-ray images as inputs and laterality classes associated with the training x-ray images as outputs.

Method <NUM> also includes receiving <NUM> an unclassified x-ray image, the laterality of which has not been classified. Method <NUM> also includes analyzing <NUM> the unclassified x-ray image using the neural network model. A laterality class of the unclassified x-ray image is then assigned <NUM> based on the analysis. If the assigned laterality class is the target laterality such as being proper, the unclassified x-ray image has a correct laterality and is outputted <NUM>. If the assigned laterality class is not the target laterality, the unclassified x-ray image is adjusted <NUM>.

The unclassified x-ray image is adjusted by flipping the unclassified x-ray image to have laterality of the target laterality. If the assigned laterality class is being flipped, the unclassified image is flipped to the target laterality. If the assigned laterality class is being flipped, the lead marker is digitally blocked out or covered up such that the lead marker does not confuse a reader. In another example, a new digital marker may be generated based on a user-defined logic, indicating the correct laterality of the image. A digital marker may be a letter "L" or "R. " A digital marker of a letter "L" may be placed on the right side of the medical image to indicate the left side of the patient. Alternatively, a digital marker of a letter "R" may be placed on the left side of the medical image to indicate the right side of the patient.

Method <NUM> also includes outputting <NUM> the corrected x-ray image. In some embodiments, method <NUM> includes concurrently outputting the corrected x-ray image and the laterality class of the input image using the neural network model. That is, the neural network model outputs both the corrected x-ray image and the laterality class of the input image.

In the exemplary embodiment, the metadata associated with the unclassified x-ray image may be updated based on the detected laterality class. The metadata associated with the output x-ray image is then generated to reflect the update.

Computing device <NUM>, post-processor <NUM>, user interface manager <NUM>, metadata editor <NUM>, and user interface <NUM> described herein may be implemented on any suitable computing device and software implemented therein.

<FIG> is a block diagram of an exemplary computing device <NUM>. In the exemplary embodiment, computing device <NUM> includes a user interface <NUM> that receives at least one input from a user. User interface <NUM> may include a keyboard <NUM> that enables the user to input pertinent information. User interface <NUM> may also include, for example, a pointing device, a mouse, a stylus, a touch sensitive panel (e.g., a touch pad and a touch screen), a gyroscope, an accelerometer, a position detector, and/or an audio input interface (e.g., including a microphone).

Moreover, in the exemplary embodiment, computing device <NUM> includes a presentation interface <NUM> that presents information, such as input events and/or validation results, to the user. Presentation interface <NUM> may also include a display adapter <NUM> that is coupled to at least one display device <NUM>. More specifically, in the exemplary embodiment, display device <NUM> may be a visual display device, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a light-emitting diode (LED) display, and/or an "electronic ink" display. Alternatively, presentation interface <NUM> may include an audio output device (e.g., an audio adapter and/or a speaker) and/or a printer.

Computing device <NUM> also includes a processor <NUM> and a memory device <NUM>. Processor <NUM> is coupled to user interface <NUM>, presentation interface <NUM>, and memory device <NUM> via a system bus <NUM>. In the exemplary embodiment, processor <NUM> communicates with the user, such as by prompting the user via presentation interface <NUM> and/or by receiving user inputs via user interface <NUM>. The term "processor" refers generally to any programmable system including systems and microcontrollers, reduced instruction set computers (RISC), complex instruction set computers (CISC), application specific integrated circuits (ASIC), programmable logic circuits (PLC), and any other circuit or processor capable of executing the functions described herein. The above examples are exemplary only, and thus are not intended to limit in any way the definition and/or meaning of the term "processor.

In the exemplary embodiment, memory device <NUM> includes one or more devices that enable information, such as executable instructions and/or other data, to be stored and retrieved. Moreover, memory device <NUM> includes one or more computer readable media, such as, without limitation, dynamic random access memory (DRAM), static random access memory (SRAM), a solid state disk, and/or a hard disk. In the exemplary embodiment, memory device <NUM> stores, without limitation, application source code, application object code, configuration data, additional input events, application states, assertion statements, validation results, and/or any other type of data. Computing device <NUM>, in the exemplary embodiment, may also include a communication interface <NUM> that is coupled to processor <NUM> via system bus <NUM>. Moreover, communication interface <NUM> is communicatively coupled to data acquisition devices.

In the exemplary embodiment, processor <NUM> may be programmed by encoding an operation using one or more executable instructions and providing the executable instructions in memory device <NUM>. In the exemplary embodiment, processor <NUM> is programmed to select a plurality of measurements that are received from data acquisition devices.

In operation, a computer executes computer-executable instructions embodied in one or more computer-executable components stored on one or more computer-readable media to implement aspects of the invention described and/or illustrated herein. The order of execution or performance of the operations in embodiments of the invention illustrated and described herein is not essential, unless otherwise specified. That is, the operations may be performed in any order, unless otherwise specified, and embodiments of the invention may include additional or fewer operations than those disclosed herein. For example, it is contemplated that executing or performing a particular operation before, contemporaneously with, or after another operation is within the scope of aspects of the invention.

At least one technical effect of the systems and methods described herein includes (a) automatic detection of laterality of an x-ray image; (b) automatic adjustment of laterality of an x-ray image; and (c) increased flexibility by providing a user interface manager to receive user inputs.

Exemplary embodiments of systems and methods of detecting and/or correcting laterality of medical images are described above in detail. The systems and methods are not limited to the specific embodiments described herein but, rather, components of the systems and/or operations of the methods may be utilized independently and separately from other components and/or operations described herein. Further, the described components and/or operations may also be defined in, or used in combination with, other systems, methods, and/or devices, and are not limited to practice with only the systems described herein.

Claim 1:
An x-ray image laterality detection system (<NUM>), comprising:
a detection computing device (<NUM>, <NUM>) comprising at least one processor (<NUM>) in communication with at least one memory device (<NUM>), wherein said at least one processor is programmed to:
receive an unclassified x-ray image, wherein the unclassified x-ray image is acquired with a lead marker (<NUM>);
execute a neural network model for analyzing x-ray images in order to analyze the unclassified x-ray image using the neural network model, wherein the neural network model (<NUM>, <NUM>, <NUM>, <NUM>, <NUM>, <NUM>) is trained with training x-ray images as inputs and observed laterality classes associated with the training x-ray images as outputs;
assign a laterality class to the unclassified x-ray image based on the analysis;
if the assigned laterality class is not a target laterality, the at least one processor is programmed to:
adjust the unclassified x-ray image by flipping the unclassified x-ray image to derive a corrected x-ray image having the target laterality; and
output the corrected x-ray image, wherein the lead marker (<NUM>) is digitally blocked out or covered up in the corrected x-ray image; and
if the assigned laterality class is the target laterality, the at least one processor is programmed to output the unclassified x-ray image,
wherein the target laterality comprises a proper laterality wherein the right side of the x-ray image depicts the left side of a patient.