Patent Description:
This invention was made with government support under AR064840 awarded by the National Instituted of Health. The government has certain rights in the invention.

An estimated <NUM> million Americans currently have artificial hip joints, or hip arthroplasty ("HA"), all of which have inevitable failure modes that depend heavily on patient-specific risk and genetic factors. In practice, most HA failures requiring revision are due to alterations in adjoining tissues (e.g., bone, synovium, muscle). These tissue alterations are best assessed using magnetic resonance imaging ("MRI"). In particular, MRI can visualize implant loosening, periprosthetic bone loss, and adverse host-mediated synovial reactions, thus providing substantial value to surgeons. This added value comes in the form of improved and earlier diagnoses of disease progression in cases of symptomatic HA.

MRI acquisitions that utilize three-dimensional multi-spectral imaging ("3D-MSI") technology can resolve the historic metal artifact challenges of imaging near HA and have expanded the potential clinical role of MRI in HA management. However, the mitigation of the physical challenge of MRI metal artifacts has exposed another challenge: the complexity of MRI interpretation in the immediate vicinity of HA. A variety of image signatures, spanning from synovial abnormalities, tissue reactions, and osteolysis are often readily identifiable near symptomatic or asymptomatic HA on 3D-MSI MRI, but require a fairly steep learning curve on the part of the interpreting radiologist or clinician. This challenge, combined with the complex manifestation of the MRI findings, results in poor characterization of these clinically relevant MRI signatures and an under-utilization of the imaging technology in many clinical orthopedic imaging environments. This under-utilization of available imaging technology is particularly relevant in imaging departments that are not solely focused on orthopedics.

Documents <CIT>, <CIT>, and <CIT> disclose a system and method for generating a feature maps that depicts a spatial distributions or patterns of a features associated with osteolysis or synovitis according to the prior art.

The present disclosure addresses the aforementioned drawbacks by providing a method for generating a feature map according to claim <NUM>.

It is another aspect of the present disclosure to provide a method for constructing and implementing a machine learning algorithm to a feature map according to claim <NUM>.

It is another aspect of the present disclosure, not claimed, to provide a method for generating feature data that indicates a thickness of a synovial membrane in a subject. The method includes accessing with a computer system, magnetic resonance imaging (MRI) data acquired from a subject. A trained machine learning algorithm that has been trained on training data to generate feature data that indicates a thickness of a synovial membrane is also accessed by the computer system. The MRI data are input to the trained machine learning algorithm in order to generate output with the computer system. The output includes feature data that indicates a thickness of a synovial membrane in the subject.

Described here are systems and methods for utilizing a machine learning algorithm implemented with a hardware processor and memory to provide image-based deep learning on 3D-MSI MRI to provide supplementary guidance for radiological identification of common soft-tissue pathologies near HA or other metallic implants or objects. In particular, preliminary data demonstrates the promising capabilities of convolutional neural networks ("CNNs") trained on a cohort of normal and pathological 3D-MSI of HA (or patients with or other metallic implants or objects) to spatially identify periprosthetic bone loss (i.e., osteolysis) and adverse synovial responses. Using this artificial intelligence ("AI") analysis engine, feature maps advising potential soft tissue and bone pathology can be constructed on a subject-specific basis and presented to radiologists and surgeons as a direct visual presentation of regional tissue abnormalities near HA or other metallic implants or objects. This computer-aided diagnosis ("CAD") approach can also serve to accelerate and broaden the adoption of advanced MRI utility in HA management by adding simplicity and intuition to application workflows.

In addition, the systems and methods described in the present disclosure could aid in the education and intuitive comprehension of tissue signatures near HA or other metallic implants or objects in large patient cohorts, allowing for the generation of important natural history datasets.

The systems and methods described in the present disclosure can also enable image-based soft-tissue pathology differentiation near hip replacements or other metallic implants or objects. Such technology can offer quantitative evidence informing the severity of tissue disease in failing hip replacements (or other joint replacements or metallic orthopedic implants) and better evaluate the necessity and mechanisms for revision procedures.

It is one aspect of the present disclosure to provide a deep learning classification model for spatially highlighting osteolysis and synovitis on 3D-MSI MRI. As one example, supervised model training can utilize a neural network classification infrastructure. The output of the neural network will be 3D synovitis and osteolysis feature maps for individual subjects. The scale on the feature maps will represent the local probability for a given pathological classification. The performance of the network model can be statistically assessed (e.g., on a pixel-wise basis) against expert radiologist image annotations.

Referring now to <FIG>, a flowchart is illustrated as setting forth the steps of an example method for generating feature maps associated with osteolysis, synovitis, or both, using a trained machine learning algorithm that has been trained on relevant training data to generate these feature maps.

The method includes accessing magnetic resonance imaging ("MRI") data with a computer system, as indicated at step <NUM>. The MRI data may include magnetic resonance signal data (e.g., k-space data), images reconstructed from k-space data, or parametric images or maps generated from magnetic resonance signal data, reconstructed images, or both. In some instances, the MRI data may also include texture feature maps generated from the magnetic resonance signal data, reconstructed images, or parametric images or maps. Texture features can include first-order statistics, second-order statistics, and other texture features known to those skilled in the art. The MRI data can be accessed by retrieving such data from a memory or other data storage device or medium. In other instances, the magnetic resonance data can be accessed by acquiring such data from a subject using an MRI system and communicating that data to the computer system.

As one example, the MRI data may include proton density-weighted images acquired with an MRI system. In other instances, however, the MRI data can include T1-weighted images, T2-weighted images, or other magnetic resonance images acquired using a desired pulse sequence to elicit the desired magnetic resonance imaging contrast. In general, the MRI data will include MSI data, which may include 2D-MSI data or 3D-MSI data, acquired using a multi-spectral imaging technique. In some examples, the MRI data may also include parametric maps indicating frequency information derived from the multi-spectral imaging (e.g., frequency bins, frequency offsets).

A trained machine learning algorithm is then accessed with the computer system, as indicated at step <NUM>. Accessing the trained machine learning algorithm may include, for instance, accessing weights, biases, or both, which have been computed or otherwise estimated by training the machine learning algorithm on training data. When the machine learning algorithm implements a neural network, accessing the trained machine learning algorithm may include retrieving or otherwise accessing the particular neural network architecture to be implemented. For instance, data pertaining to the layers in the neural network architecture (e.g., number of layers, type of layers, ordering of layers, connections between layers) may be retrieved or otherwise accessed.

In some instances, more than one trained machine learning algorithm may be accessed. For example, a first machine learning algorithm may have been trained on first training data to generate a first feature map and a second machine learning algorithm may have been trained on second training data to generate a second feature map that is different from the first feature map. As an example, the first feature map may be associated with osteolysis whereas the second feature map may be associated with synovitis.

The MRI data are then input to the one or more trained machine learning algorithms, as indicated at step <NUM> in order to generate output as feature maps associated with osteolysis, synovitis, or both. Inputting the MRI data may include inputting data from the entire field-of-view represented in the MRI data, or may include inputting data from one or more regions-of-interest ("ROIs") that have been identified in the MRI data. As mentioned, the feature maps may depict the spatial distribution or spatial patterns of features, statistics, or other parameters associated with osteolysis, synovitis, or both.

As one example, the feature maps may indicate the local probability for a given pathological classification. For instance, the feature maps may indicate the location and/or probability of each pathology. Clinically, these feature maps can be used to give radiologists a baseline off which to diagnosis. Using the systems and methods described in the present disclosure, diagnosis of both osteolysis and synovitis could be done around other implants as well.

As another example, the feature maps may indicate the type of synovitis, not just the presence or absence of synovitis. For instance, the trained machine learning algorithms may be trained to implement automatic pattern recognition to generate feature maps that classify synovitis patterns on MRI data, such as 3D-MSI data. While synovial thickness and volumetric expansion can provide useful and established quantitative metrics of synovial health, they do not necessarily specify the nature or type of the synovial reaction. Using the systems and methods described in the present disclosure, the location, type, and severity of synovitis can be estimated in the generated feature maps. Robust monitoring of a soft tissue response may benefit from the ability to track changes of synovitis based on these feature maps.

In these instances, the one or more trained machine learning algorithms are trained and implemented to classify the severity and type of synovitis. Training of these machine learning algorithms can be performed using training data that include labeled data indicating segmented and classified regions of intermediate-to-severe synovitis. These labeled data may include manually segmented and classified regions that has been performed by a radiologist. Using machine learning algorithms that have been trained to classify and localize synovitis can provide advantages for generalized usage in HA assessment and can also be used an education tool for radiologists interpreting MRI near HA.

As still another example, the output of the trained machine learning algorithm may include feature data that indicates a thickness of the synovial membrane. In these instances the trained machine learning algorithm may include an encoding path of an encoder-decoder neural network that is implemented to provide a classifier (e.g., an <NUM>-bit precision classifier) for maximal synovial thickness. In such examples, the feature data may be displayed to a user, such as on a graphical user interface. The feature data may be displayed, for instance, by displaying the feature data as an overlay on the MRI data, or may be displayed as a separate display element adjacent MRI data in a graphical user interface. The feature data may also be output in a report, which may include textual or numerical information or data.

As a non-limiting example, the systems and methods described in the present disclosure can be implemented to provide a voxel-wise synovitis classifier. Synovial segmentation can be used to focus this pattern detection network within regions of synovium or synovial expansion. In these instances, the network can be expanded to utilize the synovial mask as a secondary input to the training and inference phases. As one non-limiting example this may include implementing a region-based CNN (e.g., Mask R-CNN) network architecture to detect a bounding box of synovial area and then perform a voxel-wise synovitis identification only inside the synovial bounding box.

In some other implementations, the encoding path of an encoder-decoder neural network can be implemented to provide classification of identified synovitis regions based on synovitis type labels defined by radiologists during data curation.

These feature maps may then be displayed to a user, stored for later use or further processing, or both, as indicated at step <NUM>.

Referring now to <FIG>, a flowchart is illustrated as setting forth the steps of an example method for training one or more machine learning algorithms on training data, such that the one or more machine learning algorithms are trained to receive input as MRI data in order to generate output as feature maps associated with osteolysis and/or synovitis.

The method includes accessing MRI data that have been acquired with an MRI system, as indicated at step <NUM>. As described above, the MRI data may include magnetic resonance signal data (e.g., k-space data), images reconstructed from k-space data, or parametric images or maps generated from magnetic resonance signal data, reconstructed images, or both. The MRI data may also include texture feature maps. The MRI data can be accessed by retrieving such data from a memory or other data storage device or medium. In other instances, the magnetic resonance data can be accessed by acquiring such data from one or more subjects using an MRI system and communicating that data to the computer system.

When the MRI data includes magnetic resonance signal data, images can be reconstructed from the magnetic resonance signal data, as indicated at step <NUM>. Segmented MRI data are then generated at step <NUM> by segmenting magnetic resonance images reconstructed from, or otherwise included in, the MRI data. The segmented MRI data can be generated by manually, semi-automatically, or automatically segmenting images. In some instances, the segmented MRI data can be generated by inputting MRI data to one or more trained machine learning algorithms that have been trained to generate output as segmented MRI data. For instance, the trained machine learning algorithms can be trained to generate output as segmented MRI data in which synovium has been segmented. These segmented MRI data may, therefore, include segmented synovium information that can be used as a restricted mask for synovial classification.

Training data are assembled next, as indicated at step <NUM>. Assembling the training data may include assembling MRI data, segmented MRI data, and other relevant data. For instance, assembling the training data may include generating labeled data and including the labeled data in the training data. Labeled data may include MRI data, segmented MRI data, or other relevant data that have been labeled as belong to, or otherwise being associated with, one or more different classifications or categories. For instance, labeled data may include MRI data and/or segmented MRI data that have been labeled based on a probability of a pathological classification (e.g., osteolysis, synovitis, or both). The labeled data may include labeling all data within a field-of-view of the MRI data or the segmented MRI data, or may include labeling only those data in one or more ROIs in the MRI data and/or segmented MRI data. The labeled data may include data that are classified on a voxel-by-voxel basis, or a regional or larger volume basis.

One or more machine learning algorithms are trained on the training data, as indicated at step <NUM>. The machine learning algorithm can be any suitable machine learning algorithm, and in some instances includes a machine learning algorithm based on a neural network. The neural network may be a convolutional neural network, and in some instances may include an encoder-decoder neural network, such as an encoder-decoder neural network that implements a U-Net architecture.

The one or more trained machine learning algorithms are then stored for later use, as indicated at step <NUM>. Storing the machine learning algorithms may include storing weights, biases, or both, which have been computed or otherwise estimated by training the machine learning algorithm on the training data. When the machine learning algorithm implements a neural network, storing the trained machine learning algorithm may include storing the particular neural network architecture to be implemented. For instance, data pertaining to the layers in the neural network architecture (e.g., number of layers, type of layers, ordering of layers, connections between layers) may be stored.

Referring now to <FIG>, a flowchart is illustrated as setting forth the steps of an example method for training and implementing a machine learning algorithm for synovial segmentation and classification.

The method includes accessing MRI data, as indicated at step <NUM>. As described above, the MRI data may include magnetic resonance signal data (e.g., k-space data), images reconstructed from k-space data, or parametric images or maps generated from magnetic resonance signal data, reconstructed images, or both. The MRI data may also include texture feature maps. The MRI data can be accessed by retrieving such data from a memory or other data storage device or medium. In other instances, the magnetic resonance data can be accessed by acquiring such data from one or more subjects using an MRI system and communicating that data to the computer system. As an example, the MRI data may include data or images acquired using an isotropic intermediate weighted 3D-MSI acquisition.

Segmented MRI data are then generated or otherwise accessed, as indicated at step <NUM>. For instance, the segmented MRI data may include previously manually segmented datasets from 3D-MSI. The segmented MRI data may include images in which the synovium (e.g., the synovial envelope) have been manually segmented.

Training data are assembled at step <NUM>. Assembling the training data may include, for example, assembling the MRI data and the corresponding segmented MRI data, on which the machine learning algorithm is to be trained. This step may include assembling the data into an appropriate data structure on which the machine learning algorithm can be trained.

One or more machine learning algorithms are trained on the training data, as indicated at step <NUM>. The machine learning algorithm can be any suitable machine learning algorithm, and in some instances includes a machine learning algorithm based on a neural network. The neural network may be a convolutional neural network, and in some instances may include an encoder-decoder neural network, such as an encoder-decoder neural network that implements a U-Net architecture. In general, the same neural network architecture utilized for the osteolysis/synovitis feature map generation can also be used to generate a trained machine learning algorithm for synovial segmentation.

At step <NUM>, the one or more trained machine learning algorithms can then be implemented to generated segmented MRI data by inputting MRI data acquired from a different subject into the trained machine learning algorithm, generating output as segmented MRI data in which synovial anatomy has been segmented. When applied to MRI data, the resulting synovial segmentation predictions can be used to compute synovial volumes in an unsupervised fashion. Such automated synovial health characterization is advantageous for generating feature maps of osteolysis and/or synovitis.

In other instances, the trained machine learning algorithms can be stored for later use. Storing the machine learning algorithms may include storing weights, biases, or both, which have been computed or otherwise estimated by training the machine learning algorithm on the training data. When the machine learning algorithm implements a neural network, storing the trained machine learning algorithm may include storing the particular neural network architecture to be implemented. For instance, data pertaining to the layers in the neural network architecture (e.g., number of layers, type of layers, ordering of layers, connections between layers) may be stored.

Similar to the methods described above for the synovial volume segmentation, a machine learning algorithm can also be trained as a synovial thickness classifier model. In these implementations, the training data can include maximal synovial thickness measurements estimated or otherwise made from the accessed MRI data. In some implementations, scalar thickness measurements can be converted into an <NUM>-bit classifier table (<NUM> levels) to reduce the dynamic range of the final rectifier filter within the neural network. The encoding path of an encoder-decoder neural network, such as those described in the present disclosure, can be utilized to train the thickness classifier.

As noted, the machine learning algorithm can include a neural network, such as a convolutional neural network, or other suitable machine learning algorithm, such as support vector machines ("SVM") and other algorithms based on supervised learning, unsupervised learning, deep learning, ensemble learning, dimensionality reduction, and so on.

As noted above, in some instances the trained neural network can be an encoder-decoder neural network, such as a three-dimensional encoder-decoder deep neural network. An example of such a neural network is shown in <FIG>. The data input to the network <NUM> are the local field shift maps and the data output are estimates of the source magnetic susceptibility tensor at each input voxel. In this example, the applied encoder-decoder network architecture utilizes skip connections between the encoding path <NUM> and decoding path <NUM>, which can effectively transfer local feature information from the encoding path to the decoding path and facilitate faster training.

The encoding path <NUM> generally implements a convolutional neural network. For instance, the encoding path <NUM> can include repeated application of convolutional layers (e.g., <NUM>×<NUM>×<NUM> convolutions) each followed by a batch normalization layer and a nonlinear layer, which may be a recitified linear unit ("ReLU"). The output of each convolutional layer is a feature map that is passed to the nonlinear layer. Each feature map generally represents a particular feature extracted at all locations on the input. Each nonlinear layer is followed by a downsampling layer, which may be a pooling layer, such as a max pooling layer (e.g., a max pooling layer using stride <NUM>×<NUM>×<NUM>), an average pooling layer, an L2-norm pooling layer, or so on. The output of each nonlinear layer is a feature map that is passed to the downsampling layer. At each downsampling step, the number of feature channels in the feature map can be doubled, or otherwise increased.

The decoding path <NUM> generally includes a transposed convolutional layer with additional feature concatenation layers from the encoding layers. The output of each transposed convolutional layer is a feature map that is passed to the concatenation layer. At each transposed convolution step, the number of feature channels in the feature map can be halved, or otherwise reduced. As noted, each upsampled feature map is also concatenated with the corresponding feature map from the encoding path <NUM>. The concatenated feature map is then passed to a convolutional layer followed by a batch normalization layer and a nonlinear layer (e.g., a ReLU). The output of the convolutional layer is a feature map that is passed to the batch normalization layer, the output of which is a feature map passed to the nonlinear layer. The final layer is a convolutional layer (e.g., a <NUM>×<NUM>×<NUM> convolution) with linear activation, which is applied to output the susceptibility estimations.

In other instances, the neural network architecture shown in <FIG> can be used. Changes to the structure included a smaller image size, additional layers, and other minor parameter updates.

As one non-limiting example of an encoder-decoder neural network architecture that was implemented in an example study, each encoder step included two 3x3, 2D convolution layers followed by a single 2x2 max-pooling layer. Decoder steps had a single 2x2 up convolution followed by two 3x3, 2D convolution layers. The final layer was a single 1x1, 2D convolution which created a feature map of the same size as the input images. The model in this example had a depth of four, meaning there were a total of three max-pooling and up-sampling steps.

In this example, for each epoch, the input data were divided into batches of four, and a random distortion was applied to fifty percent of the data. Iterative performance was calculated using a binary cross entropy loss function for <NUM> epochs. Analysis of the trained model was performed on a voxel-wise basis. Synovitis prediction values were computed across entire slices for both the training, validation, and control testing image sets. Examination of a control testing cohort was performed by identifying slices in which a false positive high thresholded prediction region was found in an area of potential synovitis.

Referring now to <FIG>, an example of a system <NUM> for generating feature maps indicative of osteolysis, synovitis, or both, in accordance with some embodiments of the systems and methods described in the present disclosure is shown. As shown in <FIG>, a computing device <NUM> can receive one or more types of data (e.g., MRI data, segmented MRI data, training data) from image source <NUM>, which may be a magnetic resonance image source. In some embodiments, computing device <NUM> can execute at least a portion of an osteolysis/synovitis feature map generating system <NUM> to generate feature maps that indicate the spatial distribution or spatial patterns of features, statistics, probabilities, estimates, or parameters associated with osteolysis and/or synovitis, these feature maps being generated from data received from the image source <NUM>.

Additionally or alternatively, in some embodiments, the computing device <NUM> can communicate information about data received from the image source <NUM> to a server <NUM> over a communication network <NUM>, which can execute at least a portion of the osteolysis/synovitis feature map generating system <NUM>. In such embodiments, the server <NUM> can return information to the computing device <NUM> (and/or any other suitable computing device) indicative of an output of the osteolysis/synovitis feature map generating system <NUM>.

In some embodiments, computing device <NUM> and/or server <NUM> can be any suitable computing device or combination of devices, such as a desktop computer, a laptop computer, a smartphone, a tablet computer, a wearable computer, a server computer, a virtual machine being executed by a physical computing device, and so on. The computing device <NUM> and/or server <NUM> can also reconstruct images from the data.

In some embodiments, image source <NUM> can be any suitable source of image data (e.g., measurement data, images reconstructed from measurement data), such as an MRI system, another computing device (e.g., a server storing image data), and so on. In some embodiments, image source <NUM> can be local to computing device <NUM>. For example, image source <NUM> can be incorporated with computing device <NUM> (e.g., computing device <NUM> can be configured as part of a device for capturing, scanning, and/or storing images). As another example, image source <NUM> can be connected to computing device <NUM> by a cable, a direct wireless link, and so on. Additionally or alternatively, in some embodiments, image source <NUM> can be located locally and/or remotely from computing device <NUM>, and can communicate data to computing device <NUM> (and/or server <NUM>) via a communication network (e.g., communication network <NUM>).

In some embodiments, communication network <NUM> can be any suitable communication network or combination of communication networks. For example, communication network <NUM> can include a Wi-Fi network (which can include one or more wireless routers, one or more switches, etc.), a peer-to-peer network (e.g., a Bluetooth network), a cellular network (e.g., a <NUM> network, a <NUM> network, etc., complying with any suitable standard, such as CDMA, GSM, LTE, LTE Advanced, WiMAX, etc.), a wired network, and so on. In some embodiments, communication network <NUM> can be a local area network, a wide area network, a public network (e.g., the Internet), a private or semi-private network (e.g., a corporate or university intranet), any other suitable type of network, or any suitable combination of networks. Communications links shown in <FIG> can each be any suitable communications link or combination of communications links, such as wired links, fiber optic links, Wi-Fi links, Bluetooth links, cellular links, and so on.

Referring now to <FIG>, an example of hardware <NUM> that can be used to implement image source <NUM>, computing device <NUM>, and server <NUM> in accordance with some embodiments of the systems and methods described in the present disclosure is shown. As shown in <FIG>, in some embodiments, computing device <NUM> can include a processor <NUM>, a display <NUM>, one or more inputs <NUM>, one or more communication systems <NUM>, and/or memory <NUM>. In some embodiments, processor <NUM> can be any suitable hardware processor or combination of processors, such as a central processing unit ("CPU"), a graphics processing unit ("GPU"), and so on. In some embodiments, display <NUM> can include any suitable display devices, such as a computer monitor, a touchscreen, a television, and so on. In some embodiments, inputs <NUM> can include any suitable input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, and so on.

In some embodiments, communications systems <NUM> can include any suitable hardware, firmware, and/or software for communicating information over communication network <NUM> and/or any other suitable communication networks. For example, communications systems <NUM> can include one or more transceivers, one or more communication chips and/or chip sets, and so on. In a more particular example, communications systems <NUM> can include hardware, firmware and/or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.

In some embodiments, memory <NUM> can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor <NUM> to present content using display <NUM>, to communicate with server <NUM> via communications system(s) <NUM>, and so on. Memory <NUM> can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example, memory <NUM> can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments, memory <NUM> can have encoded thereon, or otherwise stored therein, a computer program for controlling operation of computing device <NUM>. In such embodiments, processor <NUM> can execute at least a portion of the computer program to present content (e.g., images, user interfaces, graphics, tables), receive content from server <NUM>, transmit information to server <NUM>, and so on.

In some embodiments, server <NUM> can include a processor <NUM>, a display <NUM>, one or more inputs <NUM>, one or more communications systems <NUM>, and/or memory <NUM>. In some embodiments, processor <NUM> can be any suitable hardware processor or combination of processors, such as a CPU, a GPU, and so on. In some embodiments, display <NUM> can include any suitable display devices, such as a computer monitor, a touchscreen, a television, and so on. In some embodiments, inputs <NUM> can include any suitable input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, and so on.

In some embodiments, memory <NUM> can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor <NUM> to present content using display <NUM>, to communicate with one or more computing devices <NUM>, and so on. Memory <NUM> can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example, memory <NUM> can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments, memory <NUM> can have encoded thereon a server program for controlling operation of server <NUM>. In such embodiments, processor <NUM> can execute at least a portion of the server program to transmit information and/or content (e.g., data, images, a user interface) to one or more computing devices <NUM>, receive information and/or content from one or more computing devices <NUM>, receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone), and so on.

In some embodiments, image source <NUM> can include a processor <NUM>, one or more image acquisition systems <NUM>, one or more communications systems <NUM>, and/or memory <NUM>. In some embodiments, processor <NUM> can be any suitable hardware processor or combination of processors, such as a CPU, a GPU, and so on. In some embodiments, the one or more image acquisition systems <NUM> are generally configured to acquire data, images, or both, and can include an MRI system. Additionally or alternatively, in some embodiments, one or more image acquisition systems <NUM> can include any suitable hardware, firmware, and/or software for coupling to and/or controlling operations of an MRI system. In some embodiments, one or more portions of the one or more image acquisition systems <NUM> can be removable and/or replaceable.

Note that, although not shown, image source <NUM> can include any suitable inputs and/or outputs. For example, image source <NUM> can include input devices and/or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, a trackpad, a trackball, and so on. As another example, image source <NUM> can include any suitable display devices, such as a computer monitor, a touchscreen, a television, etc., one or more speakers, and so on.

In some embodiments, communications systems <NUM> can include any suitable hardware, firmware, and/or software for communicating information to computing device <NUM> (and, in some embodiments, over communication network <NUM> and/or any other suitable communication networks). For example, communications systems <NUM> can include one or more transceivers, one or more communication chips and/or chip sets, and so on. In a more particular example, communications systems <NUM> can include hardware, firmware and/or software that can be used to establish a wired connection using any suitable port and/or communication standard (e.g., VGA, DVI video, USB, RS-<NUM>, etc.), Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.

In some embodiments, memory <NUM> can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor <NUM> to control the one or more image acquisition systems <NUM>, and/or receive data from the one or more image acquisition systems <NUM>; to images from data; present content (e.g., images, a user interface) using a display; communicate with one or more computing devices <NUM>; and so on. Memory <NUM> can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example, memory <NUM> can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments, memory <NUM> can have encoded thereon, or otherwise stored therein, a program for controlling operation of image source <NUM>. In such embodiments, processor <NUM> can execute at least a portion of the program to generate images, transmit information and/or content (e.g., data, images) to one or more computing devices <NUM>, receive information and/or content from one or more computing devices <NUM>, receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone, etc.), and so on.

In some embodiments, any suitable computer readable media can be used for storing instructions for performing the functions and/or processes described herein. For example, in some embodiments, computer readable media can be transitory or non-transitory. For example, non-transitory computer readable media can include media such as magnetic media (e.g., hard disks, floppy disks), optical media (e.g., compact discs, digital video discs, Blu-ray discs), semiconductor media (e.g., random access memory ("RAM"), flash memory, electrically programmable read only memory ("EPROM"), electrically erasable programmable read only memory ("EEPROM")), any suitable media that is not fleeting or devoid of any semblance of permanence during transmission, and/or any suitable tangible media. As another example, transitory computer readable media can include signals on networks, in wires, conductors, optical fibers, circuits, or any suitable media that is fleeting and devoid of any semblance of permanence during transmission, and/or any suitable intangible media.

Referring particularly now to <FIG>, an example of an MRI system <NUM> that can implement the methods described here is illustrated. The MRI system <NUM> includes an operator workstation <NUM> that may include a display <NUM>, one or more input devices <NUM> (e.g., a keyboard, a mouse), and a processor <NUM>. The processor <NUM> may include a commercially available programmable machine running a commercially available operating system. The operator workstation <NUM> provides an operator interface that facilitates entering scan parameters into the MRI system <NUM>. The operator workstation <NUM> may be coupled to different servers, including, for example, a pulse sequence server <NUM>, a data acquisition server <NUM>, a data processing server <NUM>, and a data store server <NUM>. The operator workstation <NUM> and the servers <NUM>, <NUM>, <NUM>, and <NUM> may be connected via a communication system <NUM>, which may include wired or wireless network connections.

The pulse sequence server <NUM> functions in response to instructions provided by the operator workstation <NUM> to operate a gradient system <NUM> and a radiofrequency ("RF") system <NUM>. Gradient waveforms for performing a prescribed scan are produced and applied to the gradient system <NUM>, which then excites gradient coils in an assembly <NUM> to produce the magnetic field gradients Gx, Gy, and Gz that are used for spatially encoding magnetic resonance signals. The gradient coil assembly <NUM> forms part of a magnet assembly <NUM> that includes a polarizing magnet <NUM> and a whole-body RF coil <NUM>.

RF waveforms are applied by the RF system <NUM> to the RF coil <NUM>, or a separate local coil to perform the prescribed magnetic resonance pulse sequence. Responsive magnetic resonance signals detected by the RF coil <NUM>, or a separate local coil, are received by the RF system <NUM>. The responsive magnetic resonance signals may be amplified, demodulated, filtered, and digitized under direction of commands produced by the pulse sequence server <NUM>. The RF system <NUM> includes an RF transmitter for producing a wide variety of RF pulses used in MRI pulse sequences. The RF transmitter is responsive to the prescribed scan and direction from the pulse sequence server <NUM> to produce RF pulses of the desired frequency, phase, and pulse amplitude waveform. The generated RF pulses may be applied to the whole-body RF coil <NUM><NUM> or to one or more local coils or coil arrays.

The RF system <NUM> also includes one or more RF receiver channels. An RF receiver channel includes an RF preamplifier that amplifies the magnetic resonance signal received by the coil <NUM> to which it is connected, and a detector that detects and digitizes the I and Q quadrature components of the received magnetic resonance signal. The magnitude of the received magnetic resonance signal may, therefore, be determined at a sampled point by the square root of the sum of the squares of the I and Q components: <MAT>.

and the phase of the received magnetic resonance signal may also be determined according to the following relationship: <MAT>.

The pulse sequence server <NUM> may receive patient data from a physiological acquisition controller <NUM>. By way of example, the physiological acquisition controller <NUM> may receive signals from a number of different sensors connected to the patient, including electrocardiograph ("ECG") signals from electrodes, or respiratory signals from a respiratory bellows or other respiratory monitoring devices. These signals may be used by the pulse sequence server <NUM> to synchronize, or "gate," the performance of the scan with the subject's heart beat or respiration.

The pulse sequence server <NUM> may also connect to a scan room interface circuit <NUM> that receives signals from various sensors associated with the condition of the patient and the magnet system. Through the scan room interface circuit <NUM>, a patient positioning system <NUM> can receive commands to move the patient to desired positions during the scan.

The digitized magnetic resonance signal samples produced by the RF system <NUM> are received by the data acquisition server <NUM>. The data acquisition server <NUM> operates in response to instructions downloaded from the operator workstation <NUM> to receive the real-time magnetic resonance data and provide buffer storage, so that data is not lost by data overrun. In some scans, the data acquisition server <NUM> passes the acquired magnetic resonance data to the data processor server <NUM>. In scans that require information derived from acquired magnetic resonance data to control the further performance of the scan, the data acquisition server <NUM> may be programmed to produce such information and convey it to the pulse sequence server <NUM>. For example, during pre-scans, magnetic resonance data may be acquired and used to calibrate the pulse sequence performed by the pulse sequence server <NUM>. As another example, navigator signals may be acquired and used to adjust the operating parameters of the RF system <NUM> or the gradient system <NUM>, or to control the view order in which k-space is sampled. In still another example, the data acquisition server <NUM> may also process magnetic resonance signals used to detect the arrival of a contrast agent in a magnetic resonance angiography ("MRA") scan. For example, the data acquisition server <NUM> may acquire magnetic resonance data and processes it in real-time to produce information that is used to control the scan.

The data processing server <NUM> receives magnetic resonance data from the data acquisition server <NUM> and processes the magnetic resonance data in accordance with instructions provided by the operator workstation <NUM>. Such processing may include, for example, reconstructing two-dimensional or three-dimensional images by performing a Fourier transformation of raw k-space data, performing other image reconstruction algorithms (e.g., iterative or backprojection reconstruction algorithms), applying filters to raw k-space data or to reconstructed images, generating functional magnetic resonance images, or calculating motion or flow images.

Images reconstructed by the data processing server <NUM> are conveyed back to the operator workstation <NUM> for storage. Real-time images may be stored in a data base memory cache, from which they may be output to operator display <NUM> or a display <NUM>. Batch mode images or selected real time images may be stored in a host database on disc storage <NUM>. When such images have been reconstructed and transferred to storage, the data processing server <NUM> may notify the data store server <NUM> on the operator workstation <NUM>. The operator workstation <NUM> may be used by an operator to archive the images, produce films, or send the images via a network to other facilities.

The MRI system <NUM> may also include one or more networked workstations <NUM>. For example, a networked workstation <NUM> may include a display <NUM>, one or more input devices <NUM> (e.g., a keyboard, a mouse), and a processor <NUM>. The networked workstation <NUM> may be located within the same facility as the operator workstation <NUM>, or in a different facility, such as a different healthcare institution or clinic.

The networked workstation <NUM> may gain remote access to the data processing server <NUM> or data store server <NUM> via the communication system <NUM>. Accordingly, multiple networked workstations <NUM> may have access to the data processing server <NUM> and the data store server <NUM>. In this manner, magnetic resonance data, reconstructed images, or other data may be exchanged between the data processing server <NUM> or the data store server <NUM> and the networked workstations <NUM>, such that the data or images may be remotely processed by a networked workstation <NUM>.

Claim 1:
A method for generating feature maps that depict spatial distributions or patterns of features associated with osteolysis or synovitis, the steps of the method comprising:
(a) accessing with a computer system, magnetic resonance imaging (MRI) data acquired from a subject by retrieving the MRI data from a memory or storage device of the computer system;
(b) accessing with the computer system, a trained machine learning algorithm that has been trained on training data to generate feature maps associated with at least one of osteolysis or synovitis;
(c) inputting the MRI data to the trained machine learning algorithm in order to generate output with the computer system, wherein the output comprises:
a first feature map that depicts the spatial distribution of a probability of locations within the subject being associated with at least one of osteolysis or synovitis;
a second feature map that depicts the spatial distribution of a classification of locations within the subject as being associated with a severity of synovitis at those location; and
a third feature map that depicts the spatial distribution of a classification of locations within the subject as being associated with a type of synovitis at those locations.