Patent Description:
Muscle tears, especially tendon tears, are a painful pathology. For example, rotator cuff tears are one of the most common causes of shoulder pain, for example, accounting for almost <NUM> million U. doctor visits in <NUM>. A meta-study estimates an overall prevalence of rotator cuff tears of <NUM>%-<NUM>% in asymptomatic individuals and up to <NUM>% in symptomatic individuals. Furthermore, the incidence of surgical treatments for rotator cuff tears is consistently increasing.

Magnetic Resonance Imaging (MRI) is considered the standard of care for muscle and/or tendon tears, e.g., rotator cuff tear, assessment and treatment planning. The upper extremities, i.e. shoulder and arm constitute <NUM>% of all MR procedures. A tear must be measured in two dimensions on different imaging planes. In addition, it has been shown that rotator cuff muscle structures are prone to fatty infiltration and atrophy after rotator cuff trauma such as tendon tears and that severe fatty infiltration and atrophy correlate with poor functional outcome of rotator cuff repair.

Consistent and accurate measurements of tears, especially rotator cuff tears and assessment of degeneration of the rotator cuff muscle structures (e.g., fatty infiltration and atrophy) are imperative and crucial for selecting the best treatment and surgical approach and impact postoperative prognosis and tear recurrence. In routine clinical practice, radiologists scroll through a large number of MR images to detect tears and then either manually measure the tears in the Picture Archiving and Communication System (PACS) or just estimate the size, and/or to classify fatty infiltration based on the Goutallier classification system and describe the degree of muscle atrophy (volume loss).

However, such techniques face certain restrictions and drawbacks. For instance, MR imaging interpretation of suspected rotator cuff tears is complex and time-consuming, requiring the analysis of several image series acquired in different imaging planes. In addition, studies report significant inter-reader but also intra-reader variabilities resulting in moderate reproducibility in tear assessment, such as quantifications of both tear sizes and the amount of fatty degeneration of the rotator cuff muscle structures.

Accordingly, there is a need for advanced techniques which mitigate or overcome the above-identified drawbacks or restrictions. There is a need for advanced techniques of assessment of muscle and/or tendon tears, such as rotator cuff tears.

Hereinafter, techniques of determining at least one characteristic of the muscle structures are disclosed.

As a general rule, the muscle structure includes at least one muscle. The muscle structure also comprises at least one tendon. The at least one muscle can contract and expand. The at least one tendon can connect the at least one muscle to a bone. Thus, the muscle structure may be surrounding or be arranged adjacently to the bone.

Various characteristics can be determined, such as at least one of the following anatomical characteristics: presence or absence of muscle and/or tendon tears, tear sizes and degeneration grades of muscles of the muscle structure, and especially measuring a size of the muscle and/or tendon tear and assessing degeneration grades of the muscles automatically and precisely.

As a general rule, it would be possible that the at least one characteristic is a classification of a property of the muscle structure into multiple predefined classes. A classification algorithm can be used.

Alternatively or additionally, the at least one characteristic can include a quantification of a property of the muscle structure. Here, it is not required to rely on predefined classes. A regression algorithm can be used.

The at least one characteristic of the muscle structure is determined by using at least two artificial neural networks and based on one or more medical images depicting a muscle structure of a patient.

A computer-implemented method is provided. The method comprises obtaining one or more medical images. The one or more medical images depict a muscle structure of a patient. The muscle structure comprises at least one muscle and at least one tendon. The method further comprises obtaining one or more medical images, the one or more medical images depicting a muscle structure of a patient, wherein the muscle structure comprises at least one muscle and at least one tendon, determining a pre-segmentation of a region of interest associated with the muscle structure, using a first one of at least two artificial neural networks, determining a segmentation of the region of interest associated with the muscle structure in the one or more medical images, wherein the segmentation is determined based on the pre-segmentation, and determining a quantification of at least one characteristic of the muscle structure using a second one of the at least two artificial neural networks based on the segmentation of the region of interest and the one or more medical images.

For example, a quantification of the at least one characteristic could be determined.

For example, the muscle structure could be a rotator cuff muscle structure. The rotator cuff muscle structure comprises multiple muscles, namely: supraspinatus, infraspinatus, teres minor, and subscapularis.

A system comprising at least one processor and at least one memory is provided. The at least one processor is configured to load program code from the at least one memory. Upon executing the program code, the at least one processor performs the computer-implemented method provided above.

A medical imaging scanner comprising a system is provided.

The system is the system provided above.

Various techniques disclosed herein generally relate to determining one or more characteristics of a muscle structure of a patient. The muscle structure includes at least one muscle and typically at least one tendon. An example of the muscle structure would be the rotator cuff. For instance, one or multiple characteristics of the muscle structure, especially those characterizing tears of the muscle structure, e.g., rotator cuff tears, can be determined. It would be possible that the characteristics of the muscle structure of different patients are determined automatically, accurately, and based on the same criteria to facilitate inter-patient comparison.

For example, it would be possible that a quantification of at least one characteristic is determined. The quantification can pertain to a numerical value defined in a continuous result space.

In other examples, it would be possible to determine a classification of at least one characteristic. Here, a discrete set of predefined classes is available and the result is a pointer to one of these predefined classes.

As a general rule, the muscle structure can include a group of muscles. The muscle structure may be close to any one of the following joints of a human body: ball and socket joints, hinge joints, condyloid joints, pivot joints, gliding joints, saddle joints, etc..

For instance, the at least one characteristic can affect multiple muscles of the muscle structure.

As a further general rule, the tear may comprise a tendon tear, a muscle fiber tear, or a strain of a tendon and/or a muscle fiber.

According to various examples, at least one characteristic of the muscle structure is determined based on data collected via non-invasive diagnosis methods, such as at least one of projection radiography, computed tomography (CT), ultrasound imaging, MRI, or any other kind of medical imaging modalities, i.e., the medical imaging data or images being processed or analyzed in this disclosure may comprise at least one of MRI image data, X-ray image data, computed tomography image data, ultrasound image data, or any other kind of medical image data. In particular, imaging data acquired by these imaging modalities may be fed to at least one trained artificial neural network, and thereby the at least one characteristic is determined by the trained artificial neural network precisely and automatically.

As a general rule, projection radiography imaging data and ultrasound imaging data utilized in this disclosure may respectively comprise images in spatial domain. CT imaging data and MR imaging data may respectively comprise reconstructed images in frequency domain, reconstructed images in spatial domain and etc. The medical imaging data may be <NUM>-D data obtained directly from a corresponding scanner, <NUM>-D reconstructed images, or <NUM>-D reconstructed slices comprising multiple voxels.

According to the disclosure, MR imaging, especially an MRI scanner with a main magnetic field of <NUM> Tesla, is preferred in evaluations of shoulder because of a greater signal-to-noise ratio (SNR) and contrast-to-noise ratio (CNR) offered by higher main magnetic field strengths due to faster acquisition time and thinner slice selection. Standard conventional MR imaging of the shoulder is acquired in three orthogonal planes: axial, coronal-oblique and sagittaloblique. Various MR scanning protocols may be utilized to perform a scan of shoulders.

As explained above, the muscle structure described in this disclosure may comprise any one or more muscles and one or more tendons. The muscle may be a human muscle and be muscles of other animals, too. Hereinafter, various techniques of this disclosure will be described in detail based on a rotator cuff as an exemplary muscle structure. , the techniques disclosed in this disclosure can be readily applied to other muscles by simply replacing the term "rotator cuff" with specific names of other muscle structures.

<FIG> schematically illustrates an exemplary MRI image <NUM> acquired in the axial plane and depicting the shoulder. The MRI image <NUM> may comprise a region of interest (ROI) <NUM>, i.e., the area inside the dash-lined rectangular. The region of interest <NUM> is associated with the rotator cuff. The MRI image <NUM> may be cropped to obtain the region of interest <NUM> and then only the region of interest <NUM> may be fed to the at least one trained artificial neural network to determine the at least one characteristic of the rotator cuff. Similarly, ROIs may be defined on other images obtained via projection radiography, CT, ultrasound imaging, or MRI. Alternatively, ROIs may be also defined on a volume, i.e. <NUM>-D.

According to various examples, the ROI <NUM> may be manually determined, e.g., by a clinician.

According to various examples, the ROI <NUM> may be determined based on at least one landmark in one or more medical images/slices, such as a rotator cuff tear, a tendon of the rotator cuff. Such a landmark may be detected by using a landmark detection algorithm configured to detect the ROI <NUM> in the one or more medical images <NUM>. Such a landmark detection algorithm may be a machine-learning algorithm.

According to the disclosure, the ROI may comprise at least one tendon comprising rotator cuff muscles, such as the tendon of the Supraspinatus muscle, the tendon of the Infraspinatus muscle, the tendon of the Teres minor muscle or the tendon of the Subscapularis muscle. The following examples may be in the context of a single tendon and/or a single muscle, i.e., the techniques disclosed hereinafter may respectively apply to each individual tendon and/or muscle based on a precise ROI detection. Accordingly, multiple ROIs may be detected in one medical image or slice.

As a further general rule, image pre-processing techniques may be applied to the images or imaging data before they are fed to the at least one trained artificial neural network, for example, cropping images or slices to obtain ROIs, down-sampling to reduce resolutions of images or slices, filtering out noises. Additionally, when the one or more medical images (or imaging data) depicting a rotator cuff of a patient comprises multiple images or slices acquired by one or multiple imaging modalities, registration may be applied to the one or more medical images (or imaging data).

According to the disclosure, the at least one characteristic of the rotator cuff may comprise presence or absence and/or position and/or a type of a tear in a tendon of the rotator cuff. The at least one characteristic may further comprise a classification and optionally a quantification of a muscle atrophy of the rotator cuff. Additionally or alternatively, the at least one characteristic may comprise a classification and optionally a quantification of a fatty infiltration of the rotator cuff. The at least one characteristic may also comprise rotator cuff tear length, width, thickness, and musculotendinous junction position.

According to the disclosure, the Goutallier classification (e.g., See: <NPL>. ) may be used to quantify the amount of fatty degeneration of the rotator cuff muscles, particularly in the context of rotator cuff tendon tears. Although originally described in shoulder CT, it is applicable and now most commonly used in MRI. It is based mainly on the percentage of atrophy and fatty degeneration of the involved muscle. The grading increases in severity and higher grades correlate with poorer function outcomes following rotator cuff tear surgical repair. The Goutallier classification system may comprise five grades as shown in Table <NUM>.

<FIG> schematically illustrates details with respect to a system <NUM> according to various examples. The system <NUM> may comprise the at least two artificial neural networks receiving the one or more medical images <NUM> depicting a rotator cuff of a patient as input and outputting, for each of the at least one characteristic, a value indicative of corresponding characteristic. The value indicative of corresponding characteristic may be written in a file <NUM>. The system <NUM> may comprise only one artificial neural network comprising multiple sub-networks, which is not part the invention.

According to various examples, the system <NUM> may comprise a convolutional neural network <NUM> which may be configured to perform a pre-segmentation (or coarse segmentation) of the one or more medical images <NUM> to determine a pixel probability map <NUM>, or directly determine a pixel mask <NUM>. The pixel mask <NUM> is obtained by applying a threshold comparison to probability values of the pixel probability map <NUM> and selecting a largest contiguous area.

As a general rule, a convolutional neural network may include one or more layers that perform convolutions. Here, a predefined kernel - the weights of which are set during a training phase - is convoluted against input values that are output by a preceding layer.

According to the disclosure, the system <NUM> may further comprise a first one <NUM> and a second one <NUM> of the at least two artificial neural networks. The first one <NUM> of the at least two artificial neural networks may be configured to determine a segmentation <NUM> of a region of interest <NUM> associated with the rotator cuff in the one or more medical images <NUM> based on the pre-segmentation, for instance based on the pixel probability map <NUM> or the pixel mask <NUM>. The segmentation <NUM> of the region of interest <NUM> determined by the first one <NUM> of the at least two artificial neural networks may be further based on the one or more medical images <NUM> or the ROI <NUM>. Additionally or alternatively, the segmentation <NUM> may comprise a bounding box which may be determined based on the pixel mask <NUM>. The second one <NUM> of the at least two artificial neural networks may be configured to determine a value indicative of the at least one characteristic based on the segmentation <NUM> of the region of interest <NUM> and the one or more medical images <NUM>.

According to various examples, when the at least one characteristic comprises multiple characteristics, the system <NUM> may comprise multiple decoder branches configured to respectively determine the multiple characteristics based on a shared set of latent features determined based on the one or more medical images <NUM>. For example, the second one <NUM> of the at least two artificial neural networks may comprise multiple decoder branches 2070a - 2070d. Each one of the multiple decoder branches 2070a - 2070d may respectively determine at least one distinct characteristic. For instance, the multiple decoder branches 2070a - 2070d may determine a size of a tear (e.g., including length, width, and thickness), a percentage of muscle atrophy, a grade of muscle fatty infiltration, and a coordinate of musculotendinous junction position, respectively.

As a general rule, the pixel probability map <NUM>, the pixel mask <NUM>, the segmentation <NUM>, and the bounding box may be <NUM>-D or <NUM>-D.

As a general rule, various kinds and types of artificial neural network can be used as the first one <NUM> and the second one <NUM> of the at least two artificial neural networks and benefit from the techniques described herein, such as convolutional neural networks, reinforcement neural networks, residual neural network, recursive neural network, recurrent neural work, Long short-term memory (LSTM) neural network etc.. For instance, it would be possible to use a deep neural network, e.g., a convolutional neural network having one or more convolutional layers performing convolutions between the input data and a kernel, to implement both the first one <NUM> and the second one <NUM> of the at least two artificial neural networks. It would also be possible to use a support vector machine, to give just a few examples. Preferably, the first one <NUM> of the at least two artificial neural networks may comprise an encoder and a decoder, see, e.g., <NPL>, or <NPL>. Preferably, the second one <NUM> of the at least two artificial neural networks may use deep reinforcement learning approach, see, e.g.,<NPL>. Additionally or alternatively, muscle atrophy and/or muscle fatty infiltration grading in the ROI <NUM> may be performed by a residual learning framework, see, e.g., <NPL>.

According to various examples, the at least two artificial neural networks may be implemented by a deep reinforcement learning approach to determine quantification of the tear, such as the tear location (i.e., coordinates of start and end points, respectively), length, width or thickness. Alternatively or optionally, two or more different artificial neural networks may be utilized to implement the at least two artificial neural networks for determining classification and optionally the quantification of the muscle atrophy or the fat infiltration.

According to the disclosure, the convolutional neural network <NUM> may share the same network architecture as that of the first one <NUM> of the at least two artificial neural networks. However, the input of the convolutional neural network <NUM> may have a lower resolution than that of the first one <NUM> of the at least two artificial neural networks. The lower resolution can mean that the size of an input vector is smaller; thus, fewer spatial points are sampled by the input vector.

According to various examples, the medical images <NUM> or the ROI <NUM> fed to the first one <NUM> and/or second one <NUM> of the at least two artificial neural networks also have a lower resolution than that of the original medical images, for example, medical images acquired by a medical imaging scanner, to reduce the computational burden.

According to the disclosure, various training methods of artificial neural networks may be applied to train the at least two artificial neural networks, such as supervised learning, un-supervised learning, semi-supervised learning, reinforce learning and etc..

For example, the convolutional neural network <NUM>, the first one <NUM> and the second one <NUM> of the at least two artificial neural networks may be trained separately using different sets of training data based on supervised learning techniques. Each training process can include determining a loss value based on a comparison between a prediction of the respective one <NUM>, <NUM> of the at least two artificial neural networks and a ground truth. A loss function can provide the loss value by performing the comparison. Based on the loss value, it is then possible to adjust the weights of the artificial neural networks. Here, an optimization algorithm, e.g., gradient descent, can be employed. Backpropagation can be an alternative.

Each set of the training data generally comprises one or more training medical images and a set of ground truth in which information relating to the region of interest <NUM> is indicated. For example, the set of training data for the convolutional neural network <NUM> may comprise annotated pixel probability maps or annotated pixel masks, the set of training data for the first one <NUM> of the at least two artificial neural networks may comprise annotated segmentation of the region of interest <NUM>, and the set of training data for the second one <NUM> of the at least two artificial neural networks may comprise clinical or real values quantifying corresponding characteristic of the rotator cuff. These labels could be assigned by domain experts, e.g., clinicians. Alternatively, the one or more training medical images for the convolutional neural network <NUM>, the first one <NUM> and the second one <NUM> of the at least two artificial neural networks may be identical.

On the other hand, the convolutional neural network <NUM>, the first one <NUM> and the second one <NUM> of the at least two artificial neural networks may be trained jointly, i.e., the three neural networks may be regarded as a whole and parameter values of the three neural networks are updated together by using, for example, back propagation in a joint optimization process based on a common loss value. This corresponds to end-to-end training.

Employing neural networks trained using appropriate sets of training data provide a robust way of processing the medical image data. Neural networks may deal with complex medical image data and provide more accurate results than other prior art computational techniques for image processing.

According to various examples, to eliminate processing steps for obtaining annotated training data - e.g., manual annotation by a domain expert -, unsupervised learning may be used to train the convolutional neural network <NUM>, the first one <NUM> and the second one <NUM> of the at least two artificial neural networks, such as using constraints as disclosed in <NPL>. Reinforcement learning may be an alternative.

According to various examples, each of the convolutional neural network <NUM>, the first one <NUM> and the second one <NUM> of the at least two artificial neural networks may be trained using different training techniques, respectively. For example, the convolutional neural network <NUM>, the first one <NUM> and the second one <NUM> of the at least two artificial neural networks may be respectively trained by using supervised learning, unsupervised learning, and semi-supervised learning.

Next, aspects with respect to a processing workflow to determine at least one characteristic of a rotator cuff will be explained in connection with <FIG>.

Specifically, the processing workflows can implement the processing discussed above in connection with <FIG>. The system <NUM> can implement the processing workflows discussed in connection with the processing pipelines <NUM> and <NUM>.

<FIG> is an exemplary clinical processing pipeline <NUM> for automatically determining a quantification of at least one characteristic of a rotator cuff tendon tear based on medical imaging data, such as MR imaging data. The quantification of at least one characteristic may comprise a size of the rotator cuff tendon tear (e.g., length, width, thickness) and/or a musculotendinous junction position.

As a general rule, while techniques will be described in connection with determining the quantification of the at least one characteristic, as a general rule, it would be possible to determine a classification of the at least one characteristic. Similar techniques as those described below can be readily applied.

The processing pipeline <NUM> may be executed by the system <NUM> of <FIG> upon loading program code. Details of the processing pipeline <NUM> are described below.

At block <NUM>, an image series (e.g., MRI series) for measurements are defined by the clinical protocol, e.g., the mediolateral dimension of a supraspinatus tendon tear, are measured, e.g., on a coronal oblique fluid-sensitive series. For a selection of an appropriate image series, the system <NUM> may use DICOM (Digital Imaging and Communications in Medicine) information. The image series may be loaded from a database. An imaging device may be controlled to acquire and provide the image series. Thereby, one or more medical images are obtained.

At block <NUM>, presence or absence and/or a type of the tear in the tendon of the rotator cuff is determined using a further AI agent, such as those disclosed in a patent application <CIT>. When performing a medical imaging scan, e.g., an MR scan, the acquired imaging data during the scan, such as image slices, may comprise imaging data depicting both healthy and diseased tissues (e.g., tendon tears). The imaging data depicting diseased tissues are particularly important from a clinical perspective. By using the further AI agent, the imaging data acquired during the scan can be automatically and precisely assigned to at least a first group and a second group respectively corresponding to both the healthy and diseased tissues based on the presence or absence of the tear in the tendon. The following-up processing of the imaging data may focus on the second group, i.e., the imaging data corresponding to the diseased tissues, and thereby consumption of computational resources is reduced. , a pre-filtering can be performed. Further, the further AI agent may determine the type of the tear in the tendon based on the second group of the acquired imaging data. Alternatively, the presence or absence and/or the type of the tear in the tendon may be determined by using a classification algorithm. The classification algorithm is configured to select a class from a list indicating: no tendon tear; a presence of a tendon tear; a partial tear; a low-grade partial tear; a high-grade partial tear; or a full tear.

Block <NUM> thus corresponds to pre-processing.

At block <NUM>, one or more medical images <NUM> depicting the rotator cuff are selected based on detection of the presence of the tear and/or the type of the tear. The one or more medical images <NUM> may be processed based on a landmark detection algorithm configured to detect a region of interest <NUM> in the one or more medical images <NUM>. Other image processing techniques, such as down-sampling and/or normalizing may be applied before and/or after detecting the ROI <NUM> by the landmark detection algorithm.

The pre-processing of blocks <NUM> and <NUM> can, as a general rule, also be combined in a single pre-processing block.

At block <NUM>, a coarse tear segmentation is performed by a convolutional neural network, such as <NUM> of <FIG>. The convolutional neural network <NUM> may provide, as an output, a pixel probability map <NUM>. To get a binary segmentation mask, e.g., a pixel mask <NUM>, a threshold is applied at, for example, <NUM>, on the pixel probability map <NUM>. The largest connected component is chosen as the final coarse segmentation mask <NUM>. The segmentation mask <NUM> is transferred to the original full resolution images. A bounding box with extra padding in all dimensions may be generated and define the ROI.

At block <NUM>, fine tear segmentation in the ROI <NUM> is performed by the first one <NUM> of the at least two artificial neural networks and thereby a segmentation <NUM> of the ROI <NUM> associated with the rotator cuff in the one or more medical images <NUM> is determined.

At block <NUM>, coordinates of the tear, such as respective coordinates of start and end points of the tear, may be extracted from a segmentation mask, e.g., the segmentation <NUM>, by finding the most distant voxels in the segmentation mask and projecting them into the slice at the center of gravity of the segmentation mask.

At block <NUM>, the coordinates of the tear are transferred to a target system, such as a PACS workstation, a PACS storage and / or a RIS (Radiological Information System).

The exemplary clinical processing pipeline <NUM> facilitates determining quantification of at least one characteristic of a rotator cuff tendon tear, such as a size of the rotator cuff tendon tear (e.g., length, width, thickness) and/or a musculotendinous junction position, based on medical imaging data and thereby significantly improves treatments and/or surgery planning of rotator cuff tears of patients. The exemplary clinical processing pipeline <NUM> utilizes AI-based techniques and may significantly reduce the time to determine such characteristics compared to manual approach, and provide consistent and reproducible results which may address both the inter-reader and intra-reader variability in current clinical practice.

<FIG> is an exemplary clinical processing pipeline <NUM> for automatically classifying and quantifying muscle quality of a rotator cuff based on medical imaging data, such as MR imaging data. The muscle quality of a rotator cuff may comprise at least one of a muscle atrophy or a fat infiltration.

The processing pipeline <NUM> may be also executed by the system <NUM> of <FIG> upon loading program code. The processing pipeline <NUM> are described below in detail.

At block <NUM>, an image series (i.e., MRI series) for classifying and quantifying muscle quality of the rotator cuff is obtained, for example, from a database of rotator cuff MRI study. It would be possible that the image series is loaded from an imaging device.

Block <NUM> generally corresponds to block <NUM>.

At block <NUM>, the scapular-Y structure of the rotator cuff is determined from the scapula bone and a type of a tear in each rotator cuff tendon is classified based on a landmark and using an AI agent. The AI agent may be those disclosed in a patent <CIT> or a patent application <CIT>.

According to various examples, multiple muscles of a muscle structure may be determined to be having muscle quality degradation. For each individual muscle, the following techniques may be applied.

At block <NUM>, one or more medical images <NUM> depicting the rotator cuff is selected based on detection of the presence of the tear and/or the type of the tear to perform muscle quality grading. The one or more medical images <NUM> may be processed based on a landmark detection algorithm configured to detect a region of interest <NUM> in the one or more medical images <NUM>. Other image processing techniques, such as down-sampling and/or normalizing and/or cropping may be applied before and/or after detecting the ROI <NUM> by the landmark detection algorithm.

Optionally, a coarse tear segmentation is performed by a convolutional neural network, such as <NUM> of <FIG>. The convolutional neural network <NUM> may provide, as an output, a pixel probability map <NUM>. To get a binary segmentation mask, e.g., a pixel mask <NUM>, a threshold is applied at, for example, <NUM>, on the pixel probability map <NUM>. The largest connected component is chosen as the final coarse segmentation mask <NUM>. The segmentation mask <NUM> is transferred to the original full resolution images. A bounding box with extra padding in all dimensions may be generated and define the ROI.

At block <NUM>, the muscle structure in the ROI is segmented by, for example, the first one <NUM> of the at least two artificial neural networks, and optionally based on the segmentation mask <NUM>, and thereby a segmentation <NUM> of the ROI <NUM> associated with the rotator cuff in the one or more medical images <NUM> is determined.

At block <NUM>, based on both the segmentation <NUM> of the ROI <NUM> and the ROI <NUM>, both the muscle atrophy and the fat infiltration are classified, respectively. Alternatively, the classification of the muscle atrophy and the fat infiltration may be further based on the classification of the tear.

At block <NUM>, based on both the segmentation <NUM> of the ROI <NUM> and the ROI <NUM>, both the muscle atrophy and the fat infiltration are quantified.

At block <NUM>, both the classification and quantification of the muscle atrophy and the fat infiltration are rendered and optionally an exam report is prepopulated.

At block <NUM>, both the classification and quantification of the muscle atrophy and the fat infiltration are transferred to a target system, such as a PACS workstation, a PACS storage and / or a RIS (Radiological Information System).

The exemplary clinical processing pipeline <NUM> facilitates automate classification and quantification of muscle quality and thereby improves postoperative evaluation of surgeries of rotator cuff tears.

The processing pipelines <NUM> and <NUM> of <FIG> are modular. This means that it is not required that all blocks are implemented. Also, further blocks may be implemented. For instance, it may be optional to upload results to the RIS and/or PACS at blocks <NUM> and <NUM>, respectively. The pre-processing at blocks <NUM>, <NUM> and <NUM>, <NUM>, respectively may be optional.

<FIG> is a flowchart of a method <NUM> according to various examples. The method <NUM> pertains to determining, by using at least two artificial neural networks, such as the at least two artificial neural networks of the system <NUM>, quantification of at least one characteristic of a muscle structure comprising a tendon, such as the rotator cuff, based on one or more medical images <NUM> depicting the muscle structure comprising the tendon of a patient.

Optional blocks are labeled with dashed lines.

The method <NUM> may be executed by a computer comprising at least one processing unit, or by the system <NUM> of <FIG> upon loading program code. Details of the method <NUM> are described below.

At block <NUM>, one or more medical images <NUM> are obtained, for example by the system <NUM> of <FIG>. The one or more medical images <NUM> depicts a muscle structure comprising a tendon of a patient.

The medical images <NUM> could be loaded from a picture archiving system (PACS). Block <NUM> could include controlling an MRI unit to acquire the images. The medical images <NUM> could be loaded from a memory. Alternatively, the medical images <NUM> may be received directly from a medical imaging scanner during a scan to perform a real-time determination of the quantification of at least one characteristic of the rotator cuff.

At block <NUM>, the quantification of at least one characteristic of the muscle structure is determined using at least two artificial neural networks. The at least two artificial neural networks may be implemented according to the system <NUM> of <FIG>.

Optionally, at block <NUM>, said obtaining of the one or more medical images <NUM> may comprise pre-processing the one or more medical images <NUM> based on a landmark detection algorithm configured to detect a region of interest <NUM> in the one or more medical images <NUM>. Such a landmark detection algorithm may be a machine-learning algorithm, see, e.g., <CIT>. The region of interest <NUM> is associated with the muscle structure. Additionally or alternatively, the image pre-processing techniques described above may be applied before and/or after detecting the ROI <NUM> by the landmark detection algorithm.

At block <NUM>, a pre-segmentation of the region of interest associated with the muscle structure is determined, for example by the convolutional neural network <NUM> of <FIG>. The convolutional neural network <NUM> may provide, as an output, a pixel probability map <NUM>. The pre-segmentation may be obtained as a pixel mask <NUM> by applying a threshold comparison to probability values of the pixel probability map <NUM> and by selecting a largest contiguous area.

The pre-segmentation of the ROI <NUM> determined by the convolutional neural network <NUM> may be based on a lowresolution version of the ROI <NUM> of the one or more medical images <NUM>, for example obtained by applying down-sampling to the one or more medical images <NUM>, which may speed up the determination of the pre-segmentation.

At block <NUM>, a segmentation <NUM> of the region of interest <NUM> associated with the muscle structure in the one or more medical images <NUM> is determined using a first one <NUM> of the at least two artificial neural networks. The segmentation <NUM> is determined based on the pre-segmentation, i.e., the pixel mask <NUM>, or may be based on pixel probability map <NUM>. The segmentation <NUM> of a region of interest <NUM> determined by the first one <NUM> of the at least two artificial neural networks is further based on the one or more medical images <NUM> and the ROI <NUM>.

Additionally or alternatively, the segmentation <NUM> may comprise a bounding box which may be determined based on the pixel mask <NUM>. As a general rule, the bounding box may be the rectangular (in <NUM>-D) or cuboid (in <NUM>-D) of smallest volume that enclosed all relevant features. The determination of the bounding box may comprise determining the location coordinates of the bounding box (e.g. x, y and z coordinates), size parameters of the bounding box (e.g. height, width and depth), and orientation parameters of the bounding box (e.g. angles θx, θy and θz which are angles about the x, y and z axes defined with respect to the medical images <NUM>) for each <NUM>-D medical image or <NUM>-D medical slices. The described coordinates and parameters may be determined such that the bounding box indicates the ROI <NUM>. According to various examples, full or original resolution of the ROI <NUM> may be selected based on the pre-segmentation, i.e., the pixel mask <NUM>. Then, the full or original resolution of the ROI <NUM> may be fed to the first one <NUM> of the at least two artificial neural networks to determine the segmentation <NUM>.

Optionally or alternatively, the method <NUM> may, at block <NUM>, further comprise using at least a second <NUM> one of the at least two artificial neural networks, determining a value indicative of the quantification of the at least one characteristic based on the segmentation <NUM> of the region of interest <NUM> and the one or more medical images <NUM>.

According to various examples, when the at least one characteristic comprises multiple characteristics, the at least two artificial neural networks, e.g., the system <NUM>, may be trained for each individual characteristic separately. , quantification of each individual characteristic may be determined by using the at least two artificial neural networks having different parameter values, for example several systems <NUM> having the same network architecture but different parameter values.

Optionally or alternatively, when the at least one characteristic comprises multiple characteristics, the quantification of the at least one characteristic may be determined jointly based on the same one of the at least two artificial neural networks, e.g., the same system <NUM>. For example, the at least two artificial neural networks may comprise multiple decoder branches 2070a-2070d configured to determine the quantification of the multiple characteristics based on the same input, such as a combination of the segmentation <NUM> and the medical images <NUM>, or in particular, based on a shared set of latent features determined based on the one or more medical images <NUM>. The multiple decoder branches 2070a-2070d may be parts of the second <NUM> one of the at least two artificial neural networks.

According to various examples, when the one or more medical images <NUM> are or comprise multi-slice images, the method <NUM> may further comprise, at block <NUM>, determining a reference point of the region of interest <NUM> based on the segmentation <NUM>, for example, the location of the detected landmarks may be used as the reference point of the ROI <NUM>; and at block <NUM>, selecting a given slice from multiple slices of the one or more medical images <NUM> based on the reference point. Then, the quantification of the at least one characteristic may be determined based on an appearance of the muscle structure in the given slice.

According to the disclosure, for example, coordinates of a tear, such as respective coordinates of start and end points of the tear, may be extracted from a segmentation mask, e.g., the segmentation <NUM>, by finding the most distant voxels in the segmentation mask and projecting them into the slice at the center of gravity of the segmentation mask. Alternatively, the coordinates of the tear may be determined by first determining the center of gravity of the segmentation mask, then selecting the image slice at the center of gravity and measuring the two most distant voxels of the segmentation mask only in the selected image slice. Alternatively or optionally, the coordinates of the tear may be determined based on distances between the two most distant voxels of each slice of the segmentation mask, e.g., selecting the one with the largest length to determine the coordinates, or using a further machine-learning algorithm to determine the best slice to extract the coordinates.

Optionally or alternatively, the at least two artificial neural networks may comprise a first artificial neural network configured to determine the presence or absence and/or the type of the tear of the muscle structure. The at least two artificial neural networks may further comprise a second artificial neural network configured to determine the classification and optionally the quantification of the muscle atrophy or the fat infiltration. The second artificial neural network may obtain, as an input, an output of the first artificial neural network.

According to various examples, after determining the quantification of the at least one characteristic of the muscle structure, the method <NUM> may further comprise at least one of rendering a visualization of the muscle structure including the value of the at least one characteristic, prepopulating an exam report comprising the value of the at least one characteristic, or sending the value of at least one characteristic to the PACS.

According to the disclosure, the method <NUM> facilitates automatic and precise determinations of the quantification of at least one characteristic of the muscle structure by using at least two artificial neural networks. The method <NUM> may significantly reduce the time to determine such quantification of characteristics compared to manual approach, and provide consistent and reproducible results which may address both the inter-reader and intra-reader variability in current clinical practice. Thus, treatments and surgery planning are improved greatly.

<FIG> is a block diagram of a system <NUM> according to various examples. The system <NUM> provides a functionality of determining quantification of at least one characteristic of the muscle structure based on the method <NUM>.

The system <NUM> may comprise at least one processor <NUM>, at least one memory <NUM>, and at least one input/output interface <NUM>. The at least one processor <NUM> is configured to load program code from the at least one memory <NUM> and execute the program code. Upon executing the program code, the at least one processor <NUM> performs the method <NUM>.

According to the disclosure, a medical imaging scanner, such as a CT scanner, an MRI scanner, an ultrasound scanner, or an x-ray scanner, may comprise the system <NUM> of <FIG>. The medical imaging scanner may determine quantification of at least one characteristic of the muscle structure while performing a scan of a shoulder of a patient.

Alternatively, the system <NUM> may be embedded in or connected with the medical imaging scanner and thereby the medical imaging scanner may be also configured to perform the method <NUM>.

Claim 1:
A computer-implemented method (<NUM>), comprising:
- obtaining (<NUM>) one or more medical images (<NUM>), the one or more medical images depicting a muscle structure of a patient, wherein the muscle structure comprises at least one muscle and at least one tendon,
- determining (<NUM>) a pre-segmentation (<NUM>) of a region of interest (<NUM>) associated with the muscle structure,
- using a first one (<NUM>) of at least two artificial neural networks, determining (<NUM>) a segmentation (<NUM>) of the region of interest (<NUM>) associated with the muscle structure in the one or more medical images (<NUM>), wherein the segmentation (<NUM>) is determined based on the pre-segmentation (<NUM>), and
- determining (<NUM>) a quantification of at least one characteristic of the muscle structure using a second one (<NUM>) of the at least two artificial neural networks based on the segmentation (<NUM>) of the region of interest (<NUM>) and the one or more medical images (<NUM>).