Patent Description:
Computed tomography (CT) imaging ("conventional CT") is a widely used and known approach for imaging a subject. Spectral computed tomography (CT) imaging is an imaging modality that extends the capabilities of conventional CT.

Dual-energy (DE) CT is one specific configuration of spectral CT that utilizes two attenuation values acquired simultaneously at two photon energies. These attention values can be used to solve the photoelectric and Compton contribution that consists of the mass attenuation coefficient of a material. It is therefore possible to identify an unknown material by its value of photoelectric and Compton contribution. This imaging modality works especially well in materials, such as iodine, that have k-edge energy close to the mean value of a diagnostic energy range. Because any two linearly independent sums of two basis functions span the entire attenuation coefficient space, any material can be represented by a linear combination of two other materials, so called basis materials, such as water and iodine.

Recent technical advances have developed several approaches for performing dual-energy CT accusation such as dual-source, fast kVp switching, and dual-layer detector configurations.

Quantitative imaging has been of increasing interest to the medical imaging community. Spectral CT modalities supports this trend, as the additional spectral information improves the quantitative information that can be measured about the scanned object and its material composition.

For example, document <CIT> discloses a computed tomography scanner that is configured to generate spectral volumetric image data.

Diagnostic radiology aims to perform complete detection of all findings or abnormalities during an imaging examination, and perform accurate assessment and/or diagnosis of the same. However, error rates during radiologic examinations are relatively high. Overall, approximately <NUM>% of abnormal radiographic studies are missed, and approximately <NUM>% of radiologic interpretations rendered by radiologists in daily practice contain errors. As a result, there is a clear and increasing demand to improve the diagnostic reading environment and tools for aiding a clinician to review and assess CT images.

According to examples in accordance with an aspect of the invention, there is provided a computer-implemented method of controlling an alert signal responsive to differences between spectral CT image data and predicted spectral CT image data.

The computer-implemented method comprises: obtaining non-spectral CT image data and spectral CT image data, each representing a same region of interest of the subject; processing the non-spectral CT image data, using a machine-learning algorithm, to produce predicted spectral CT image data, the predicted spectral CT image data being a prediction of the spectral CT image data derived from the non-spectral CT image data; processing the spectral CT image data and the predicted spectral CT image data to identify any differences between the spectral CT image data and the predicted spectral CT image data; and controlling an alert signal responsive to any identified differences.

The present disclosure proposes an approach for generating an alert signal that indicates whether there is a potential anatomical feature or abnormality that cannot be clearly seen in non-spectral CT image data, but may still have some diagnostic importance. Embodiments are based on the realization that differences between "true" spectral CT image data and predicted spectral CT image data (generated using non-spectral CT image data) represent anatomical features or abnormalities that cannot be readily identified in the original non-spectral CT image data. In other words, a difference effectively represents a measure of imperceptibility of a particular (anatomical) feature of potential clinical relevance.

By generating an alert signal responsive to such differences, this information can be used (for instance) to alert a clinician as to a possible presence and/or position of a potential anatomical feature or abnormality not clearly visible (to the human eye) in the non-spectral CT image data.

Differences between the spectral CT image data and the predicted CT image data may thereby effectively represent anatomical features that are identifiable in the spectral CT image data, but which are not identifiable in the predicted CT spectral CT image data and/or the non-spectral CT image data.

A further advantage of the proposed approach is that the machine-learning algorithm does not require user-annotated training data. Thus, an unsupervised learning approach can be used, which significantly increases an available amount of training data and reduces professional and/or clinician time for annotating the training data. In this way, accurate identification of potentially clinically relevant areas can be achieved without the need for a machine-learning method that makes use of a supervised learning technique.

In some examples, the spectral CT image data comprises a set of one or more spectral CT images; the predicted CT image data comprises a respective set of one or more predicted spectral CT images, each predicted spectral CT image corresponding to a respective spectral CT image; and the step of identifying any differences between the spectral CT image data and the predicted CT image data comprises, for each spectral CT image, identifying any differences between the spectral CT image and the corresponding predicted spectral CT image.

The step of identifying any differences between the spectral CT image and the corresponding predicted spectral CT image may comprise processing the spectral CT image and the corresponding predicted spectral CT image using one or more functions to produce a difference image that identifies a presence and position of differences between the spectral CT image and the corresponding predicted spectral CT image.

The identified presence and position of differences may be used, for instance, to alert a clinician as to the presence of potential anatomical features or abnormalities that might otherwise be overlooked by the clinician. In particular the position of differences may be used to identify a most relevant spectral CT image for review by the clinician.

Optionally, the step of processing the spectral CT image and the corresponding predicted spectral CT image comprises, for each pixel/voxel of the spectral CT image, determining a difference between the pixel/voxel of the spectral CT image and the corresponding pixel/voxel of the predicted spectral CT image.

In some examples, the step of processing the spectral CT image and the corresponding predicted spectral CT image comprises: performing a standard deviation filtering function on the spectral CT image to produce a standard deviation spectral CT image having a same resolution as the spectral CT image; and for each pixel/voxel of the spectral CT image, dividing the determined difference by a value derived from the value of the corresponding pixel/voxel of the standard deviation spectral CT image.

The value derived from the value of the corresponding pixel/voxel of the standard deviation spectral CT image may be equal to the value of the corresponding pixel/voxel of the standard deviation spectral CT image or equal to the value of the corresponding pixel/voxel of the standard deviation spectral CT image plus a predetermined value. This latter embodiment helps to reduce a likelihood of dividing by zero, e.g. reduce the likelihood of a divide-by-zero error occurring. The predetermined value may be very small (e.g. <<NUM> or less than <NUM>% of a maximum possible value for a pixel of a spectral CT image).

In at least one embodiment, the step of processing the spectral CT image and the corresponding predicted CT image comprises processing the spectral CT image and the corresponding predicted CT image using a cross-correlation or mutual information function.

The step of controlling the alert signal may comprise controlling the alert signal responsive to the presence and position of differences between the spectral CT image and the corresponding predicted CT image in any difference map.

The set of one or more spectral CT images may comprise a plurality of spectral CT images. In some examples, the method further comprises identifying, based on any identified differences, one or more of the plurality of spectral CT images that contains a representation of the cause of the identified difference.

The method may further comprise controlling a user-perceptible output responsive to the alert signal.

The computer implemented method may further comprising obtaining input parameter data, the input parameter data defining one or more parameters of the acquisition protocol used, by a CT imaging system, in acquiring the non-spectral CT image data and/or the spectral CT image data, preferably wherein the step of processing the non-spectral CT image data comprises processing the non-spectral CT image data and the input parameter data using the machine-learning algorithm to produce the predicted spectral CT image data.

In other words, the machine-learning algorithm may receive as input (features) the non-spectral CT image data and the input parameter data.

In at least one example, the computer-implemented method further comprises obtaining reconstruction parameter data, the reconstruction parameter data defining one or more parameters of the reconstruction protocol used, by a CT imaging system, to process raw CT data to produce the non-spectral CT image data and/or the spectral CT image data, preferably wherein the step of processing the non-spectral CT image data comprises processing the non-spectral CT image data and the reconstruction parameter data using the machine-learning algorithm to produce the predicted spectral CT image data.

Of course, in some embodiments, the step of processing the non-spectral CT image data comprises processing the non-spectral CT image data, the input parameter data and the reconstruction parameter data (using the machine-learning algorithm) to produce the predicted spectral CT image data. In other words, the machine-learning algorithm may receive as input (features) the non-spectral CT image data, the input parameter data and/or the reconstruction parameter data.

The method may further comprise obtaining subject data, the subject data defining one or more parameters of the subject, wherein the step of processing the non-spectral CT image data comprises processing the non-spectral CT image data and the subject data using the machine-learning algorithm to produce the predicted spectral CT image data.

Of course, it will be appreciated that the step of processing the non-spectral CT image data comprises processing the non-spectral CT image data and one or more of: the input parameter data, the reconstruction parameter data and/or the subject data (using the machine-learning algorithm) to produce the predicted spectral CT image data. In other words, the machine-learning algorithm may receive as input (features) the non-spectral CT image data and one or more of: the input parameter data, the reconstruction parameter data and/or the subject-data.

There is also proposed a computer program product comprising computer program code means which, when executed on a computing device having a processing system, cause the processing system to perform all of the steps of the method as herein described. The computer program product may be formed on a (non-transitory) storage medium.

There is also proposed a processing system for controlling an alert signal responsive to differences between spectral CT image data and predicted spectral CT image data.

The processing system is configured to: obtain non-spectral CT image data and spectral CT image data, each representing a same region of interest of the subject; process the non-spectral CT image data, using a machine-learning algorithm, to produce predicted spectral CT image data, the predicted spectral CT image data being a prediction of the spectral CT image data derived from the non-spectral CT image data; process the spectral CT image data and the predicted spectral CT image data to identify any differences between the spectral CT image data and the predicted spectral CT image data; and control an alert signal responsive to any identified differences.

The processing system may obtain the non-spectral and spectral CT image data at an input interface. In particular examples, the processing system may obtain the non-spectral and spectral CT image data from a memory/storage unit and/or a CT imaging system (that generates the CT image data).

The processing system may be configured to output, e.g. at an output interface, the alert signal to another system. As one example, the processing system may be configured to control a user interface using the alert signal to thereby control a user-perceptible output responsive to the alert signal.

There is also proposed a system comprising the processing system and the user interface.

The invention provides a mechanism for identifying potential areas of clinical relevance that are not readily visible in non-spectral CT image data. The non-spectral CT image data is processed using a machine-learning method to produce predicted spectral CT image data. The predicted spectral CT image data is compared to (actual) spectral CT image data to identify any differences between the two. Any identified differences are used to control an alert signal, for instance, to draw a clinician or other user's attention to possible areas of clinical relevance that are not readily visible in the non-spectral CT image data.

The present disclosure is based on the realization that features that are not readily visible in non-spectral CT image data will not be accurately generated in a prediction of spectral CT image data from the non-spectral CT image data. This facilitates identification of missing features or areas of potential clinical relevance in the non-spectral CT image data. By using a machine-learning method to produce predicted spectral CT image data (e.g. rather than attempting to directly detect potentially missing features) a non-supervised training technique can be used to train the machine-learning network, significantly reducing a professional input required and/or significantly increasing an amount of available data for training the machine-learning method.

Embodiments of the invention may be employed in any CT imaging environment, e.g. for use in clinical environments. In particular, embodiments may be employed to aid radiographers or other clinicians in assessing the condition of a patient, by drawing the clinician's attention to potentially clinically relevant areas that might otherwise have been overlooked.

<FIG> schematically illustrates a system <NUM> including an imaging system <NUM> such as a CT scanner configured for conventional CT imaging and spectral (multi-energy) imaging. The imaging system <NUM> includes a generally stationary portion <NUM> and a rotating portion <NUM>, which is rotatably supported by the stationary portion <NUM> and rotates around an examination region <NUM> about a z-axis. A subject support <NUM>, such as a couch, supports an object or subject in the examination region <NUM>.

A radiation source <NUM>, such as an x-ray tube, is rotatably supported by the rotating portion <NUM>, rotates with the rotating portion <NUM>, and emits radiation that traverses the examination region <NUM>.

To facilitate spectral imaging, in one instance, the radiation source <NUM> includes a single broad spectrum x-ray tube. In another instance, the radiation source <NUM> includes a single x-ray tube configured to switch between at least two different emission voltages (e.g., <NUM> kVp and <NUM> kVp) during scanning. In yet another instance, the radiation source <NUM> includes two or more x-ray tubes configured to emit radiation having different mean spectra. In still another instance, the radiation source <NUM> includes a combination thereof.

A radiation sensitive detector array <NUM> subtends an angular arc opposite the radiation source <NUM> across the examination region <NUM>. The radiation sensitive detector array <NUM> detects radiation traversing the examination region <NUM> and generates an electrical signal(s) (projection data) indicative thereof.

To facilitate spectral imaging, where the radiation source <NUM> includes a single broad spectrum x-ray tube, the radiation sensitive detector array <NUM> includes energy-resolving detectors (e.g., direct conversion photon counting detectors, at least two sets of scintillators with different spectral sensitivities (multi-layer), etc.). With kVp switching and multi-tube configurations, the detector array <NUM> can include single layer detectors, direct conversion photon counting detectors, and/or multi-layer detectors. The direct conversion photon counting detectors may include a conversion material such as CdTe, CdZnTe, Si, Ge, GaAs, or other direct conversion material. An example of multi-layer detector includes a double decker detector such as the double decker detector described in <CIT>, and entitled "Double Decker Detector for Spectral CT.

A reconstructor <NUM> receives projection data (and/or spectral projection data) from the detector array <NUM> and reconstructs non-spectral CT image data and spectral volumetric image data such as sCCTA image data, a high-energy image, a low energy image, a photoelectric image, a Compton scatter image, an iodine image, a calcium image, a virtual non-contrast image, a bone image, a soft tissue image, and/or other basis material image. The reconstructor <NUM> is also able to reconstruct non-spectral volumetric data, e.g., by combining spectral projection data and/or spectral volumetric image data or by processing non-spectral projection data (e.g. if the radiation source is controlled to operate in a non-spectral mode). Generally, the non-spectral projection data will include data for a single energy level or energy range. Generally, the spectral projection data and/or spectral volumetric image data will include data generated during an imaging process in which at least two different energies and/or energy ranges are detected and distinguished from one another by the radiation sensitive detector array.

A processing system <NUM> is configured to receive and process image data from the reconstructor <NUM>. The processing system <NUM> is a processing system according to one embodiment of the invention. In other embodiments, a processing system may be provided to receive such information from a separate system, e.g. a memory or storage system.

The processing system <NUM> includes a processor <NUM> (e.g., a microprocessor, a controller, a central processing unit, etc.) and a computer readable storage medium <NUM>, which excludes non-transitory medium, and includes transitory medium such as a physical memory device, etc. The computer readable storage medium <NUM> includes instructions <NUM> for an image processor <NUM>. The processor <NUM> is configured to execute the instructions <NUM>. The processor <NUM> may additionally be configured to execute one or more computer readable instructions carried by a carrier wave, a signal and/or other transitory medium. In a variation, the processor <NUM> and the computer readable storage medium <NUM> are part of another processing system, which is separate from the processing system <NUM>.

In some examples, the processing system <NUM> is configured to serve as an operator console. The console <NUM> may include a human readable output device such as a monitor and an input device such as a keyboard, mouse, etc. Software resident on the console <NUM> may allow the operator to interact with and/or operate the scanner <NUM> via a graphical user interface (GUI) or otherwise.

The processing system <NUM> is configured to process non-spectral CT image data and spectral CT image data. Both types of image data represent a same region of interest of the subject. The non-spectral CT image data is processed (using a machine-learning algorithm) to produce predicted spectral CT image data, i.e. a prediction of what the spectral CT image data contains. The predicted spectral CT image data and the spectral CT image data are processed together to identify any differences. An alert signal is controlled (and preferably output) by the processing system responsive to any identified differences. This approach effectively facilitates identification of the presence of areas in the non-spectral CT image data that fail to contain information of potential clinical importance (i.e. areas that contain information found in the spectral CT image data, but which are absent or unclear in the non-spectral CT image data).

<FIG> is a flowchart illustrating a (computer-implemented) method <NUM> according to an embodiment of the invention. The method <NUM> may be performed by a processing system, such as the processing system <NUM> described with reference to <FIG>.

The method <NUM> comprises a step <NUM> of obtaining non-spectral CT image data 201A and spectral CT image data <NUM> (which could be relabeled "base spectral CT image data" where appropriate). At the step <NUM> of obtaining the CT image data both: non-spectral CT image data 201A and spectral CT image data <NUM> can be obtained during the same CT image acquisition action and can be derivable from the CT image data acquired during said action. Both the non-spectral and spectral CT image data represent a same region of interest of the subject. For instance, both types of obtained image data may represent the entire imaged area of the subject.

The non-spectral and spectral CT image data may comprise one or more 2D or 3D images, i.e. be formed of 2D or 3D image data. Each 2D/3D image may be formed of a plurality of pixels or voxels. Of course, higher dimensionality images could also be used, e.g. where a fourth dimension represents time. In this scenario, the smallest addressable element representing a particular area in space of each high dimensionality image is a "voxel". Of course, each element may contain a plurality of channels or values (e.g. RGB values or the like).

The non-spectral CT image data may be "conventional" CT image data, i.e. image data generated from a single range of X-ray radiation frequencies (e.g. a broadband X-ray radiation). By way of example only, the non-spectral CT image data may comprise one or more non-spectral CT images.

The spectral CT image data may comprise one or more image datasets generated using a spectral imaging approach, such as sCCTA image data, a high-energy image, a low energy image, a photoelectric image, a Compton scatter image, an iodine image, a calcium image, a virtual non-contrast image, a bone image, a soft tissue image, and/or other basis material image. Other forms of content for spectral CT image data will be readily apparent to the person skilled in the art. Thus, the spectral CT image data may comprise one or more spectral CT images.

Step <NUM> may be performed by an input interface of the processing system. The non-spectral and spectral CT-image data may be obtained from a reconstructor (i.e. that originally generated both types of image data) or a memory/storage system (e.g. that stores image data generated by a reconstructor).

The method <NUM> than performs a step <NUM> of processing (at least) the non-spectral CT image data using a machine-learning algorithm to produce predicted spectral CT image data <NUM>. The predicted spectral CT image data <NUM> attempts to predict the "true" content of the spectral CT image data. Thus, the spectral CT image data <NUM> represents the "true" spectral CT image data and the predicted spectral CT image data <NUM> represents the predicted content of this "true" spectral CT image data. Thus, predicted spectral CT image data is a prediction of the (actual) spectral CT image data obtained in step <NUM>.

The machine-learning algorithm thereby attempts to predict the content of the spectral CT image data from (at least) the non-spectral CT image data. Such a machine-learning method has an advantage in that it only needs to be trained on pairs of non-spectral CT image data and corresponding spectral CT image data. Thus, the machine-learning method used in methods of the invention can be trained using an unsupervised learning approach. This eliminates the need to prepare large reliable annotated training data, which is a very challenging, tedious, time-consuming and costly task.

Step <NUM> may use additional information in processing the non-spectral CT image data to produce the predicted spectral CT image data. Thus, step <NUM> may comprise processing, using a machine-learning method, the non-spectral CT image data and the additional information in order to generate the predicted spectral CT image data.

The additional information may contain features or information that indicate potential characteristics that would affect spectral CT image data. Thus, the accuracy of the machine-learning methods inference/prediction will be improved through the use of such additional information.

The machine-learning method thereby receives, as input, at least the non-spectral CT image data (and optionally, additional information) and provides, as output, the predicted spectral CT image data. The data received as input may act as input features for the machine-learning method.

In some embodiments, the additional information may comprise input parameter data 201B. The input parameter data defines one or more parameters of the acquisition protocol used, by a CT imaging system, in acquiring the non-spectral CT image data and/or the spectral CT image data. This may comprise, for instance, information or one or more indicators responsive to: a scan type (e.g. single-slice, multi-slice or helical); body part; mA (or milliamperes, being a measure of a current that is provided to an X-ray tube); mAs (or millampere-seconds, being a product of the current provided to the X-ray tube and the amount of time that current is provided to the X-ray tube); kVp (kilovoltage peak); rotation time (of the rotating portion)' collimation (of the X-ray beam produced by the X-ray tube); and/or pitch (being a measure of overlap between x-ray beams provided to the subject from a same angle).

In some embodiments, the additional information comprises reconstruction parameter data. The reconstruction parameter data defines one or more parameters of the reconstruction protocol used, by a CT imaging system, to process raw CT data to produce the non-spectral CT image data and/or the spectral CT image data. This may comprise, for instance, information or one or more indicators responsive to: reconstruction filter used; reconstruction algorithm used (e.g. FBP, iDose or IMR); slice thickness; slice increment; matrix size and/or field of view.

In some embodiments, the additional information comprises subject data. The subject data defines one or more parameters of the subject (of the image). The subject data may, for instance, comprise (where the subject is undergoing a clinical observation) information or one or more indicators responsive to: health data of the subject (e.g. medical record information); demographic/personal data of the subject (e.g. age, gender, weight etc.); medical history of the subject (e.g. family history) and so on.

Examples of suitable machine-learning algorithms and approaches will be provided later in this disclosure.

The method <NUM> further comprises a step <NUM> of identifying any differences between the spectral CT image data <NUM> and the predicted spectral CT image data <NUM>. Thus, the spectral CT image data is compared to the predicted spectral CT image data to identify differences or inconsistencies between the two sets of image data.

The differences may be differences that meet one or more predetermined criteria. The differences need not be mathematical differences between the spectral CT image data and the predicted spectral CT image data, but may represent features present in one set of image data, but not in the other set of image data and/or features highly visible (e.g. having a high contrast, e.g. a contrast beyond some predetermined threshold, compared to nearby areas) in one set of image data, but not in the other set of image data.

In particular examples, step <NUM> may comprise identifying any potentially relevant clinical findings that are present (or readily visible) in the spectral CT image data, but are absent (or not readily visible) in the predicted spectral CT image data. Here, "readily visible" means easily distinguishable to the human eye, e.g. clinical findings that are represented by values in the predicted spectral CT image that are within a predetermined range of values of the surroundings of feature in the predicted spectral CT image data (i.e. are not sufficiently (i.e. with respect to some predetermined threshold) contrasted from the other elements of the predicted spectral CT image).

In particular, any identified differences between the predicted spectral CT image data and the spectral CT image data may represent, or may be processed to identify, areas of predicted/potential clinical relevance. This may comprise, for example, performing one or more grouping steps (e.g. clustering or connected component algorithms or the like) on identified differences and/or filtering operations to identify areas of significance (e.g. significant differences or significantly sized groups of differences).

It is herein recognized that differences between the spectral CT image data and the predicted spectral CT image data may represent areas of the subject that contain clinically important information which is not (clearly) represented in the non-spectral CT image data. In other words, a finding that could not be identified/seen or could barely be seen in the conventional CT images will not be accurately generated by the spectral generative network.

Working examples of step <NUM> will be provided later in this disclosure.

The method <NUM> further comprises a step <NUM> of controlling an alert signal SA responsive to any identified differences. Thus, the content of the alert signal is dependent upon any identified differences. Put another way, step <NUM> may comprise processing any identified differences to generate the alert signal. Step <NUM> may comprise outputting the alert signal, e.g. via an output interface of a processing system.

In some examples, step <NUM> comprises controlling the alert signal to contain or carry one or more indicators of the identified differences. It will be appreciated that the position of differences (between actual and predicted spectral CT image data) can be mapped to corresponding positions of non-spectral CT image data. The alert signal may be controlled to carry one or more indicators of a position of the identified differences with respect to the non-spectral CT image data.

In other examples, step <NUM> comprises controlling the alert signal to contain an indicator of whether or not any differences have been identified. This information may be used to alert a clinician or other user that differences have been detected, and that closer attention to the CT image data may therefore be required.

In some examples, step <NUM> comprises controlling the alert signal to contain an indicator of a number of identified differences. This information may be used to alert a clinician or other user as to a scale of possible areas of clinical relevance that are not (readily) visible from the non-spectral CT image data.

In some examples, step <NUM> or <NUM> may further comprise identifying one or more parts of the spectral CT image data that contain a representation of any/each identified difference. For instance, if the spectral CT image data is formed of a plurality of spectral CT images, step <NUM> may comprise identifying, for each identified difference, a spectral CT image that contains a representation of the difference, e.g. the spectral CT image for which a contrast between the area of the spectral CT image representing the difference compared to that areas surroundings is the greatest and/or exceeds some predetermined threshold.

The alert signal may be used to control a user interface, e.g. to provide a user-perceptible output (such as a visual, audio or haptic output) responsive to the alert signal. In some examples, the alert signal may be passed to another processor or processing system for further processing, e.g. to trigger the automatic acquisition of additional CT image data or to flag, e.g. set a flag of, a medical record of the subject. In some examples, the alert signal may carry alert data for storage, e.g. alongside the image data, so that the alert data may trigger the generation of a user-perceptible output during later review of the stored image data.

Steps <NUM> to <NUM> may be performed by a processing system, such as the processing system <NUM> described with reference to <FIG>.

As previously mentioned, the alert signal may be used to control a user interface. In one example, as illustrated in <FIG>, the method <NUM> may comprise a step <NUM> of controlling a display (i.e. a visual output) responsive to the alert signal. Step <NUM> may be performed by the processing system or a separate processor (e.g. of the user interface).

By way of example, if the alert signal carries one or more indicators of a position of the identified differences with respect to the non-spectral CT image data, then step <NUM> may comprise controlling a display to provide a visual representation of the non-spectral CT image data together with a representation of the identified difference(s), e.g. by providing an predetermined graphic or visual indicator to overlay the corresponding location(s) of the non-spectral CT image data.

By way of another example, if the alert signal contains an indicator of whether or not any differences have been identified, step <NUM> may comprise controlling the display to provide a visual representation (e.g. using different colors) as to whether or not any differences have been identified.

<FIG> illustrates one approach for processing non-spectral CT image data 201A (and optionally additional information) to produce predicted spectral CT image data <NUM>.

A machine-learning algorithm <NUM> is used to process non-spectral CT image data (and optionally additional information). Thus, the machine-learning algorithm <NUM> receives, as input, non-spectral CT image data 201A (and optionally additional information 201B, 201C, 201D) and produces predicted non-spectral data <NUM>.

The illustrated machine-learning algorithm <NUM> is a neural network. The structure of a neural network (or artificial neural network) is inspired by the human brain. Neural networks are comprised of layers <NUM>, <NUM>, each layer comprising a plurality of neurons. Each neuron comprises a mathematical operation. In particular, each neuron may comprise a different weighted combination of a single type of transformation (e.g. the same type of transformation, sigmoid etc. but with different weightings). In the process of processing input data, the mathematical operation of each neuron is performed on the input data to produce a numerical output, and the outputs of each layer in the neural network are fed into the next layer sequentially. The first layer <NUM> receives input data, and the final layer <NUM> provides the output data (e.g. the predicted spectral CT image data).

More specifically, the illustrated machine-learning algorithm operates according to a fully (connected) convolutional (neural) network configuration. In this configuration, input data could be of arbitrary size and the network is able to produce correspondingly sized output data with efficient inference and learning. In some configurations, features across layers of the neural network are fused to define a nonlinear local to-global representation that could be optimized or training end-to-end.

In some examples, rather than using a fully connected convolutional network, a convolutional neural network with a sliding-window configuration could also be also used. Although this process may be more computationally efficient, it is expected to have inferior performance.

The illustrated machine-learning algorithm comprises of the following portions.

A U-net portion <NUM>, which is effectively a U-net network and is formed of a contracting path <NUM> and an expansive path <NUM>.

In the contracting path, input data (comprising the non-spectral CT image data and optionally the (reshaped) additional information) is encoded using a series of one or more convolution processes (not shown), e.g. at least two convolution processes or at least three convolution processes. Each convolution process comprises applying at least one (e.g. two or more) convolutions (using a filter of size NxN or NxNxN, where N is at least <NUM>, e.g. as least <NUM>). The dimensionality of the filter may be dependent upon (e.g. equal to) the dimensionality of the non-spectral CT image data. Each convolution in the convolution process may be followed by a batch normalization process and an activation function process, e.g. such as that which can be applied by a rectified linear unit (ReLU) or similar. The last convolution of each convolution process is done with stride M (where M><NUM> and (optionally) M<=N) for down-sampling/downscaling.

In the expansive path, the down-sampled/downscaled convolutional output is decoded using a series of one or more deconvolution processes, e.g. at least two deconvolution processes or at least three deconvolution processes. There may be a same number of deconvolution processes as convolution process, e.g. so that each deconvolution process corresponds (i.e. operates at a same level to) a corresponding deconvolution process. Each deconvolution process comprises applying at least one (e.g. two or more) deconvolutions (using a filter of size NxN or NxNxN, where N is at least <NUM>, e.g. as least <NUM>). Each deconvolution in the deconvolution process may be followed by a batch normalization process and an activation function process, e.g. such as that which can be applied by a rectified linear unit (ReLU) or similar. The last deconvolution of each deconvolution process is done with stride M (where M><NUM> and (optionally) M<=N) for down-sampling/downscaling.

At the final layer <NUM> a <NUM>×<NUM>×<NUM> convolution may be used to provide the final decoder result, i.e. the output data.

In some examples, one or skip connections <NUM> may be used to connect layers from the contracting path to corresponding layers of the expansive path. This approach combines better localized, high resolution features from the contracting path combined with the upsampled output of the expansive path, which can produce a more accurate and localized output. The use of skip connections in a neural network architecture will be known to the skilled person. Additional skip connections <NUM> (not shown) may be used in the neural network.

The number of convolutions performed in each convolution process in the contracting path may be (<NUM>^<NUM>)^s. c, where s=<NUM>,<NUM>,<NUM> is the position of the convolution process in the series of convolution processes, e.g. where the first convolution process has value s = <NUM>, and c is a network control parameter. Experimental analysis has shown that, for use in the present disclosure, improved performed is achieved in an overcomplete setting, in which c><NUM>.

The number of deconvolutions performed in each deconvolution process may be equal to the number of convolutions performed in the convolution process at the same level as the corresponding deconvolution process.

The neural network <NUM> may also comprise a reshaping process <NUM>, which may be performed if additional information 201B, 201C, 201D is used to generate the predicted spectral CT image data.

The reshaping process <NUM> may be configured to preprocess and reshape the additional information for use in the U-net portion <NUM> of the neural network. This may comprise, for instance, performing two or more fully connected convolutions on the additional information and reshaping the processed information to produce.

The reshaping may comprise reshaping the processed information to one or more representations having a same size as the non-spectral CT image data. The reshaped information and non-spectral CT image data is then provided to the U-net portion, e.g. in the form of a plurality of channel(s).

Although the machine-learning algorithm is a spectral generative network in the illustrated examples, other forms of machine-learning algorithms could be used.

More generally, a machine-learning algorithm is any self-training algorithm that processes input data in order to produce or predict output data. Here, the input data comprises non-spectral CT image data (and optional additional information) and the output data comprises predicted spectral CT image data.

Other suitable machine-learning algorithms for being employed in the present invention will be apparent to the skilled person. Examples of suitable machine-learning algorithms include decision tree algorithms and artificial neural networks. Other machine-learning algorithms such as logistic regression, support vector machines or Naive Bayesian models are suitable alternatives.

Methods of training a machine-learning algorithm are well known. Typically, such methods comprise obtaining a training dataset, comprising training input data entries and corresponding training output data entries. An initialized machine-learning algorithm is applied to each input data entry to generate predicted output data entries. An error between the predicted output data entries and corresponding training output data entries is used to modify the machine-learning algorithm. This process can be repeated until the error converges, and the predicted output data entries are sufficiently similar (e.g. ±<NUM>%) to the training output data entries.

For example, where the machine-learning algorithm is formed from a neural network, (weightings of) the mathematical operation of each neuron may be modified until the error converges. Known methods of modifying a neural network include gradient descent, backpropagation algorithms and so on.

The training input data entries correspond to sets of example non-spectral CT image data (and optional, corresponding additional information). The training output data entries correspond to spectral CT image data (i.e. representing the same region of interest as the non-spectral CT image data).

One advantage of the proposed approach is that the machine-learning algorithm can be trained in a non-supervised manner. In other words, the machine-learning algorithm does not require professional or clinical input in order to correctly train itself. In this way, generation of an alert signal (effectively identifying the position and/or presence of potential areas of clinical oversight) can be provided in a non-supervised manner. This significantly reduces a difficulty and cost in producing such a mechanism, as well as significantly increasing the amount of available data for training the machine-learning algorithm to be used in the proposed approach, leading to a more accurate machine-learning algorithm.

<FIG> illustrates a method <NUM> for identifying any differences between spectral CT image data <NUM> and predicted spectral CT image data <NUM> (generated/derived from non-spectral CT image data).

Here, the spectral CT image data comprises one or more spectral CT images, and the predicted spectral CT image data comprises a corresponding number of one more predicted spectral CT images. Each predicted spectral CT image corresponds to a spectral CT image (e.g. represents a prediction of the content of the corresponding spectral CT image).

The method <NUM> effectively identifies, for each of the plurality of spectral CT images, any differences between said CT image and its corresponding predicted CT image. Identified differences may then be processed to identify areas of (predicted) potential clinical importance or relevance (e.g. by applying one or more grouping and/or filtering steps on identified differences). The areas of potential clinical importance or relevance may then be used to control an alert signal, e.g. to draw a clinicians attention to areas of potential clinical importance or to perform additional imaging of the area of potential clinical importance.

The method <NUM> comprises a step <NUM> of generating one or more difference maps or difference images by processing the spectral CT image data <NUM> and the predicted spectral CT image data <NUM>. A map may effectively be difference image data that identifies the presence and/or position of any differences between the CT image data and the predicted spectral CT image data.

Where the spectral CT image data comprises a single CT image, the difference image data may comprise generating a single difference map by directly processing the spectral CT image and the predicted CT image.

Where the spectral CT image data comprises a plurality of spectral CT images, this process <NUM> may comprise generating a plurality of difference maps, before generating a composite difference map by weighting and/or combining the difference maps together. This effectively identifies the presence and/or position of any differences between any of the spectral CT images and the predicted spectral CT images.

In some examples, the composite difference map is not generated (even when the spectral CT image data comprises a plurality of spectral CT images), so that a plurality of difference maps undergo further processing.

In one example, step <NUM> comprises calculating a difference image (e.g. for each spectral CT image) by determining, for each pixel of the spectral CT image, a magnitude of a difference between that pixel (value(s)) and the pixel (value(s)) of the predicted spectral CT image.

In another example, step <NUM> comprises calculating a difference image (e.g. for each spectral CT image) using the following function: <MAT> where Mapi represents a pixel i of the difference image Map, Ii represents the spectral CT image I, <MAT> represents a pixel i of the predicted CT image Î, stdfilt(. ) represents a standard deviation filtering function and ε represents a small value (e.g. <<NUM>), which is used to reduce a likelihood that a divide by zero error will occur.

One example standard deviation filtering function is one that process an input image so that a value of each output pixel/voxel is the standard deviation of a neighborhood (e.g. the adjacent <NUM> pixels around the corresponding input pixel/voxel or the neighboring <NUM> pixels around the corresponding input pixel/voxel).

In another example, step <NUM> comprises calculating a difference image (e.g. for each spectral CT image) using the following function: <MAT>.

In yet another example, step <NUM> comprises calculating a difference image (e.g. for each spectral CT image) using the following function: <MAT> where ncc(. ) is a normalized cross-correlation filtering between two images, here: the spectral CT image and the predicted spectral CT image.

In yet another example, step <NUM> comprises calculating a difference image (e.g. for each spectral CT image) using the following function: <MAT> where mi(. ) is a mutual information filtering function performed between two images.

Where multiple difference images are generated, e.g. for each spectral CT image, the method <NUM> may further comprise a process of combining the multiple difference images to produce a composite difference image (e.g. as part of step <NUM>), which may act as the difference image for the rest of the process of <NUM>.

In one example, combining the multiple difference images may comprise simply summing the difference images together, i.e. by adding the values of each corresponding pixel/voxel to one another. In another example, combining the multiple difference images may comprise performed a weighted sum of the difference images. In yet another example, a single one of the difference images may be selected as the "composite difference image", e.g. the difference image that contains the greatest summed pixel value (of all pixels in the difference image).

In another example, the composite difference image is generated by selecting, for each pixel/value of the composite difference image, the maximum value of any of the corresponding (e.g. weighted) pixels/voxels of the difference images. Thus, for a pixel/voxel at position (a,b,c) in the composite difference image, the maximum value of the pixel/voxel at position (a,b,c) in any of the (e.g. weighted) difference images is selected as the value for that pixel/voxel in the composite difference image.

A composite difference image CMap may be calculated using the following equation: <MAT> where max(. ) is a maximum function, wi represents a weight and Map represents a difference map.

The (composite) difference image may then be processed in a step <NUM> to identify any groups or clusters in the difference image. This process effectively establishes whether there are any clinically relevant areas of the difference image.

Step <NUM> may comprise, for instance, processing the difference image using a grouping or clustering algorithm to identify any groups or clusters. In some examples, only pixels/voxels having a value above a predetermined threshold (e.g. tr) may form part of a group or cluster. Thus, step <NUM> may comprise processing the difference image using a thresholding grouping/clustering algorithm.

One example of a suitable grouping/clustering algorithm is a k-means clustering algorithm or a density-based clustering algorithm. Another suitable example is a segmentation algorithm.

Yet another example of a suitable grouping algorithm is a connected components algorithm. A connected component algorithm identifies connected groups of pixels/voxels. The connected component algorithm may be configured to identify connected groups of pixels/voxels that have a value that reaches or exceeds some predetermined threshold value, i.e. is a thresholded connected component algorithm.

The identified groups and/or clusters may then be subject to a filtering process in step <NUM>. Thus, step <NUM> may comprise filtering or cleaning any identified groups and/or clusters using a filtering process.

The filtering process <NUM> may comprise, for instance, removing any groups/clusters that are below a predetermined size. By way of example, if a group is a connected component, the filtering process may comprise removing any connected components having a size below some predetermined size (e.g. less than <NUM>% or <NUM>% or <NUM>% or <NUM>% of the total number of pixels of the difference image). This approach reduces the likelihood that noise or clinically irrelevant information is identified.

The filtering process <NUM> may comprise removing any groups/clusters for which a maximum value contained in the group/cluster is below some second predetermined threshold (e.g. <NUM>% or <NUM>% of the maximum value for the (composite) difference image). This approach increases the likelihood that only significant groups are identified as being of potential clinical relevance.

The filtering process <NUM> may comprise removing any groups/clusters for which fewer than a predetermined number of pixels/voxels (e.g. <NUM>% or <NUM>% or <NUM>% of the total number of pixels of the difference image) have a value that is above some third predetermined threshold (e.g. <NUM>% or <NUM>% of the maximum value for the (composite) difference image). This approach also increases the likelihood that only significant groups are identified as being of potential clinical relevance.

In some examples, step <NUM> does not comprise a step of generating a composite difference image. Rather, if multiple difference maps are generated for a respective plurality of spectral CT images, each difference may be processed separately by following the procedure of steps <NUM> and <NUM> on each difference map. This approach identifies any suitable groups/clusters in each difference image.

Use of a composite difference map may provide a more efficient mechanism for identifying differences, as the greatest differences contribute to the difference map which is then processed. However, separately processing each difference image may produce more robust/accurate identification of clusters/groups that represent clinically relevant areas (e.g. reduce a likelihood that noise will impact the identification of suitable clusters/groups).

The output of the illustrated process <NUM> is a set of (zero or more) groups/clusters that each represent an area of potential clinical relevance that is present in the spectral CT image data, but not present or readily apparent in the predicted spectral CT image data. This effectively identifies areas of the non-spectral CT image data that fail to contain or display potentially clinically relevant or important features of the subject that are present or readily apparent in the spectral CT image data.

The set of groups/clusters thereby act as the identified differences between the spectral CT image data and the predicted spectral CT image data. The identified differences thereby act as identified area of potential clinical relevance (that are not present or readily visible in the original non-spectral CT image data).

It will be appreciated that the process <NUM> effectively identifies a position and size of the area of potential clinical relevance. As the non-spectral CT image data and the spectral CT image data represent a same area of the subject, a position and size of an area in the spectral CT image data can be mapped to a corresponding size and position in the non-spectral CT image data.

The alert signal may be configured to carry an indicator of the size and/or position of each identified area of potential clinical relevance (i.e. each group/cluster). This facilitates identification of areas that require closer or further investigation for performing a clinical assessment.

<FIG> also illustrates a process <NUM> for identifying (for each group/cluster identified in process <NUM>) a spectral CT image that contains or (best) depicts the said group/cluster. This process <NUM> can be performed where the spectral CT image data contains a plurality of spectral CT images.

In this way, each identified area of potential clinical relevance is mapped to a particular spectral CT image, and in particular, mapped to a clinical image that provides the area of potential clinical relevance in the "cleanest" way.

One approach for carrying out process <NUM> could be to identify the spectral CT image for which the area that contains the said group/cluster has the lowest contrast-to-noise ratio (CNR).

For instance, process <NUM> may comprise processing each group/cluster in turn.

A CNR metric is obtained, in step <NUM> for the position of the group/cluster in the spectral CT image. The spectral CT image having the highest CNR metric at/around this position is then selected in step <NUM> as the spectral CT image for the group/cluster.

One approach for calculating a CNR metric is to employ the following equation: <MAT> where G represents the pixel/voxel locations of the group/cluster of pixels/voxels; Sj represents the jth spectral CT image; <MAT> is represents the jth predicted spectral CT image generated by the machine-learning method. In this way Sj[G] and <MAT>[G] are the pixel values in G of Sj and <MAT>, correspondingly.

Process <NUM> is repeated until a spectral CT image is identified for each group/cluster identified in process <NUM>.

The alert signal may be (further) controlled to carry, for each identified group/cluster, an identification of the spectral CT image identified by process <NUM>. This information could, for instance, be used to control a display to provide visual representations of the identified spectral CT images and the corresponding locations and/or sizes of the identified groups/clusters with respect to the appropriate spectral CT images.

Thus, the alert signal may be controlled to provide an indicator of, for each identified group/cluster (i.e. each identified difference), an identification of the corresponding spectral CT image (that best/clearly contains a representation of the difference) and a position and/or size and/or shape of the group with respect to that corresponding spectral CT image.

<FIG> demonstrates an underlying recognition of the present disclosure. <FIG> illustrates two CT images of a same area or region of the subject, containing a representation of the gallbladder (upper-left corner) and the surrounding anatomy.

In particular, <FIG> illustrates a first image <NUM>, which is a non-spectral CT image, i.e. a "conventional" CT image obtained using conventional CT image techniques and reconstructions approaches.

<FIG> also illustrates a second image <NUM>, which is a spectral CT image, namely a monochromatic 40keV image. An identified area <NUM> of the second image contains a gallstone which is not visible in the non-spectral CT image <NUM>.

This example illustrates how potentially clinically relevant features or areas are unidentifiable in the non-spectral CT image, but can be easily identified in the spectral CT image.

<FIG> can also be used to understand one example of an embodiment in which a visual representation of a spectral CT image <NUM> is configured to provide a visual representation of an identified area <NUM>. In this instance, the circle <NUM> may represent a visual indicator of an identified area.

<FIG> illustrates a system <NUM> according to an embodiment of the invention.

The system <NUM> comprises a processing system <NUM>. The processing system <NUM> is configured for controlling an alert signal SA responsive to differences between spectral CT image data and predicted spectral CT image data.

In particular, the processing system <NUM> is configured to: obtain non-spectral CT image data and spectral CT image data, each representing a same region of interest of the subject; process the non-spectral CT image data, using a machine-learning algorithm, to produce predicted spectral CT image data, the predicted spectral CT image data being a prediction of the spectral CT image data derived from the non-spectral CT image data; process the spectral CT image data and the predicted spectral CT image data to identify any differences between the spectral CT image data and the predicted spectral CT image data; and control an alert signal responsive to any identified differences.

The processing system <NUM> may be configured to obtain the non-spectral and/or spectral CT image data from a medical imaging system <NUM> and/or a memory <NUM>. The processing system may be configured to control a user interface <NUM>, e.g. a display, using the alert signal SA.

The medical imaging system <NUM>, the memory <NUM> and the user interface <NUM> form optional components of the system <NUM>.

By way of further example, <FIG> illustrates an example of a processing system <NUM> within which one or more parts of an embodiment may be employed. The processing system <NUM> provides an example of a suitable processing system for use in previously described embodiments.

Various operations discussed above may utilize the capabilities of the processing system <NUM>. For example, one or more parts of a system for processing non-spectral and spectral CT image data may be incorporated in any element, module, application, and/or component discussed herein. In this regard, it is to be understood that system functional blocks can run on a single computer or may be distributed over several computers and locations (e.g. connected via internet).

The processing system <NUM> includes, but is not limited to, PCs, workstations, laptops, PDAs, palm devices, servers, storages, and the like. Generally, in terms of hardware architecture, the processing system <NUM> may include one or more processors <NUM>, memory <NUM>, and one or more I/O devices <NUM> that are communicatively coupled via a local interface (not shown). The local interface can be, for example but not limited to, one or more buses or other wired or wireless connections, as is known in the art. The local interface may have additional elements, such as controllers, buffers (caches), drivers, repeaters, and receivers, to enable communications. Further, the local interface may include address, control, and/or data connections to enable appropriate communications among the aforementioned components.

The processor <NUM> is a hardware device for executing software that can be stored in the memory <NUM>. The processor <NUM> can be virtually any custom made or commercially available processor, a central processing unit (CPU), a digital signal processor (DSP), or an auxiliary processor among several processors associated with the processing system <NUM>, and the processor <NUM> may be a semiconductor based microprocessor (in the form of a microchip) or a microprocessor.

The software in the memory <NUM> may include one or more separate programs, each of which comprises an ordered listing of executable instructions for implementing logical functions. The software in the memory <NUM> includes a suitable operating system (O/S) <NUM>, compiler <NUM>, source code <NUM>, and one or more applications <NUM> in accordance with exemplary embodiments. As illustrated, the application <NUM> comprises numerous functional components for implementing the features and operations of the exemplary embodiments. The application <NUM> of the processing system <NUM> may represent various applications, computational units, logic, functional units, processes, operations, virtual entities, and/or modules in accordance with exemplary embodiments, but the application <NUM> is not meant to be a limitation.

If the processing system <NUM> is a PC, workstation, intelligent device or the like, the software in the memory <NUM> may further include a basic input output system (BIOS) (omitted for simplicity). The BIOS is a set of essential software routines that initialize and test hardware at startup, start the O/S <NUM>, and support the transfer of data among the hardware devices. The BIOS is stored in some type of read-only-memory, such as ROM, PROM, EPROM, EEPROM or the like, so that the BIOS can be executed when the processing system <NUM> is activated.

When the processing system <NUM> is in operation, the processor <NUM> is configured to execute software stored within the memory <NUM>, to communicate data to and from the memory <NUM>, and to generally control operations of the processing system <NUM> pursuant to the software. The application <NUM> and the O/S <NUM> are read, in whole or in part, by the processor <NUM>, perhaps buffered within the processor <NUM>, and then executed.

The skilled person would be readily capable of developing a processing system for carrying out any herein described method. Thus, each step of the flow chart may represent a different action performed by a processing system, and may be performed by a respective module of the processing system.

Embodiments may therefore make use of a processing system. The processing system can be implemented in numerous ways, with software and/or hardware, to perform the various functions required. A processor is one example of a processing system which employs one or more microprocessors that may be programmed using software (e.g., microcode) to perform the required functions. A processing system may however be implemented with or without employing a processor, and also may be implemented as a combination of dedicated hardware to perform some functions and a processor (e.g., one or more programmed microprocessors and associated circuitry) to perform other functions.

Examples of processing system components that may be employed in various embodiments of the present disclosure include, but are not limited to, conventional microprocessors, application specific integrated circuits (ASICs), and field-programmable gate arrays (FPGAs). In various implementations, a processor or processing system may be associated with one or more storage media such as volatile and non-volatile computer memory such as RAM, PROM, EPROM, and EEPROM. The storage media may be encoded with one or more programs that, when executed on one or more processors and/or processing systems, perform the required functions. Various storage media may be fixed within a processor or processing system or may be transportable, such that the one or more programs stored thereon can be loaded into a processor or processing system.

It will be understood that disclosed methods are preferably computer-implemented methods. As such, there is also proposed the concept of a computer program comprising code means for implementing any described method when said program is run on a processing system, such as a computer. Thus, different portions, lines or blocks of code of a computer program according to an embodiment may be executed by a processing system or computer to perform any herein described method. In some alternative implementations, the functions noted in the block diagram(s) or flow chart(s) may occur out of the order noted in the figures.

Claim 1:
A computer-implemented method (<NUM>) of controlling an alert signal (SA) responsive to differences between spectral CT image data (<NUM>) and predicted spectral CT image data (<NUM>), the computer-implemented method comprising:
obtaining (<NUM>) non-spectral CT image data (201A) and spectral CT image data (<NUM>), each representing a same region of interest of a subject; and
processing (<NUM>) the non-spectral CT image data, using a machine-learning algorithm (<NUM>), to produce predicted spectral CT image data (<NUM>), the predicted spectral CT image data being a prediction of the spectral CT image data derived from the non-spectral CT image data;
the method being characterized by processing (<NUM>) the spectral CT image data and the predicted spectral CT image data to identify any differences between the spectral CT image data and the predicted spectral CT image data; and
controlling (<NUM>) an alert signal responsive to any identified differences.