Patent Description:
Mobile X-ray, also referred to as portable X-ray, is an important image acquisition modality. It allows to take X-ray in places where stationary X-ray imaging is not practical, e.g., emergency department (ED), intensive care unit (ICU), etc. Mobile X-ray targets mainly patients lying in bed. In contrast to stationary X-ray, acquiring an X-ray image on portable X-ray more challenging since there are more degrees of freedom, than for stationary X-ray image.

<FIG> shows the basic working principle of a mobile X-ray machine <NUM>. First, the technician places an X-ray detector <NUM> behind a patient, shown as bed-patient in <FIG>. Then, the technician manually sets the position of an X-ray emitter <NUM> so that the desired field of view can be captured by the X-ray detector <NUM>. Finally, the technician acquires an image by activating the X-ray emitter <NUM>. There are a plenty of parameters that can seriously affect the image quality, such as the position of X-ray detector <NUM> (e.g., coordinates and rotation angles) and the position of the X-ray emitter (e.g., distance to patient, rotation angles, and voltage).

<CIT> relates to generating a positioning signal to facilitate positioning of one or more of a patient, X-ray source, or detector during an image acquisition.

Thus, there may be a need to acquire mobile X-ray images of better quality. The object of the present invention is solved by the subject-matter of the appended independent claims, wherein further embodiments are incorporated in the dependent claims.

According to a first aspect of the present invention, there is provided a computer-implemented method for determining a position of an X-ray emitter of a mobile X-ray device, comprising:.

In other words, the present disclosure proposes a method that can automatically locate the optimal X-ray emitter position for an image using either a reference X-ray image, such as a prior high-quality image from the same patient or a reference image from an atlas, deemed suitable for this imaging case. With the method disclosed herein, a reduction of the radiation dose received by the patients may be achieved by lowering the number of retakes. Additionally, reduced turnaround times may be achieved by avoiding low quality images being send to the PACS and rejected at the time of their review. Furthermore, because of the multiple job repetitions may be reduced, the costs may also be reduced.

This will be explained in detail hereinafter and in particular with respect to the examples shown in <FIG>.

The step of generating a virtual X-ray image of the internal body structure further comprises:.

This will be explained in detail hereinafter and in particular with respect to the example shown in <FIG>.

According to an embodiment of the present invention, the camera comprises a three-dimensional (3D) camera and/or a two-dimensional (2D) camera. The distance from the X-ray emitter to the patient is determined based on depth information in the camera-acquired image or a neural network based depth estimation.

This will be explained in detail hereinafter and in particular with respect to step <NUM> shown in <FIG>.

According to an embodiment of the present invention, the step of registering the virtual X-ray image of the internal body structure with the reference X-ray image of the internal body structure further comprises:.

According to an embodiment of the present invention, the at least one parameter for adjusting the position of the X-ray emitter includes one or more of:.

According to an embodiment of the present invention, the computer-implemented method further comprises:.

According to an embodiment of the present invention, the reference X-ray image comprises one or more of:.

According to a second aspect of the present invention, there is provided an X-ray emitter position determination device comprising a processing unit configured to perform the steps of the method according to the first aspect and any associated example.

According to a third aspect of the present invention, there is provided an X-ray imaging system, comprising:.

According to an embodiment of the present invention, the tracker device comprises one or more of: a marker device, a gyroscope, and an antenna.

According to an embodiment of the present invention, the mobile X-ray device further comprises a mobile X-ray robotic arm to support the X-ray emitter. The X-ray emitter position determination device is configured to provide a control signal to control the mobile X-ray robotic arm of the mobile X-ray device to adjust the position of the X-ray emitter.

According to an embodiment of the present invention, the mobile X-ray device further comprises a display configured to display an instruction provided by the X-ray emitter position determination device to guide a user to manually adjust the position of the X-ray emitter.

The predicted directions are used on the X-ray machine: they either can be displayed for the manual correction by the technician, or an automated correction by a mobile X-ray robotic arm (if present) can be realized.

According to another aspect of the present invention, there is provided a computer program product comprising instructions which, when the program is executed by a processing unit, cause the processing unit to carry out the steps of the method disclosed herein.

According to a further aspect of the present invention, there is provided a computer-readable medium having stored thereon the computer program product.

In clinical practice, mobile X-ray systems may be limited in terms of image quality compared to their stationary counterparts, i.e., bad field of view, missed or cut anatomy. Bad images may lead to image retakes, which increase the radiation dose for the patient.

Towards this end, a method, apparatus, and system are provided with the aim of improving the image quality and overcoming one or more of the above-mentioned problems of mobile X-ray during the image acquisition stage. In particular, a system is proposed that can automatically acquire mobile X-ray images of better quality. The system disclosed herein may automatically locate the optimal X-ray emitter position for an image using either a prior high-quality image from the same patient, or a reference image from an atlas, deemed suitable for this imaging case.

<FIG> illustrates an exemplary X-ray imaging system <NUM> according to an embodiment of the present invention. The X-ray imaging system <NUM> comprises a mobile X-ray device <NUM> with an X-ray emitter <NUM>. As shown in <FIG>, the mobile X-ray device <NUM> further comprises a chassis <NUM> that supports an arm, e.g., a robotic arm <NUM>, and having a system with wheels <NUM> for manual or motorized movement which allow the equipment to be transported. The robotic arm <NUM> may move horizontally and/or vertically and support at its end a head assembly <NUM>, where the X-ray emitter <NUM> is located.

The X-ray imaging system <NUM> further comprises a camera <NUM> mountable to the mobile X-ray device <NUM> and configured to capture a camera-acquired image from a patient in an X-ray imaging session. The camera <NUM> is registered to an origin of the X-ray emitter <NUM>. For example, as shown in <FIG>, the camera <NUM> may be located in the head assembly <NUM>. In some examples, the camera <NUM> may be an embedded camera. In some other examples, the camera <NUM> may be removably mounted to the head assembly <NUM>. In some examples, the camera <NUM> may be a two-dimensional (2D) camera configured for capturing one scene by using one photographing lens and one image sensor. The obtained image is called a 2D image. In some examples, the camera <NUM> may be a three-dimensional (3D) camera for capturing 3D images. The 3D camera may be a range camera, which produces a 2D image showing the distance to points in a scene from a specific point. The 3D camera may be e.g., a stereo camera, which is a type of camera with two or more lenses with separate image sensors or film frame for each lens.

The X-ray imaging system <NUM> shown in <FIG> further comprises a portable X-ray detector <NUM>, which may be arranged behind a patient in order to measure the flux, spatial distribution, spectrum, and/or other properties of X-rays. The portable X-ray detector <NUM> may be fitted with a tracker device <NUM>, such as a gyroscope, antennas, a marker or other devices, which help to localize the portable X-ray detector's position relative to the X-ray emitter <NUM>.

The X-ray imaging system <NUM> shown in <FIG> further comprises an X-ray emitter position determination device <NUM> configured to determine at least one parameter to adjust a position of the X-ray emitter <NUM>. In general, the X-ray emitter position determination device <NUM> may comprise various physical and/or logical components for communicating and manipulating information, which may be implemented as hardware components (e.g., computing devices, processors, logic devices), executable computer program instructions (e.g., firmware, software) to be executed by various hardware components, or any combination thereof, as desired for a given set of design parameters or performance constraints.

In some implementations, the X-ray emitter position determination device <NUM> may be embodied as, or in, a device, such as the mobile X-ray device <NUM> shown in <FIG> or mobile device. The X-ray emitter position determination device <NUM> may comprise one or more microprocessors or computer processors, which execute appropriate software. The processing unit of the apparatus <NUM> may be embodied by one or more of these processors. The software may have been downloaded and/or stored in a corresponding memory, e.g., a volatile memory such as RAM or a non-volatile memory such as flash. The software may comprise instructions configuring the one or more processors to perform the functions as described herein.

It is noted that the X-ray emitter position determination device <NUM> may be implemented with or without employing a processor, and also may be implemented as a combination of dedicated hardware to perform some functions and a processor (e.g., one or more programmed microprocessors and associated circuitry) to perform other functions. For example, the functional units of the X-ray emitter position determination device <NUM> may be implemented in the device or apparatus in the form of programmable logic, e.g., as a Field-Programmable Gate Array (FPGA). In general, each functional unit of the apparatus may be implemented in the form of a circuit.

Although <FIG> may show that the X-ray emitter position determination device <NUM> is embodied in the mobile X-ray device <NUM>, it will be appreciated that in some implementations the X-ray emitter position determination device <NUM> may be embodied as, or in, a mobile device, e.g., tablet computer.

The X-ray emitter position determination device <NUM> is configured to perform the method disclosed herein. The method will be described in detail hereinafter and in conjunction with the flowchart shown in <FIG>.

<FIG> illustrates a flowchart describing a computer-implemented method <NUM> for determining a position of an X-ray emitter of a mobile X-ray device. The method <NUM> may be implemented as a device, module or related component in a set of logic instructions stored in a nontransitory machine- or computer-readable storage medium such as random access memory (RAM), read only memory (ROM), programmable ROM (PROM), firmware, flash memory, etc., in configurable logic such as, for example, programmable logic arrays (PLAs), field programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), in fixed-functionality hardware logic using circuit technology such as, for example, application specific integrated circuit (ASIC), complementary metal oxide semiconductor (CMOS) or transistor-transistor logic (TTL) technology, or any combination thereof. For example, computer program code to carry out operations shown in the method <NUM> may be written in any combination of one or more programming languages, including an object oriented programming language such as JAVA, SMALLTALK, C++, Python, or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. For example, the exemplary method may be implemented as the apparatus <NUM> shown in <FIG>.

In step <NUM>, the method <NUM> comprises a step of receiving (i) a camera-acquired image originating from a camera that monitors a patient in an X-ray imaging session, wherein the camera is registered to an origin of the X-ray emitter, (ii) a reference X-ray image of an internal body structure to be examined in the X-ray examination, and (iii) position information of an X-ray detector.

The camera-acquired image may originate from the camera <NUM> shown in <FIG> that monitors the patient in an X-ray imaging session. In some examples, the camera <NUM> may be a video camera configured to send the real-time video stream to the apparatus <NUM>, which may have a videoprocessing unit to process the real-time video stream. In some examples, the camera <NUM> may be a camera configured to capture images and send the images to the apparatus <NUM>, which may have an imageprocessing unit to process the images. In some examples, the camera-acquired image may comprise one or more 3D images. In some examples, the camera-acquired image may comprise one or more 2D images.

The camera <NUM> is registered to an origin of the X-ray emitter <NUM>. <FIG> illustrates an exemplary registration process. When an object is imaged, its representation is stored in a matrix of pixels, which can be addressed by their coordinates x, y. Usually, the origin (i.e., the <NUM>, <NUM> point) is located in the upper left corner of the matrix, with the x axis going from left to right, and the y axis from top to bottom. Unless the object and the camera are rigidly attached to each other, imaging the object twice will result in two different matrices, with two different coordinate systems. In the example of <FIG>, the nose of the patient on image "a" is located further on the right and down, than on image "b". It is possible to register the two images together to evaluate the coordinate transformation, which permits to transform one image into the other. Since the imaging planes is perpendicular to the axis of the camera, the transformation of the images directly translates into the camera movements required to reacquire one image if the second is available.

In the present disclosure it is proposed to use image registration in two separated instances:.

The reference X-ray image of an internal body structure to be examined in the X-ray examination may be downloaded from a Picture Archiving and Communication System (PACS). The reference X-ray image may be a previous scan of the patient of a good quality or some reference image or atlas of a good quality. In some examples, the reference X-ray image may be acquired from a different patient.

The position information of an X-ray detector <NUM> may be acquired from the tracker device <NUM>, e.g., a marker device, a gyroscope, an antenna, or any combination thereof, which is attached to the X-ray detector <NUM>.

In step <NUM>, the method <NUM> further comprises a step of detecting and localizing a plurality of anatomical landmarks in the camera-acquired image. In some examples, a model of the patient may be determined prior to imaging in order to conform the imaging parameters to the patient anatomy. The model may include locations of anatomical landmarks, such as shoulders, pelvis, torso, knees, etc. An acquired surface image of a patient may be compared against a library of pre-modeled surface images to determine a model corresponding to the patient. The determination may be performed by a neural network which is trained based on the library of pre-modeled surface images. In some examples, the location of a head, shoulder, torso, knee, and ankle may be determined based on a 2D or 3D image. Neural networks may be trained to detect the landmarks automatically. For example, supervised learning can be applied for anatomical landmark localization. See for example <NPL>.

In step <NUM>, the method <NUM> further comprises a step of determining a distance from the X-ray emitter to the patient based on the camera-acquired image. If the camera is a 3D camera, the distance from the X-ray emitter to the patient may be determined based on depth information in the camera-acquired image. If the camera is a 2D camera, the distance from the X-ray emitter to the patient may be determined based on a neural network based depth estimation. For example, self-supervised learning may be applied for the distance or depth estimation. See for example <NPL>.

In step <NUM>, the method <NUM> further comprises a step of generating a virtual X-ray image of the internal body structure based on the plurality of detected anatomical landmarks, the distance from the X-ray emitter to the patient, and the position information of the X-ray detector.

<FIG> shows a flow diagram describing one implementation of step <NUM>.

In step <NUM> of step <NUM>, a pseudo density image of the patient is generated based on the plurality of detected anatomical landmarks.

In step <NUM> of step <NUM>, the pseudo density image of the patient is projected to the X-ray detector based on a camera position of the camera, the distance from the X-ray emitter to the patient, and position information of the X-ray detector using a cone-beam projection to obtain the virtual X-ray image of the internal body structure.

In other words, a virtual X-ray image may be generated in two steps. Frist, anatomical landmarks are used to produce a pseudo density image of the patient e.g., by a neural network. Such network can be trained using annotated pairs of images, e.g., CT images and the corresponding masks. The neural network may be a convolutional network. Adversarial training may be used to train this model. See for example <NPL>). The resulted pseudo density image is then projected to the detector given the camera position, distance to patient, and the relative detector position using cone-beam projection. The obtained image is a virtual X-ray image.

Turning back to <FIG>, in step <NUM>, the method <NUM> further comprises a step of registering the virtual X-ray image of the internal body structure with the reference X-ray image of the internal body structure
In some examples, a pre-trained neural network may be applied to register the virtual X-ray image of the internal body structure with the reference X-ray image of the internal body structure. The pre-trained neural network has been trained to generate a residual parameter of a camera position of the camera from the virtual X-ray image, the reference X-ray image, and the plurality of detected anatomical landmarks. The residual parameter of the camera position is usable to transform the virtual X-ray image to the reference X-ray image. For example, a neural network may have been trained in the following way. The neural network takes three inputs: a reference X-ray image, a virtual X-ray image, and an anatomical landmarks map. The output of the model is a residual parameter of the camera position. Such parameters can be applied to transform virtual image towards reference image. The required images for training are artificially sampled from pseudo CT projects using static detector parameters and flexible emitter. The registration results are used to predict the directions on how to adjust the emitter's position. The neural network may be a convolutional network. Self-supervised learning can be applied to train this model. See for example <NPL>.

In step <NUM>, the method <NUM> further comprises a step of determining at least one parameter to adjust the position of the X-ray emitter based on a result of registration. In other words, the registration results are used to predict the directions on how to adjust the emitter's position. The at least one parameter may include one or more of a parameter for adjusting the distance from the X-ray emitter to the patient and a parameter for adjusting a rotation angle of the X-ray emitter.

In some examples, the apparatus <NUM> may provide, based on the at least one determined parameter, an instruction signal to guide a user to manually adjust the position of the X-ray emitter. In some examples, the instruction signal may be a voice signal for guiding a technician to manually correct the position of the X-ray emitter. In some examples, the instruction signals may be a displayed instruction used to guide a technician to manually correct the position of the X-ray emitter. In some other examples, the apparatus <NUM> may provide based on the at least one determined parameter, a control signal usable to control a mobile X-ray robotic arm <NUM> of the mobile X-ray device <NUM> to adjust the position of the X-ray emitter <NUM>.

<FIG> shows an exemplary working principle of the exemplary X-ray imaging system <NUM> shown in <FIG>.

The mobile X-ray machine <NUM> is placed in front of the patient (indicated with "a"). The X-ray detector <NUM> is placed behind the patient.

The technician may download from the PACS a reference X-ray image (indicated with "b"). It may be a previous scan of a good quality or some reference image or atlas of a good quality. The reference X-ray image is provided to the apparatus <NUM>. The camera <NUM> sends a camera-acquired image, e.g., the real-time video stream or images, to the apparatus <NUM>.

The apparatus <NUM> may comprise a first neural network, also referred to as neural network A, to predict anatomical landmarks and the distance to the patient (indicated with "c") based on the camera-acquired image in the patient video.

The apparatus <NUM> may comprise a second neural network, also referred to as neural network B, to predict a virtual X-ray image using the anatomical landmarks (indicated with "d") and the X-ray emitter (indicated with "f") and detector position (indicated with "e"). A virtual X-ray image is generated in two steps. 3D anatomical landmarks from neural network A are being used to produce a pseudo density image of the patient by neural network B. Such network can be trained using annotated pairs of images, e.g., CT images and the corresponding masks. The resulted pseudo density image is being projected to the detector given the camera position, distance to patient, and the relative detector position using cone-beam projection. The obtained image is a virtual X-ray image.

The apparatus <NUM> may comprise a third neural network, also referred to as neural network C, to register the virtual X-ray (indicated with "h") with the reference X-ray image (indicated with "i") using the anatomical landmarks (indicated with "g"). The neural network C has been trained in the following way. It takes three inputs: a reference X-ray image, a virtual X-ray image and an anatomical landmarks map. The output of the model is a residual parameter of the camera position. Such parameters can be applied to transform virtual image towards reference image. The required images for training are artificially sampled from pseudo CT projects using static detector parameters and flexible emitter parameters. The registration results are used to predict the directions on how to adjust the emitter's position.

The predicted directions (indicated with "j") are used on the X-ray machine: they either can be displayed for the manual correction by the technician, or an automated correction by a mobile X-ray robotic arm (if present) can be realized.

The method, apparatus, and system as disclosed herein can automatically locate an optimal X-ray emitter position for an image using either a prior high-quality image from the same patient, or a reference image from an atlas, deemed suitable for this imaging case, and can automatically acquire mobile X-ray images of better quality. The method, apparatus, and system as disclosed herein may lower the number of retakes and therefore achieve a reduction of the radiation dose received by the patients. The method, apparatus, and system as disclosed herein may avoid low quality images being sent to the PACs and rejected at the time of the review and therefore achieve reduced turnaround times. The method, apparatus, and system as disclosed herein may also reduce costs, because of the multiple job repetitions may be reduced.

In another exemplary embodiment of the present invention, a computer program or a computer program element is provided that is characterized by being adapted to execute the method steps of the method according to one of the preceding sfs45kk, on an appropriate system.

The computer program element might therefore be stored on a computer unit, which might also be part of an embodiment of the present invention. This computing unit may be adapted to perform or induce a performing of the steps of the method described above. Moreover, it may be adapted to operate the components of the above described apparatus. The computing unit can be adapted to operate automatically and/or to execute the orders of a user. The data processor may thus be equipped to carry out the method of the invention.

Claim 1:
A computer-implemented method for determining a position of an X-ray emitter of a mobile X-ray device, comprising:
- receiving (<NUM>) (i) a camera-acquired image originating from a camera that monitors a patient in an X-ray imaging session, wherein the camera is registered to an origin of the X-ray emitter, (ii) a reference X-ray image of an internal body structure to be examined in the X-ray examination, and (iii) position information of an X-ray detector;
- detecting (<NUM>) and localizing a plurality of anatomical landmarks in the camera-acquired image;
- determining (<NUM>) a distance from the X-ray emitter to the patient based on the camera-acquired image;
characterised in that
- generating (<NUM>) a virtual X-ray image of the internal body structure based on the plurality of detected anatomical landmarks, the distance from the X-ray emitter to the patient, and the position information of the X-ray detector;
- producing (<NUM>) a pseudo density image of the patient based on the plurality of detected anatomical landmarks; and
- projecting (<NUM>) the pseudo density image of the patient to the X-ray detector based on a camera position of the camera, the distance from the X-ray emitter to the patient, and position information of the X-ray detector using a cone-beam projection to obtain the virtual X-ray image of the internal body structure,
- registering (<NUM>) the virtual X-ray image of the internal body structure with the reference X-ray image of the internal body structure; and
- determining (<NUM>) at least one parameter to adjust the position of the X-ray emitter based on a result of registration.