Patent Description:
Radiation therapy involves medical procedures that selectively expose certain areas of a human body, such as cancerous tumors, to high doses of radiation. The intent of the radiation therapy is to irradiate the targeted biological tissue such that the harmful tissue is destroyed. During a radiation therapy, a radiation source may be rotated around a patient to deliver radiation from different angles at target region inside the patient. The radiation source may be mounted on an arm or a ring gantry. In certain radiation therapy, the patient support supporting the patient may also be moved. Despite careful treatment planning, during a medical procedure, a collision may occur between a moving part of a medical device and a patient. For example, the gantry of the radiation machine and the patient may possibly collide during radiation therapy. As the dose delivery plans become more complex, the combination of a rotating gantry during treatment and couch movement for non-coplanar beams delivery has increased the chance of collisions.

While gantry mounted laser guard system and room mounted scanners have been used to detect possible collisions, these methods have disadvantages. While a gantry mounted laser guard system has the ability to detect when a plane defined by laser scanning is intersected by an intruding object, it does not work in many situations where the couch is rotated.

In the case of a room-based scanning system, a room mounted scanner creates a profile of the patient on the couch and the profile is added to the knowledge base of the machine with software that keeps track of machine parts movement. A room-based scanning system is model-based and thus requires that the model be updated with every couch movement. In addition, from certain angles the scanner may not be able to re-profile the patient. An article by <NPL>, discloses several techniques for interactively performing occlusion and collision detection between static real objects and dynamic virtual objects in augmented reality. <CIT> discloses a collision protection system for a medical imaging system. Applicant of the subject application determines that it may be desirable to develop a collision avoidance system that functions irrespective of the position and movement of the couch and other parts of the medical device.

A method of detecting a possible collision in a medical procedure, includes: obtaining a reference depth image; obtaining an input depth image; determining a composite image using at least a part of the reference depth image and at least a part of the input depth image, wherein the act of determining the composite image is performed using a processing unit; and determining whether there is a possible collision between an object and a patient based on the composite image.

Optionally, the reference depth image is generated based on a time-of-flight technique.

Optionally, the reference depth image is generated using infrared illumination.

Optionally, the act of obtaining the reference depth image and the input depth image are performed by the processing unit receiving the reference depth image and the input depth image.

Optionally, the method further includes generating the reference depth image using a depth sensing camera.

Optionally, the input depth image comprises a real-time depth image obtained using a depth sensing camera.

Optionally, the depth sensing camera is mounted so that its position relative to a patient support is fixed, and is viewing the patient from the foot-to-head direction.

Optionally, the act of determining the composite image comprises performing a subtraction using the at least a part of the reference depth image and the at least a part of the input depth image.

Optionally, the method further includes identifying an outline of the patient in the reference depth image.

Optionally, the method further includes determining a detection zone based on an outline of the patient in the reference depth image.

Optionally, the part of the at least a part of the reference depth image comprises image data in the detection zone.

Optionally, the act of determining whether there is a possible collision comprises determining whether a value of an image data in the composite image is greater or less than a threshold.

Optionally, the method further includes obtaining a visible image of the patient.

Optionally, the method further includes displaying the visible image together with the composite image in a screen.

Optionally, the act of determining whether there is a possible collision between the object and the patient is based on both the composite image and the visible image.

Optionally, the method further includes obtaining an infrared image.

Optionally, the method further includes displaying the infrared image together with the composite image in a screen.

Optionally, the act of determining whether there is a possible collision between the object and the patient is based on both the composite image and the infrared image.

Optionally, the method further includes generating a warning signal, generating a control signal to stop an operation of a medical device, or both, in response to the determined possible collision.

Optionally, the method further includes: obtaining an additional reference depth image; obtaining an additional input depth image; and determining an additional composite image using at least a part of the additional reference depth image and at least a part of the additional input depth image.

Optionally, the reference depth image and the input depth image are generated using a first depth sensing camera, and the additional reference depth image and the additional input depth are generated using a second depth sensing camera.

Optionally, the method further includes generating the reference depth image using a depth sensing camera that is mounted to a patient support.

Optionally, the act of determining whether there is a possible collision comprises using an intrusion detection zone, wherein a shape of the intrusion detection zone is independent of a movement of a patient support supporting the patient.

An apparatus for detecting a possible collision in a medical procedure, includes: a depth sensing camera for providing a reference depth image, and an input depth image; and a processing unit configured for: determining a composite image using at least a part of the reference depth image and at least a part of the input depth image, and determining whether there is a possible collision between an object and a patient based on the composite image.

Optionally, the depth sensing camera is configured to generate the reference depth image based on a time-of-flight technique.

Optionally, the depth sensing camera is configured to generate the reference depth image using infrared illumination.

Optionally, the processing unit is configured to use the input depth image as a real-time depth image.

Optionally, the apparatus further includes a securing mechanism for securing the depth sensing camera in a fixed position relative to a patient support, wherein the depth sensing camera is oriented for viewing the patient from the foot-to-head direction.

Optionally, the processing unit is configured for determining the composite image by performing a subtraction using the at least a part of the reference depth image and the at least a part of the input depth image.

Optionally, the processing unit is further configured for identifying an outline of the patient in the reference depth image.

Optionally, the processing unit is further configured for determining a detection zone based on an outline of the patient in the reference depth image.

Optionally, the at least a part of the reference depth image comprises image data in the detection zone.

Optionally, the processing unit is configured for determining whether there is a possible collision by determining whether a value of an image data in the composite image is greater or less than a threshold.

Optionally, the processing unit is also configured to obtain a visible image of the patient.

Optionally, the processing unit is configured to output both the visible image and the composite image for display on a screen.

Optionally, the processing unit is configured for determining whether there is a possible collision between the object and the patient based on both the composite image and the visible image.

Optionally, the processing unit is also configured to obtain an infrared image.

Optionally, the processing unit is configured to output both the infrared image and the composite image for display on a screen.

Optionally, the processing unit is configured for determining whether there is a possible collision between the object and the patient based on both the composite image and the infrared image.

Optionally, the processing unit is further configured for generating a warning signal, generating a control signal to stop an operation of a medical device, or both, in response to the determined possible collision.

Optionally, the apparatus further includes an additional depth measuring camera for generating an additional reference depth image and an additional input depth image; wherein the processing unit is configured for determining an additional composite image using at least a part of the additional reference depth image and at least a part of the additional input depth image.

Optionally, the depth sensing camera is mounted to a patient support.

Optionally, the processing unit is configured to use an intrusion detection zone to determine whether there is a possible collision between the object and the patient, and wherein a shape of the intrusion detection zone is independent of a movement of a patient support supporting the patient.

A computer product includes a non-transitory medium storing a set of instructions, an execution of which by a processing unit causes a method for detecting a possible collision in a medical procedure to be performed, the method comprising: obtaining a reference depth image; obtaining an input depth image; determining a composite image using at least a part of the reference depth image and at least a part of the input depth image; and determining whether there is a possible collision between an object and a patient based on the composite image.

A method of detecting a possible collision in a medical procedure, includes: obtaining a reference depth image; using the reference depth image to determine a region of interest; obtaining a reference optical image; obtaining an input optical image; determining a composite image using the reference optical image and the input optical image, wherein the act of determining the composite image is performed using a processing unit; and determining whether there is a possible collision between an object and a patient based on at least a part of the composite image that corresponds with the region of interest.

Optionally, the reference optical image comprises a reference infrared image.

Optionally, the act of determining the composite image comprises performing an image subtraction.

Optionally, the act of determining whether there is a possible collision comprises determining whether an absolute value of a pixel in the composite image exceeds a threshold.

Optionally, the pixel in the composite image corresponds to a position in the region of interest.

Optionally, a shape of the region of interest is independent of a movement of a patient support supporting the patient.

A system for detecting a possible collision in a medical procedure, includes: a depth sensing camera for obtaining a reference depth image; an optical camera for obtaining a reference optical image, and obtaining an input optical image; and a processing unit configured for using the reference depth image to determine a region of interest, determining a composite image using the reference optical image and the input optical image, and determining whether there is a possible collision between an object and a patient based on at least a part of the composite image that corresponds with the region of interest.

Optionally, the processing unit is configured to perform an image subtraction to determine the composite image.

Optionally, the processing unit is configured to determine whether an absolute value of a pixel in the composite image exceeds a threshold.

A computer product includes a non-transitory medium storing a set of instructions, an execution of which by a processing unit causes a method for detecting a possible collision in a medical procedure to be performed, the method comprising: obtaining a reference depth image; using the reference depth image to determine a region of interest; obtaining a reference optical image; obtaining an input optical image; determining a composite image using the reference optical image and the input optical image; and determining whether there is a possible collision between an object and a patient based on at least a part of the composite image that corresponds with the region of interest.

Other and further aspects and features will be evident from reading the following detailed description.

The drawings illustrate the design and utility of embodiments, in which similar elements are referred to by common reference numerals. These drawings are not necessarily drawn to scale. In order to better appreciate how the above-recited and other advantages and objects are obtained, a more particular description of the embodiments will be rendered, which are illustrated in the accompanying drawings. These drawings depict only exemplary embodiments and are not therefore to be considered limiting in the scope of the claims.

It should be noted that the figures are not drawn to scale and that elements of similar structures or functions are represented by like reference numerals throughout the figures.

<FIG> illustrates a radiation treatment system <NUM>. The system <NUM> includes an arm gantry <NUM>, a patient support <NUM> for supporting a patient <NUM>, and a control system <NUM> for controlling an operation of the gantry <NUM> and delivery of radiation. The system <NUM> also includes a radiation source <NUM> that projects a beam <NUM> of radiation towards the patient <NUM> while the patient <NUM> is supported on support <NUM>, and a collimator system <NUM> for changing a cross sectional shape of the radiation beam <NUM>. The radiation source <NUM> may be configured to generate a cone beam, a fan beam, or other types of radiation beams in different embodiments. Also, in other embodiments, the source <NUM> may be configured to generate proton beam as a form of radiation for treatment purpose. Also, in other embodiments, the system <NUM> may have other form and/or configuration. For example, in other embodiments, instead of an arm gantry <NUM>, the system <NUM> may have a ring gantry <NUM>.

In the illustrated embodiments, the radiation source <NUM> is a treatment radiation source for providing treatment energy. In other embodiments, in addition to being a treatment radiation source, the radiation source <NUM> can also be a diagnostic radiation source for providing diagnostic energy for imaging purpose. In such cases, the system <NUM> will include an imager, such as the imager <NUM>, located at an operative position relative to the source <NUM> (e.g., under the support <NUM>). In further embodiments, the radiation source <NUM> may be a treatment radiation source for providing treatment energy, wherein the treatment energy may be used to obtain images. In such cases, in order to obtain imaging using treatment energies, the imager <NUM> is configured to generate images in response to radiation having treatment energies (e.g., MV imager). In some embodiments, the treatment energy is generally those energies of <NUM> kilo-electron-volts (keV) or greater, and more typically <NUM> mega-electron-volts (MeV) or greater, and diagnostic energy is generally those energies below the high energy range, and more typically below <NUM> keV. In other embodiments, the treatment energy and the diagnostic energy can have other energy levels, and refer to energies that are used for treatment and diagnostic purposes, respectively. In some embodiments, the radiation source <NUM> is able to generate X-ray radiation at a plurality of photon energy levels within a range anywhere between approximately <NUM> keV and approximately <NUM> MeV. In further embodiments, the radiation source <NUM> can be a diagnostic radiation source. In such cases, the system <NUM> may be a diagnostic system with one or more moving parts. In the illustrated embodiments, the radiation source <NUM> is carried by the arm gantry <NUM>. Alternatively, the radiation source <NUM> may be located within a bore (e.g., coupled to a ring gantry).

In the illustrated embodiments, the control system <NUM> includes a processing unit <NUM>, such as a processor, coupled to a control <NUM>. The control system <NUM> may also include a monitor <NUM> for displaying data and an input device <NUM>, such as a keyboard or a mouse, for inputting data. The operation of the radiation source <NUM> and the gantry <NUM> are controlled by the control <NUM>, which provides power and timing signals to the radiation source <NUM>, and controls a rotational speed and position of the gantry <NUM>, based on signals received from the processing unit <NUM>. Although the control <NUM> is shown as a separate component from the gantry <NUM> and the processing unit <NUM>, in alternative embodiments, the control <NUM> can be a part of the gantry <NUM> or the processing unit <NUM>.

In some embodiments, the system <NUM> may be a treatment system configured to deliver treatment radiation beam towards the patient <NUM> at different gantry angles. During a treatment procedure, the source <NUM> rotates around the patient <NUM> and delivers treatment radiation beam from different gantry angles towards the patient <NUM>. While the source <NUM> is at different gantry angles, the collimator <NUM> is operated to change the shape of the beam to correspond with a shape of the target tissue structure. For example, the collimator <NUM> may be operated so that the shape of the beam is similar to a cross sectional shape of the target tissue structure. In another example, the collimator <NUM> may be operated so that different portions of the target tissue structure receive different amount of radiation (as in an IMRT procedure).

As shown in <FIG> and <FIG>, the medical system <NUM> also includes a collision detection system <NUM>, which includes a depth measuring camera <NUM>, a securing mechanism <NUM> for securing the depth measuring camera <NUM> relative to an object, and a support <NUM>.

The depth sensing camera <NUM> is configured to sense depths and to generate signals representing the depths. In some embodiments, the depth sensing camera <NUM> may use structured light for depth measurement (e.g., a Kinect camera). In other embodiments, the depth sensing camera <NUM> may use time-of-flight method for depth measurement (e.g., Mesa SR4000, or the new Microsoft Kinect2 camera). In further embodiments, the depth sensing camera <NUM> may be any device that is capable of sensing depth using any known techniques. It should be noted that the term "camera", as used in this specification, may be any device, and should not be limited to a device that provides "image" signals. For example, in some embodiments, the depth sensing camera <NUM> may be configured to provide depth signals, which may or may not be considered image signals, regardless of whether such depth signals are displayed in image form or not. A depth signal may be any signal indicating a depth or distance, or any signal from with a depth or distance may be derived. By means of non-limiting examples, the signal may be an infrared signal, an ultrasound signal, etc. In some embodiments, the dimensions of the depth sensing camera <NUM> may be small enough to be non-intrusive to the treatment process when mounted during use. For example, in some embodiments, the camera <NUM> may have a dimension of <NUM> inch x <NUM> inch x <NUM> inch. In other embodiments, the camera <NUM> may have other dimensions, such as those larger or smaller than the example provided above, as long as the use of the camera <NUM> does not interfere with the treatment procedure.

Also, in some embodiments, the depth sensing camera <NUM> may be infrared-based, in which cases, the depth may be sensed by the camera <NUM> using infrared. In some embodiments, such depth sensing camera <NUM> may be configured to output infrared video images from which depth images are formed. In some embodiments, these infrared video images may have exactly the same field of view as the depth images. Thus, the infrared video images may be used together with the depth images to determine whether there is a possible collision.

Furthermore, in some embodiments, the depth sensing camera <NUM> may include an infrared emitter, a color sensor, and an infrared depth sensor. The infrared depth sensor is configured to sense depth based on infrared signals output by the infrared emitter. The color sensor is configured to sense visible image.

In some embodiments, the depth sensing camera <NUM> may have a detection (or frame) rate of <NUM> per second or higher. In other embodiments, the detection / frame rate may be less than <NUM> per second.

The support <NUM> may be a post, a bracket, a beam, an arm, etc., for supporting the depth sensing camera <NUM>. The securing mechanism <NUM> may be located at the support <NUM>. Also, in some embodiments, the support <NUM> may optionally have one or more moveable parts to allow a position and/or an orientation of the depth sensing camera <NUM> to be adjusted relative to the support <NUM> (or relative to the patient <NUM> or another reference location). In some embodiments, the support <NUM> itself may be movable relative to the support <NUM> in order to adjust the camera position (e.g., longitudinally) relative to the patient. In further embodiments, the support <NUM> may be a base with a tilt motor, which allows the camera <NUM> to be tilted in one, two, or three, degrees of movement relative to the base. In other embodiments, the support <NUM> is not needed, and the collision detection system <NUM> may not include the support <NUM>.

In the illustrated embodiments, the securing mechanism <NUM> is configured to secure the depth measuring camera <NUM> to a bracket <NUM> at the foot of the support <NUM>. The bracket <NUM> may be considered as a part of the collision detection system <NUM> in some embodiments. Alternatively, the bracket <NUM> may be considered as a part of the patient support <NUM>. In some embodiments, the bracket <NUM> may have an opening to allow the patient's feet to go through it if necessary (<FIG>). For example, the bracket <NUM> may have an opening height that is more than <NUM> inches, such as <NUM> inches, <NUM> inches, <NUM> inches, etc., in order to allow the patient's feet to enter therethrough.

In other embodiments, the bracket <NUM> is optional, and the securing mechanism <NUM> may be configured to secure the depth measuring camera <NUM> directly to the patient support <NUM>, or to other component(s) of the medical system <NUM>. In further embodiments, the securing mechanism <NUM> may be configured to secure the depth measuring camera <NUM> to a room, such as to a ceiling, a wall, or a floor. In still further embodiments, the securing mechanism <NUM> may be configured to secure the depth measuring camera <NUM> to a structure that is not a part of the medical system <NUM>. The securing mechanism <NUM> may be a clamp for grasping an object, a screw for insertion into a screw slot located in an object to which the depth measuring camera <NUM> is to be secured against, a snap-and-fit type connector, a hook-and-loop type connector, or any of other types of securing mechanism. In still further embodiments, the securing mechanism <NUM> is not required, and the collision detection system <NUM> does not include the securing mechanism <NUM>. For example, in other embodiments, the support <NUM> may be a base, and the base may be placed on a flat surface that supports the depth sensing camera <NUM> during use.

In the illustrated embodiments, the depth sensing camera <NUM> is mounted above the top surface of the support <NUM>. The height of the depth sensing camera <NUM> may be adjustable so that the head of the patient <NUM> is visible over his/her belly. Also, the orientation of the depth sensing camera <NUM> may be adjustable to adjust a viewing angle (e.g., relative to a horizontal surface). It should be noted that mounting the depth sensing camera <NUM> so that its position is fixed relative to the support <NUM> is advantageous because such configuration allows the depth sensing camera <NUM> to move with the patient support <NUM> (and therefore the patient <NUM>) irrespective of the movement of the support <NUM>.

As shown in <FIG>, the collision detection system <NUM> also includes a processing unit <NUM> communicatively coupled to the depth sensing camera <NUM>. The processing unit <NUM> is configured to process signals transmitted from the depth sensing camera <NUM>, and to determine whether there is a possible collision between the patient and an object based on the signals. In some embodiments, the processing unit <NUM> may be a processor, such as an ASIC processor, a FPGA processor, a general purpose processor, or any of other types of processor. Also, the processing unit <NUM> may include hardware, software, or combination of both. Also, in some embodiments, the processing unit <NUM> may be the same as the processing unit <NUM>, or a component of the processing unit <NUM>. In other embodiments, the processing unit <NUM> may be considered to be a part of the treatment system <NUM>, and not a part of the collision detection system <NUM>.

<FIG> illustrates a method <NUM> for determining whether there is a possible collision using the system <NUM> in accordance with some embodiments. First, the patient <NUM> is placed on the patient support <NUM>, and is positioned in a desired location relative to the treatment system <NUM> in a patient setup procedure. Also, the depth sensing camera <NUM> is mounted so that its position is fixed relative to the support <NUM>. The height of camera <NUM> and its orientation are then adjusted such that the camera <NUM> is looking towards the patient body (e.g., from one end where the feet are located towards the head, or vice versa), and can capture images of areas surrounding the patient <NUM>.

After the patient setup procedure is completed, the depth sensing camera <NUM> is activated to capture a reference depth image while the patient <NUM> is on the support <NUM>. As used in this specification, the term "image" may refer to any group of data (e.g., depth values), which may or may not be displayed in image form. For example, the data may be stored in a non-transitory medium, and/or may be processed by the processing unit <NUM> without being displayed to a user for viewing. The reference depth image is then transmitted to the processing unit <NUM>. After the processing unit <NUM> obtains the reference image (item <NUM>), the processing unit <NUM> then processes the reference image to create one or more intrusion detection zone (item <NUM>). In some embodiments, the intrusion detection zone(s) may be stored in a non-transitory medium for later processing. <FIG> illustrates an example of a reference image <NUM> generated using the depth sensing camera <NUM>. As shown in the example, the reference image <NUM> includes an outline/boundary <NUM> of the patient <NUM>. In some embodiments, the boundary <NUM> of the patient <NUM> may be identified automatically by the processing unit <NUM> using image processing techniques. In other embodiments, the boundary <NUM> of the patient <NUM> may be determined manually by a user, who views the depth image, manually identifies the patient boundary <NUM>, and draws the boundary <NUM> using a graphical user interface. In some embodiments, all of the area in the reference image within the outline <NUM> may be considered an intrusion detection zone <NUM>. Thus, the intrusion detection zone <NUM> may be considered a portion of the reference depth image. In other embodiments, the processing unit <NUM> may generate a larger outline <NUM> surrounding the patient outline <NUM>, and the area within the larger outline <NUM> may be considered an intrusion detection zone <NUM> (<FIG>). In some embodiments, the width between the patient outline <NUM> and the larger outline <NUM> may be determined to account for possible patient movement and safety margin. In further embodiments, the processing unit <NUM> may perform an image logical subtraction between the area formed by the larger outline <NUM> and the area formed by the patient outline <NUM>, and the difference image may then be used as an intrusion detection zone <NUM> (<FIG>). As shown in the figure, in such cases, the intrusion detection zone <NUM> includes only a strip of the buffer area around the patient outline <NUM>. In further embodiments, the intrusion detection zone <NUM> may be defined as the area in the reference depth image between the expanded larger outline <NUM>, and a smaller outline that is contracted from the patient outline <NUM>. It should be noted that as used in this specification, the term "reference depth image" may refer to the entire frame captured by the camera <NUM>, or may refer to a subset of the entire frame captured by the camera <NUM> (such as an intrusion detection zone <NUM>). Also, in some embodiments, the intrusion detection zone <NUM> itself may be considered a reference depth image.

In some cases, if the system <NUM> includes the monitor <NUM>, the depth image provided by the camera <NUM>, and/or the intrusion detection zones <NUM> generated by the processing unit <NUM>, may be displayed on the monitor <NUM>.

It should be noted that the outline for creating the intrusion detection zone <NUM> is not limited to that of a patient, and that the outline may include at least a part of a device, such as a part of a treatment machine and/or a patient support. In other embodiments.

Returning to <FIG>, after the intrusion detection zone <NUM> is created, a treatment procedure may then begin. In some embodiments, the treatment procedure is carried out using the system <NUM> of <FIG>. In such cases, the gantry <NUM> may be rotated around the patient <NUM>, and the radiation source <NUM> may deliver radiation from different gantry angles towards the patient <NUM>. During the treatment procedure, the depth sensing camera <NUM> captures a real time (input) depth image and transmits the real time image to the processing unit <NUM>. As used in this specification, the term "input depth image" or "real time depth image" may refer to the entire frame captured by the camera <NUM>, or may refer to a subset of the entire frame (such as an area that corresponds with the intrusion detection zone <NUM>). The processing unit <NUM> receives the real time depth image (item <NUM>), and determines whether there is a possible collision based on the real time depth image and the intrusion detection zone <NUM> (item <NUM>). Because the intrusion detection zone <NUM> includes at least a portion of the reference depth image, or is derived from the reference depth image, or may be considered to be a feature in the reference depth image, or may be considered a reference depth image itself, the determination of a possible collision may also be considered as basing on at least a part of the reference depth image. Items <NUM>-<NUM> are repeated so that the processing unit <NUM> receives multiple real time depth images at different times during the treatment procedure, and determines whether there is any possible collision based on those real time depth images during the course of the treatment procedure.

<FIG> illustrates a method <NUM> to implement item <NUM> in some embodiments. During the treatment procedure, after the processing unit <NUM> receives a real time depth image, the processing unit <NUM> compares the real time depth image with the reference depth image by performing a subtraction between the reference depth image and the real time depth image to get a subtraction image (item <NUM>). In some embodiments, the act of comparing the real time depth image with the reference depth image may be performed by comparing a portion of the real time depth image corresponding to the area of the intrusion detection zone <NUM>, with a portion of the reference depth image where the intrusion detection zone <NUM> is located. The processing unit <NUM> then analyzes the subtraction image in the intrusion detection zone <NUM> to determine whether there is a possible collision (item <NUM>). In the illustrated embodiments, the processing unit <NUM> determines that there is a possible collision between the patient <NUM> and an object when a difference between the reference depth image and the real time depth image falls below certain specified threshold. In some embodiments, the difference may be obtained by subtracting the real time depth image from the reference depth image, or vice versa. Also, in some embodiments, an absolute difference of the pixel values may be used so that it does not matter whether the real time depth image is subtracted from the reference depth image or vice versa.

<FIG> illustrate an example of the method <NUM>, and how an image subtraction between an input depth image and the reference depth image may be used to detect a possible collision. Because the depth sensing camera <NUM> is configured to sense depth, the two-dimensional image it produces has an array of pixel values that indicate respective distances between the camera <NUM> (i.e., the sensors in the camera <NUM>) and different objects (or different parts of an object). For example, as shown in <FIG>, an arbitrary origin may be set to be at the camera, and so an object that is further away from the camera <NUM> will have a corresponding sensed depth by the camera <NUM> that has a higher value than another object that is closer to the camera <NUM>. Thus, a pixel with a smaller value indicates that the sensed object is closer to the depth sensing camera <NUM> compared to a pixel that has a larger value.

<FIG> shows an example of a reference depth image <NUM> having an intrusion detection zone <NUM> represented by pixel values that is generated using the depth sensing camera <NUM> and the processing unit <NUM>. The reference depth image <NUM> may be generated before a treatment session, or during the treatment session. In the example, the reference depth image <NUM> is generated by using a depth image from the depth sensing camera <NUM> positioned near a foot of the patient <NUM> and looking towards the head of the patient <NUM>, like that shown in <FIG>, and by processing the depth image using the processing unit <NUM> to create an intrusion detection zone <NUM> (which has a shape that looks like that shown in <FIG>). The pixel values near the top of the intrusion detection zone <NUM> in the reference depth image <NUM> have relatively higher values because they correspond with the location where the head of the patient <NUM> is located, which is further away from the depth sensing camera <NUM>. On the other hand, the pixel values near the bottom of the intrusion detection zone <NUM> in the reference depth image <NUM> have relatively lower values because they correspond with the location where the feet of the patient <NUM> are located, which are closer towards the depth sensing camera <NUM>. Thus, the intrusion detection zone <NUM> is "three-dimensional" in the sense that it represents a region in a three-dimensional space for which collision detection is to be performed. In the illustrated example, the intrusion detection zone <NUM> has one strip of pixel values along the outline of the intrusion detection zone <NUM>. In other embodiments, there may be more pixels in the intrusion detection zone <NUM>, depending on how wide the intrusion detection zone <NUM> is set. Also, in other embodiments, instead of having one intrusion detection zone <NUM>, the processing unit <NUM> may be configured to, or a user may, determine multiple intrusion detection zones <NUM>. In some embodiments, one or more two-dimensional masks may be applied to the depth image from the depth image camera <NUM> to create the one or more detection zones <NUM> for the reference depth image <NUM>.

<FIG> shows a real time input depth image <NUM> represented by pixel values that are generated using the depth sensing camera <NUM> and the processing unit <NUM>. The real time depth image <NUM> is generated during the treatment session. Only values corresponding to the detection zone <NUM> are presented because the processing unit <NUM> is configured to analyze signals in the detection zone <NUM> for detecting possible collision. However, it should be understood that a complete real time input depth image <NUM> may include a two dimensional array of values that correspond to the view of the depth sensing camera <NUM>. In other embodiments, the detection zone <NUM> may include all pixel values within the patient outline <NUM> (like that shown in <FIG>), or within the margin <NUM> (like that shown in <FIG>), in which cases, there will be pixel values in the middle region of the detection zone <NUM>. Thus, as used in this specification, the term "input depth image" or "real time input depth image" may refer to the entire frame provided by the camera <NUM> or a subset of such entire frame (e.g., an area that corresponds to the intrusion detection zone <NUM>).

In the illustrated example, during treatment, the processing unit <NUM> subtracts depth values corresponding to the intrusion detection zone <NUM> in the real time depth image <NUM> from the depth values in the intrusion detection zone <NUM> in the reference depth image <NUM> in order to detect a possible collision. <FIG> shows the result of the subtraction. A detection of possible collision has occurred when the difference in value between the reference depth image <NUM> and the real time depth image <NUM> falls below a certain threshold. In the illustrated example, the threshold is set as <NUM>, which may be stored in a non-transitory medium for use by the processing unit <NUM>. Thus, any subtraction values with a number less than <NUM> represent a situation in which an object is too close to the patient to imply that a possible collision may be about to happen. As shown in the figure, some of the pixel values in the intrusion detection zone <NUM> in the subtraction image <NUM> have a value of <NUM>, which is below the prescribed threshold of <NUM>. When that occurs, the processing unit <NUM> may determine that there is a possible collision that may be about to happen. This is because when a moving object (such as the arm gantry <NUM>, the radiation source <NUM>, or the collimator <NUM> of the treatment system <NUM>) is moving towards the patient <NUM>, the object may appear within the field of view by the camera <NUM>, and may be closer to the camera <NUM> than the patient <NUM> based on the position of the camera <NUM> (like the situation shown in the left of <FIG>). Thus, a depth value in the intrusion detection zone <NUM> representing the depth from the camera <NUM> to the detected colliding object will be less than the depth value in the same point of the intrusion detection zone <NUM> in the reference image (i.e., when the colliding object is not present). As a result, when the real-time depth image (with the colliding object detected) is subtracted from the reference depth image, the difference value will be a positive number. The difference value represents a proximity between the detected colliding object and the patient <NUM>. If the detected object is far from the patient <NUM>, the difference value will have a relatively larger value. On the other hand, if the detected object is moved so that it is closer to the patient <NUM>, the difference value will have a relatively smaller value. If the difference value is below the prescribed threshold, it may represent the situation in which the detected moving object is too close to the patient to imply a possible collision that may be about to take place. Thus, as illustrated in the above example, the processing unit <NUM> may be configured to determine a possible collision by examining the values (intensity) of the various difference values in the intrusion detection zone <NUM> to see if a threshold has been violated.

In the illustrated embodiments, the processing unit <NUM> is configured to automatically detect a possible collision. In other embodiments, a user may also participate in determining whether there is a possible collision. For example, in some cases, the monitor <NUM> may be used to continuously display the real time depth images and/or the subtraction images representing comparisons of the reference depth image and the real time depth images during the procedure, so that the person operating the treatment procedure may view the images and identify possible collisions.

In some embodiments, when the processing unit <NUM> determines that there is a possible collision that may be about to happen, the processing unit <NUM> may generate a signal to stop the treatment system <NUM>, generate a visual and/or audio warning signal, or a combination of the above.

It should be noted that the pixel values are arbitrarily chosen based on a certain coordinate frame (e.g., located at the depth sensing camera <NUM> so that the coordinate is <NUM> at the camera). In other embodiments, the pixel values in the reference depth image and the real time depth images may be based on other coordinate frames. For example in other embodiments, the pixel values in an image may be based on a coordinate frame so that a pixel with a smaller value indicates that the sensed object is further from the camera <NUM> compared to a pixel that has a larger value.

Also, in other embodiments, instead of subtracting the real time depth image <NUM> from the reference depth image <NUM>, the processing unit <NUM> may be configured to subtract the reference depth image <NUM> from the real time depth image <NUM>. In such cases, the processing unit <NUM> may be configured to determine that a possible collision may be about to occur if there is a difference pixel value in the intrusion detection zone <NUM> that is above a certain prescribed threshold, such as -<NUM> in the illustrated example. Following the above example, if the reference depth image <NUM> is subtracted from the real time depth image <NUM>, some of the difference pixel values in the intrusion detection zone <NUM> will have a value of -<NUM>, which is higher than the threshold of -<NUM>. In such cases, the processing unit <NUM> may then determine that a possible collision may be about to happen. Also, as discussed, in some embodiments, an absolute difference of the pixel values may be used so that it does not matter whether the real time depth image is subtracted from the reference depth image or vice versa.

In some embodiments, the depth sensing camera <NUM> may have the ability to acquire optical images (e.g., infrared or visible images), simultaneously in addition to depth images. In other embodiments, there may be two separate cameras, one capturing depth images and the other capturing optical images. If depth and optical images are both acquired during the procedure, the processing unit <NUM> may display both images next to each other on the monitor <NUM>, or superimpose the two images on top of each other, to show how the depth image corresponds with the optical image. Also, in some embodiments, the processing unit <NUM> may perform analysis using optical images to determine whether there is a possible collision. For example, in some embodiments, the camera <NUM> may capture an infrared image (i.e., based on infrared emitter of the depth camera) of the patient <NUM> after the patient <NUM> has been set up on the support <NUM>, and the infrared image is then transmitted to the processing unit <NUM> for use as a reference image. During a treatment procedure, real-time infrared input images are provided by the camera <NUM>, and are transmitted to the processing unit <NUM>. The processing unit <NUM> may compare the real-time infrared images with the reference infrared image to determine whether there is a possible collision that may be about to happen.

In some embodiments, both optical images and depth images may be used by the processing unit <NUM> to determine whether there is a possible collision. For example, in some embodiments, both the reference depth image and a reference optical image may be provided by the camera <NUM> (or by separate depth sensing camera and optical image capturing camera). The reference depth image and the reference optical image may be generated at the same time or at different respective times. During a treatment procedure, real-time optical input images and real-time depth images are provided by the camera(s), and are transmitted to the processing unit <NUM>. The processing unit <NUM> may compare the real-time optical images with the reference optical image, as well as the real-time depth images with the reference depth image, to determine whether there is a possible collision that may be about to happen. For example, the processing unit <NUM> may compare real-time optical image V1 generated at time t1 with the reference optical image RV, as well as real-time depth image D1 generated at time t1 with the reference depth image RD, to determine whether there is a possible collision for time t1. Then, the processing unit <NUM> may compare real-time optical image V2 generated at time t2 with the reference optical image RV, as well as real-time depth image D2 generated at time t2 with the reference depth image RD, to determine whether there is a possible collision for time t2. As treatment continues, the processing unit <NUM> processes the images at different times to continuously detect possible collision.

As discussed, in some embodiments, the camera <NUM> may use an infrared emitter (illuminator), and the infrared images from which the depth data are derived may be output to the processing unit <NUM>. In some embodiments, both the infrared images and depth images may be used by the processing unit <NUM> to determine whether there is a possible collision. For example, in some embodiments, both the reference depth image and a reference infrared image may be provided by the camera <NUM> (or by separate depth sensing camera and infrared camera). The reference depth image and the reference infrared image may be generated at the same time or at different respective times. During a treatment procedure, real-time infrared input images and real-time depth images are provided by the camera(s), and are transmitted to the processing unit <NUM>. The processing unit <NUM> may compare the real-time infrared images with the reference visible image, as well as the real-time depth images with the reference depth image, to determine whether there is a possible collision that may be about to happen. For example, the processing unit <NUM> may compare real-time infrared image F1 generated at time t1 with the reference infrared image RIF, as well as real-time depth image D1 generated at time t1 with the reference depth image RD, to determine whether there is a possible collision for time t1. Then, the processing unit <NUM> may compare real-time infrared image F2 generated at time t2 with the reference infrared image RIF, as well as real-time depth image D2 generated at time t2 with the reference depth image RD, to determine whether there is a possible collision for time t2. As treatment continues, the processing unit <NUM> processes the images at different times to continuously detect possible collision.

In some embodiments, the real-time optical image is subtracted from the reference optical image, or vice versa. However the intrusion zone mask derived from the reference depth image is used as the region of interest (ROI) imposed on the subtraction image in order to check whether the absolute value of the pixels of the subtraction image exceeds a threshold. In other words, the depth image is used to determine a ROI for analyzing the optical image. This is especially effective when the optical image is produced by the camera infrared emitter, and therefore is not vulnerable to changing shadows created by the room ambient light and moving parts of the device.

Thus, as shown in <FIG>, in some embodiments, a method <NUM> of detecting a possible collision in a medical procedure, includes: obtaining a reference depth image (item <NUM>); using the reference depth image to determine a region of interest (item <NUM>); obtaining a reference optical image (e.g., visible image, or infrared image) (item <NUM>); obtaining an input optical image (item <NUM>); and determining whether there is a possible collision between an object and a patient based on the optical image and the reference optical image (item <NUM>). In some embodiments, item <NUM> may be implemented by determining a composite image using the reference optical image and the input optical image, wherein the act of determining the composite image is performed using a processing unit. The processing unit may then determine whether there is a possible collision between an object and a patient based on at least a part of the composite image that corresponds with the region of interest. In some embodiments, the act of obtaining the reference depth image may be achieved by the processing unit receiving the reference depth image. In other embodiments, the act of obtaining the reference depth image may be achieved by a depth sensing camera generating the reference depth image. Similarly, in some embodiments, the acts of obtaining the reference optical image and the input optical image may be achieved by the processing unit receiving these images. In other embodiments, the acts of obtaining the reference optical image and the input optical image may be achieved by an optical camera generating these images. Also, in some embodiments, the composite image may be determined by the processing unit performing an image subtraction between the reference optical image and the input optical image. In addition, in some embodiments, the region of interest may be determined in the same or similar manner as that for the intrusion detection zone <NUM> discussed previously. Furthermore, in some embodiments, the act of determining whether there is a possible collision includes determining whether an absolute value of a pixel in the composite image exceeds a threshold. The pixel in the composite image may correspond to a position in the region of interest (which region of interest is imposed on the composite image). In still further embodiments, the act of obtaining input optical image may be repeated to obtain additional input optical images (e.g., real time optical images). Also, the act of determining composite image may be repeated for the additional input optical images to obtain additional composite images, and the act of determining whether there is a possible collision that is about to take place may be repeated based on the additional composite images during the course of a medical procedure. This way, during the medical procedure, the patient may be monitored in real time to prevent an object from colliding against the patient.

In one or more embodiments, the depth image and the optical image may be superimposed / overlaid to obtain a composite image that shows both depth and visible image.

In the above embodiments, the intrusion detection system <NUM> is described as having one depth sensing camera <NUM>. In other embodiments, the intrusion detection system <NUM> may include multiple depth sensing cameras <NUM> to provide better coverage of the areas surrounding the patient <NUM>.

In should be noted that the collision detection system <NUM> is not limited to being used during treatment by a radiation treatment device, and may be used in other types of treatment procedures, or any of other types of procedures which may or may not involve radiation.

<FIG> is a block diagram illustrating an embodiment of a computer system <NUM> that can be used to implement various embodiments described herein. For example, the computer system <NUM> may be configured to implement the method of <FIG> in accordance with some embodiments. Also, in some embodiments, the computer system <NUM> may be used to implement the processing unit <NUM> of <FIG> and/or the processing unit <NUM> of <FIG>. Computer system <NUM> includes a bus <NUM> or other communication mechanism for communicating information, and a processor <NUM> coupled with the bus <NUM> for processing information. The processor <NUM> may be an example of the processor <NUM> of <FIG>, an example of the processor <NUM> of <FIG>, or an example of any processor described herein. The computer system <NUM> also includes a main memory <NUM>, such as a random access memory (RAM) or other dynamic storage device, coupled to the bus <NUM> for storing information and instructions to be executed by the processor <NUM>. The main memory <NUM> also may be used for storing temporary variables or other intermediate information during execution of instructions to be executed by the processor <NUM>. The computer system <NUM> further includes a read only memory (ROM) <NUM> or other static storage device coupled to the bus <NUM> for storing static information and instructions for the processor <NUM>. A data storage device <NUM>, such as a magnetic disk or optical disk, is provided and coupled to the bus <NUM> for storing information and instructions.

The computer system <NUM> may be coupled via the bus <NUM> to a display <NUM>, such as a cathode ray tube (CRT), for displaying information to a user. An input device <NUM>, including alphanumeric and other keys, is coupled to the bus <NUM> for communicating information and command selections to processor <NUM>.

In some embodiments, the computer system <NUM> can be used to perform various functions described herein. According to some embodiments, such use is provided by computer system <NUM> in response to processor <NUM> executing one or more sequences of one or more instructions contained in the main memory <NUM>. Those skilled in the art will know how to prepare such instructions based on the functions and methods described herein. Such instructions may be read into the main memory <NUM> from another computer-readable medium, such as storage device <NUM>. Execution of the sequences of instructions contained in the main memory <NUM> causes the processor <NUM> to perform the process steps described herein. One or more processors in a multi-processing arrangement may also be employed to execute the sequences of instructions contained in the main memory <NUM>. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions to implement the various embodiments described herein. Thus, embodiments are not limited to any specific combination of hardware circuitry and software.

The term "computer-readable medium" as used herein refers to any medium that participates in providing instructions to the processor <NUM> for execution. Such a medium may take many forms, including but not limited to, non-volatile media, volatile media, and transmission media. Non-volatile media includes, for example, optical or magnetic disks, such as the storage device <NUM>. A non-volatile medium may be considered an example of non-transitory medium. Volatile media includes dynamic memory, such as the main memory <NUM>. A volatile medium may be considered an example of non-transitory medium. Transmission media includes coaxial cables, copper wire and fiber optics, including the wires that comprise the bus <NUM>. Transmission media can also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.

Common forms of computer-readable media include, for example, a floppy disk, a flexible disk, hard disk, magnetic tape, or any other magnetic medium, a CD-ROM, any other optical medium, punch cards, paper tape, any other physical medium with patterns of holes, a RAM, a PROM, and EPROM, a FLASH-EPROM, any other memory chip or cartridge, a carrier wave as described hereinafter, or any other medium from which a computer can read.

Various forms of computer-readable media may be involved in carrying one or more sequences of one or more instructions to the processor <NUM> for execution. For example, the instructions may initially be carried on a magnetic disk of a remote computer. A modem local to the computer system <NUM> can receive the data on the telephone line and use an infrared transmitter to convert the data to an infrared signal. An infrared detector coupled to the bus <NUM> can receive the data carried in the infrared signal and place the data on the bus <NUM>. The bus <NUM> carries the data to the main memory <NUM>, from which the processor <NUM> retrieves and executes the instructions. The instructions received by the main memory <NUM> may optionally be stored on the storage device <NUM> either before or after execution by the processor <NUM>.

The computer system <NUM> also includes a communication interface <NUM> coupled to the bus <NUM>. The communication interface <NUM> provides a two-way data communication coupling to a network link <NUM> that is connected to a local network <NUM>. For example, the communication interface <NUM> may be an integrated services digital network (ISDN) card or a modem to provide a data communication connection to a corresponding type of telephone line. As another example, the communication interface <NUM> may be a local area network (LAN) card to provide a data communication connection to a compatible LAN. In any such implementation, the communication interface <NUM> sends and receives electrical, electromagnetic or optical signals that carry data streams representing various types of information.

Claim 1:
A computer implemented
method of detecting a possible collision between an object of a treatment system (<NUM>) and a patient (<NUM>) during a medical procedure, characterised by comprising:
obtaining a reference depth image (<NUM>) generated using a depth sensing camera (<NUM>), the depth sensing camera (<NUM>) having a position that is fixed relative to a patient support (<NUM>);
processing the reference depth image (<NUM>) to create one or more intrusion detection zones (<NUM>), wherein the act of processing the reference depth image (<NUM>) is performed using a processing unit (<NUM>);
obtaining, during the medical procedure, a real time input depth image (<NUM>) using the depth sensing camera (<NUM>);
determining a composite image (<NUM>) using at least a part of the reference depth image (<NUM>) corresponding to an area of the intrusion detection zone (<NUM>) and at least a part of the input depth image (<NUM>) corresponding to the area of the intrusion detection zone (<NUM>), wherein the act of determining the composite image is performed using the processing unit (<NUM>);
determining, during the medical procedure, whether there is a possible collision between an object of the treatment system (<NUM>) and a patient (<NUM>) supported on the patient support (<NUM>) based on the composite image (<NUM>); and
when it is determined that there is a possible collision between the object of the treatment system (<NUM>) and the patient (<NUM>) supported on the patient support (<NUM>), generating a control signal to stop an operation of the treatment system (<NUM>) to prevent the possible collision between the object and the patient (<NUM>).