Patent ID: 12207960

DETAILED DESCRIPTION

Various embodiments of projection mapping a radiation suite are described. It is to be understood that the disclosure is not limited to the particular embodiments described. An aspect described in conjunction with a particular embodiment is not necessarily limited to that embodiment and can be practiced in any other embodiments.

Various embodiments are described with reference to the figures. It should be noted that some figures are not necessarily drawn to scale. The figures are only intended to facilitate the description of specific embodiments, and are not intended as an exhaustive description or as a limitation on the scope of the disclosure. Further, in the figures and description, specific details may be set forth in order to provide a thorough understanding of the disclosure. It will be apparent to one of ordinary skill in the art that some of these specific details may not be employed to practice embodiments of the disclosure. In other instances, well known components may not be shown or described in detail in order to avoid unnecessarily obscuring embodiments of the disclosure.

All technical and scientific terms used herein have the meaning as commonly understood by one of ordinary skill in the art unless specifically defined otherwise. As used in the description and appended claims, the singular forms of “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise. The term “or” refers to a nonexclusive “or” unless the context clearly dictates otherwise. The term “first” or “second” etc. may be used to distinguish one element from another. The use of the term “first” or “second” should not be construed as in any particular order unless the context clearly dictates otherwise. Further, the singular form of “first” and “second” include plural references unless the context clearly dictates otherwise.

As used herein, the term “image” includes either a still image or a video feed, and includes either a pictorial image and/or a text image.

Disclosed herein is a system and method for projection mapping a radiation suite to provide an augmented reality or a calming environment. An image file can be projection mapped to a three-dimensional surface on a radiation system and/or room. The projected images or videos are particularly suited for calming or relaxing the patient, reducing fear and anxiety of the patient, and thus preventing or reducing the patient movement during setup and treatment or imaging. The image file can be dynamically altered in response to e.g. the changing location of the patient walking in the room or in response to the movement of the gantry and/or couch of the radiation system. A soundtrack and/or a scent related to the image content may be provided concurrently to create a greater emotional impact on the patient.

FIG.1schematically illustrates a radiation suite100according to embodiments of the disclosure. As shown, the radiation suite100includes a room120, a radiation system140, and an image projection system160. The image projection system160is operable to map an image file to a projection surface on the radiation system140and/or the room120and project the mapped image file on the projection surface, creating a calming environment for a patient. The radiation suite100may also include a scanner162operable to measure the room geometry for projection mapping or to track movement of the patient and/or radiation system for creating a dynamic interaction between the projected image and a moving object.

The room120may include a floor122, a ceiling124, and one or more walls126. At least a surface portion of the floor122, ceiling124, and walls126may constitute a projection surface displaying an image projected by the image projection system160. In some embodiments, all of the exposed surfaces of the floor122, ceiling124, and walls126may constitute a projection surface displaying an image projected by the image projection system160. As shown, the room120may include planar surfaces such as flat floor122, flat ceiling124, flat walls126. The room120may also include curved or angled surfaces such as a curved ceiling portion. The room120may further include non-coplanar surfaces such as adjoining walls126, adjoining ceiling124and walls126, adjoining floor122and walls126etc. The image projection system160, as will be described in greater detail below, may map an image file to the complex three-dimensional display surfaces of the room120, and the radiation system140as will be described below, such that the image projected on the display surfaces does not exhibit optical distortion. As used in the following description and appended claims, the term “projection surface” includes reference to a combination of a plurality of surfaces of various shapes, sizes, geometries, and contours, which collectively constitute a complex three-dimensional display surface.

The radiation system140may be a treatment system, an imaging system, or a simulation system. The radiation system140may be an x-ray radiation system, a gamma-ray radiation system, or a proton or heavy ion radiation system and so forth. The imaging system may be any kind of modalities including computed tomography (CT), magnetic resonance (MR), single photon emission computed tomography (SPECT), and so forth. The radiation system140may be a system that can perform both treatment and imaging. Indeed, the principles and inventive ideas described in the disclosure can apply to any kind of radiation systems. Further, it will be appreciated by one of ordinary skill in the art that the principles and inventive ideas can also apply to non-radiation medical procedures and/or facilities such as a doctor's office.

The exemplary radiation system140shown inFIG.1includes a gantry142and a couch144. The gantry142may enclose a radiation source (not shown) producing radiation and various components or devices modulating and measuring the properties of the radiation produced. The couch144supports a patient during irradiation. As shown, the gantry142may be a ring gantry, which includes a bore146for receiving at least a portion of the patient. A radiation source may be enclosed inside the ring gantry142and rotated about an isocenter. In some embodiments, the gantry may be in the form of a rotatable C-arm, which may rotate a radiation source mounted inside the gantry about an isocenter. In alternative embodiments, the gantry may be in the form of a robotic arm, carrying and moving a radiation source in multiple motion degrees of freedom. The couch144may also be moved in multiple motion degrees of freedom as well-known in the art.

The gantry142and the couch144may include complex three-dimensional surfaces, including planar, curved, angled surfaces of various shapes and sizes, non-coplanar, non-continuous surfaces (bore in the gantry), and linear, arcuate or circular edges, and so forth. For example, the front surface of the gantry142may include a combination of planar, curved or angled surfaces, non-coplanar, non-continuous surfaces, and linear, arcuate, and circular edges. The front or top surface of the couch144may also include a combination of planar and curved surfaces of various shapes and sizes, and linear and arcuate edges. In embodiments of the disclosure, at least a surface portion of the gantry142and/or the couch144may constitute a projection surface displaying an image projected by the image projection system160. In some embodiments, all of the exposed surface of the gantry142and the couch144may constitute a projection surface displaying an image projected by the image projection system160. As described above, the room120may also include complex three-dimension surface portions. The image projection system160, as will be described in greater detail below, may map an image file to the complex three-dimensional projection surface on the radiation system and/or the room such that the projected image matches the structural features or geometry of the projection surface and does not exhibit optical distortion.

Still referring toFIG.1, the image projection system160is operable to map an image file to at least a surface portion of the radiation system140, a surface portion of the room120, or combined surface portions of the radiation system140and room120, which constitutes a complex three-dimensional projection surface. The image projection system160may include a computer170, and one or more projectors164operably connected with the computer170. The computer170executes a mapping software171, which functions to map an image file to a selected projection surface on the radiation system140and/or the room120. The one or more projectors164, which may be controlled by the computer170, are operable to project the mapped image file to the selected projection surface.

The computer170may be a general purpose or special purpose computer, and may include a processor172, a memory173, input/output devices174, a monitor and user interface175, and various ports (not shown) for receiving and sending data signals between the computer170and projectors164and/or scanner162. The computer170may be operably connected with the projectors164and/or scanner162wirelessly or via a connector cable such as HDMI cable, USB cable, fiber-optic cable, or the like. The computer170can be connected to the Internet177, e.g. to an image content provider for purchasing or downloading image data files. Alternatively, various image data files may be locally stored in the memory173of the computer170or stored in a content server which may be accessed by the computer170. The computer170may include software necessary to perform various functions, including a projection mapping software171to be executed by a user through the user interface175.

According to embodiments of the disclosure, the image data file to be projection mapped may include all kinds of calming images. By way of example, images of a waterfall and/or mountain scenery may have a calming effect on some patients. Some patients may find gardens and/or rain forests more relaxing. To many children, images of animals, birds, fish, space, or the like may be more effective in distracting them from their treatment or imaging by a radiation machine. In some embodiments, an image file may include a plurality of image portions which collectively form a theme having an emotional impact on the patient. For example, an image file may include image portions to be projected to the floor, ceiling, and walls of the room120, collectively creating a background e.g. a calm lake surrounded by a scenery mountain and waterfalls under a blue sky, and image portions to be projected to the radiation system140creating an appearance or illusion of e.g. a boat. A library of various image files of different categories can be made available for a patient to select before treatment or imaging, and a particular selected image file may be mapped and projected to the radiation suite for the particular patient.

The image file may include still images such as still pictures, texts photographs, or moving images such as video clips, movies, and so forth. The image may contain pictorial images and/or texts providing information such as the patient's name, hospital name and logo etc. Depending on the nature of the projected images, sounds and/or smells may be added or simultaneously added with the projected images to enhance the patient's sensory experience or make the projected illusion more realistic. The sounds can be added by a separate audio system (not shown), which may be coupled to and controlled by the computer170, or an audio system built into the projector164or computer170. The smells can be introduced by a gaseous system (not shown) controlled by the computer170.

The mapping software171is operable to map an image file to a selected projection surface on the radiation system140and/or room120, which may have complex, three-dimensional contour or geometry. By using a mapping software, the projected image can be mapped or aligned or matched to the complex contour or geometry of the projection surface without or with reduced optical distortion. As will be described in greater detail below, the mapping software171may also alter an image file in response to movement of a patient walking in the room and/or a rotating gantry or moving couch, providing a dynamic interaction between the moving object and the projected image. The mapping software171may be loaded to the computer170via a computer readable medium such as a CD-ROM, flash drive, or other external memory medium, or downloaded through the Internet177.

The mapping software171run by the computer170can be a commercially available mapping software or custom created. Various projection mapping software are commercially available, including MadMapper® developed by GarageCube SA of Switzerland, TouchDesigner® developed by Derivative of Canada, VPT 7 developed by HC Gilje of Norway, and so forth. The mapping software171may be executed by the processor172via the user interface and monitor175. Briefly and in general, after a projection surface on the radiation system140and/or room120is determined and an image file chosen or created, the mapping software171can be run by the processor172via the user interface175to map the image file to the projection surface, or to align the projected image with various geometric features of the projection surface including the shape, size, contour, or position of the projection surface, allowing the projected image to properly match the geometry and contour of the projection surface without showing or with reduced visual or optical distortion. The mapping steps or procedures by the mapping software171may involve warping, masking, edging, constraining, splitting, or blending of the image or other tools such as rotating, scaling, skewing, positioning, and so forth, depending on the mapping software used. The coordinates of the radiation system140and the room120in relation to the projectors164may be provided to the mapping software171for mapping. Adjustments to the projectors164including their orientations can be made manually or by remote control during the mapping process. The properly mapped image file may be stored in the memory173of the computer170. The mapped image file may be fed to the projectors164and projected to the projection surface on the radiation system140and/or room120. Alternatively, the mapped image file may be saved in a memory device such as a CD, USB drive or the like, which can be inserted into a port in the projectors for projection.

The one or more projectors164, which are communicably connected with and controlled by the computer170, are operable to project a mapped image file to the selected projection surface on the radiation system140and/or the room120. Projectors are well-known in the art. Briefly and in general, the projectors164contain optical, mechanical, and electrical components or circuitry necessary to project an image file. In some embodiments of the disclosure, short throw or ultra-short throw projectors are used. Short throw or ultra-short throw projectors allow projection of big images in tight spaces. The term “throw” in projector terminology refers to the distance between the projector lens and the projection surface. A short throw projector has a small throw ratio, or a ratio of the throw to the width of the projection surface, generally less than 1. An ultra-short throw projector has a throw ratio generally less than 0.4. Various short throw and ultra-short throw projectors are commercially available. One example of the projectors164is LG PF1000U ultra-short throw projector manufactured by LG of Korea.

The location and number of the projectors164depend on the size and geometry of the selected projection surface on the radiation system140, and/or the projection surface on the floor122, the ceiling124, and the walls126of the room120. In situations where one projector cannot cover all of the selected projection surface on the radiation system140and/or the room120, several projectors may be used. For example, several projectors164may be mounted on the walls126, the ceiling124, and the floor122to map the entire surface of the radiation suite100including the radiation system140. In such situations, mapping steps or processes using the mapping software171can be performed for all of the projectors164, and the projection of image files by different projectors164can be coordinated.

Still referring toFIG.1, in some embodiments, the image projection system160may include a scanner162operable to measure the room120, the radiation system140, and other objects in the room. The measured information about the coordinates, geometry, or contour of the room120, the radiation system140or other objects in the room120can be used by the mapping software171in projection mapping an image file to the radiation system and/or the room. In situations where one scanner162cannot cover all of the selected projection surface on the radiation system140and/or the room120, several scanners may be used.

The scanner162may be communicably connected with the computer170wirelessly or via a connector cable and controlled by the computer170. The scanner162may scan the room120and send the scanned data to the computer170for processing. The computer170may include a suitable scanning software176processing the scanned data and reconstructing a spatial or three-dimensional configuration representative of the contour or geometry of the room120, the radiation system140or other object in the room120, to be used by the mapping software171in projection mapping an image file.

Various kinds of optical or laser scanners are known in the art and commercially available. One example of the scanner162is an infrared (IR) 3D scanner. Briefly and in general, an IR scanner may include an infrared light source, a scanning optics, and a photodetector or light sensor. A light beam emitted from the light source can be rotated or steered by the optics to optically scan the room and the radiation system and other objects in the room. The light beam reflected back from the various objects can be detected by the photodetector, which provides output signal data. The scanner162may collect a variety of data points with respect to the room120, the radiation system140and other objects in the room120, including distance information for each object in its surrounding environment, a grey scale value (i.e., a measure of the intensity of light) for each distance measurement value, and coordinates (e.g., x, y, and z) for each distance measurement value. The collected data are sent to the computer170and processed by a scanning software176to generate a three dimensional (3D) reconstructed image of the scanned environment with measurements.

In some embodiments, the scanner162and the scanning software176run in the computer170may detect a dynamic object in the room, and the mapping software171can compensate for movement of the dynamic object in projection mapping an image file. For example, the scanner162and the scanning software176may operate to track movement of a patient walking in the room or of the rotating gantry and/or moving couch, which may alter the projection surface in the radiation suite. In response, the mapping software171may alter the image file, providing a dynamic interaction between the moving object and the projected image. The image file may be altered by e.g. intermixing with another image file to create a composite image file.

By way of example, the scanner162and scanning software176run in the computer170may detect in real time a patient walking through an augmented water-filled floor122. In response, the mapping software171may alter the projected image or video by e.g. intermixing another image file or an overlay to create a composite image file. The projectors164project the composite image file, creating a projected image e.g. water moving around the patient's feet. Sounds of moving water may be added concurrently to enhance the patient's sensory experience or make the projected illusion more realistic. In another example, the scanner162and scanning software176run in the computer170may detect in real time the gantry rotating in an augmented star- and/or planet-filled space. In response to rotation of the gantry, the mapping software171may alter the image file or create a composite image file, projecting e.g. a shooting star passing the gantry. It should be noted that the above examples are provided for illustration. The type of feedback may be specific to the video and image contents.

FIG.2is a flowchart illustrating an exemplary method for projection mapping a radiation suite according to some embodiments of the disclosure. The method is described herein in conjunction withFIG.1which depicts a radiation system140in a radiation room120. It will be appreciated by one of ordinary skill in the art that the method can also be implemented in other medical facilities, such as a surgical room without a radiation system or a doctor's office, to provide a calming environment. According to the method, one or more scanners162, and one or more projectors164are mounted in a radiation room120(step202). The one or more projectors164may be mounted on the side walls126adjacent to an entrance to the radiation room120so that images are projected or displayed on the radiation system142generally in the viewpoint of the patient entering the room. The projectors164may also be mounted in other suitable locations such as on the ceiling124and/or the floor122of the room120. Other considerations for projectors' mounting locations include the existence of an optical path between the projectors and the surfaces onto which images are to be projected. The locations of the projectors may also depend on the contents of the images that have been produced. In some embodiments, the image file may be custom-produced after the projectors are mounted at selected locations. If desired, multiple projectors may be mounted to cover the entire radiation room.

At step204, one or more scanners162operate to scan the room120, including the floor122, ceiling124, walls126, the radiation system140and other objects in the room120. The scanning data is sent to a computer170, which runs a scanning software176processing the scanning data to determine the contour or geometry of the radiation room120and the radiation system140, and their coordinates in relation to the projectors164.

At step206, the computer170provides information about the geometry of the room120and the radiation system140to a projection mapping software171. The projection mapping software171functions to map an image file to a selected surface or the entire surface of the radiation system140and/or the room120by e.g. warping, masking, edging and/or other means provided by the mapping software until the projected image matches the geometry and contour of the projection surface (step208). One example of projection mapping software is MadMapper®. The mapped image file may be stored in the memory of the computer170. The mapped image file can be fed to the projectors164and projected to the selected surface or entire surface of the radiation system140and the room120(step210). Alternatively, the mapped image file may be saved in a memory device such as a CD, USB drive, or the like, which can be inserted into a port in the projectors for projection.

The entry of a patient may change the geometry or the projection surface of the radiation room (step212). The movement of the patient may be detected or tracked in real time by the scanner162and the scanning software176run in the computer170. The computer170provides information about the movement of the patient or the change of geometry of the radiation room caused by the patient's movement to the projection mapping software171(step214). In response to the change of geometry of the radiation room120, the mapping software171may alter the image file e.g. by intermixing another image file to create a composite image file (step216). The altered image file is projected to the radiation room120including the radiation system140(step210), creating a dynamic interaction between the patient and the augmented reality.

Referring toFIGS.3A and3B, an alternative embodiment of the disclosure will now be described.FIGS.3A and3Bshow a radiation system140, which may be located in a radiation suite100comprising an image projection system160as described above in connection withFIG.1. The radiation system140comprises a ring gantry142enclosing a radiation source (not shown) and a couch144supporting a patient148. The ring gantry142may include a bore146for receiving at least a portion of the patient148. In addition to the projectors164described above in connection with the image projection system160shown inFIG.1, an additional projector166may be mounted in the room120to project an image file to the surface168defining the bore146, allowing the patient148lying on the couch144to watch during treatment or imaging. The projector166may project an image file in the direction of the patient's viewpoint during treatment or imaging since this is the position the patient148remains in for the longest time.

The projector166may be mounted on the wall or floor, and operably connected with and controlled by the computer170. The image file may be mapped to the bore surface168, which may be curved, using the mapping software171run by the computer170as described above. The image file may be adapted for relaxing the patient148. By way of example, for pediatric patients, images of animals, birds, fish, or space etc. or a cartoon movie may be projected on the surface168to distract children from their treatment or imaging by the radiation machine.

Alternatively, the image file projected by the projector166may be adapted for visual gating learning and feedback during treatment or imaging. Therefore, according to embodiments of the disclosure, the projector166or the image projection system160may be operably connected with a gating system (not shown inFIGS.3A and3B) provided with the radiation system140. To irradiate a target or tumor that is not stationary, “gating” is normally utilized to block the radiation beam whenever the tumor is out of position. A gating system may include an imaging device such as an infrared camera continuously tracking one or more motion surrogates such as markers, implanted sensors or the like directly correlated to the motion and/or position of the tumor. A computer processes the signals from the surrogates to correlate the motion of the surrogates with the motion or position of the tumor in real time, and generate gating signals synchronized to the motion of the tumor to the control of the radiation system. According to embodiments of the disclosure, the projector166may project an image file to the bore surface168that prompts the patient148to maintain a desired physiological movement pattern. For example, a slider image or a graphical signal chart that simultaneously displays visual feedback of the physiological movement of the patient and a desired range or window of the movement can be projected. For respiration activity, the slider or the graphical signal chart may include a movable bar or other suitable symbol that moves in response to the patient's inhale-exhale movements. This provides visual prompting and feedback regarding the respiration activity. Verbal prompting may be employed in conjunction with the visual prompting to assist in controlling, maintaining, or manipulating the physiological activity of interest. Such verbal promptings can be computer-activated promptings to instruct a patient to breath in and breath out. U.S. Pat. No. 9,232,928 B2 entitled “Method and System for Predictive Physiological Gating” describes various embodiments of patient feedback and visual gating learning, the disclosure of all of which is incorporated herein by reference in its entirety.

Embodiments of projection mapping a radiation suite have been described. Those skilled in the art will appreciate that various other modifications may be made within the spirit and scope of the invention. All these or other variations and modifications are contemplated by the inventors and within the scope of the invention.