Patent Description:
For a given 3D image data set, image rendering techniques are used to produce a 2D image from a given viewpoint by making assumptions about the optical properties of tissue being imaged under a light source of a predefined color and intensity. Currently, image rendering techniques for ultrasound imaging systems rely on a directional light source located at a fixed distance or infinity. The incoming light direction may be presented to a user by an arrow on a trackball-controlled dedicated sphere widget. In addition to rotating the 3D volume, the user may change the direction of incoming light from the simulated light source.

<FIG> is a schematic illustration of an example of an existing image rendering technique <NUM>. A 3D data set <NUM> may have been acquired by an ultrasound probe or other imaging technique. The 3D data set <NUM> may include data corresponding to a 3D volume in a body. The 3D data set <NUM> may include a region of interest <NUM>. The region of interest <NUM> may be a portion of an object (e.g., wall of blood vessel, valve of heart) or may be an entire object (e.g., tumor, fetus). When rendering an image of the 3D data set <NUM> including the region of interest <NUM>, a simulated light source may be used to provide shadows and reflections on one or more surfaces within the 3D data set <NUM>, for example, a surface <NUM> of the region of interest <NUM>, which may provide depth perception for a user. The simulated light source may be a directional light source <NUM>. The directional light source <NUM> may transmit light only in a direction indicated by arrow <NUM>. The user may be permitted to select a position of the directional light source <NUM> at a fixed distance <NUM> from the 3D data set <NUM>. A 2D projection of the 3D data set <NUM> may be rendered relative to display image plane <NUM>, based on a virtual observer observing the 3D data set <NUM> from a viewpoint indicated by arrow <NUM>. Display image plane <NUM> may be aligned with the X-Y plane of the 3D data set <NUM>. Arrow <NUM> may be perpendicular to image plane <NUM>. That is, a virtual observer may be considered to be "looking" through the image plane <NUM> at the 3D data set <NUM> through the depth of the 3D data set <NUM> indicated by the Z-axis. The 2D projection at display image plane <NUM> of the 3D data set <NUM> may be provided as an image to a user on a display.

Although the user may move the directional light source <NUM> about the 3D data set <NUM>, locating the directional light source <NUM> outside of a rendered volume may cause object self-shadowing and make it difficult to illuminate structures of the region of interest <NUM>. Details of the volume and/or region of interest <NUM> may be obscured. Anatomic details inside concave cavities may not be visible without cropping of the 3D data set <NUM> or other significant adjustments.

<FIG> is an example of an image <NUM> rendered from a 3D data set using an external directional light source. The image <NUM> displays a fetus <NUM> within a uterus <NUM>. Many anatomical structures of the fetus <NUM> are obscured by shadows cast by the uterus <NUM> based on an image rendering technique using a directional light source located outside the uterus <NUM>. This may inhibit the user, which may be a sonographer, obstetrician, or other clinician, from making a diagnosis or being able to navigate within the volume defined by the 3D data set.

<CIT> discloses an ultrasonic diagnostic apparatus with a light source setting section, a rendering control section, and a display control section. The light source setting section, on the basis of the shape of a region of interest which is included in three-dimensional image data collected by an ultrasonic probe, sets a direction in which the region of interest is depicted. The rendering control section controls to generate the rendering image in which the region of interest is depicted in the direction set by the light source setting section. The display control section controls to display the rendering image on a display section.

<CIT> discloses a method, system and medical imaging device include accessing a 3D medical imaging dataset and generating a volume-rendered image from the 3D medical imaging dataset. Generating the volume-rendered image includes calculating a shading for the volume-rendered image based on a first light source, a second light source, and a third light source. The second light source and the third light source are both positioned differently than the first light source. The method, system, and medical imaging device also include displaying the volume-rendered image.

In one embodiment, an imaging system is defined according to claim <NUM>.

In another embodiment, a method is defined according to claim <NUM>, that may include receiving a selection of a simulated light source for rendering a 2D projection image of a 3D data set, wherein the 3D data set may be constructed from ultrasound echoes received from a volume of a subject, receiving an indication, responsive to user input, of an in-plane position of the simulated light source in a plane corresponding to a projection plane of the 2D projection image, automatically determining a depth position of the simulated light source on an axis normal to the projection plane, calculating surface shading information of a surface of.

The following description of certain exemplary embodiments is merely exemplary in nature and is in no way intended to limit the invention, which is defined only by the appended claims. or its applications or uses. In the following detailed description of embodiments of the present systems and methods, reference is made to the accompanying drawings which form a part hereof, and in which are shown by way of illustration specific embodiments in which the described systems and methods may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the presently disclosed systems and methods, and it is to be understood that other embodiments may be utilized and that structural and logical changes may be made without departing from the scope of the present system as defined by the appended claims. Moreover, for the purpose of clarity, detailed descriptions of certain features will not be discussed when they would be apparent to those with skill in the art so as not to obscure the description of the present system. The following detailed description is therefore not to be taken in a limiting sense, and the scope of the present system is defined only by the appended claims.

In some applications, it may be desirable to render an image from a 3D data set using a simulated light source positioned within the 3D data set. In some applications, it may be desirable to render an image from a 3D data set using a simulated light source within a region of interest within the 3D data set. In some applications, it may be desirable for the simulated light source to be a multidirectional light source. For example, the simulated light source may be modeled as a sphere that projects light from the entire surface of the sphere in all directions. In another example, the simulated light source may be modeled as a point source that projects light in all directions. Allowing a user to place the simulated light source within the 3D data set may provide rendered images that are less obscured by shadows and/or other artifacts that are generated when an image is rendered with a simulated directional light source located outside the 3D data set. Compared to lighting with an external light source, the close-range lighting may provide better local depth perception of shape and curvature of objects. An image rendered with a simulated light source within the 3D data set may provide an image that is easier for a clinician or other user to interpret. This may improve the ability of the clinician or other user to make a diagnosis and/or navigate within the 3D data set.

In an illustrative example, a clinician may conduct an ultrasound exam on a patient and acquire a 3D data set from the patient (e.g., a fetus in utero). The imaging system may render an image of a 2D projection of the 3D data set with a simulated multidirectional light source. The clinician may move the light source within the 3D data set, and the imaging system may adjust the rendered image based in part on the new position of the light source. For example, the clinician may touch a touch screen displaying the rendered image along with a visual cue for the light source (e.g., orb, square, X, etc.) and "drag" the light source to different positions within the image. The clinician may move the light source to investigate different areas of interest. Continuing with this example, the clinician may move the light source to highlight contours of the face of the fetus to check for a cleft pallet. The clinician may then move the light source to illuminate the spine to check for deformities. The clinician may choose to control the location of the light source in the image plane (e.g., an in-plane position, X-Y plane position) as well as the depth of the light source in the 3D data set (e.g., along an axis perpendicular to a plane of the in-plane position, along a Z-axis) or the imaging system may automatically set the depth position of the light source in the 3D data set. The clinician may control the light source during the ultrasound exam or during review of stored images after an exam.

<FIG> shows a block diagram of an ultrasound imaging system <NUM> constructed in accordance with the principles of the present disclosure. Although an ultrasound imaging system is shown in explanatory examples of embodiments of the invention, embodiments of the invention may be practiced with other medical imaging modalities. Other modalities may include, but are not limited to, magnetic resonance imaging and computed tomography. The ultrasound imaging system <NUM> in <FIG> includes an ultrasound probe <NUM> which includes a transducer array <NUM> for transmitting ultrasonic waves and receiving echo information. A variety of transducer arrays are well known in the art, e.g., linear arrays, convex arrays or phased arrays. The transducer array <NUM>, for example, can include a two dimensional array (as shown) of transducer elements capable of scanning in both elevation and azimuth dimensions for 2D and/or 3D imaging. The transducer array <NUM> is coupled to a microbeamformer <NUM> in the ultrasound probe <NUM> which controls transmission and reception of signals by the transducer elements in the array. In this example, the microbeamformer <NUM> is coupled by the probe cable to a transmit/receive (T/R) switch <NUM>, which switches between transmission and reception and protects the main beamformer <NUM> from high energy transmit signals. In some embodiments, for example in portable ultrasound systems, the T/R switch <NUM> and other elements in the system can be included in the ultrasound probe rather than in a separate ultrasound system base. The transmission of ultrasonic beams from the transducer array <NUM> under control of the microbeamformer <NUM> is directed by the transmit controller <NUM> coupled to the T/R switch <NUM> and the beamformer <NUM>, which receive input from the user's operation of the user interface or control panel <NUM>. The user interface <NUM> may include one or more input devices including one or more user interface elements (e.g., buttons, track ball, rotary encoder, or a soft control provided on a touch screen). In some embodiments, one or more of the user interface elements may include one or more graphical user interface (GUI) elements, which may be provided on a touch screen. One of the functions controlled by the transmit controller <NUM> is the direction in which beams are steered. The partially beamformed signals produced by the microbeamformer <NUM> are coupled to a main beamformer <NUM> where partially beamformed signals from individual patches of transducer elements are combined into a fully beamformed signal.

The beamformed signals are coupled to a signal processor <NUM>. The signal processor <NUM> can process the received echo signals in various ways, such as bandpass filtering, decimation, I and Q component separation, and harmonic signal separation. The signal processor <NUM> may also perform additional signal enhancement such as speckle reduction, signal compounding, and noise elimination. The processed signals are coupled to a B-mode processor <NUM>, which can employ amplitude detection for the imaging of structures in the body. The signals produced by the B-mode processor <NUM> are coupled to a scan converter <NUM> and a multiplanar reformatter <NUM>. The scan converter <NUM> arranges the echo signals in the spatial relationship from which they were received in a desired image format. For instance, the scan converter <NUM> may arrange the echo signal into a two dimensional (2D) sector-shaped format, or a pyramidal three dimensional (3D) image. In some embodiments, the scan converter <NUM> may generate a 3D data set from the echo signal. The multiplanar reformatter <NUM> can convert echoes which are received from points in a common plane in a volumetric region of the body into an ultrasonic image of that plane, as described in <CIT>). A volume renderer <NUM> converts the echo signals of a 3D data set into a projected 3D image as viewed from a given reference point, e.g., as described in <CIT>). In some embodiments, the volume renderer <NUM> may receive input from the user interface <NUM>. The input may include the given reference point (e.g., viewpoint of a virtual observer), position of a simulated light source, and/or properties of the simulated light source for the rendered projected image. In some embodiments, the volume renderer <NUM> may determine an in-plane and/or depth position of the simulated light source automatically. In some embodiments, the volume renderer <NUM> may calculate surface shading information for one or more surfaces in the 3D data set based at least in part, on the position and/or properties of the simulated light source. The 2D or 3D images are coupled from the scan converter <NUM>, multiplanar reformatter <NUM>, and volume renderer <NUM> to an image processor <NUM> for further enhancement, buffering and temporary storage for display on an image display <NUM>. The image processor <NUM> may render visual cues for the simulated light source (e.g., orb, halo) in some embodiments. In some embodiments, the visual cues may be rendered by the volume renderer <NUM>. The graphics processor <NUM> can generate graphic overlays for display with the ultrasound images. These graphic overlays can contain, e.g., standard identifying information such as patient name, date and time of the image, imaging parameters, and the like. For these purposes the graphics processor receives input from the user interface <NUM>, such as a typed patient name. The user interface can also be coupled to the multiplanar reformatter <NUM> for selection and control of a display of multiple multiplanar reformatted (MPR) images.

According to an embodiment of the disclosure, the ultrasound probe <NUM> may be configured to receive ultrasound echoes from a subject to image a volume of the subject. The scan converter <NUM> may receive the ultrasound echoes and generate a 3D data set. As described above, the ultrasound echoes may be pre-processed by the beamformer <NUM>, signal processor <NUM>, and/or B-mode processor prior to being received by the scan converter <NUM>. The 3D data set may include values for each point (e.g., voxel) in the imaged volume. The values may correspond to echo intensity, tissue density, flow rate, and/or material composition. Based on the values in the 3D data set, the scan converter <NUM> and/or volume renderer <NUM> may define one or more surfaces within the imaged volume. The surfaces may represent a boundary between two different objects (e.g., fetus and uterus) or materials (e.g., bone and muscle), or regions (e.g., different flow rates in a vessel) within the imaged volume. In some embodiments, the surfaces may be isosurfaces.

When rendering a 2D projection image of the 3D data set, the volume renderer <NUM> may receive a location of a simulated light source relative to the 3D data set. In some embodiments, the location of the simulated light source may be pre-programmed by the imaging system <NUM>. The simulated light source may default to a pre-programmed location, e.g., upon activation of a volume rendering mode, and in some cases the light source may be movable by the user while in the volume rendering mode. In some embodiments, the location of the simulated light source may be received via user interface <NUM>, which may include input devices having one or more input elements configured to receive user input. For example, the user interface <NUM> may include a touch screen with a graphical user interface (GUI) that allows a user to set a location of the simulated light source anywhere within and/or proximate to the 3D data set. As an example, the graphical user interface (GUI) may provide one or more GUI elements that enable the user to set the location of the simulated light source. In some examples, a GUI element (e.g., a light orb) may additionally provide a visual cue as to the location of the light source in relation to the volume. In other examples, the GUI element may be an input widget whereby the user may be able to specify the location (e.g., specify X, Y, Z coordinates) of the light source. Other examples of GUI elements may be used. In yet further examples, the user input may be received via a mechanical control (e.g., a trackball or a rotary encoder on a control panel) which in the volume rendering mode may be specifically associated with and configured to generate manipulation commands for moving the light source. In some embodiments, only the in-plane position (e.g., X and Y coordinates) may be received via the user interface <NUM>, and the volume renderer <NUM> may automatically determine a depth position (e.g., Z coordinate) of the simulated light source. The depth position may be determined based, at least in part, on a pre-set distance from a surface in the 3D data set. The pre-set distance may be pre-programed and/or user configurable. For example, the pre-set distance may be stored in memory and the volume renderer may be programmed to use the pre-set distance as a default value when determining the depth dimension default unless the default value is modified by a user. In some embodiments, the user interface may provide a user interface element configured to receive user input for specifying the pre-set distance.

The volume renderer <NUM> may calculate surface shading information for one or more surfaces within the 3D data set, based, at least in part, on the location of the simulated light source relative to the 3D data set. The surface shading information may include information regarding the brightness of any given pixel representing a surface of the 3D dataset in a rendered 2D projection image, which information may provide three-dimensionality to the otherwise 2D rendered image. In addition to the location of the light source relative to the surface, the surface shading information may be based on properties of the volume adjacent to the surface (e.g., the value of voxels interposed between the light source and the surface). For example, when calculating the shading information for a given surface, the volume renderer <NUM> may take into account the density of tissue interposed between the simulated light source and the rendered outer surface. When the simulated light source is located in front of a surface of the imaged volume, only zero-value voxels may be interposed between the light source and the surface and an illuminated region on the surface may have a high luminosity or brightness than in instances in which the simulated light source is behind the surface and thus spaced from the surface by non-zero value voxels. Light transmittance through the zero-value voxels of the regions surrounding the rendered 3D dataset may be approximated, by known light simulation techniques, to be similar to light transmittance through air, thus light transmittance through non-zero value voxels may be reduced to approximate transmittance through tissue which is denser than air. Thus, when the simulated light source is located behind a surface enclosing a volume of the 3D data set having a density higher than a surrounding volume, the surface shading information calculated by the volume renderer <NUM> may be different than when the simulated light source is located in front of the surface. For example, the surface shading information may include fewer reflections and appear to "glow" from within when the simulated light source is located behind the surface while the surface shading information may be such that the surface appears more opaque when the simulated light source is located in front of the surface. As will be appreciated, density and other properties of an object positioned in front of a light source will affect the light transmittance through the object, thus the volume renderer <NUM> is configured to account for the density of material disposed between the light source and the surface being rendered.

Although reference is made to surface shading, the volume renderer <NUM> may or may not explicitly extract surfaces from the 3D dataset for calculating surface shading information. For example, the volume renderer <NUM> may calculate shading information for every voxel within the 3D dataset (e.g., volumetric shading). As previously mentioned, the shading information for each voxel may be based at least in part on the distance of the voxel from the simulated light source, the density of the voxel, and/or density of surrounding voxels. The resulting shading information for the 3D dataset may provide the appearance of 3D surfaces within the 3D dataset to a user. For simplicity, the shading information of surfaces of objects and/or areas of interest within the 3D dataset will be referred to as surface shading information without regard to the manner in which it is calculated by the volume renderer <NUM>.

The surface shading information may be used by the volume renderer <NUM> to render the 2D projection image. The rendered 2D projection image may be provided by the volume renderer <NUM> to the image processor <NUM> in some embodiments. The rendered 2D projection image may be provided to the display <NUM> for viewing by a user such as a clinician. In some examples, the rendering by the volume renderer <NUM> and the resulting 2D projection image provided on the display <NUM> may be updated responsive to user inputs via the user interface <NUM>, for example to indicate movement (e.g., translation or rotation) of the volume, movement of the simulated light source in relation to the volume, and/or other changes to parameters associated with the various rendering constructs in the rendering. For example, the volume renderer is configured, responsive to movement of the simulated light source via the user input, to automatically render the simulated light source at a location corresponding to the in-plane position and a depth position determined by the volume renderer. In some embodiments, the depth position is set based at least in part on contours of the first surface.

<FIG> is a schematic illustration of an image rendering technique <NUM> according to an embodiment of the disclosure. In some embodiments, the image rendering technique <NUM> may be performed by an imaging system such as ultrasound imaging system <NUM>. A 3D data set <NUM> may have been acquired by an ultrasound probe, such as ultrasound probe <NUM> shown in <FIG>. In other examples, the 3D dataset <NUM> may have been acquiring using a different medical imaging modality (e.g., CT, MRI, etc.). The 3D data set <NUM> may include data corresponding to a 3D volume in a body. The 3D data set <NUM> may include a region of interest <NUM>. The region of interest <NUM> may be a portion of an object (e.g., wall of blood vessel, valve of heart) or may be an entire object (e.g., tumor, fetus). In some embodiments, the 3D data set <NUM> may include multiple regions of interest <NUM>. A 2D projection image of the 3D data set <NUM> may be rendered relative to display image plane <NUM>, based on a virtual observer observing the 3D data set <NUM> from a viewpoint indicated by arrow <NUM>. Display image plane <NUM> may be aligned with an X-Y plane. The vector indicated by arrow <NUM> may pass through image plane <NUM>. That is, a virtual observer may be considered to be "looking" through the image plane <NUM> at the 3D data set <NUM> through the depth of the 3D data set <NUM> indicated by the Z-axis, which is orthogonal to the X-Y plane. Although shown perpendicular to image plane <NUM>, arrow <NUM> may be at some other angle relative to image plane <NUM> (e.g., <NUM>, <NUM>, <NUM> degrees). The 2D projection image at display image plane <NUM> of the 3D data set <NUM> may be provided as an image to a user on a display, such as display <NUM> shown in <FIG>.

When rendering an image of the 3D data set <NUM> including the region of interest <NUM>, a simulated light source <NUM> may be used to calculate surface shading information to render shadows and reflections on one or more surfaces within the 3D data set <NUM>, for example, a surface <NUM> of the region of interest <NUM>, which may provide depth perception for a user. The surface shading information may be based, at least in part, on the position of the simulated light source <NUM> relative to the 3D data set <NUM> and/or region of interest <NUM>. In some embodiments, the simulated light source <NUM> may be a multidirectional light source. The light source <NUM> may transmit light in all directions as indicated by arrows <NUM>. Unlike the light source <NUM> shown in <FIG>, the user may be permitted to select a position of the light source <NUM> outside of or anywhere within the 3D data set <NUM>. As shown in the embodiment illustrated in <FIG>, the light source <NUM> is within the 3D data set <NUM> at a depth less than a depth of the region of interest <NUM>. That is, the light source <NUM> is at a depth along the Z-axis between the region of interest <NUM> and the virtual observer looking from a direction indicated by arrow <NUM>. In some embodiments, the user may select a position of the simulated light source <NUM> in the image plane <NUM> and the imaging system may automatically determine a depth position of the simulated light source <NUM>.

<FIG> is an example image <NUM> rendered using the image rendering technique <NUM> shown in <FIG>. The image <NUM> is rendered from the same 3D data set as image <NUM> shown in <FIG>, a fetus <NUM> within a uterus <NUM>. In some embodiments, the simulated light source may be rendered as an emissive material in the image. In the example shown in image <NUM>, the simulated light source is rendered as a glowing orb <NUM>. The glowing orb <NUM> is rendered within the 3D data set within the uterus <NUM>. As a result, the uterus <NUM> does not cast shadows that obscure the fetus <NUM>. In contrast with the fetus <NUM> in <FIG>, the left arm, right shoulder, and torso of fetus <NUM> may be discerned. These same features are obscured by uterine shadows in the image <NUM> shown in <FIG>.

As mentioned previously, the light source <NUM> is not limited to a set distance from the 3D data set <NUM>. <FIG> is a schematic illustration of a variety of example possible positions of the light source 405a-e according to embodiments of the disclosure. As shown in <FIG>, the light source <NUM> may be rendered at varying positions in the image plane <NUM> (e.g., different positions on the X-Y plane) and at different depths within the 3D data set <NUM> (e.g., along the Z-axis). For example, the light source 405a is in the position shown in <FIG>, and light source 405b is at the same depth as light source 405a, but at a different point in image plane <NUM> in the 3D data set <NUM>. Positioning the light source <NUM> in front of the region of interest <NUM> may allow a user to discern features on the surface <NUM> of the region of interest <NUM> and/or surrounding area. Light source 405c is at both a different point on the image plane <NUM> and at a different depth in the 3D data set <NUM>. As shown in <FIG>, light source 405c is at a deeper depth than the region of interest <NUM> with reference to the image plane <NUM>. Positioning the light source <NUM> behind the region of interest <NUM> may allow the user to make at least a qualitative determination of the thickness and/or density of the region of interest <NUM>. The light source <NUM> may even be placed within the region of interest <NUM>, as shown by light source 405d. Positioning the light source <NUM> within the region of interest <NUM> may allow the user to observe more subtle contours and depths of different components within the region of interest <NUM>. The position of the light source <NUM> is not limited to the 3D data set <NUM>. Light source 405e shows an example of a position outside the 3D data set <NUM>. The example positions are shown for explanatory purposes only, and the light source <NUM> is not limited to the positions shown in <FIG>. There may be alternative and/or additional advantages to different positions of the light source <NUM> than those described above. For example, the user may position the light source <NUM> to avoid casting shadows from other anatomy and/or portions of the region of interest <NUM>.

Although not shown in <FIG>, the simulated light source <NUM> may be a directional light source rather than a multidirectional light source. In some embodiments, a user may be able to toggle between multidirectional and directional modes. A directional light source within the 3D data set <NUM> may be desirable in some applications. For example, a user may want to highlight a particular area within the 3D data set while minimizing the illumination to other areas, which may reduce distractions (e.g., a "spotlight" effect).

<FIG> is an illustration of a portion of an ultrasound system <NUM> that may be used to implement an embodiment of the disclosure. The ultrasound system <NUM> may include a user interface <NUM> and a display <NUM>. In some embodiments, user interface <NUM> may be used to implement user interface <NUM> shown in <FIG>. The display <NUM> may be used to implement display <NUM> shown in <FIG> in some embodiments. The user interface <NUM> may include one or more input devices including one or more user interface elements. For example, user interface <NUM> may include a touch screen <NUM>, one or more rotary controls <NUM>, a track ball <NUM>, and buttons <NUM>. In some embodiments, the buttons <NUM> may include arrow keys and/or a QWERTY keyboard. In some embodiments, the display <NUM> may also be part of the user interface <NUM>. For example, the display <NUM> may be implemented using a touch screen. A user may have the option of using the display <NUM>, the touch screen <NUM>, and/or other controls included in the user interface <NUM> to position the simulated light source in a rendered image and/or control other properties of the simulated light source (e.g., directional vs. multidirectional, intensity, color). In yet further examples, the input device may include a touchless interface configured to receive user inputs without the user physically contacting the touch screen or mechanical controls of the system <NUM>.

A user may control the position of the simulated light source in a rendered image via a user interface such as the user interface <NUM> shown in <FIG>. In some embodiments, the user may use the track <NUM> ball and the rotary control <NUM>. The user may select an in-plane position (e.g., an X-Y coordinate) on the image plane with the track ball <NUM> and select a depth position (e.g., a coordinate on the Z-axis) with the rotary control <NUM> to set the position of the simulated light source. In some embodiments, an individual rotary control may be provided for each degree of freedom (e.g., an X-axis control, a Y-axis control, and a Z-axis control) to set the position of the simulated light source. In some embodiments, the user may use buttons <NUM>, such as arrow keys, to select a position (e.g., X-Y-Z coordinate) of the simulated light source. In some embodiments, the user may select an in-plane position of the simulated light source and the imaging system may automatically determine a depth position of the simulated light source for the selected in-plane position.

In some embodiments, the user interface <NUM> or an input element of the user interface includes a graphical user interface (GUI). For example, the display <NUM> and/or touch screen <NUM> may include a GUI. In some embodiments, the user may use the touch screen <NUM> to position the simulated light source. A variety of gestures on the touch screen <NUM> may be used to select a position of the simulated light source. For example, the user may tap the touch screen <NUM> at a location to set the in-plane position and/or touch a rendered light orb in the image displayed on the touch screen <NUM> and "drag" it to an in-plane position by moving their finger along the touch screen <NUM>. Each point on the touch screen <NUM> may coincide with each point of the image plane of a rendered 2D projection image. These gestures are provided only as examples, and other gestures may be used to set the position of the simulated light source in the 3D data set (e.g., control buttons provided on touch screen). In some embodiments, a user may position the simulated light source using one or a combination of user input methods. For example, a user may set a position of the simulated light source using the touch screen and then "fine tune" the position using the track ball and/or rotary control. In some embodiments, the user interface <NUM> may include additional and/or alternative user input controls (e.g., slide control, motion sensor, stylus) for positioning the simulated light source. In some embodiments, the user may use the user interface <NUM> to control properties of the simulated light source. For example, a user may set an intensity and/or color of the light source.

<FIG> is an illustration of a rendered image <NUM> on a display <NUM> according to an embodiment of the disclosure. Display <NUM> of <FIG> or display <NUM> of <FIG> may be used to implement display <NUM> in some embodiments. In some embodiments, the display <NUM> may include a GUI and the simulated light source <NUM> may be rendered with visual cues to assist a user in interpreting the position of the light source in the 3D data set. As shown in <FIG>, the simulated light source <NUM> may be rendered in the image <NUM> as smaller in size as the light source is positioned farther away from the image plane in the 3D data set. In some embodiments, the image plane aligns with the display <NUM>. As shown in <FIG>, the light source <NUM> would appear to be moving further into the page. In this example, light source 815a is closest to the image plane and light source 815c is furthest from the image plane. Changing the size of the light source <NUM> in the image <NUM> may provide a visual cue indicating a depth of the light source <NUM> along the Z-axis in the 3D data set and may assist a user in interpreting the position of the light source within the 3D data set.

<FIG> is an illustration of a rendered image <NUM> on a display <NUM> according to an embodiment of the disclosure. Display <NUM> of <FIG> or display <NUM> of <FIG> may be used to implement display <NUM> in some embodiments. In some embodiments, the display <NUM> may include a GUI and the simulated light source <NUM> may be rendered in the image <NUM> with a halo <NUM>. The halo <NUM> may allow a user to visually locate the light source <NUM> in the image <NUM>. In some embodiments, the halo <NUM> may allow the user to locate the light source <NUM> when the light source <NUM> is positioned outside the field of view of the image <NUM>. In some embodiments, a user may activate or deactivate the halo <NUM>. That is, the user may control whether or not the halo <NUM> is rendered around the light source <NUM> in the image <NUM>. In some embodiments, the halo <NUM> may automatically disappear after the light source <NUM> has been stationary for a period of time (e.g., half a second, two seconds, ten seconds). In some embodiments, the user may deactivate the visual cue of the light source <NUM>. By deactivate, it is not meant that the user chooses to remove the lighting rendered from the light source <NUM> from the image <NUM>, but that the user turns off the rendering of the visual cue of the light source <NUM> in the image <NUM> (e.g., the orb). In some embodiments, the rendering of the visual cue of the light source <NUM> may automatically disappear after the light source <NUM> has been stationary for a period of time (e.g., half a second, two seconds, ten seconds). Activating and deactivating the halo <NUM> and/or rendering of the light source <NUM> may allow for the user to observe the image <NUM> without interference from the visual cues for positioning the light source <NUM>. Visual cues such as the orb and/or halo may be rendered by a volume renderer and/or image processor of an imaging system. For example, volume renderer <NUM> and image processor <NUM> of ultrasound imaging system <NUM> shown in <FIG> may be used to implement an embodiment of the disclosure.

A simulated light source that may be placed anywhere within and/or surrounding a 3D data set may provide additional illumination options for images rendered from the 3D data set. The simulated light source may be a multidirectional light source in some embodiments. These additional options may allow for rendering of images that are less prone to self-shadowing by other anatomical features and better definition of surfaces and/or thicknesses of tissues. However, in some applications, a user may not want to select an in-plane position and/or depth position of the simulated light source. The user may find navigating through the entire 3D data set to select a depth position time consuming and/or disorienting. In some embodiments, a user may choose an option that positions the simulated light source to a set distance from a region of interest and/or surface of the region of interest. That is, as a user moves the simulated light source through the image plane, the depth position of the simulated light source may automatically adjust based on contours of the surface of the region of interest such that a distance between the simulated light source and the region of interest and/or surface is maintained. For example, as a user moves the light source along an image of a spine, the light source may appear to "float" over the vertebrae, following the contours of the spine, remaining a set distance away from the spine. This automatic depth selection mode may be preferable when a user is conducting a cursory review of images and/or the user is less experienced with imaging systems.

<FIG> is a schematic illustration of an image rendering technique <NUM> according to an embodiment of the disclosure. In some embodiments, image rendering technique <NUM> may be an embodiment of image rendering technique <NUM> wherein a depth position of a simulated light source is automatically determined by an imaging system. Image rendering technique <NUM> may be performed by an imaging system such as ultrasound imaging system <NUM> shown in <FIG>. A 3D data set <NUM> may have been acquired by an ultrasound probe, such as ultrasound probe <NUM> shown in <FIG>, or other input device. The 3D data set <NUM> may include data corresponding to a 3D volume in a body of a subject. The 3D data set <NUM> may include a region of interest <NUM>. The region of interest <NUM> may be a portion of an object (e.g., wall of blood vessel, valve of heart) or may be an entire object (e.g., tumor, fetus). A 2D projection of the 3D data set <NUM> may be rendered relative to display image plane <NUM>, based on a virtual observer observing the 3D data set <NUM> from a viewpoint indicated by arrow <NUM>. Display image plane <NUM> may be aligned with the X-Y plane. The vector indicated by arrow <NUM> may pass through image plane <NUM>. That is, a virtual observer may be considered to be "looking" through the image plane <NUM> at the 3D data set <NUM> through the depth of the 3D data set <NUM> indicated by the Z-axis. Although shown perpendicular to image plane <NUM>, arrow <NUM> may be at some other angle relative to image plane <NUM> (e.g., <NUM>, <NUM>, <NUM> degrees). The 2D projection at display image plane <NUM> of the 3D data set <NUM> may be provided as an image to a user on a display, such as display <NUM> shown in <FIG> or display <NUM> shown in <FIG>.

In some embodiments, for a given position of a simulated light source <NUM> in the display image plane <NUM> (e.g., an X-Y coordinate), a ray <NUM> may be cast into the 3D data set <NUM> along arrow <NUM>. In some embodiments, arrow <NUM> may be along an axis orthogonal to the image plane <NUM> (e.g., along a Z-axis). The ray may be cast into the 3D data set <NUM> until it finds a non-zero density point (e.g., voxel) that may be a portion of an imaged object in the 3D data set <NUM> (e.g., a surface of an anatomical feature), such as a surface <NUM> of the region of interest <NUM>. In some embodiments, the closest non-zero density point may be found by interpolation. A distance <NUM> along the ray <NUM> back towards the virtual observer away from the non-zero density point may be calculated. The imaging system may then position the simulated light <NUM> source at a depth position that is distance <NUM> from the surface <NUM> of the region of interest <NUM>. The distance <NUM> may be calculated by an image processor and/or a volume renderer in some embodiments. For example, image processor <NUM> and/or volume renderer <NUM> of ultrasound imaging system <NUM> shown in <FIG> may be used to implement an embodiment of the disclosure. In some embodiments, another processor may be used to calculate the distance <NUM>.

The distance <NUM> may be pre-programmed or it may be set by the user. The distance <NUM> may range from the equivalent of <NUM>-<NUM> millimeters in a volume from which the 3D data set <NUM> was acquired. Larger or smaller distances may be used for the distance <NUM> of the light source <NUM> from the object, based in part on the application. For example, larger distances between the light source and the object may be used when viewing an entire fetus and smaller distances may be used when viewing a heart valve. In some embodiments, the distance <NUM> of the light source <NUM> from the surface <NUM> of the region of interest <NUM> may be based, at least in part, on a quality criterion. For example, a distance <NUM> is selected that minimizes the amount of shadowing in the vicinity of the target of the ray <NUM> on the surface <NUM> of the region of interest <NUM>. In another quality metric example, a distance <NUM> is selected that maximizes a dynamic range of lighting intensity in the 2D projection image.

The distance <NUM> between the light source <NUM> and region of interest <NUM> is maintained as the in-plane position of the light source <NUM> is changed in the image plane <NUM>. That is, the depth position of the light source <NUM> may automatically be adjusted. The imaging system may automatically "scan" the light source <NUM> along the surface of the region of interest <NUM> and/or a user may control the position of the light source <NUM> in the image plane <NUM> via a user interface (e.g., "drag" an orb rendered in the image via a touch screen, tap a desired location on a touch screen for the light source, manipulate a track ball, etc.). In some embodiments, the distance <NUM> may be maintained while the region of interest <NUM> is rotated relative to the image plane <NUM> and/or the region of interest <NUM> moves. For example, the region of interest <NUM> may move over time when the 3D data set <NUM> includes multiple 3D data sets corresponding to different periods of time (e.g., four dimensional image, real time imaging, time elapsed loop). In another example, the light source <NUM> may remain stationary relative to a user and/or virtual observer while the 3D data set <NUM> is rotated relative to the user and/or virtual observer.

In some embodiments, a volume renderer and/or image processor may determine the depth position of the simulated light source <NUM>. In some embodiments, the volume renderer and/or image processor may determine the shading information for the surface <NUM> for rendering a 2D projection image at image plane <NUM>. In some embodiments, the volume renderer and/or image processor dynamically determines the depth position of the simulated light source <NUM>, shading information, and render the 2D projection image as the in-plane position of the simulated light source <NUM> is altered.

<FIG> are schematic illustrations of a user positioning a simulated light source according to an embodiment of the disclosure. In some embodiments, the 3D data set may include multiple regions of interest (e.g., lesions along a vessel wall) and/or objects (e.g., vertebrae, interventional devices). A user moves a simulated light source to different positions in order to illuminate each of the regions of interest. When the image rendering technique <NUM> is employed, a set distance between each region of interest and the simulated light source may be maintained automatically. As shown in <FIG>, a user <NUM> may position a simulated light source <NUM> within a 3D data set <NUM> within a plane <NUM>. In some embodiments, plane <NUM> may correspond to an image plane of a 2D projection image. <FIG> shows a user positioning the simulated light source <NUM> in the plane <NUM> by placing a finger on a touch screen displaying a projected image rendered from the 3D data set <NUM>. However, other methods of positioning the light source <NUM> may be used such as those discussed in reference to <FIG>. A depth position of the light source <NUM> within the 3D data set <NUM> may be automatically determined based on a distance from a surface <NUM> of a first region of interest <NUM>. The distance may be calculated according to one of or a combination of methods described above in reference to <FIG>. When the user "drags" the light source <NUM> to a new position in the plane <NUM> and/or selects a new location in the plane <NUM> on or near region of interest <NUM>, the light source <NUM> may move along the path <NUM>, which defines a set distance from the surface <NUM> of the region of interest <NUM>.

As shown in <FIG>, when the user positions the light source <NUM> in the plane <NUM> on or near a second region of interest <NUM>, the depth position of the light source <NUM> may automatically adjust so that the light source <NUM> is the same distance from a surface <NUM> of the second region of interest <NUM> that it was from the surface <NUM> of the first region of interest <NUM>. In some embodiments, the set distance from the surface <NUM> of the region of interest <NUM> may be different from the set distance from the surface <NUM> of the region of interest <NUM>. For example, if the distance between the simulated light source <NUM> is determined based, at least in part, on a quality metric, the value of the quality metric calculated for region of interest <NUM> may be different than the value of the quality metric calculated for region of interest <NUM>. When the user "drags" the light source <NUM> to a new position in the plane <NUM> and/or selects a new location in the plane <NUM> on or near the region of interest <NUM>, the light source <NUM> may move along the path <NUM>, which defines a set distance from the surface <NUM> of the region of interest <NUM>. The technique <NUM> illustrated in <FIG> and <FIG> may allow a user to illuminate multiple areas of an image in succession without having to manually adjust the depth of the light source <NUM>. In some applications, this may save the user time as the user only has to choose an in-plane position for the light source <NUM>.

Features described with reference to image rendering technique <NUM> in <FIG> may be applied to the image rendering technique <NUM> in <FIG> and <FIG>. For example, the light source <NUM>, <NUM> may be rendered in the image as an emissive object (e.g., glowing orb). The rendering of the light source <NUM>, <NUM> in the image may be activated and/or deactivated automatically and/or by user selection. The light source <NUM>, <NUM> may be rendered with a halo, which may be activated and/or deactivated automatically and/or by user selection. In some embodiments, a user may toggle between the image rendering technique <NUM> and image rendering technique <NUM>. For example, a clinician may survey a rendered image using image rendering technique <NUM> and sweep the light source across several areas of interest in succession. When the clinician spots an area of particular interest (e.g., lesion), the clinician may switch to image rendering technique <NUM> to "fine tune" the position of the light source to examine the area in greater detail.

<FIG> is a flowchart of a method <NUM> for positioning a simulated light source within a 3D data set for rendering 2D projections from a perspective of a virtual observer of the 3D data set according to an embodiment of the disclosure. In some embodiments, method <NUM> may be implemented using the image rendering technique <NUM> illustrated in <FIG> and the ultrasound imaging system shown in <FIG>. In some embodiments, a user may select a position of a simulated light source in a 3D data set prior to rendering of a 2D projection image of the 3D data set. In some embodiments, an imaging system may render a 2D projection image from a 3D data set with an initial default light source in a default position. The default light source and position may be pre-programmed into the imaging system and/or may be set by a user. In some embodiments, the default light source may be an external directional light source at fixed distance from the data set. In some embodiments, the default light source may be a multidirectional light source positioned within or near the 3D data set. At Step <NUM>, an imaging system may receive a selection of a simulated light source for rendering a 2D projection image of a 3D data set. In some embodiments, a user may select a simulated light source. The user may select the light source via a user interface such as user interface <NUM> in <FIG> or user interface <NUM> in <FIG>. In some embodiments, the user may navigate through a user interface to enter a lighting control mode of the imaging system. In some embodiments, the user may tap a button or a touch screen to select the light source. Optionally, the user and/or imaging system may activate a visual cue of the light source at Step <NUM>. That is, the user may choose to have the light source rendered in the image as an object (e.g., an orb). In some embodiments, the light source may be rendered in the image by default. Optionally, the user and/or imaging system may activate a halo around the light source at Step <NUM>. In some embodiments, the light source may be rendered with a halo by default. In some embodiments, the user may prefer to render the image without the halo.

At Step <NUM>, the imaging system may receive an indication, responsive to user input, of an in-plane position of the simulated light source in a plane corresponding to a projection plane of the 2D projection image (e.g., image plane <NUM> of <FIG>). The user may select an in-plane position for the light source. The in-plane position may correspond to a position in the image plane in some embodiments. At Step <NUM>, a depth position of the simulated light source on an axis normal to the projection plane (e.g., Z-axis) may be automatically determined by the imaging system. In some embodiments, the depth position may be based on a set distance between the simulated light source and a surface in the region of interest. The depth position may correspond to the depth within the 3D data set in relation to the image plane. In some embodiments, Step <NUM> and Step <NUM> may be performed in reverse order. In some embodiments, Step <NUM> and <NUM> may be performed simultaneously. The user may select the in-plane position and depth position by using a track ball, a touch screen, and/or another method and/or user interface such as those described above in reference to <FIG>. The imaging system may then calculate surface shading information for one or more surfaces in the 3D data set based on the in-plane and depth positions at Step <NUM>. At Step <NUM>, the imaging system may render the 2D projection image including the shading information on a display. In some embodiments, the imaging system may re-render the image as the in-plane position of the light source is moved by the user. That is, the light and shadows of the image may dynamically change as the position of the light source is altered (e.g., the depth position and surface shading information may be recalculated). This may allow the user to quickly compare potential positions of the light source and/or investigate features of the image by illuminating portions of the image in sequence. For example, the user may move the light source along a spinal column to examine each vertebra.

Once the light source is in position, the halo, if rendered, may be deactivated at Step <NUM>. In some embodiments, the user may choose to deactivate it (e.g., via a user interface). In some embodiments, the imaging system may automatically stop rendering the halo when the light source is stationary for a period of time. Alternatively, the halo may continue to be rendered. This may be desirable when the user has chosen a position for the light source that is outside the field of view. Optionally, at Step <NUM>, the visual cue for the light source may be deactivated. That is, the object rendered as the light source in the image may be removed from the image. The imaging system may deactivate the visual cue for the light source automatically or the user may choose to deactivate the visual cue for the light source. Deactivating the visual cue for the light source may be advantageous when the user wishes to observe minute features illuminated in the image near the light source.

Method <NUM> may be performed during image acquisition in some embodiments. For example, the imaging system may render images from a 3D data set acquired from a matrix array ultrasound transducer during an ultrasound exam. Method <NUM> may be performed on a 3D data set stored on an imaging system or other computing device (e.g., computer, hospital mainframe, cloud service). For example, a radiologist may review images rendered from a 3D data set acquired during a prior exam.

Although method <NUM> is described with reference to a single light source, all or portions of method <NUM> may be performed and/or repeated for multiple light sources. For example, a user may set a first light source at a first region of interest and a second light source at a second region of interest. This may allow the user to quickly highlight features of the 3D data set.

<FIG> are examples of rendered spinal column images 1300a-c according to an embodiment of the disclosure. <FIG> shows spinal column image 1300a with a simulated light source <NUM> rendered as a glowing orb. Spinal column image 1300a may have been rendered with the simulated light source <NUM> in a default position. After spinal column image 1300a has been rendered, a user and/or an imaging system may adjust a position of the simulated light source <NUM>. <FIG> shows spinal column image 1300b with simulated light source <NUM> over a vertebra <NUM>. The simulated light source <NUM> may be a set distance from a surface of the vertebra <NUM>. The user may have adjusted the in-plane position of the simulated light source <NUM> from the default position shown in image 1300a to the current position over the vertebra <NUM> shown in image 1300b. The user may have adjusted the in-plane position of the simulated light source <NUM> using one or more of the methods described previously in reference to <FIG>. The imaging system may have automatically adjusted a depth position of the simulated light source <NUM> such that the simulated light source <NUM> is the set distance from the surface of the vertebra <NUM>. <FIG> shows spinal column image 1300c with simulated light source <NUM> over a vertebra <NUM>. The simulated light source <NUM> may be the set distance from a surface of the vertebra <NUM>. The user may have adjusted the in-plane position of the simulated light source <NUM> from the position over vertebra <NUM> shown in image 1300b to the current position over the vertebra <NUM> shown in image 1300c. The user may have adjusted the in-plane position of the simulated light source <NUM> using one or more of the methods described previously in reference to <FIG>. For example, the user may have dragged the light source <NUM> along a direction indicated by arrow <NUM> using a touch screen. The imaging system may have automatically adjusted a depth position of the simulated light source <NUM> such that the simulated light source <NUM> is the set distance from the surface of the vertebra <NUM>. The imaging system may have automatically adjusted a depth position of the simulated light source <NUM> as the user dragged the light source <NUM> along the spinal column image 1300c between vertebra <NUM> and vertebra <NUM> such that the set distance between the light source <NUM> and the surfaces shown in spinal image 1300c. The imaging system may have adjusted the depth position of light source <NUM> using one or more techniques described in reference to <FIG>. This may allow a clinician to quickly scan the spinal column and inspect each vertebra.

In various embodiments where components, systems and/or methods are implemented using a programmable device, such as a computer-based system or programmable logic, it should be appreciated that the above-described systems and methods can be implemented using any of various known or later developed programming languages, such as "C", "C++", "FORTRAN", "Pascal", "VHDL" and the like. Accordingly, various storage media, such as magnetic computer disks, optical disks, electronic memories and the like, can be prepared that can contain information that can direct a device, such as a computer, to implement the above-described systems and/or methods. Once an appropriate device has access to the information and programs contained on the storage media, the storage media can provide the information and programs to the device, thus enabling the device to perform functions of the systems and/or methods described herein. For example, if a computer disk containing appropriate materials, such as a source file, an object file, an executable file or the like, were provided to a computer, the computer could receive the information, appropriately configure itself and perform the functions of the various systems and methods outlined in the diagrams and flowcharts above to implement the various functions. That is, the computer could receive various portions of information from the disk relating to different elements of the above-described systems and/or methods, implement the individual systems and/or methods and coordinate the functions of the individual systems and/or methods described above.

In view of this disclosure it is noted that the various methods and devices described herein can be implemented in hardware, software and firmware. Further, the various methods and parameters are included by way of example only and not in any limiting sense. In view of this disclosure, those of ordinary skill in the art can implement the present teachings in determining their own techniques and needed equipment to affect these techniques, while remaining within the scope of the invention.

Claim 1:
An ultrasound imaging system comprising:
a scan converter configured to generate a three dimensional (3D) data set from ultrasound echoes received from a subject when imaging a volume of the subject;
a volume renderer configured to calculate surface shading information of a first surface of the 3D data set based, at least in part, on a location of a simulated light source, and render a two dimensional (2D) projection image of the 3D data set which includes the shading information; and
a user interface comprising:
a display configured to display the 2D projection image; and
an input device comprising a user interface element configured to receive first user input indicative of an in-plane position of the simulated light source within a projection plane of the 2D projection image, wherein the volume renderer is further configured, responsive to movement of the simulated light source via the user input, to automatically render the simulated light source at a location corresponding to the in-plane position and a depth position determined by the volume renderer, and wherein the depth position is set based at least in part on a quality criterion and at least in part on contours of the first surface such that a distance between the simulated light source and the first surface is maintained, wherein the quality criterion represents minimizing an amount of shadowing in a vicinity of the first surface of a ray of the simulated light source or maximizing a dynamic range of lighting intensity in the 2D projection image, wherein the depth position corresponds to a location along an axis normal to the projection plane at a set distance from the first surface of the 3D dataset, wherein the set distance is a constant distance from different positions on the first surface.