Graphical virtual controls of an ultrasound imaging system

A method includes receiving, at a console (132) of an ultrasound imaging system (102), a first signal indicative of actuation of a touch control (128) of a touch screen user interface (122) of the ultrasound imaging system, wherein the touch control is one of a plurality of different touch controls of the ultrasound imaging system. The method further includes identifying, with a processor of the ultrasound imaging system, the touch control from the plurality of different touch controls based on the first signal. The method further includes identifying, with the processor of the ultrasound imaging system, a graphic representation of the touch control based on the identification of the touch control. The method further includes visually displaying, with the processor of the ultrasound imaging system, the graphic representation on a display monitor concurrently with displaying an image.

RELATED APPLICATION

This application is a national filing of PCT application Serial No. PCT/IB2014/059836, filed Mar. 14, 2014, published as WO2015/136336 on Sep. 17, 2015. This application claims priority to PCT application Serial No. PCT/IB2014/059836, published as WO2015/136336 on Sep. 17, 2015.

TECHNICAL FIELD

The following generally relates to ultrasound imaging and more particular to displaying, on a display monitor, a virtual control or representation mimicking an active touch screen user interface control and operation thereof.

BACKGROUND

An ultrasound imaging system has included an ultrasound probe with a transducer, a console with an integrated or external display monitor, and a user interface. The transducer transmits an ultrasound signal into a field of view and receives echoes produced in response to the signal interacting with structure therein. The echoes are conveyed to the console and are processed, producing images of the scanned structure, which may be visually presented through the display monitor.

The display monitor may include a cathode ray tube (CRT), a liquid crystal display (LCD), a light emitting diode (LED) display, and/or other type of display. The display monitor has been a “dumb” monitor with no processor or processing capabilities and that is simply an output device that displays images and other information (e.g., transducer frequency, gain, etc.). The user interface has included a keyboard or keypad with mechanical depressable buttons and/or a flat touch screen area (e.g., LCD, CRT, etc.).

The display monitor has been placed in an upright vertical position so that the clinician can look at images and/or the other information displayed via the display monitor. The user interface has been placed in a generally horizontal position, approximately perpendicular to the upright vertical position. This arrangement is similar to that of a desktop computer monitor and corresponding keyboard. The user controls features such as gain, zoom, pans, etc. with the controls of the user interface.

Unfortunately, with such a user interface/display monitor arrangement, where the user interface includes a touch screen with a flat surface, it may not be readily easy for the user to locate and/or operate touch sensitive controls of the user interface while observing an image and/or patient. Rather, the user may have to look away from the image and/or patient and down at the user interface to find and/or operate the control, adding complexity and inefficiency to the procedure.

SUMMARY

Aspects of the application address the above matters, and others.

In one aspect, a method includes receiving, at a console of an ultrasound imaging system, a first signal indicative of actuation of a touch control of a touch screen user interface of the ultrasound imaging system, wherein the touch control is one of a plurality of different touch controls of the ultrasound imaging system. The method further includes identifying, with a processor of the ultrasound imaging system, the touch control from the plurality of different touch controls based on the first signal. The method further includes identifying, with the processor of the ultrasound imaging system, a graphic representation of the touch control based on the identification of the touch control. The method further includes visually displaying, with the processor of the ultrasound imaging system, the graphic representation on a display monitor concurrently with displaying an image.

In another aspect, an ultrasound imaging system includes a touch screen user interface including a touch panel active region with one or more touch controls, an ultrasound scanner console including a transducer controller, an echo processor and a display controller, and a display monitor, including at least two display regions. The display controller, in response to activation of one of the one or more touch controls, renders a virtual graphical control, which represents the one of the touch controls, in one of the display regions.

In another aspect, a computer readable storage medium is encoded with computer readable instructions which, when executed by a computer processor, cause the computer process to: display, in a display region of a display monitor of an ultrasound imaging system, a graphic that visually mimics an operation of a touch control of a touch screen user interface of the ultrasound imaging system, wherein the touch screen user interface and the display monitor are separate and distinct elements of the ultrasound imaging system.

Those skilled in the art will recognize still other aspects of the present application upon reading and understanding the attached description.

DETAILED DESCRIPTION

The ultrasound imaging system102includes a probe104with a one-dimensional (1D) or two-dimensional (2D) transducer array106with at least one transducer element108. Suitable array configurations include, but are not limited to, linear, curved (e.g., concave, convex, etc.), circular, etc., full populated or sparse, etc. The probe104further includes a console interface110, which may include a connector (e.g., an electro-mechanical device for joining electrical circuits) and/or wireless transceiver.

The ultrasound imaging system102further includes a display monitor112. The display monitor112can be a cathode ray tube (CRT), a liquid crystal display (LCD), a light emitting diode (LED), and/or other display monitor. The display monitor112includes a display area114with a multiple sub-display regions1141, . . . ,114N, where N is a positive integer. The display monitor122further includes a console interface110, which may include a connector (e.g., an electro-mechanical device for joining electrical circuits) and/or wireless receiver.

The display monitor112can be affixed to a support120(as shown inFIG. 1) such as a stand, a bracket, etc. that rests on or is affixed to a surface (e.g., a desk, a table, etc.) and holds the display monitor112in a generally upright vertical viewing position. In another instance, the support120can be a wall bracket that mounts to a wall, a ceiling, etc., thereby indirectly mounting the display monitor112thereto. Other supports are also contemplated herein. The support120may be configured to rotate, tilt, translate, and/or otherwise move, which selectively allows for spatially orienting the region124.

The ultrasound imaging system102further includes a touch screen user interface122. The touch screen user interface122includes a touch panel124. At least a sub-portion of the touch panel124includes one or more active regions126with one or more touch sensitive control(s)128such as a time-gain control (TGC), a mode (e.g., 2D, 3D, 4D, etc.) selector control, a CINE control, a measurement control, a zoom control, a depth control, a focus control, etc. The touch panel214may include a resistive, a capacitive, an acoustic, an infrared, an optical, a piezoelectric, and/or other region. Furthermore, the touch panel214may include an LCD, thin film transistor (TFT) LCD, organic light-emitting diode (OLED), and/or other display.

The one or more controls128, in one instance, are kept in a normally deactivate state, but are activatible. A deactivated control128, when actuated by a gesture (e.g., a press, a swipe, a touch, etc.) on the control128with one or more fingers, a stylus, a glove, etc., transitions to an active state. An activate control128, when operated, generates an electrical signal corresponding to the operation of the control128. Generally, when a control128is active, one or more other non-active controls transition to a non-activatible state. In a variation, one or more controls can be kept in a normally active state. In this variation, the normally active control may transition to a de-active state in response to activation of another control, a gesture over the control, an input signal including a control signal that deactivates the control, etc.

As an example of activation and operation, a gain control is activated by touching the control and then the gain is increased or decreased (i.e., operated) through the control through a predetermined gesture. For example, circling clockwise, sliding over in direction, etc. the gain control with a suitable object increases gain, and circling counter-clockwise, sliding in another direction, etc. over the gain control with the suitable object decreases gain. A reset control may be provided to quickly return on or more of the touch controls128to the initial or starting conditions. An active control can be configured to automatically transition back to a deactive state, e.g., in response to lapse of a pre-determined length of time of inactivity, activation of another control126, on demand, etc.

Briefly turning toFIGS. 2, 3 and 4, non-limiting examples of the touch screen user interface122are illustrated.

InFIG. 2, the controls128include circular shaped controls202including one control2021with one diameter and another control2022with a different larger diameter, a ring shape control204with a non-active area206in the hole of the ring, concentric controls208including one control2081surrounded by another control2081, a rectangular control210with rounded edges, and curved rectangular controls212with rounded edges. A control214includes a plurality of touch activated slides2161,2162,2163, . . . ,216N, collectively referred to herein as virtual slides216, which reside in a recess218in the touch panel124. Again, the controls128can be activated and used by pressing, tapping, swiping, etc.

Turning toFIG. 3, the controls128include a two-dimensional (2D) array302of rectangular controls304and other controls306. InFIG. 4, the controls128include controls402of a computer keyboard as well as other controls128, such as a search box404, and custom configure controls406. Another example touch screen user interface122includes a combination ofFIGS. 2, 3 and/or 4, optionally with alternative and/or additional controls. Still other controls can also include physical mechanical controls such as a trackball, a track pad, etc. Other examples of touch screen controls are described in application Ser. No. 13/748,653, filed on Jan. 24, 2013, entitled “Ultrasound Imaging System,” and assigned to B-K Medical Aps, which is incorporated herein by reference in its entirety.

Returning toFIG. 1, the touch screen user interface122may also include visual indicators (e.g., lights, etc.), audible indicators (e.g., speakers, etc.), tactile indicators (e.g., vibration, etc.), other controls (e.g., physical mechanical buttons, physical mechanical slides, physical mechanical rotary knobs, etc.), an image display region, etc. The touch screen user interface122further includes a console interface130, which may include a connector (e.g., an electro-mechanical device for joining electrical circuits) and/or wireless transceiver.

The ultrasound imaging system102further includes an ultrasound scanner console132. The console132includes a probe interface134, a display monitor interface136, and a touch screen user interface138, which, respectively are complementary to the console interfaces110,118and130. For example, the probe interface134is complementary to the console interface110in that the probe interface134and the console interface110physically engage and provide an electrical pathway between the probe104and the ultrasound scanner console132. For instance, the console interface110may include a female connector and the probe interface134may include a male connector, wherein the connectors physically engage and physically connect electrodes.

The ultrasound scanner console132further includes a transducer controller140. The transducer controller140controls excitation of the at least one transducer element108. The transducer controller140also controls detection of echoes with the at least one transducer element108. In a variation, the excitation and detection can be through separate components such as transmit and receive circuitry. The console132further includes an echo processor142(e.g., microprocessor, central processing unit, etc.) that processes detected echoes. Such processing may include generating an image, estimating flow velocity, and/or processing. The ultrasound scanner console132further includes a physical memory device (memory)144, which can be used to store ultrasound data.

A display controller146controls the information visually presented in at least one of the display regions114of the display monitor112. By way of example, in one non-limiting instance, the display controller146renders an ultrasound image in at least one of the display regions114. In another example, the display controller146renders a graphical representation of one of the controls128in at least one of the display regions114. As described in greater detail below, for the latter, the display controller146identifies activation and/or operation of a touch control128and renders the graphical representation in the at least one of the display regions114, where the graphical representation may include alpha-numeric information and/or graphics, shows a current value and/or any changes thereto, shows movement of the actual control, etc.

Generally, the graphical representation provides a virtual control of a touch control128, on the display monitor112, that mirrors or mimics the touch control128of the touch screen user interface122and operation of the touch control128through the touch screen user interface122. The graphical representation shows in the display monitor112, for example, where the user's finger (or other object) is located on the touch screen user interface122with respect to the control128, thereby allowing the user to adjust a control128without looking at the control128on the touch screen user interface122. In one instance, this provides a more intuitive adaptation of the touch controls, and may reduce complexity and inefficiency, relative to a configuration in which the display controller146does not render virtual controls on a display region of the display monitor112.

FIGS. 5 and 6illustrate non-limiting examples of the ultrasound imaging system102. In the non-limiting examples, the display monitor112, the touch screen user interface122and the ultrasound scanner console132are integrated and part of respective mobile carts500and600, which include movers502and602such as wheels, casters, etc. In another configuration, the ultrasound imaging system102rests on a table, desk, etc., and does not include movers and is not integrated into a cart, is also contemplated herein. In this configuration, the ultrasound imaging system102may be portable in that a user can pick it up and carry it around or not. Further, the illustrated probe104does not limit the type of probe and shown for explanatory purposes.

FIG. 7schematically illustrates an example of the display controller146.

The display controller146includes a user interface (US) signal sensor702. The user interface signal sensor702senses when a touch control128of the touchscreen user interface122is activated and/or operated. By way of example, as described herein, activating a control128or operating the control128with an activate control128(e.g., through touch, etc.) invokes the control128to generate a signal, which is conveyed to the ultrasound scanner console132. The user interface signal sensor702senses actuation and/or operation of a control128through this signal.

The display controller146further includes a control identifier704, which, based on the sensed signal, determines which control128of the controls128generated the signal. Known and/or other approaches for identifying the control128can be used. For example, with a capacitive based touchscreen, when an object touches the control128, a measureable change in capacitance occurs, and the location of the touch, and hence, the function corresponding to the control128, can be determined.

The display controller146further includes visual graphic library706, which can be stored in memory144and/or other memory. The visual graphic library706, in the illustrated embodiment, includes instructions for generating a different graphic representation for each of the controls128. In general, the graphic representation for a control128graphics mimics the appearance and/or operation of the control128on the touchscreen user interface122.

The display controller146further includes virtual control renderer708. The virtual control renderer708renders the graphic representation on the display region114not displaying the image. The virtual control renderer708also removes graphic representation from the display region114. The display controller146further includes virtual control updater710. The virtual control renderer708updates the graphic representation on the display region114in response to a signal indicating operation such as a change (e.g., an increase in gain) invoked by the control128.

The display controller146further includes a timer712. The timer712is invoked to start a count (increment or decrement) based on a pre-determined time duration in response to receiving a control activating signal, re-start the count in response to receiving a change in value signal for the activated control, start a count for another control in response to a control activating signal for another control, cease the count in response to the pre-determined time duration lapsed before another change in value signal is received.

The display controller146can be implemented by a computer processor (e.g., a microprocessor, a central processing unit, etc.) executing computer readable instructions stored on computer readable storage medium (which exclude transitory medium) such as hardware memory. Additionally or alternatively, at least one of the computer readable instructions is carried by a signal, a carrier wave and/or other transitory medium.

FIGS. 8-13illustrate example operation of the display controller;

InFIG. 9, a user210actuates the control8023by pressing on the control8023. In this example, in response thereto, the controls802and804transition to a state in which they cannot be activated, and a value line902, a current value904, end limits906and908(e.g., minimum and maximum values), and a value selector910are rendered above the control8023, and a value selector912, a current value914, and minimum and maximum values916and918are rendered in the display region114N. In this example, the selectors910and920are at a center position.

InFIG. 10, the user210moves the selector910by touching the selector910and sliding their finger across the touch screen user interface122. Concurrently therewith, the selector912in the display region114Nmoves in coordination therewith, and the current values904and914increment by the same amount. InFIG. 11, the user moves the selector910in the opposite direction. Concurrently therewith, the selector912moves in coordination therewith, and the current values904and914decrement.

InFIG. 12, the user210ceases touching the touch screen user interface122. InFIG. 13, after lapse of a pre-determined time duration in which the user210does not operate the control8023, the value line902, the current value904, the end limits906and908, the value selector910, the value selector912, the current value914, and the minimum and maximum values limits916and918, are removed from touch screen user interface122and the display region1142(or are no longer rendered), and the controls802and804transition to a state in which they are activatible.

FIGS. 14-19illustrate other example operation of the display controller;

InFIG. 15, the user210actuates the set of slide controls804by pressing on the region806. In this example, in response thereto, the controls802transition to a state in which they cannot be used, and a virtual slide control1502, including virtual adjusters15021,15022, . . .1502N, is rendered in the display region114N. In this example, the virtual adjuster15021,15022, . . .1502Nare each at a center position.

InFIG. 16, the user210moves the adjuster8041by touching the adjuster8041and sliding their finger across the touch screen user interface122. Concurrently therewith, the virtual adjuster15041in the display region114Nmoves in coordination therewith. InFIG. 17, the user210moves the adjuster8042by touching the adjuster8042and sliding their finger across the touchscreen user interface122. Concurrently therewith, the virtual adjuster15042in the display region114Nmoves in coordination therewith.

InFIG. 18, the user ceases touching the controls804. InFIG. 19, after lapse of a pre-determined time duration in which the user does not operate any of the controls804, the virtual control1502and the virtual adjusters15021,15022, . . .1502Nare removed from the display region114N, and the controls802transition to a state in which they are activatible.

InFIGS. 8-13, the virtual controls are placed under the image. InFIGS. 14-19, the virtual controls are placed to one side of the image. In other embodiments, the virtual controls are place on top, on multiple sides of, within, etc. the image.

FIG. 20illustrates example method for displaying a virtual control mimicking a control of the touch screen user interface;

At2002, a touchscreen control of a touchscreen user interface of an ultrasound scanning apparatus generates a signal in response to actuation of the touchscreen control.

At2004, a display controller of an ultrasound console of the ultrasound scanning apparatus receives the signal.

At2006, the display controller identifies the function assigned to the touchscreen control

At2008, the display controller identifies a visual graphic for the control, where the visual graphic mimics the appearance and/or operation of the control on the touchscreen user interface.

At2010, the display controller renders the visual graphic on a display monitor.

At2012, the display controller updates the displayed visual graphic in coordination with operation of the control on the touchscreen user interface, where the update mimics the appearance and/or operation of the control on the touchscreen user interface.

FIG. 21illustrates example method for removing a displayed virtual control after lapse of a predetermined inactivity time; and

At2102, a touchscreen control of a touchscreen user interface of an ultrasound scanning apparatus generates a signal in response to actuation of the touchscreen control.

At2104, a display controller of an ultrasound console of the ultrasound scanning apparatus receives the signal.

At2106, the display controller identifies the function assigned to the touchscreen control

At2108, the display controller identifies a visual graphic for the control, where the visual graphic mimics the appearance and/or operation of the control on the touchscreen user interface.

At2110, the display controller renders the visual graphic on a display monitor.

At2112, the display controller starts a timer.

At2111, the display controller removes the graphic from the display monitor in response to lapse of a predetermine time duration in which a subsequent signal is not received from the control.

FIG. 22illustrates example method for resetting a timer in response to subsequent use of the control of the touch screen user interface.

At2202, a touchscreen control of a touchscreen user interface of an ultrasound scanning apparatus generates a signal in response to actuation of the touchscreen control.

At2204, a display controller of an ultrasound console of the ultrasound scanning apparatus receives the signal.

At2206, the display controller identifies the function assigned to the touchscreen control

At2208, the display controller identifies a visual graphic for the control, where the visual graphic mimics the appearance and/or operation of the control on the touchscreen user interface.

At2210, the display controller renders the visual graphic on a display monitor.

At2212, a timer is started.

At2214, it is determined if the timer time out.

If, the timer timed out, then at2116, the display controller removes the graphic from the display monitor in response to lapse of a predetermine time duration in which a subsequent signal is not received from the control.

If the timer did not time out, then at2218, the touchscreen control generates a subsequent signal in response to subsequent actuation of the touchscreen control.

At2220, the display controller of the US console receives the subsequent signal.

At2222, the display controller recognizes the control as the active control.

At2224, the display controller updates the displayed visual graphic in coordination with operation of the control on the touchscreen user interface based on the subsequent signal

At2226, the timer is reset, and act2214is repeated.

At least a portion of the methods discussed herein may be implemented by way of computer readable instructions, encoded or embedded on computer readable storage medium (which excludes transitory medium), which, when executed by a computer processor(s), causes the processor(s) to carry out the described acts. Additionally or alternatively, at least one of the computer readable instructions is carried by a signal, carrier wave or other transitory medium.

The application has been described with reference to various embodiments. Modifications and alterations will occur to others upon reading the application. It is intended that the invention be construed as including all such modifications and alterations, including insofar as they come within the scope of the appended claims and the equivalents thereof.