Patent Description:
Interpretation of medical images can include quantitative measurements, such as the dimensions of a lesion. For example, in Response Evaluation Criteria In Solid Tumors (RECIST), a measurement is taken across a lesion at a widest point in an image slice or view. That is, the widest part of the lesion is measured. The measurements over time are then used to grade the tumor or provide a status of the tumor, such as progressing or stable. The grade or status of the tumor is typically, according to formal guidelines, then used to determine treatment options for a patient.

Healthcare professionals, such as a radiologist, receive medical images for patients and interpret or read the images. Healthcare professionals are under time pressures to interpret the medical images. Measurements, such as RECIST measurements are typically made by marking a start and end point. However, due to the resolution of a display device displaying the image and a precision of a pointer, such as a mouse cursor relative to the displayed image, placement of end points can introduce error in the measurements. A typical approach to address this is to use a zoom feature. The zoom feature enlarges the image on the display device within physical constraints of the display device, which provide for more precisely placed end points relative to the boundaries of the measured lesion.

The zooming in and out on images interrupts the workflow, and adds time to the process. The zooming in on an image additionally removes from view a portion of the image to accommodate the zoomed image within the constraints of the display device or a window within the display device, which provides context and interpretation relative to the image as a whole. For example, in an image with multiple lesions, zooming in on an image for measurements of a particular lesion, may remove other lesions from view, for which then the healthcare practitioner visually and mentally rescans after zooming back out to reestablish context. The context can include, for example, which lesions are measured and which is next to be measured, adding additional time for the rescanning and reestablishing context.

One approach is to add a tool bar or drop down menu, which adds additional inputs, such as mouse clicks, to select an enlargement and/or measurement tool. Over time this approach adds overhead from the repetitive additional inputs similar to that of the zooming in and out on images.

Another approach to interpretation and measurements is image processing, which automatically identifies and segments lesions in an image using computer based algorithms. The segmented lesions are identified with a contour line from which measurements can be taken. The contoured lesions are presented superimposed on the image, which allow the healthcare professional to adjust the contour lines before a measurement is made, that is, dragging or moving a contour line to better conform to a shape of a lesion. Identifying a better placement of the contour line is often approached with zooming in on the contoured lesion, which presents similar issues to the zooming in for a measurement described above.

<CIT> discloses a system and a method for allowing a user to selectively view an automatically zoomed region of a displayed initial viewable area. Typically, the automatically zoomed region is deployed by the user when placing mark-ups in the viewable area. The automatically zoomed region may be displayed by itself, or it may be displayed simultaneously with an un-zoomed portion of the initial viewable area. After the user places a mark-up in the viewable area, the automatically zoomed region is closed, and the initial viewable area is returned to the displayed viewable area.

Aspects described herein address the above-referenced problems and others.

The following describes a context sensitive magnifying glass displayed within view of a medical image displayed on a display device. The context sensitive magnifying glass includes a localized enlargement of a region of interest in the displayed medical image. Inputs can be received marking points in the localized enlargement of the region of interest, which are mapped to dimensions of the medical image. Invoking the context sensitive magnifying glass can be in response to currently displayed information and/or inputs from one or more input devices.

According to the present invention a medical imaging system and a method of displaying medical images are presented as defined in the claims.

With reference to <FIG>, an embodiment of a medical imaging system <NUM> with a context sensitive magnifying view engine is schematically illustrated. The system <NUM> includes a user interface <NUM>, which displays a view of a medical image <NUM> on a display device <NUM> and provides a moveable indicator <NUM>. The moveable indicator <NUM> identifies a moveable point positioned in the displayed medical image <NUM> according to a first input by one or more input devices <NUM>, such as a mouse, keyboard, microphone, touch screen, light pen, combinations and the like. The user interface <NUM> maps the position of the moveable indicator <NUM> in a first coordinate system to a second coordinate system of the medical image.

For example, a medical image in a DICOM format can include dimensions which correspond to anatomical positions and sizes of a patient in a first coordinate system, and the display of the medical image changes the dimensions in a second coordinate system, such as with a view or window including the displayed medical image <NUM> with a zoom factor which enlarges the medical image to fit within a display area <NUM> of the display device <NUM>. The coordinates, according to the display device <NUM>, the displayed medical image <NUM>, and a position of the moveable indicator <NUM> are mapped to the coordinate system of the medical image in the DICOM format. Thus, for example, when measuring a lesion <NUM> in the displayed medical image <NUM>, the measurement is according to the anatomical distances indicated by the DICOM format, and not the physical dimensions of the display device <NUM> with the magnified portion of the lesion <NUM>.

The displayed medical image <NUM> can be received directly from a medical imaging scanner <NUM>, such as a computed tomography (CT) scanner, a magnetic resonance (MR) scanner, a positron emission tomography (PET) scanner, single photon emission computed tomography (SPECT) scanner, ultrasound (US) scanner, combinations thereof, and the like. The displayed medical image <NUM> can be received from a storage subsystem <NUM>, such as a Picture Archiving and Communication System (PACS), radiology information system (RIS), Electronic Medical Record (EMR), Hospital Information System (HIS) and the like. The medical image can include a two or more dimensional medical image, which is displayed in a two dimensional view on the display device <NUM> as the displayed medical image <NUM>.

A magnifying view engine <NUM> generates a localized enlargement of a region of interest <NUM> within the displayed medical image <NUM> relative to a current position of the moveable indicator <NUM>. The magnifying view engine <NUM> determines attributes of the localized enlargement of the region of interest, which include a size, a zoom factor, a position and a shape. A localized enlargement includes a magnified area of the displayed medical image <NUM>, which is less than the whole image, such an area less than ten percent of the area of the displayed medical image <NUM>. In some embodiments, the attributes are determined based on user preferences or defaults. In some embodiments, the attributes are determined based on displayed information with the display medical image <NUM>, such as end points of a measurement or a width of area contours of a segmented lesion.

For example, the size of the localized enlargement of the region of interest can be proportional to a distance between existing end points, the zoom factor of the localized enlargement of the region of interest can be inversely proportional to the distance, and/or the position of the localized enlargement of the region of interest can include one of the end points within the displayed medical image. In another example, the distance corresponds to a maximum width across a contoured segmented lesion.

The attributes of the localized enlargement of a region of interest <NUM> can be modified with inputs from the one or more input devices <NUM>. For example, an input from a mouse wheel can increase or decrease the zoom factor of the localized enlargement of a region of interest <NUM> currently displayed. The user interface <NUM> adjusts the mapping to the first coordinate system of the medical image according to the moveable indicator <NUM> from a third coordinate system within the localized enlargement of a region of interest <NUM>, which is different from the second coordinate system of the displayed medical image <NUM>.

A contrast engine <NUM> can modify the contrast of the localized enlargement of the region of interest differently from the region of interest in the displayed medical image. In some instances, the contrast can be increased from the non-enlarged displayed region of interest such that the edges of the lesion <NUM> are more readily apparent. In some instances, the difference in contrast from the surrounding non-enlarged areas of the displayed medical image <NUM> adds to the focus on the contrasted areas, such as increasing visual attention, while maintaining context relative to the displayed medical image <NUM> as a whole.

An intent detection engine <NUM> can generate a first signal that invokes the magnifying view engine <NUM> to generate the localized enlargement of a region of interest <NUM> and a second signal that invokes the magnifying view engine <NUM> to remove the localized enlargement of a region of interest <NUM>. The first and second signals can be implemented using computer call or interrupt functions known in the art. The signals can be evaluated based on rules that map the inputs to the signal.

The first signal can be generated based on currently displayed information, an input and/or attributes from the input, such as a selection of the end point or contour, or a selection of the end point or contour in combination with other inputs. In some instances, the first signal indicates an intended response indicative of a healthcare practitioner attempting to slightly adjust a position of the existing end point or contour. In some instances, the intent is inferred from small and/or proximate movements of the one or more input devices <NUM> and corresponding movement of the moveable indicator <NUM>. In some instances, unnecessarily invoking the localized enlargement of a region of interest <NUM> is avoided because for some healthcare practitioners the existing end point is correctly placed, initially, approximately <NUM>% of the time, and unneeded extra inputs and/or actions to remove the localized enlargement of a region of interest <NUM> for correctly placed end points are avoided, while extra inputs and/or actions to invoke the localized enlargement of a region of interest <NUM> for an incorrectly placed end point are minimized.

The second signal can be generated based on an input command or positioning the moveable indicator <NUM> outside of the localized enlargement of a region of interest <NUM>. For example, the magnifying view engine <NUM> is signaled in response to a mouse position, received as an input that correspond to the moveable indicator <NUM>, which is no longer within the localized enlargement of a region of interest <NUM>. In another example, in response to a keyboard command received by the intent detection engine <NUM>, the magnifying view engine <NUM> is signaled to remove the localized enlargement of a region of interest <NUM>. Changes in position of the existing end point or contour are displayed corrected in the non-enlarged displayed region of interest <NUM> after removal of the localized enlargement of a region of interest <NUM>. In other words, the changes to an existing end point indicated in the magnified area are mapped from the magnified area to the anatomical coordinate system, and after removal of the magnified area are mapped from the anatomical coordinate system to the displayed image coordinate system.

The user interface <NUM>, the magnifying view engine <NUM>, the contrast engine <NUM>, and the intent detection engine <NUM> are suitably embodied by one or more configured processors, such as one or more processors <NUM> of a computing device <NUM>. The configured processor(s) <NUM> executes at least one computer readable instruction stored in computer readable storage medium, such as the memory <NUM> of the computing device <NUM>, which excludes transitory medium and includes physical memory and/or other non-transitory medium to perform the disclosed phase reconstruction, segmentation, mask construction, vessel enhancement, registration, motion estimation, and motion compensated reconstruction techniques. The configured processor may also execute one or more computer readable instructions carried by a carrier wave, a signal or other transitory medium. The computing device <NUM> can comprise a workstation, laptop, tablet, smart phone, body worn computing device, server, combinations and the like. The lines between components represented in the diagram represent communications paths, which can be wired or wireless.

The computing device <NUM> includes the display device <NUM>, such as a computer display, projector, body worn display, and the like, and one or more input devices <NUM>, such as a mouse, keyboard, microphone, touch or gesture interface, and the like. The computing device <NUM> includes the one or more processors <NUM>, such as a digital processor, a microprocessor, an electronic processor, an optical processor, a multi-processor, a distribution of processors including peer-to-peer or cooperatively operating processors, client-server arrangement of processors, and the like.

With reference to <FIG>, an example display of the localized enlargement of a region of interest <NUM> with an end point <NUM> at an extremity of a lesion <NUM> is diagrammatically illustrated. The localized enlargement of the region of interest <NUM> is positioned within the displayed medical image <NUM>. In some instances, the localized enlargement of a region of interest <NUM> includes only an enlarged portion of the lesion <NUM>, contour, and/or one existing end point of a measurement.

The moveable indicator <NUM> with an input, such as a mouse click, selects an existing end point <NUM>, such as a previously entered or existing end point, one of two end points used to measure the lesion <NUM> or an existing contour line. A context is established, which includes the displayed information of an existing end point that is then selected. In some embodiments, the context of selecting an existing end point generates the signal invokes the localized enlargement of a region of interest <NUM>. In some instances, the context reduces the number of inputs and/or actions by the healthcare professional to accurately position an end point, such as automatically zooming in on a particular area of the displayed medical image <NUM> in a separate window within a window without losing context of a place within the displayed medical image <NUM>.

In some embodiments, the context that generates the first signal to invoke the localized enlargement of a region of interest <NUM> is in combination of selecting the existing end point and additional inputs, such as in combination with a speed of the moveable indicator, a maximum speed of the moveable indicator, an average speed of the moveable indicator <NUM>, a maximum distance of the current position of the moveable indicator <NUM> from the selected existing end point, a depressed key or a depressed key for a duration of the one or more input devices <NUM>, and/or a duration of the moveable indicator <NUM> in a fixed position relative to the selected existing end point. The durations can be compared with a predetermined threshold value that is set greater than a normal response time.

With reference to <FIG>, an example display of the localized enlargement of a region of interest <NUM> with a point <NUM> indicating movement of a segmented lesion contour <NUM> is diagrammatically illustrated. In some embodiments, the first signal is generated to invoke the localized enlargement of a region of interest <NUM> based on the displayed information, such as the contour <NUM> of the segmented lesion. For example, a mouse click within an area defined by the contour <NUM> invokes the localized enlargement of a region of interest <NUM>. In some embodiments, the attributes of the localized enlargement of a region of interest <NUM> are determined such that the entire contour <NUM> is visible within the localized enlargement of a region of interest <NUM>. Changing or repositioning the contour or a portion thereof can be according to algorithms known in the art.

A change in or movement of the contour <NUM> can be indicated with the moveable indicator <NUM>. For example, selection of a point along the contour with a drag operation can be used to modify the contour. In some embodiments, the input which moves the contour <NUM> and the input which moves the localized enlargement of a region of interest <NUM> are based on currently displayed information. For example, a mouse click on a point of the contour <NUM> and a drag moves the contour <NUM>, while a mouse click within the localized enlargement of a region of interest <NUM> not within the area defined by the contour <NUM> and a drag moves the localized enlargement of a region of interest <NUM>. Movements of the one or more input devices <NUM> repositioning a contour can invoke the localized enlargement of a region of interest <NUM> similarly to movements re-positioning the existing end point <NUM> of <FIG>.

With reference to <FIG>, an example display of the localized enlargement of a region of interest <NUM> within a view of an example medical image selecting an existing end point at an extremity of a lesion is illustrated. The displayed medical image includes two measurements: an "A" measurement of <NUM> millimeters (mm), and a "B" measurement of <NUM>. The localized enlargement of a region of interest <NUM> is invoked with a selection of one end point of the two end points of the "A" measurement indicated by a dotted line. The first end point is located outside the localized enlargement of a region of interest <NUM>. The second (selected) end point is located within the localized enlargement of a region of interest <NUM>. The moveable indicator <NUM> is indicated by an unfilled rectangular shape with a bordered dotted line. The center of the rectangular shape represents a point. In some instances, the unfilled rectangular shape allows the enlarged areas of the displayed medical image <NUM> to be visually apparent around the point that is not obscured by the moveable indicator <NUM>. The localized enlargement of a region of interest <NUM> includes a border which separates and distinguishes the enlarged region of interest from remaining areas of the displayed medical image <NUM>.

Rules can be used to invoke the localized enlargement of a region of interest <NUM>, which evaluate attributes of one or more inputs and currently displayed information to generate a signal to invoke the localized enlargement of a region of interest <NUM>. For example, with an input of a mouse click corresponding to a selected end point, X = yes, in <FIG>. That is, the moveable indicator <NUM> is positioned to correspond to an existing end point currently displayed when the mouse is clicked. An attribute of the input includes the duration, Y, in milliseconds during which the user modifies the location of the start or end point of the measurement. That is, the end point is being moved to a new position with a drag and the duration of the drag is measured in milliseconds. Another attribute of the input includes an average speed of a mouse tracking, Z mm/s, for a pre-determined window of milliseconds. That is, the attribute of input includes the average speed of the moveable indicator across the displayed medical image <NUM>.

An example rule can include: if X = yes and Y > <NUM>, then invoke localized enlargement of a region of interest <NUM> or generate the first signal. In some instances, the healthcare professional has paused after selecting an end point, which suggests that magnification would aid. Another example can include if X = yes, Z < threshold and Y > <NUM>, then invoke localized enlargement of a region of interest <NUM> or generate the first signal. In some instances the healthcare professional is moving the moveable indicator <NUM> slowly for a period of time, in small increments, or in close proximity to the originally positioned end point, which suggests that magnification would aid.

In some embodiments, parameters such as Y and/or Z are configurable. In some embodiments, the system <NUM> can learn to optimize one or more parameters by analyzing a database of user interactions. Parameter values can be determined and/or modified using statistical or machine learning techniques that minimizes a "false positive" rate, that is, a percentage of cases of invoking localized enlargement of a region of interest <NUM> without any input indicative of moving a selected end point or fixing a new end point.

With reference to <FIG>, an embodiment of a method of displaying medical images with a context sensitive localized enlargement of a region of interest is flowcharted. At <NUM> a view of a medical image is displayed on the display device <NUM> and the moveable indicator <NUM> provides a moveable point positioned in the displayed medical image <NUM>. A first input by one or more input devices <NUM> selects an existing end point or position of a contour within the displayed medical image, such as a mouse tracking. The displayed medical image <NUM> can include other displayed information, such as one or more existing end points and/or existing contours.

At <NUM>, a first signal is generated that requests the generated localized enlargement of the region of interest <NUM> within the displayed medical image <NUM> according to a second input. The second input can include attributes of a speed of the moveable indicator <NUM>, a time duration, and/or a distance of the moveable indicator <NUM> from the original or selected existing end point or position selected on the existing contour. The first signal can be in response to selecting an existing end point or contour. The first signal can be in response to selecting an existing end point or contour in combination with another input and/or other input attributes, such as the speed, time duration or distance. The generation of the first signal can use rules that map the second input to the first signal.

At <NUM>, in response to the first signal, the localized enlargement of the region of interest <NUM> within the displayed medical image <NUM> relative to a current position of the moveable indicator <NUM> is generated. Generating the localized enlargement of the region of interest <NUM> includes determining attributes of a size, a zoom factor, a position and a shape of the localized enlargement of the region of interest <NUM>. The attributes of the localized enlargement of the region of interest <NUM> can be determined according to a distance between the selected end point and another end point, a width across an area defined by the existing contour, and/or a user or system parameter. The generated localized enlargement of the region of interest <NUM> can include modifying the contrast of the localized enlargement of the region of interest <NUM> differently from the region of interest in the displayed medical image <NUM>.

At <NUM>, a third input can be received, which moves an existing end point or contour in the localized enlargement of the region of interest <NUM>. The moved point or contour is mapped from a coordinate system of the localized enlargement of the region of interest <NUM> to a coordinate system of the medical image. The third input can include indicating the moved point superimposed on the localized enlargement of the region of interest <NUM>, such as a release of a mouse button. The third input can include modifying a contour currently displayed in the localized enlargement of the region of interest <NUM>.

At <NUM>, a second signal is generated to remove the localized enlargement of the region of interest <NUM> based on a fourth input. The fourth input can be a command, such as a keyboard command. The fourth input can include attributes of the first input, such as a position of the moveable indicator <NUM> being external to the localized enlargement of the region of interest <NUM>. In some instances, the fourth input can include another inspection of the first input.

At <NUM>, a measurement can be computed, such as a distance between two end points, an area of a contour, a distance based on a contour, and the like. The computed measurement can include displaying the measurement superimposed on the displayed medical image <NUM>. The computed measurement can include a visual indicator, such as a dotted line between end points.

The above may be implemented by way of computer readable instructions, encoded or embedded on computer readable storage medium, which, when executed by a computer processor(s), cause the processor(s) to carry out the described acts. Additionally or alternatively, at least one of the computer readable instructions is carried by a signal, carrier wave or other transitory medium.

Claim 1:
A system for displaying medical images (<NUM>), comprising:
a user interface (<NUM>) configured to display a view of a medical image on a display device (<NUM>) and to provide a moveable indicator (<NUM>) identifying a moveable point positioned in the displayed medical image; and
a magnifying view engine (<NUM>) configured to generate a localized enlargement of a region of interest within the displayed medical image in response to a selection of an existing end point or an existing contour in the displayed medical image according to a first input by one or more input devices and indicated by a current position of the moveable indicator, wherein the existing end point includes an end point of two previously entered end points of a measurement of a previously segmented lesion, and the existing contour includes a contour of a previously segmented lesion,
wherein the magnifying view engine is further configured to determine a size, a zoom factor, and a shape of the localized enlargement of the region of interest according to a first distance between the selected existing end point and another end point or according to a second distance of a width across an area defined by the existing contour.