Patent Description:
In ultrasound imaging, an operator of a system uses a probe to obtain ultrasound images of a patient. The images captured by the system may be viewed, printed or included in a patient report for diagnosis and record keeping. In addition, select images may be included in a report that is used to bill the patient or their insurance for the services rendered.

Depending on the examination procedure being performed, the number and subject of the images required in a report of the examination may be standardized or defined. For example, an internal exploratory scan may require that images of the patient's liver, bladder, intestines, kidneys and stomach be obtained. Similarly, a prenatal scan may require images and measurements of various anatomic structures of a fetus. In a typical single operator examination, a physician or an ultrasound technician uses the imaging system to obtain all the images needed to complete the examination. These images are typically selected by the user during the exam, stored in a buffer memory and must be reviewed after the examination is complete to mark or otherwise identify images to be used in creating a record of the scan. In the case of a procedure, such as an ultrasound-guided regional anesthesia injection, where the care provider cannot stop mid procedure or has no free hands to control the system, a second person may assist in controlling the system settings and collecting the needed images during the procedure.

Document <CIT> discloses a system adapted to acquire ultrasound image data for at least a portion of a body organ; utilize data defining a reference plane for the body organ to define at least one other plane with respect to the reference plane; and display automatically and substantially simultaneously at least two ultrasound images corresponding to at least one of the reference plane and the data defining the at least one other plane.

To improve on the systems described above, the disclosed technology relates to an imaging system that includes a cine buffer memory to store image frames as they are produced by the imaging system. A processor analyzes the image frames stored in the cine buffer and presents one or more selected image frames to the operator for approval. In one example of the disclosure, the image frames selected are image frames that are obtained at equal time intervals during the examination. According the the invention, defined in claim <NUM>, image frames are be selected by comparing the image frames in the cine buffer with one or more target image frames. The target image frames may be determined by the type of scanning procedure being performed, based on previous operator selections when performing the same or similar type of scanning procedure. In another example of the disclosure, image frames that are presented are selected by determining whether the image frames meet a particular quality metric, such as where a measurement is most clearly seen or image frames that contain a particular anatomical structure or interventional instrument (e.g., a needle), or are obtained at a particular point in an examination procedure. In the invention of claim <NUM>, image frames are selected for presentation based on how different the image frames are from previous image frames stored in the cine buffer.

A programmed processor in the imaging system allows an operator to approve the selection of one or more of the presented images for inclusion in a report or summary of the scanning procedure. Alternatively, the operator can select their own images by, for example stepping through and selecting one or more image frames stored in the cine buffer. Optionally, the operator can make measurements on these selected image frames before storing them depending on the clinical protocol for the targeted exam/procedure.

As described above, the disclosed technology relates to imaging systems and in particular to ultrasound imaging systems. In some described embodiments, an ultrasound imaging system includes a programmed processor that is configured to select one or more image frames that are captured during an examination of a patient. An operator can approve the selected image frames or can choose alternative image frames. Once the image frames are approved, the processor can mark or otherwise designate the approved image frames for inclusion in a report of the examination for the patient's records and/or for billing purposes. Although the disclosed technology is illustrated in connection with an ultrasound imaging system, it will be appreciated that the technology can be employed with other types of imaging systems such as, but not limited to, MRI systems, video endoscopy systems or other medical imaging systems that capture many images during an examination procedure.

As shown in <FIG>, a simplified block diagram of an ultrasound imaging system <NUM> includes a transducer <NUM> that is configured to send ultrasound signals into a body and to detect the corresponding echo signals. The transducer <NUM> may be a single-element transducer or may consist of one- or two-dimensional linear or phased array of transducer elements. The transducer <NUM> may be connected to a high voltage multiplexer/de-multiplexer (HV mux/demux) <NUM> that is used select individual or groups of transducer elements in the transducer <NUM>. In the case of a phased-array, signals to be transmitted by the transducer <NUM> are generated by a transmit beamformer <NUM> that adjusts the timing of the signals in order to direct the signals in a particular direction and to focus the signals at a particular depth in the tissue. Alternatively, unfocused (plane) waves can be transmitted by the transducer. Signals from the transmit beamformer <NUM> are amplified by one or more high-voltage amplifiers <NUM> before being applied to the HV mux/demux <NUM> and the transducer <NUM>.

A transmit/receive (T/R) switch <NUM> operates to disconnect the receive electronics of the ultrasound system from the transducer when the higher powered transmit pulses are being transmitted. The T/R switch <NUM> is closed when the imaging system is to detect the returning echo signals. Signals received by the T/R switch <NUM> are amplified by low-noise receive amplifiers <NUM> that implement a gain function that typically varies according to the depth from which the echo signals originate. For a directional ultrasound system, the outputs of the receive amplifiers <NUM> feed a receive beamformer <NUM> that delays and sums the amplified received echo signals. In most ultrasound systems, the analog received signals are converted to corresponding digital signals, after amplification, with a number of analog to digital converters (not shown) that are positioned between the receive amplifiers <NUM> and a receive beamformer <NUM> in the signal path.

In one embodiment, a system processor <NUM>, which could be implemented as one or more programmed microprocessors, is configured to execute program instructions that are stored in an internal or external computer readable memory (not shown) to control the operation of the ultrasound imaging system.

Beamformed ultrasound signals produced by the receive beamformer <NUM> are delivered to an image processor <NUM>. The image processor <NUM>, which may include one or more general purpose microprocessors (including the system processor <NUM>), one or more digital signal processors (DSP), one or more graphics processor units (GPU), application-specific integrated circuits (ASIC) or the like, converts the raw, beamformed signals into a two-dimensional image frame of pixel data that can be stored in memory and shown to an operator on a video monitor or other display <NUM>. The image frames produced by the image processor <NUM> are stored in a buffer memory <NUM> (also known as a cine buffer), which in one embodiment is operated as a circular buffer of memory elements that stores a select number of frames as they are produced during an examination. In one embodiment, the cine buffer <NUM> can store <NUM>-<NUM> minutes of data at <NUM> frames/sec or <NUM>-<NUM> image frames of ultrasound data or more. In one embodiment, once the cine buffer <NUM> is full, the oldest image frame in the buffer is overwritten with a new image frame in a circular fashion. A memory <NUM> is used to store the image frames for archival purposes. The contents of the memory <NUM> may be transferred to a remote patient records keeping system after an examination is complete. In some embodiments, at least some of the image frames that are stored in the memory <NUM> are compressed to save space and therefore may lack some detail compared with the image frames that are stored in the cine buffer <NUM>.

A number of operator inputs <NUM> such as keys, buttons, knobs, voice commands, gestures or software-configured controls, such as touch screen controls or the like, allow an operator to change the operating characteristics of the ultrasound machine and to input commands to the system processor <NUM>.

In one embodiment, an operator begins an ultrasound examination procedure by using the input controls <NUM> to select an examination type from a number of pre-defined examination procedures that are shown on the display <NUM> or may have a dedicated control on a keyboard of the system. Exemplary types of examinations may include an internal organ exploratory scan, a neonatal scan, a cardiac scan, a carotid artery scan etc. Each of these scan types may be associated with particular views and measurements that are to be captured by the ultrasound machine operator. For example, a neonatal scan may require views and measurements of a baby's heart, neck tissue, femur length and other views and measurements. In one embodiment, the views and measurements required by the various examination types are stored in a knowledge base within a memory of the ultrasound system.

The operator uses a control <NUM> (e.g. an on-screen button, footswitch, control on an imaging probe etc.) to start the ultrasound machine capturing ultrasound image frames. These image frames are produced and stored in the cine buffer <NUM> until the operator uses a control <NUM> to halt the imaging process. As will be appreciated, the cine buffer is constructed to store several thousand image frames of ultrasound data. In the past, the operator had to review/search through all of the stored image frames to select which frames would be included in a patient's record and/or submitted for billing purposes.

To aid an operator in selecting one or more of the image frames, a processor such as the system processor <NUM>, a DSP or a GPU executes a number of programmed instructions to analyze the image frames that are stored in the cine buffer <NUM> and to select one or more image frames as possible frames for review and approval by an operator. The selected image frames are displayed for the operator and the operator can either approve the selection or can select different image frames. In one embodiment, the approved image frames are marked for inclusion in a patient report of an examination, for use in billing, or both. In one embodiment, the approved image frames are stored in the memory <NUM> with lossless compression or with little or no compression compared to the unselected or non-approved images frames generated during the examination in order to retain more image detail.

<FIG> illustrates a display <NUM> in which the system processor <NUM> has selected and presented a number of possible image frames 150a, 150b, 150c, 150d for the operator to review and approve. The presented image frames are shown to the operator on the display <NUM> and operator can approve one or more of the displayed image frames for inclusion in a patient or billing report. Alternatively, the operator can select one or more other image frames for inclusion into the reports if the operator does not like the image frames that have been selected by the system processor <NUM>. In one embodiment, the operator can view image frames that were recorded in the cine buffer near any of the displayed image frames by selecting, for example, image frame 150c on the display <NUM>. In one embodiment, the system processor <NUM> determines the address of the selected image frame by for example, recalling an index pointer <NUM> that is associated with the image frame 150c and that indicates the address of the image frame in the cine buffer <NUM>. The system processor then uses the determined address to recall and display a number of image frames from the cine buffer that are stored just before and/or just after the selected image frame 150c. By using an advance or rewind control on the operator inputs <NUM>, the operator can view frames that are recorded before or after the image frame 150c. In this manner, the operator can select an image frame that may show a desired view or measurement in a way that is better or more clear than is shown in the image frame 150c that was selected by the system processor <NUM>.

The system processor can select image frames in a number of different manners or in accordance with a number of different metrics. In one example of the disclosure, the system processor selects image frames that are equally spaced in the cine buffer for display to the user. For example, if an examination generates <NUM> image frames that are stored in the cine buffer <NUM>, the system processor may select image frames numbered <NUM>, <NUM>, <NUM> and <NUM> as the image frames that can be approved for possible inclusion into the patient reports. The number of frames that are initially selected may be chosen by the operator or may be defined by the procedure type.

In one embodiment, a knowledge base <NUM> is maintained on a computer readable media that may be internal to the ultrasound machine or may be accessible by the system processor <NUM> by a wired or wireless communication link. The system processor recalls parameters from the knowledge base <NUM> that are associated with the examination type that is in progress. These parameters can include such things as the number of image frames to be included in a report about the procedure, the views desired in each of the image frames or the measurements desired in an image frame. For example, a nerve block procedure may require three or more different image frames to be recorded that include views of a needle approaching a target nerve, the needle at the position of the target nerve and anesthetic being delivered around the target nerve.

In one embodiment, an image database <NUM> is maintained on a computer readable media that may be internal to the ultrasound machine or may be accessible by the system processor <NUM> by a wired or wireless communication link. The image database <NUM> stores representative image frames that show various anatomical structures that can be compared against an image frame that is stored in the cine buffer in order to detect a match. For example, if a procedure requires an image of a patient's liver in a particular orientation, the image database <NUM> includes an image frame that is coded as representing a liver in a particular orientation and that is required by a particular type of examination. If the operator is performing a liver scan procedure, the system processor recalls one or more of the target image frames from the database <NUM> and compares the target image frames with the image frames that are stored in the cine buffer to identify those image frames that most closely match the target image frames.

In some embodiments, the target image frames are associated with narrative information about the frame. The narrative information may include the type of tissue being imaged and one or more parameters of the ultrasound machine used to obtain the image. Such parameters can include the operating mode of the ultrasound machine (B-mode, Doppler Mode, Power mode etc.) as well as power settings, pulse repetition rate, focal depth, probe used etc. The system processor can pre-populate a narrative associated with the selected image frame using the corresponding operating parameters of the ultrasound machine that were used to obtain the image frame so that the operator doesn't have to enter the narrative values manually.

In some embodiments, the processor may execute instructions that perform a machine learning algorithm to compare image frames stored in the cine buffer with image frames that were selected by the operator for a previous examination. Those image frames that bear the closest resemblance to the previously selected image frames can be presented to the operator for possible inclusion into the patient report or billing record (or other report). In this way, the processor can predict likely images that will be selected based on the user's previous actions. In some embodiments, the processor executes instructions to determine which images require measurements to be taken and cause the processor to display a graphic representing a caliper or other measurement tool on the images. The placement of the caliper graphic can be based on an image recognition of the anatomical features contained in the images. For example, an image of a femur can be analyzed and the processor can display the graphic representing the caliper along the length of the femur (or across the femur width if that is the measurement to be detected). The user can use the position of the caliper to obtain a tissue measurement or can vary the location of the caliper if they desire.

<FIG> show two possible alternative methods of selecting image frames for approval by an operator of the ultrasound processing system. As shown in <FIG>, a cine buffer <NUM> stores a number of frames F1 - FN that are stored when the examination first begins and continues until the examination ends. The system processor is configured to execute programed steps to determine the number of frames stored in the cine buffer for the particular examination. In one embodiment, the system processor keeps an internal count that is increased each time an image frame is added to the cine buffer. The system processor also keeps a record of the first memory address in the cine buffer that stores a first image frame for the current examination. The desired number of image frames to be presented for approval by the operator is either received from the operator or from the parameters associated with the particular type of examination being performed. In one embodiment, the system processor divides the total number of frames stored in the cine buffer by the number of image frames desired uses the result to determine the address of the equally spaced image frames in the cine buffer. In yet another embodiment, the system processor keeps a record of the first address used in the cine buffer and subtracts the first address from the last address used to store an image frame in the cine buffer for an examination. The result is the number of memory addresses used to store the image frames in the cine buffer. An index into the cine buffer for the equally spaced image frames can be determined by the processor based on the first and last address, the number of memory addresses used by each image frame (assumed to be the same for each image frame) and the number of image frames desired to be presented.

The system processor <NUM> or other image processor analyzes the image frames stored in the cine buffer for one or more image frames that bear the closest resemblance to one or more target image frames associated with the examination being performed. In one embodiment, the system processor <NUM> is configured to execute program instructions to determine the number and type of image frames that are required by a particular type of examination from the knowledge base <NUM>. The system processor then recalls one or more target image frames from the image database <NUM>. Image frames in the cine buffer <NUM> are then compared to the target image frames to find the closest matches. The closest matches are then presented to the operator for approval.

There are a number of well-known image comparison techniques that can be used by the system processor to compare the image frames in the cine buffer with a target image frame. For example, a simple least squares comparison of the pixel brightness for the pixels in an image frame stored in the cine buffer memory and the target image frames can be used. In another embodiment, image frames can be classified by a histogram of brightness values and a comparison made to determine if the two histograms are similar. In yet another embodiment, two-dimensional Fourier transforms of the image frames are compared to determine how similar the image frames are. Other more complicated and well-known image comparison algorithms such as SIFT, PCA-SIFT and SURF could also be used to compare image frames. Image frames that most closely match the target image frames are selected and presented to the operator to approve or reject.

If an examination requires imaging different portions of the body, then the proposed image frames can be selected by the system processor based on how different the image frames are. For example, if an examination is to image the heart, lungs and liver, the procedure begins by obtaining image frames of the heart which should look fairly similar to each other. Once the operator begins imaging another part of the body such as the lungs, the image frames will look quite different from the image frames of the heart. The system processor therefore executes program instructions that compare sequential frames from the cine buffer and selects an image frame that differs significantly from a previous frame. The sequential image frame comparison may also be used in conjunction with a comparison against one or more target image frames recalled from the image database. For example, if an examination starts with imaging the heart and then moves to the lungs, the first frame that looks substantially different than the image frames of the heart may be compared to a target image frame of a lung. Subsequent frames can also be compared to the target image frame of the lung so that not every frame in the cine buffer need be compared with the target frame.

In yet another embodiment, the ultrasound system stores locally or recalls from a remote computer system, image frames that were selected by the operator on one or more previous occasions when the operator performed the same type of ultrasound examination that is currently being performed. The previously selected image frames are then used as target image frames for comparison against the image frames that are stored in the cine buffer. Those image frames that bear the closest resemblance to the previously selected target image frames are presented by the system processor for the operator to approve or reject.

In yet another embodiment, the system processor analyzes the image frames stored in the cine buffer for the presence of an object. The object can be an anatomical structure such as a particular portion of the heart, a particular organ or portion thereof (e.g. a liver) whose ultrasound image can be characterized by shape or by echo characteristics. Alternatively, the object can be a surgical instrument, such as a needle, or drug such as anesthetic, that can be detected in an image based on its shape or echo characteristics. For example, needle detection typically involves the detection of a bright reflector in a linear path within an image frame. The image frames in the cine buffer that contain the object are presented to the operator for approval or rejection.

In yet another embodiment, image frames are selected that meet some predefined image criteria. For example if an image of a heart valve needs to occupy a certain size in an image frame, then the image processor can analyze the images for the presence of the heart valve and can calculate whether the size of the heart valve in the image meets the predefined criteria. If an image frame meets both these criteria, the image frame is presented to the operator for approval or rejection. Other image criteria such a Doppler measurement that exceeds or is below a threshold could also be used.

In yet another embodiment, image frames are selected that were obtained with some defined operating criteria of the ultrasound machine. For example, image frames may be required where the image depth is between <NUM> and <NUM>. If the image frames are recorded with meta data in the cine buffer that indicates the depth of focus, then the system processor, or other image processor, can select one or more image frames that were obtained at the desired depth of focus. Other imaging parameters can include power level of signals applied to the transducer or other characteristics that can be modified to obtain an image frame and are recorded by the ultrasound system.

Once the proposed image frames are identified, the image frames are presented to the operator. The operator can accept or reject the proposed image frames for inclusion in a patient or billing report. If one or more of the proposed the frames are rejected, the operator can select another frame(s) by browsing the frames stored in the cine buffer. As indicated above, in one embodiment, the operator can jump to a location in the cine buffer by selecting a proposed frame in order to view the image frames stored before and after the proposed image frame. The ultrasound system also provides the operator with one or more hardware or software controls by which the operator can scroll through the cine buffer in a manner similar to a video (e.g. run, pause, stop, fast forward, rewind etc.) and a control to select an image frame.

Image frames that are accepted are marked by the system processor for inclusion in a patient or other report. Marking can be performed in a variety of manners such as by changing meta data associated with an image frame. Alternatively, the system processor can keep a list of image frames associated with a particular examination that are to be included in a patient or other report.

In one embodiment, once the selected image frames have been identified, the image frames (which may or may not include the non-selected image frames) can be stored in the memory <NUM> or other memory for archival purposes. In one embodiment, the selected image frames are stored with a greater resolution than the non-selected frames to preserve as much detail as possible in the selected frames.

Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded on computer storage medium for execution by, or to control the operation of, data processing apparatus.

A computer storage medium can be, or can be included in, a computer-readable storage device, a computer-readable storage substrate, a random or serial access memory array or device, or a combination of one or more of them. The computer storage medium also can be, or can be included in, one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices).

The term "processor" encompasses all kinds of apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, a system on a chip, or multiple ones, or combinations, of the foregoing. The apparatus also can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or a combination of one or more of them.

To provide for interaction with a user, embodiments of the subject matter described in this specification can be implemented on a imaging system having a display device, e.g., an LCD (liquid crystal display), LED (light emitting diode), or OLED (organic light emitting diode) monitor, for displaying information to the operator and a keyboard and a pointing device, e.g., a mouse or a trackball, by which the operator can provide input to the computer. In some implementations, a touch screen can be used to display information and to receive input from a user. Other kinds of devices can be used to provide for interaction with a operator as well; for example, feedback provided to the operator can be any form of sensory feedback, e.g., visual feedback, auditory feedback, or tactile feedback; and input from the operator can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with an operator by sending documents to and receiving documents from a device that is used by the user; for example, by sending web pages to a web browser on a user's client device in response to requests received from the web browser.

From the foregoing, it will be appreciated that specific embodiments of the invention have been described herein for purposes of illustration, but that various modifications may be made without deviating from the scope of the invention. Accordingly, the invention is not limited except as by the appended claims.

Claim 1:
An ultrasound imaging system comprising:
a transducer (<NUM>) that is configured to transmit ultrasound signals to and receive ultrasound signals from a region of interest;
a receive circuitry (<NUM>, <NUM>) that is configured to convert the received ultrasound signals into first image frames of ultrasound data;
a display (<NUM>) to display the first image frames of ultrasound data;
a cine buffer (<NUM>) to store the first image frames of ultrasound data produced during an examination; and
a processor (<NUM>) that is configured to
receive an indication of a type of examination being performed by the operator;
analyze the first image frames stored in the cine buffer;
compare sequential first image frames from the cine buffer;
select one or more first image frames of the first image frames from the cine buffer that differ from a previous image frame that was stored in the cine buffer before the one or more first image frames;
recall one or more target image frames associated with the type of examination performed by the operator;
compare the first image frames from the cine buffer to the one or more target image frames; and
select the first image frames from the cine buffer that match with the one or more target image frames for presentation to the operator.