Patent Description:
Ultrasound is a useful, non-invasive imaging technique capable of producing real time images of internal structures within tissue. Ultrasound imaging has an advantage over X-ray imaging in that ultrasound imaging does not involve ionizing radiation. Some mobile ultrasound scanners, including app-based ultrasound scanners, require an add-on device that can act as both as a display and a control device. Examples of these add-on devices are mobile phones, tablets, laptops or desktop computers.

When using some ultrasound scanners, whether mobile or not, users are traditionally expected to select a preset depending on the part of the anatomy that is to be scanned. The preset is associated with a set of parameters that instruct the ultrasound scanner how to acquire and process the ultrasound data. The set of parameters for each preset is usually optimized for the particular body part to which the preset relates. There may be upwards of a hundred different parameters (including, for example, frequency, focal zones, line density, whether harmonic imaging is on, and the like) for each preset depending on the ultrasound scanner.

In some cases, for example in an emergency room, in a field hospital, or if a user is unfamiliar with the particular ultrasound scanner, the preset may be incorrectly selected. This may happen for various reasons. For example, this may unintentionally done by the operator, or it may be left on a prior setting, or it may be not set and the scanner left in a default mode. Additionally or alternatively, if different areas of the body need to be scanned in one session, the user may forget to switch the preset when moving to a different body area. The result may be, for example, that the ultrasound image that is generated is not optimal, and/or the ultrasound scanner uses more power than necessary.

There is therefore a need to ensure that a preset of an ultrasound scanner is correctly selected for the part of the anatomy that is being scanned.

The above background information is provided to reveal information believed by the applicant to be of possible relevance to the present invention. The embodiments discussed herein may address and/or ameliorate one or more of the aforementioned drawbacks identified above. The foregoing examples of the related art and limitations related thereto are intended to be illustrative and not exclusive. Other limitations of the related art will become apparent to those of skill in the art upon a reading of the specification and a study of the drawings herein.

<CIT> discloses an adaptive ultrasonic spatial compounding method in which the number of component ultrasonic images which are to be spatially compounded is varied in response to the type of scanning procedure.

<CIT> discloses a method and system for automatic control of image quality in imaging of an object using, for example, an ultrasound system.

<CIT> discloses an ultrasound scanner is equipped with one or more fuzzy control units that can perform adaptive system parameter optimization anywhere in the system.

The following drawings illustrate embodiments of the invention and should not be construed as restricting the scope of the invention in any way.

The term "AI model" means a mathematical or statistical model that may be generated through artificial intelligence techniques such as machine learning. For example, the machine learning may involve inputting labeled or classified data into a neural network algorithm for training, so as to generate a model that can make predictions or decisions on new data without being explicitly programmed to do so. Different software tools (e.g., TensorFlow™, PyTorch™, Keras™) may be used to perform machine learning processes.

The term "depth" when relating to an ultrasound image refers to a measure of how far into the structure being scanned (e.g., tissue or a phantom) a given ultrasound image shows.

The term "module" can refer to any component in this invention and to any or all of the features of the invention without limitation. A module may be a software, firmware or hardware module, and may be located, for example, in the ultrasound scanner, a display device or a server.

The term "network" can include both a mobile network and data network without limiting the term's meaning, and includes the use of wireless (e.g. <NUM>, <NUM>, <NUM>, <NUM>, WiFi™, Wi-MAX™, Wireless USB (Universal Serial Bus), Zigbee™, Bluetooth™ and satellite), and/or hard wired connections such as local, internet, ADSL (Asymmetrical Digital Subscriber Line), DSL (Digital Subscriber Line), cable modem, T1, T3, fiber-optic, dial-up modem, television cable, and may include connections to flash memory data cards and/or USB memory sticks where appropriate. A network could also mean dedicated connections between computing devices and electronic components, such as buses for intra-chip communications.

The term "operator" (or "user") may refer to the person that is operating an ultrasound scanner (e.g., a clinician, medical personnel, a sonographer, ultrasound student, ultrasonographer and/or ultrasound technician).

The term "processor" can refer to any electronic circuit or group of circuits that perform calculations, and may include, for example, single or multicore processors, multiple processors, an ASIC (Application Specific Integrated Circuit), and dedicated circuits implemented, for example, on a reconfigurable device such as an FPGA (Field Programmable Gate Array). A processor may perform the steps in the flowcharts and sequence diagrams, whether they are explicitly described as being executed by the processor or whether the execution thereby is implicit due to the steps being described as performed by the system, a device, code or a module. The processor, if comprised of multiple processors, may be located together or geographically separate from each other. The term includes virtual processors and machine instances as in cloud computing or local virtualization, which are ultimately grounded in physical processors.

The term "scan convert", "scan conversion", or any of its grammatical forms refers to the construction of an ultrasound media, such as a still image or a video, from lines of ultrasound scan data representing echoes of ultrasound signals. Scan conversion may involve converting beams and/or vectors of acoustic scan data which are in polar (R-theta) coordinates to cartesian (X-Y) coordinates.

The term "system" when used herein, and not otherwise qualified, refers to a system for controlling the settings of an ultrasound scanner using data obtained in control data frames acquired between image data frames, the system being a subject of the present invention. The system may include a scanner and a display device, or a scanner, display device and a server.

The term "ultrasound control data frame" (or "control data frame" for brevity) refers to a frame of ultrasound data that is captured by an ultrasound scanner. The ultrasound control data frame has the form of multiple lines of data that each represent echoes of ultrasound. Ultrasound control data frames may all be acquired with consistent, reference scan parameters, unlike ultrasound data frames, which may be acquired with different parameters depending on the settings of the ultrasound scanner. Ultrasound control data frames are not usually converted to viewable image frames.

The term "ultrasound data frame" (or "image data frame") refers to a frame of ultrasound data that is captured by an ultrasound scanner. The ultrasound data frame typically has the form of multiple lines of data that each represent echoes of ultrasound. Ultrasound data frames are usually acquired with different sets of scan parameters, where each set depends on which preset of the ultrasound scanner is selected. Ultrasound data frames are usually converted to viewable image frames for viewing by an operator of the ultrasound scanner.

The term "ultrasound image frame" (or "image frame") refers to a frame of post-scan conversion data that is suitable for rendering an ultrasound image on a screen or other display device.

Referring to <FIG>, an exemplary system <NUM> is shown for controlling the settings of an ultrasound scanner <NUM> (hereinafter "scanner" for brevity) dependent on interspersed control data frames. The system <NUM> includes an ultrasound scanner <NUM> with a processor <NUM>, which is connected to a non-transitory computer readable memory <NUM> storing computer readable instructions <NUM>, which, when executed by the processor <NUM>, may cause the scanner <NUM> to provide one or more of the functions of the system <NUM>. Such functions may be, for example, the acquisition of ultrasound data, the processing of ultrasound data, the conversion of ultrasound data, the transmission of ultrasound data or images to a display device <NUM>, the detection of operator inputs to the scanner <NUM>, and/or the switching of the settings of the scanner <NUM>.

Also stored in the computer readable memory <NUM> may be computer readable data <NUM>, which may be used by the processor <NUM> in conjunction with the computer readable instructions <NUM> to provide the functions of the system <NUM>. Computer readable data <NUM> may include, for example, configuration settings for the scanner <NUM>, such as presets that instruct the processor <NUM> how to collect and process the ultrasound data for a given body part. Such a preset may be selected, for example, depending on the processing of a control data frame against an AI model that is stored in the computer readable data <NUM>. A preset may include numerous different parameters for the scanner <NUM>.

The scanner <NUM> includes a communications module <NUM> connected to the processor <NUM>. In the illustrated example, the communications module <NUM> wirelessly transmits signals to and receives signals from the display device <NUM> along wireless communication link <NUM>. The protocol used for communications between the scanner <NUM> and the display device <NUM> may be WiFi™ or Bluetooth™, for example, or any other suitable two-way radio communications protocol. The scanner <NUM> may operate as a WiFi™ hotspot, for example. Communication link <NUM> may use any suitable wireless network connection. In some embodiments, the communication link between the scanner <NUM> and the display device <NUM> may be wired. For example, the scanner <NUM> may be attached to a cord that may be pluggable into a physical port of the display device <NUM>.

The display device <NUM> may be, for example, a laptop computer, a tablet computer, a desktop computer, a smart phone, a smart watch, spectacles with a built-in display, a television, a bespoke display or any other display device that is capable of being communicably connected to the scanner <NUM>. The display device <NUM> may host a screen <NUM> and may include a processor <NUM>, which is connected to a non-transitory computer readable memory <NUM> storing computer readable instructions <NUM>, which, when executed by the processor <NUM>, cause the display device <NUM> to provide one or more of the functions of the system <NUM>. Such functions may be, for example, the receiving of ultrasound data that may or may not be pre-processed; scan conversion of ultrasound data that is received into a ultrasound images; processing of ultrasound data in control data frames and/or image data frames; the display of an ultrasound image on the screen <NUM>; the display of a user interface; the control of the scanner <NUM>; and/or the storage, application, reinforcing and/or training of an AI model.

Also stored in the computer readable memory <NUM> may be computer readable data <NUM>, which may be used by the processor <NUM> in conjunction with the computer readable instructions <NUM> to provide the functions of the system <NUM>. Computer readable data <NUM> may include, for example, settings for the scanner <NUM>, such as presets for acquiring ultrasound data depending on the analysis of control data frames; settings for a user interface displayed on the screen <NUM>; and/or one or more AI models. Settings may also include any other data that is specific to the way that the scanner <NUM> operates or that the display device <NUM> operates.

It can therefore be understood that the computer readable instructions and data used for controlling the system <NUM> may be located either in the computer readable memory <NUM> of the scanner <NUM>, the computer readable memory <NUM> of the display device <NUM>, and/or both the computer readable memories <NUM>, <NUM>.

The display device <NUM> may also include a communications module <NUM> connected to the processor <NUM> for facilitating communication with the scanner <NUM>. In the illustrated example, the communications module <NUM> wirelessly transmits signals to and receives signals from the scanner <NUM> on wireless communication link <NUM>. However, as noted, in some embodiments, the connection between scanner <NUM> and display device <NUM> may be wired.

Referring to <FIG>, shown there generally is a schematic diagram showing a series of control and image frames and their analysis, according to an embodiment of the present invention. In an embodiment, an ultrasound image feed is acquired by obtaining ultrasound data frames that are converted to viewable image frames. During acquisition of the ultrasound image feed, additional ultrasound control data frames are acquired that are interspersed amongst the ultrasound data frames. The ultrasound control data frames use reference scan parameters that are consistent, regardless of whatever preset or settings the ultrasound scanner is set to. The ultrasound control data frames are not converted to image frames for display, and instead are used to control the settings of the ultrasound scanner. This is illustrated in <FIG>.

Furthermore, the acquisition of the ultrasound control data frames may not necessarily interrupt the regular refresh rate of the displayed ultrasound image feed.

In some cases, ultrasound data frames that are acquired are converted to optimized viewable image frames, which are processed against an additional AI model that identifies anatomical features in the optimized viewable image frames. These features are then highlighted on the displayed, optimized image frames.

In <FIG>, consecutive image data frames <NUM>, <NUM>, <NUM> are shown, followed by a control data frame <NUM>, which in turn is followed by two further image data frames <NUM>, <NUM>. The image data frames <NUM>-<NUM> are illustrated as a series of vertical scan lines that represent the image data that may be acquired by the scanner <NUM>. In the example, the image data frames <NUM>-<NUM> are also shown as pre scan-converted scan lines that, for example, have not yet been converted to the reflect a curvature of the transducer array of the ultrasound scanner <NUM>. Image data frame <NUM> is acquired using a first set of parameters, for example parameters that are set by preset P1. Image data frame <NUM> may then be scan converted into ultrasound image frame <NUM> for viewing. Likewise, image data frame <NUM> is acquired using the parameters for preset P1 and is also scan converted into ultrasound image frame <NUM> for viewing. Image data frame <NUM> is also acquired using the parameters for preset P1 and converted into ultrasound image frame <NUM> for viewing. Preset P1 is, in this example, not optimal for the acquisition of ultrasound data for the body part <NUM> displayed in ultrasound image frames <NUM>, <NUM>, <NUM>.

After the acquisition of a number of image data frames <NUM>, <NUM>, <NUM>, a control data frame <NUM> is acquired. The control data frame <NUM> uses reference parameters RP for acquiring the ultrasound data in the control data frame <NUM>. In general, the reference parameters RP are configured to be consistent, regardless of whatever preset or settings the ultrasound scanner is set to. This may mean that the reference parameters are different from the parameters of the preset P1 and the other presets that the scanner <NUM> may be capable of using. For example, as illustrated, the reference parameters used to acquire the control data frame <NUM> have a shallower depth of scan than the parameters for preset P1 used in the data frames <NUM>, <NUM>, <NUM>. Also, there are fewer data lines (e.g., less line density, as shown via the sparser vertical scan lines) in the control data frame <NUM> than the parameters for preset P1 used in the image data frames <NUM>, <NUM>, <NUM>.

The control data frame <NUM> are not converted to an image frame, but instead is input to an AI model <NUM> for processing. The AI model <NUM> may be provided on the scanner <NUM>, the display device <NUM> and/or a server that is accessible to either the scanner <NUM> or the display device <NUM>. The result of the processing against the AI model <NUM> is the prediction of a preset that is most suitable for the body part <NUM> that is being scanned. In the example of <FIG>, processing the control data frame <NUM> with the AI model results in a prediction that the control data frame <NUM> corresponds to a scan of a heart. As a result of this prediction, the AI model <NUM> may output an instruction to the scanner <NUM> to set a preset P2 (e.g., a cardiac preset) that is optimized for scanning a heart <NUM>. It can be seen that the parameters for P2 (e.g. as illustrated, the depth and density of the vertical scan lines) are different from the parameters for preset P1. As illustrated, they are also different from reference parameters RP for the control data frame <NUM>.

While not illustrated, other example parameters that may be different in different presets are the ultrasound energy and/or the average power usage of the scanner <NUM>. For example, it may be the case that the AI model <NUM> predicts the suitable preset for the control data frame <NUM> is an obstetrics/gynecology preset or an ocular preset, meaning that the control data frames <NUM> may be scanning fetal tissue or eye tissue. Since these presets are generally also associated with limiting the power output of the ultrasound energy used during scanning to enhance safety for more sensitive tissue types, changing to the preset by the AI model <NUM> may also involve lower the ultrasound energy used for acquisition. In this manner, the embodiments described herein may provide enhanced safety measures, in addition to providing enhancements to operator workflow that reduce the manual step of selecting a preset.

After having switched the scanner <NUM> to use the preset P2 predicted by the AI model <NUM>, the scanner <NUM> may continuing acquiring image data frames <NUM>, <NUM> in sequence using parameters for preset P2. These image data frames <NUM>, <NUM> are then scan converted to ultrasound image frames <NUM>, <NUM> respectively for viewing. As illustrated, the image frames <NUM>, <NUM> acquired using preset P2 may be better optimized for viewing of the imaged anatomy (e.g., when acquiring images using the cardiac preset, the heart is shown more fully in view with clearer lines) as compared to image frames <NUM>, <NUM>, <NUM> acquired using preset P1.

To predict the settings that would suitable for a new control data frame <NUM>, the AI model <NUM> may previously be generated using machine learning methods. For example, this may involve training the AI model with one or more datasets containing different classes of control data frames that have been labeled as being associated with various presets P1, P2, P3, P4 (shown in <FIG> with various label icons). These various presets may generally correspond to different types of anatomical features <NUM>, <NUM>, <NUM>, <NUM>. For example, the preset P1 may generally be for scanning lungs <NUM>, preset P2 may generally be for scanning cardiac features <NUM>, preset P3 may generally be for scanning bladders <NUM> , and preset P4 may generally be for scanning abdominal features such as kidneys <NUM> or livers <NUM>. In <FIG>, the anatomical features <NUM>, <NUM>, <NUM>, <NUM>, <NUM> that the various presets P1-P4 are respectively associated with are shown in dotted outline for illustrative purposes to provide a pictorial representation of the anatomical features; but such pictorial representations are not viewable ultrasound image frames.

The embodiments herein may generally involve using the manually-selected preset under which an ultrasound data frame or an ultrasound image frame is acquired to train an AI model to predict the preset that would be suitable for a new ultrasound data frame or ultrasound image frame. While using such data to train an AI model may be workable, it is recognized there may be a mismatch between the training data (which were already manually-selected to be acquired under an optimal preset) and the new data that the AI model is to provide a prediction on (which may be acquired under a set of different unknown parameters). This mismatch may reduce the reliability of the predictions made by the AI model.

To improve the reliability of the AI model, in some embodiments, consistent reference parameters RP are used to acquire the training data that is labeled and inputted into the AI model, as well as the new data that AI model is to predict the preset for. For example, this is shown in <FIG> where the various classes of the control data frames <NUM>, <NUM>, <NUM>, <NUM> in the one or more datasets are acquired using consistent reference parameters RP (which are generally similar to the reference parameters RP used to acquire the new control data frame <NUM> that the AI model <NUM> is to predict a preset for).

Notably, the consistent reference parameters RP are used even though the different presets P1-P4 (as associated with the different anatomical features <NUM>, <NUM>, <NUM>, <NUM>, <NUM>) typically have their own associated different optimal parameters. By configuring both the training control data frames <NUM>, <NUM>, <NUM>, <NUM> and a new control data frame <NUM> to be acquired using consistent reference parameters RP, there is no mismatch between the acquisition parameters of the training data and the data for which the AI model is to provide a prediction on. This may enhance the operation of the machine learning algorithms so that the AI model <NUM> can generate more reliable predictions about the preset that is suitable for a given new control data frame. This may also allow certain steps of normalizing the training data (e.g., for scan depth, resolution, contrast, brightness, image enhancements, noise reduction and the like) to be minimized. In various embodiments, the consistent reference parameters RP may have one or more of: a fixed depth, a fixed number of acquisition lines, fixed focal zones, a fixed sampling rate, fixed gain, fixed beamformer parameters, or fixed application of filters.

As shown in <FIG>, the training is performed on control data frames <NUM>, <NUM>, <NUM>, <NUM> similar to how a control data frame <NUM> (as opposed to a viewable ultrasound image frame) is fed into the AI model <NUM> for prediction. By performing the machine learning on pre-scan converted data frames <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, the act of scan converting control data frames into viewable ultrasound image frames may be avoided. This may enhance computational efficiency by reducing the computational effort required to perform scan conversion. Also, since pre-scan converted control data frames generally have a smaller memory footprint than post-scan converted ultrasound image frames, this may allow for increased throughput of various machine learning processes.

Notwithstanding, it is not required that the machine learning techniques described herein be performed on pre-scan converted data. In some embodiments, the methods described herein may be performed on post-scan converted images (e.g., on image data after the control data frames <NUM>, <NUM>, <NUM>, <NUM>, <NUM> are scan converted). In various embodiments, data frames (whether control data frames <NUM>, <NUM>, <NUM>, <NUM> or scan converted image data frames ) used as training data for the AI model <NUM> may include greyscale data. In various embodiments, these data frames used for training may also include Doppler data.

In various embodiments, the training control data frames <NUM>, <NUM>, <NUM>, <NUM> may be acquired using the same model of ultrasound scanner <NUM> on which the AI model <NUM> will be deployed. However, in some embodiments, the model of ultrasound scanner used to acquire the training control data frames <NUM>, <NUM>, <NUM>, <NUM> may differ from the model of ultrasound scanner <NUM> on which the AI model <NUM> is deployed (e.g., different manufacturer or design). This may be possible because consistent reference parameters RP are used for both the training control data frames <NUM>, <NUM>, <NUM>, <NUM> and the new control data frame <NUM>.

In some embodiments, the different models of ultrasound scanners <NUM> may even have different transducer array footprints (e.g., linear, curvilinear, or microconvex, or phased array) and/or different frequency ranges. This may be accomplished by configuring the control data frame <NUM> to use reference parameters RP that only acquire data lines from a center portion of the transducer array that are common to all transducer array footprint types, and by selecting an imaging frequency and depth that is common or overlaps amongst the different scanner types. Since the machine learning may be performed on pre-scan converted control data frames so that do not reflect any curvature of the transducer array footprint, using consistent reference parameters RP may allow the control data frame <NUM> acquired by the various scanner types to share enough common characteristics, so as to allow their various classifications to be applicable to other scanner models.

By controlling the parameters in this manner, the generated AI model <NUM> may be sufficiently robust to predict presets for a new control data frame <NUM> acquired from a scanner model <NUM> that is different from that which is used to acquire the training control data frames <NUM>, <NUM>, <NUM>, <NUM>.

As illustrated in <FIG>, the AI model <NUM> may be trained with classes of control data frames <NUM>, <NUM>, <NUM>, <NUM> that correspond to presets P1-P4 for scanning a single type of anatomy (e.g., for lungs <NUM>, cardiac <NUM>, or bladders <NUM>), or multiple types of anatomy (e.g., an abdomen preset which may suitable for scanning kidneys <NUM> and livers <NUM>).

In various embodiments, the different classes of ultrasound control data frames used for the AI model may generally include ultrasound data acquired for one or more anatomical features; such anatomical features including a lung, a heart, a liver, a kidney, a bladder, an eye, a womb, a thyroid gland, a breast, a brain, an artery, a vein, a muscle, an embryo, a tendon, a bone, a fetus, a prostate, a uterus, an ovary, testes, a pancreas, or a gall bladder.

In various embodiments, the different presets for which there may be labeled training control data frames may generally include presets for at least two of abdomen, cardiac, bladder, lung, obstetrics/gynecology, transcranial, superficial, thyroid, vascular, musculoskeletal, breast, ocular, prostate, fertility, or nerve.

Referring still to <FIG>, the control data frames <NUM> that are collected are interspersed amongst the image data frames <NUM>, <NUM>, <NUM>, <NUM>, <NUM>, and they may continue to be acquired in an interspersed fashion as the ultrasound scan proceeds. For example, there may be one control data frame <NUM> acquired for every three image data frames <NUM>, <NUM>, <NUM>. Other interspersion rates are possible, and an interspersion rate may change during use of the scanner <NUM> (e.g., in some embodiments, control data frames <NUM> may be interleaved with image data frames <NUM>, <NUM>, <NUM>, <NUM>, <NUM>).

The acquisition of the control data frames <NUM> may be configured to have minimal impact on the refresh rate of the image frames <NUM>, <NUM>, <NUM>, <NUM>, <NUM>. In part this is because the processing of the control data frame <NUM> with the AI model <NUM> is performed on pre-scan converted image data, before being scan converted and placed into the image buffer. As a result, the image buffer may only have (not necessarily at the same time) image frames <NUM>, <NUM>, <NUM>, <NUM>, <NUM> that are going to be displayed, at regular intervals, so that the image refresh rate is uniform and the image feed appears smooth, without the acquisition and processing of the control data frames <NUM> causing any significant pause or interruption. The average acquisition rate of the image data frames, however, should be about equal to the refresh rate of the image that is displayed so that there is always enough data in the image buffer that the image refresh rate is not interrupted.

In various embodiments, the consistent reference parameters RP may be optimized to sufficiently allow for the machine learning processes described herein, while reducing the impact of the control data frames <NUM> on the image quality of ultrasound image frames <NUM>, <NUM>, <NUM>, <NUM>, <NUM>. For example, the various parameters of the reference parameters RP may be configured to consume less resources (e.g., fewer lines, focal zones, or the like) so as to reduce their acquisition time, and their potential negative impact on frame rate.

Additionally or alternatively, in some embodiments, the reference parameters RP can be configured to minimize acquisition time of the control data frame <NUM> (e.g., by reducing the number of acquisition lines in a control data frame <NUM> versus the number of acquisition lines in an image data frame <NUM>, <NUM>, <NUM>, <NUM>, <NUM>).

In some embodiments, during operation of the scanner <NUM>, the user may be presented with a set of presets and an auto-preset option. While each preset is suited for a particular part of the anatomy, the auto-preset option, if selected, will automatically predict and select the optimum preset using the AI model <NUM> as described herein.

In some embodiments, the capture and analysis of the control data frame <NUM> may occur fast enough for the settings of the scanner <NUM> to be changed in time for the acquisition of the immediately following image data frame <NUM>.

In some cases, the preset on the scanner <NUM> may be changed based on the predicted preset for a single control data frame <NUM>. However, in some embodiments, the preset on the scanner <NUM> may be changed only after multiple control data frames <NUM> result in the same predicted preset. In this latter scenario, different configurations are possible. For example, it may be required that some consecutive number of control data frames <NUM> predict the same preset prior to the scanner <NUM> changing its preset. In another embodiment, the scanner <NUM> may change its preset after some percentage (e.g., <NUM>-<NUM>%) of a past number of control data frames <NUM> provide the same predicted preset form the AI model <NUM>.

Referring to <FIG>, a flowchart shows an exemplary process undertaken by the system <NUM> (as shown in <FIG>), in which the scanner settings are updated as a result of analysis of a control data frame <NUM>. In discussing <FIG>, reference will also be generally made to the sequence of various frames shown in <FIG>. In step <NUM>, an image data frame counter is set to zero (i=<NUM>). In step <NUM>, an image data frame <NUM> is acquired using whatever the current settings of the scanner <NUM> are. For example, the settings may be default settings, a particular preset, or settings that have been made manually. In step <NUM>, the data in the image data frame <NUM> are scan converted into a form suitable for display, following which the image frame <NUM> is displayed in step <NUM>. After the display of the image frame <NUM>, the image data frame counter may be incremented in step <NUM>. In step <NUM>, the current value i of the data frame counter is compared to a limit value n. If the value of the data frame counter is not yet equal to the limit value n, then the process reverts to step <NUM> in which another image data frame <NUM> is acquired. Steps <NUM>-<NUM> are repeated, with subsequent image data frames being acquired, scan converted and displayed until the image data frame counter equals the limit value (i=n) in step <NUM>.

When a series of n image data frames have been acquired (i=n), the process moves onto step <NUM>, in which a control data frame <NUM> is acquired. The control data frame <NUM> is acquired using the reference scan parameters, or reference settings, which are typically not the same as whatever the current settings were for step <NUM>. In step <NUM>, the control data present in the control data frame <NUM> is analyzed, for example by processing it against the AI model <NUM>. As a result of this processing, the AI model <NUM> predicts, in step <NUM>, the optimal settings for the scanner <NUM> for the particular body part that is currently being scanned. In step <NUM>, the settings of the scanner <NUM> are updated. Updating the settings of the scanner <NUM> entails changing from one preset to another preset (e.g., changing the settings from one value or set of values to another value or set of values). In other cases, updating the settings of the scanner <NUM> entails changing the existing settings of the scanner <NUM> to those of a preset. In still other cases, the updating of the scanner settings is to confirm that the present settings are already optimal and do not yet need to be changed.

After the scanner settings have been updated in step <NUM>, which may or may not involve an actual change of the settings, the process repeats from step <NUM>.

Referring to <FIG>, a flowchart is shown for training the AI model <NUM>. When discussing <FIG> below, reference will also be made to certain elements of <FIG>. In step <NUM>, a preset of the scanner <NUM> is manually selected according to a type of body part that is to be scanned. In step <NUM>, the scanner <NUM> acquires image data frames (e.g. <NUM>, <NUM>) and control data frames <NUM> that are interspersed amongst the image data frames. The control data frames <NUM> are then labeled, in step <NUM>, as being associated with the manually selected preset. The labeled control data frames are then sent to a server, in step <NUM>, where they are saved as training control data frames (e.g., control data frames <NUM>, <NUM>, <NUM>, <NUM> as shown in <FIG>). These training control data frames are accessible to the AI model <NUM> for further training and/or re-enforcement. In step <NUM>, the AI model <NUM> is trained using the labeled control data frames that are stored in the server.

A common challenge in machine learning activities is obtaining labeled data that can be used to train an AI model <NUM>. For example, if using a supervised learning technique, traditionally, human involvement may be needed to label which control data frames should be associated with which presets to generate suitable dataset that can be used to train the AI model. Such manual review and labeling is laborious, so as to make it difficult to create a large and robust dataset. However, by applying the method of <FIG> and inserting a control data frame <NUM> into the regular scanning activity performed by operators when using a scanner <NUM>, the manual selection of the preset by the operators may be used as the human data labeling activity. For example, the method of <FIG> may be deployed on scanners <NUM> that do not have the AI model <NUM> enabled, so as to collect training data based on the presets selected by the operator. Then, once sufficient training data has been obtained and the AI model <NUM> trained, the AI model <NUM> may be deployed to enable the AI model-enabled preset prediction methods described herein.

Referring to <FIG>, a flowchart is shown for using and reinforcing the AI model <NUM>. In discussing <FIG>, reference will again also be made to the elements of <FIG>. In some embodiments, during continued use of the ultrasound scanner <NUM> after a preset P2 has been predicted by the AI model <NUM> based on control data frame <NUM>, subsequently acquired ultrasound control data frames are used for further training or reinforcement of the AI model <NUM>.

In step <NUM> of <FIG>, the control data in a control data frame <NUM> is processed against the AI model <NUM>. In step <NUM>, the AI model <NUM> predicts the optimal preset, and the preset of the scanner <NUM> is changed in step <NUM>. On a continuing basis, the control data acquired in subsequent interspersed control data frames is monitored may be step <NUM>, by processing it against the AI model <NUM>. For example, in the example of <FIG>, acts <NUM>-<NUM> may be performed on a control data frame <NUM> against AI model <NUM>. Referring back to <FIG>, if, in step <NUM>, after a period of time, the subsequent control data frames still correspond to the changed preset, then the subsequently obtained control data frames are labeled, in step <NUM>, as corresponding to the preset changed to in step <NUM>. In step <NUM>, the labeled control data frames are then sent for storage in a location that is accessible by the AI model <NUM> for further training or reinforcement.

In some embodiments, in the auto-preset mode, the user interface on the display device <NUM> may be configured to show the new preset each time the preset is changed, and display an option for the user to cancel the change for a few seconds after the change. If used, the cancellation may be considered to be training data for the AI model <NUM>. In the case of cancellation, the control data frames collected after the cancellation may be labeled as not corresponding to the preset predicted by the AI model <NUM> and sent to the server. If, within some period of time, a user manually selects a different preset after cancellation and the control data frames <NUM> continue to be similar to what was being acquired prior to cancellation, then the user-selected preset may also serve as training data for the AI model <NUM>.

For example, it may be the case that the class of cardiac preset-related data in the original dataset used to train the AI model <NUM> lacks a four-chamber cardiac view, such that if control data frames for such an image is acquired for the first time during scanning, the AI model <NUM> fails to accurately predict use of the cardiac preset. If the user manually selects the cardiac preset while control data frames for such an image is being acquired (e.g., after cancelling selecting of a preset predicted by the AI model <NUM>), then the control data frames for the four-chamber cardiac view may be added as training data so that the AI model <NUM> may learn that such control data frames are associated with a cardiac preset.

Referring to <FIG>, a flowchart is shown for identifying particular features in an ultrasound image <NUM>, <NUM>. In discussing <FIG>, reference will also be made to the elements of <FIG>. Once the AI model <NUM> has predicted the optimal preset for the scanner <NUM>, and the scanner <NUM> has been set to the predicted preset, then it may be possible to analyze the image frame <NUM>, <NUM> that is used for viewing by an operator of the scanner <NUM>. This further analysis may use an additional, separate AI model, which may be referred to as a micro-AI model, with the AI model <NUM> being referred to as a macro-AI model. The system <NUM> therefore may use two different AI models simultaneously, each applied to a different level of the ultrasound image acquisition process, the first being applied to control data frames <NUM> and the second being applied to image frames <NUM>, <NUM> after the settings have been optimized using the first AI model.

In step <NUM> (which may, for example, be equivalent to multiple instances of step <NUM> in the method of <FIG> after cycling through steps <NUM> or <NUM>), the system <NUM> may continue to acquire image data frames <NUM>, <NUM> after the preset of the scanner <NUM> has been updated. As each image data frame is acquired, the scanner <NUM> may scan convert the image data frames to viewable image frames <NUM>. <NUM>, in step <NUM>. The viewable image frames <NUM>, <NUM> may then be processed against a second AI model (step <NUM>) that is trained to identify (e.g., segment) anatomical features in the image frames <NUM>, <NUM>. In step <NUM>, the second AI model may identify one or more anatomical features in the image frames <NUM>, <NUM>. The image frames <NUM>, <NUM> may then be displayed with additional highlights, in step <NUM>, to show where the identified anatomical features are in the images. Optionally, the highlighted areas may be annotated with the name of the anatomical feature or features that have been identified.

Referring to <FIG>, a system <NUM> is shown in which there are multiple similar or different scanners <NUM>, <NUM>, <NUM> connected to their corresponding display devices <NUM>, <NUM>, <NUM> and either connected directly, or indirectly via the display devices, to a network <NUM>, such as the internet. The scanners <NUM>, <NUM>, <NUM> may be connected onwards via the network <NUM> to a server <NUM>.

The server <NUM> includes a processor <NUM>, which is connected to a non-transitory computer readable memory <NUM> storing computer readable instructions <NUM>, which, when executed by the processor <NUM>, cause the server <NUM> to provide one or more of the functions of the system <NUM>. Such functions are, for example, the receiving of ultrasound data that may or may not be pre-processed, the scan conversion of ultrasound data that is received into an ultrasound image, the processing of ultrasound data in control data frames or image data frames, the control of the scanners <NUM>, <NUM>, <NUM>, and/or machine learning activities related to one or more AI models. Such machine learning activities may include the training and/or reinforcing of one or more AI models.

Also stored in the computer readable memory <NUM> may be computer readable data <NUM>, which are used by the processor <NUM> in conjunction with the computer readable instructions <NUM> to provide the functions of the system <NUM>. Computer readable data <NUM> includes, for example, settings for the scanners <NUM>, <NUM>, <NUM> such as preset parameters for acquiring ultrasound data depending on the analysis of control data frames, settings for user interfaces displayed on the display devices <NUM>, <NUM>, <NUM>, and one or more AI models. For example, one AI model may be the AI model <NUM> that is used to analyze the control data frames <NUM>, while another AI model may be used to analyze image frames <NUM>, <NUM> for identifying anatomical features in the image frames <NUM>, <NUM>. Settings may also include any other data that is specific to the way that the scanners <NUM>, <NUM>, <NUM> operate or that the display devices <NUM>, <NUM>, <NUM> operate.

It can therefore be understood that the computer readable instructions and data used for controlling the system <NUM> may be located either in the computer readable memory of the scanners <NUM>, <NUM>, <NUM>, the computer readable memory of the display devices <NUM>, <NUM>, <NUM>, the computer readable memory <NUM> of the server <NUM>, or any combination of the foregoing locations.

As noted above, even though the scanners <NUM>, <NUM>, <NUM> may be different, the control data frames that are captured by them are all captured with consistent reference parameters RP, so that each control data frame acquired may be used by the AI model <NUM> for training, without any special pre-processing of the captured data. Likewise, the control data frames acquired by the individual scanners <NUM>, <NUM>, <NUM> may all be processed against the AI model <NUM> directly for prediction of the optimal presets and/or for reinforcement of the AI model <NUM>.

In some embodiments, AI models <NUM> present in the scanner <NUM> may be updated from time to time from an AI model present in the server <NUM>.

In some embodiments, the analysis of the control data frames may be performed using a rules-based engine rather than an AI model.

Embodiments of the invention may be implemented using specifically designed hardware, configurable hardware, programmable data processors configured by the provision of software (which may optionally include 'firmware') capable of executing on the data processors, special purpose computers or data processors that are specifically programmed, configured, or constructed to perform one or more steps in a method as explained in detail herein and/or combinations of two or more of these. Examples of specifically designed hardware are: logic circuits, application-specific integrated circuits ("ASICs"), large scale integrated circuits ("LSIs"), very large scale integrated circuits ("VLSIs") and the like. Examples of configurable hardware are: one or more programmable logic devices such as programmable array logic ("PALs"), programmable logic arrays ("PLAs") and field programmable gate arrays ("FPGAs"). Examples of programmable data processors are: microprocessors, digital signal processors ("DSPs"), embedded processors, graphics processors, math co-processors, general purpose computers, server computers, cloud computers, main computers, computer workstations, and the like. For example, one or more data processors in a control circuit for a device may implement methods as described herein by executing software instructions in a program memory accessible to the processors.

While processes or blocks are presented in a given order, alternative examples may perform routines having steps, or employ systems having blocks, in a different order, and some processes or blocks may be deleted, moved, added, subdivided, combined, and/or modified to provide alternative or subcombinations. Each of these processes or blocks may be implemented in a variety of different ways. Also, while processes or blocks are at times shown as being performed in series, these processes or blocks may instead be performed in parallel, or may be performed at different times.

The embodiments may also be provided in the form of a program product. The program product may include any non-transitory medium which carries a set of computer-readable instructions which, when executed by a data processor, cause the data processor to execute a method of the invention. Program products according to the invention may be in any of a wide variety of forms. The program product may include, for example, non-transitory media such as magnetic data storage media including floppy diskettes, hard disk drives, optical data storage media including CD ROMs, DVDs, electronic data storage media including ROMs, flash RAM, EPROMs, hardwired or preprogrammed chips (e.g., EEPROM semiconductor chips), nanotechnology memory, or the like. The computer-readable signals on the program product may optionally be compressed or encrypted.

Where a component (e.g. software, processor, support assembly, valve device, circuit, etc.) is referred to above, unless otherwise indicated, reference to that component (including a reference to a "means") should be interpreted as including as equivalents of that component any component which performs the function of the described component (i.e., that is functionally equivalent), including components which are not structurally equivalent to the disclosed structure which performs the function in the illustrated exemplary embodiments of the invention.

Specific examples of systems, methods and apparatus have been described herein for purposes of illustration. These are only examples. The technology provided herein can be applied to systems other than the example systems described above. Many alterations, modifications and additions are possible within the practice of this invention. This invention includes variations on described embodiments that would be apparent to the skilled addressee, including variations obtained by: mixing and matching of features, elements and/or acts from different embodiments; combining features, elements and/or acts from embodiments as described herein with features, elements and/or acts of other technology. In some embodiments, the components of the systems and apparatuses may be integrated or separated. In other instances, well known elements have not been shown or described in detail and repetitions of steps and features have been omitted to avoid unnecessarily obscuring the invention. Screen shots may show more or less than the examples given herein. Accordingly, the specification is to be regarded in an illustrative, rather than a restrictive, sense.

It is therefore intended that the appended claims and claims hereafter introduced are interpreted to include all such modifications, permutations, additions, omissions and sub-combinations as may reasonably be inferred. The scope of the claims should not be limited by the embodiments set forth in the examples but should be given the broadest interpretation consistent with the description as a whole.

Unless the context clearly requires otherwise, throughout the description and the claims, the following applies:.

In general, unless otherwise indicated, singular elements may be in the plural with no loss of generality. The use of the masculine can refer to masculine, feminine or both.

The terms "comprise", "comprising" and the like are to be construed in an inclusive sense, as opposed to an exclusive or exhaustive sense, that is to say, in the sense of "including, but not limited to".

The terms "connected", "coupled", or any variant thereof, means any connection or coupling, either direct or indirect, between two or more elements; the coupling or connection between the elements can be physical, logical, or a combination thereof.

The words "herein," "above," "below" and words of similar import, when used in this application, refer to this application as a whole and not to any particular portions of this application.

The word "or" in reference to a list of two or more items covers all of the following interpretations of the word: any of the items in the list, all of the items in the list and any combination of the items in the list.

Claim 1:
A method for controlling settings of an ultrasound scanner, the method comprising:
acquiring an ultrasound image feed according to a first preset by sequentially obtaining ultrasound data frames that are converted to viewable image frames; and
during acquisition of the ultrasound image feed:
acquiring, using reference scan parameters, ultrasound control data frames that are interspersed amongst the ultrasound data frames, the reference scan parameters being consistently used for the interspersed ultrasound control data frames regardless of scan parameters that are used for acquiring the ultrasound image feed, and wherein the ultrasound control data frames are not converted to viewable image frames; and
using the ultrasound control data frames to control the settings of the ultrasound scanner, said control of the settings of the ultrasound scanner comprising changing the first preset to a second preset different from the first preset;
wherein prior to changing the first preset to the second preset, at least the latest of the ultrasound control data frames is processed against an artificial intelligence model to predict a suitable preset for the ultrasound image feed, and the predicted suitable preset is used as the second preset that the ultrasound scanner is changed to.