Patent ID: 12243227

DETAILED DESCRIPTION

The following detailed description of the present disclosure is made with reference to the accompanying drawings, in which particular embodiments for practicing the present disclosure are shown for illustrative purposes. These embodiments are described in sufficiently detail for those skilled in the art to practice the present disclosure. It should be understood that various embodiments of the present disclosure are different but do not need to be mutually exclusive. For example, particular shapes, structures, and features described herein in connection with one embodiment may be embodied in another embodiment without departing from the spirit and scope of the present disclosure. It should be further understood that changes may be made to the positions or placement of individual elements in each disclosed embodiment without departing from the spirit and scope of the present disclosure. Accordingly, the following detailed description is not intended to be taken in limiting senses, and the scope of the present disclosure, if appropriately described, is only defined by the appended claims along with the full scope of equivalents to which such claims are entitled. In the drawings, similar reference signs denote the same or similar functions in many aspects.

The terms as used herein are general terms selected as those being now used as widely as possible in consideration of functions, but they may differ depending on the intention of those skilled in the art or the convention or the emergence of new technology. Additionally, in certain cases, there may be terms arbitrarily selected by the applicant, and in this case, the meaning will be described in the corresponding description part of the specification. Accordingly, it should be noted that the terms as used herein should be interpreted based on the substantial meaning of the terms and the context throughout the specification, rather than simply the name of the terms.

Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

Real-Time Acoustic Simulation Method Based on Artificial Intelligence

FIG.1is a flowchart illustrating the real-time acoustic simulation method based on artificial intelligence according to an embodiment.

Referring toFIG.1, to begin with, the step of acquiring medical image data of a patient's target area to be treated is performed (S100). The medical image data is used to determine ultrasound parameters or generate an acoustic simulation image, and may be used to train an artificial intelligence model.FIG.2shows the type of exemplary medical image data. As shown, the medical image data includes a computed tomography (CT) image or a magnetic resonance imaging (MRI) image. According to an embodiment, the method may further include preprocessing the acquired image data, and the preprocessing process includes a process of dividing each image into a region of interest and background and removing the background that is not necessary for training.

Subsequently, the step of determining ultrasound parameters related to the output of an ultrasonic transducer based on the medical image data (S200) is performed. In this step, a variety of variables of ultrasound that will be outputted from the transducer, such as frequency, waveform, and intensity, are determined based on the information, such as the shape and thickness of the skull, which can be identified from the CT or MRI image.

Subsequently, the step of inputting the determined ultrasound parameters to a numerical model and generating a numerical model-based acoustic simulation image for a specific position of the ultrasonic transducer (S300) is performed. The numerical model is configured to perform a preset calculation on an input value and output a corresponding result value, and in the specification, the numerical model refers to a system that outputs simulation results of the sonication path and focal position based on the input medical image and the determined ultrasound parameter (factor related to the output of the transducer) information.

FIG.3shows a process of acquiring the acoustic simulation results from the input CT image using the numerical model. As shown, when the CT image ICTshowing the ultrasonic transducer disposed at the specific position and the skull shape is inputted to the numerical model, estimation results of the travel path, acoustic pressure level, and focal position of the output ultrasound of a specific parameter from the transducer of the corresponding position are outputted in the form of an image. The image acquired by the above-described process is referred to as a ‘numerical model-based acoustic simulation image SN’. As described above, the numerical model-based simulation requires a relatively long time of about 30 seconds to 10 minutes per transducer position.

Subsequently, the step of training the artificial intelligence model using the medical image data and the numerical model-based acoustic simulation image (S400) is performed. Machine learning is used for a computer to cluster or classify things or data, and typically includes support vector machines (SVM) and Neural Networks. The present disclosure employs a Generative Adversarial Network (GAN) model that improves the performance by adversarial training of a generator and a discriminator.

The artificial intelligence model according to an embodiment may be trained to extract features from an input image, generate an acoustic simulation image corresponding to the input image through trained nonlinear mapping, and output more precise results by comparing the generated simulation image with the real simulation image based on the numerical model.

FIG.4shows a process of acquiring the acoustic simulation results from the input CT image using the artificial intelligence model. In the same way as the numerical model, when the CT image ICTshowing the ultrasonic transducer disposed at the specific position and the skull shape is inputted to the artificial intelligence model, estimation results of the sonication path, acoustic pressure level, and focal position of the output ultrasound of a specific parameter from the transducer of the corresponding position are outputted in the form of an image. The image acquired by the above-described process is referred to as an ‘artificial intelligence model based acoustic simulation image SAI’. In the same way as the numerical model, the estimated acoustic simulation results are outputted, but the corresponding image is created through nonlinear mapping based on training, not direct calculation, so the required time is much shorter than the numerical model, which makes real-time simulation possible.

FIG.5shows a 3D image of the acoustic simulation results based on the artificial intelligence model according to an embodiment. As shown, information that changes as the output ultrasound of the transducer goes through the skull, such as the sonication path, focal position, and acoustic pressure level at each position, is displayed in color.

FIG.6shows a process of training the artificial intelligence model according to an embodiment. Referring toFIG.6, the training process of the artificial intelligence model includes (i) inputting the medical image data ICTand the numerical model based acoustic simulation image SNas training data; (ii) generating the acoustic simulation image SAIcorresponding to the specific position of the ultrasonic transducer using a generator; (iii) acquiring error data by comparing the acoustic simulation image SAIgenerated by the generator with the numerical model based acoustic simulation image SNusing a discriminator; and (iv) training the generator using the error data. The artificial intelligence model has improved performance by iteratively performing the training step, thereby reducing differences between the acoustic simulation image SAIgenerated by the generator and the numerical model-based acoustic simulation image SN.

According to an embodiment, the generator may include multiple layers, including at least one convolutional layer and at least one transposed convolutional layer. The convolutional layer receives input medical image data and outputs a feature map that emphasizes the features of a region of interest. Specifically, the convolutional layer outputs the feature map that emphasizes the features of an image area by multiplying the input data with filters while the filters move with a predetermined stride. As the image goes through the convolutional layer, the width, height, and depth of the image gradually decrease and the number of channels increases. The values of the filters include weight parameters, and the values of the filters are randomly set at the initial step, and in the training step, they are updated for optimization through error backpropagation (updating the weight parameters by propagating the output error of the output layer to the input layer).

The transposed convolutional layer is a layer that learns a process of synthesizing the feature maps extracted by the convolutional layer into a target output image and restoring the size (upsampling). The transposed convolutional layer outputs the feature maps by multiplying input data with filters while the filters move with a predetermined stride, and transposes the input/output size as opposed to the convolutional layer. That is to say, as the image goes through the transposed convolutional layer, the width, height, and depth of the image gradually increase, and the number of channels decreases. A new image (for example, an acoustic simulation image) is generated based on the extracted features by transposing the convolutional layer.

According to an embodiment, the convolutional layer or the transposed convolutional layer of the generator may be used together with batch normalization for normalizing the data distribution of the feature maps and an activation function for determining the range of each output value. The batch normalization serves to prevent overfitting in which the filter values (weights) of convolution or transposed convolution fit training data well and perform worse on test data in the training process, thereby stabilizing the training process. The feature maps are normalized through the mean and the standard deviation (as much as batch fed into the model) to stabilize the data distribution of the feature maps. After training, the actual input test data is also equally normalized through the mean and the standard deviation stored during the training process, thereby generating an output image from data having a different distribution from the training data more stably.

When combined with the convolutional layer or the transposed convolutional layer, the activation function determines the range of the output value that will be passed from each layer to another layer and sets a threshold value for deciding which values will be passed. Additionally, nonlinearity is added to a deep learning model, and by the addition of the nonlinearity, the layers of the deep learning model become deeper, and derivative values are very small and converge to 0, thereby reducing the Gradient vanishing effect that the weight parameters are not updated. The activation function may include, for example, ReLu activation function that when input data is equal to or smaller than 0, outputs 0, and when input data is larger than 0, preserves the value, LeakyReLu activation function, which plays a similar role to the ReLu activation function, but when the input value is smaller than 0, multiplies the input value by 0.1 to have a non-zero output, and when the input value is larger than 0, preserves the value, Tan h activation function that causes input data to have a value between −1 and 1, and Sigmoid activation function that causes input data to have a value between 0 and 1.

The discriminator may include at least one convolutional layer to receive the input acoustic simulation image generated by the generator and output a feature map that emphasizes features of a region of interest. As opposed to the generator, each convolutional layer of the discriminator is configured to receive the input acoustic simulation image and output the feature map that emphasizes features of a region of interest. In the same way as the generator, each convolutional layer may be used together with batch normalization for normalization of the data distribution of the feature map and an activation function for determining the range of each output value.

According to an embodiment, the generator may include at least one residual block layer, and since the deeper layers of the model make optimization more difficult, the residual block layer serves to facilitate the model training. The residual block layer is repeated between the convolutional layer (Encoder), which reduces the width and height of the image and widens channelwise, and the transposed convolutional layer (Decoder), which restores the width and height of the image and the channels into the original dimensions. One residual block includes convolution-batch normalization-ReLu activation-convolution-batch normalization, and here, the convolution outputs an image having the same width, height, depth, and channel size as the input image through adjustment of the filters and the stride values. That is, it is aimed at passing the input data to the next layer with minimized information loss, but not extracting or restoring the features of the input data. For example, the residual block is set in such a way that the input x value is added to the output of the residual block, which induces the learning of a difference F(x) between input x and output H(x), not the output H(x) of the input data x. Accordingly, the previously learned input data x is taken as it stands and added to the output, so that only residual information F(x) is learned, thereby simplifying the model training process.

The generator is trained through differences between the output simulation images of the discriminator. That is, the simulation image SAIgenerated by the generator is compared with the simulation image SNgenerated based on the numerical model, and the comparison results are returned to the generator to improve the performance of the generator so as to output subsequent results that are more similar to the numerical model-based simulation image.

As described above, in the GAN model, the generator may synthesize an image through nonlinear mapping, and the discriminator may classify the generated image and a target image to be compared and generate a more precise image (i.e., a synthetic image close to the target image to be compared) with the increasing adversarial training iterations.

Referring back toFIG.1, the step of generating an artificial intelligence model-based acoustic simulation image for an arbitrary position of the ultrasonic transducer using the trained artificial intelligence model (S500) is performed. Here, the arbitrary position refers to all transducer positions at which numerical model-based simulation has not been performed. That is, as a result of training the artificial intelligence model based on the simulation results for some preset positions, it is possible to output corresponding simulation images in real-time for all unlearned positions.

FIG.7shows the possible position range of the ultrasonic transducer and the positions at which simulation is performed using the numerical model. The blue area indicates the full range of motion of the transducer for ultrasound treatment, and the white points indicate a position (a center position of the transducer) at which ultrasound simulation is performed using the numerical model. The position indicated by the red arrow is a position at which numerical model-based simulation has not been performed, and with the trained artificial intelligence model, it is possible to know acoustic simulation results for the uncalculated position.

FIG.8shows that the acoustic simulation results of the artificial intelligence model get closer to the simulation results based on the numerical model with the increasing training iterations. The simulation results when training is performed for 100, 200, and 360 iterations are shown in that order from the top. As shown, when training is performed at a small number of iterations (Epoch 100), there is a large difference between simulation (Simulation) results based on actual calculation and simulation (AI estimation) results generated by the artificial intelligence model, but as the number of iterations increases (Epoch 360), the generated simulation image gets closer to the simulation image based on actual calculation. As such, it can be seen that when training is iteratively performed over a predetermined number of iterations or more, there is almost no difference (Diff) between the two images.

FIGS.9A and9Bshow the acoustic simulation results with a position change of the ultrasonic transducer and differences between them. The two results have gone through the sufficiently trained artificial intelligence model. As shown inFIGS.9A and9B, it can be seen that even when the position of the transducer is changed, there is almost no difference between the numerical model-based simulation results (Simulation) and the artificial intelligence model-based simulation results (AI estimation).

FIGS.10A and10Bare 3D images of the acoustic simulation results based on artificial intelligence with a position change of the ultrasonic transducer. As the position of the transducer changes from A to B, the simulation results of the travel path and focal position of ultrasound and the acoustic pressure level at each position are differently measured. As a result of the experiment, the acoustic simulation results are outputted in real-time with a position change of the transducer.

The real-time acoustic simulation method based on artificial intelligence described above may be implemented as an application or in the format of program instructions that may be executed through a variety of computer components and recorded in computer-readable recording media. The computer-readable recording media may include program instructions, data files, and data structures alone or in combination.

Examples of the computer-readable recording media include hardware devices specially designed to store and execute the program instructions, for example, magnetic media such as hard disk, floppy disk and magnetic tape, optical media such as CD-ROM and DVD, magneto-optical media such as floptical disk, and ROM, RAM and flash memory.

Examples of the program instructions include machine code generated by a compiler as well as high-level language code that can be executed by a computer using an interpreter. The hardware device may be configured to act as one or more software modules to perform the processing according to the present disclosure, and vice versa.

According to the above-described embodiments, it is possible to acquire acoustic simulation results for an arbitrary transducer position at which simulation has never been performed before using the trained artificial intelligence model. The existing method that performs simulation through direct calculation needs to perform simulation for as many transducer positions as possible, and requires a few minutes or more for each position, which makes real-time simulation impossible. By contrast, according to the method of an embodiment, it is possible to perform real-time acoustic simulation for positions not included in the existing training data (i.e., positions at which calculation-based simulation has not been performed) at a very high speed using the artificial intelligence model.

Ultrasound Treatment System Based on Artificial Intelligence

FIG.11shows the structure of an ultrasound treatment system based on artificial intelligence according to an embodiment.

Referring toFIG.11, the ultrasound treatment system10based on artificial intelligence according to an embodiment includes an input unit100to receive input medical image data of a target area to be treated, an ultrasound output unit200to output ultrasound to the target area, a location tracking unit300to acquire position information of the ultrasound output unit200in real-time, a processing unit400to generate a real-time acoustic simulation image with a position change of the ultrasound output unit200using an artificial intelligence model, and a display unit500to display the real-time acoustic simulation image.

The input unit100may receive an input CT or MRI image, and carry out additional preprocessing of the input image. The preprocessing process may include dividing the image into a region of interest and background through a mask and removing the background.

The ultrasound output unit200is a component that outputs ultrasound based on preset ultrasound parameters, includes a single ultrasonic transducer or a series of ultrasonic transducers to convert alternating current energy to mechanical vibration, and generates and outputs ultrasound according to the set value such as acoustic pressure, waveform, and frequency. The output ultrasound overlaps to form an ultrasound beam which in turn converges at a target focus point to remove or stimulate the target tissue. According to an embodiment, the ultrasound output unit200is configured to output high-intensity focused ultrasound to thermally or mechanically remove the target tissue or low-intensity focused ultrasound to stimulate the target tissue without damage.

The location tracking unit300detects a position change of the ultrasound output unit200using a component embedded or mounted in the ultrasound output unit200such as an accelerometer and a gyroscope, and transmits real-time position information to the processing unit400.

The processing unit400includes at least one processor for performing the real-time acoustic simulation method based on artificial intelligence according to one of the above-described embodiments, and generates an acoustic simulation image corresponding to the transducer position using the position information of the ultrasound output unit200received from the location tracking unit300.

The display unit500displays the generated acoustic simulation image on the display to allow the surgeon to perform ultrasound treatment while observing the target tissue.

According to the configuration of the ultrasound treatment system as described above, it is possible to acquire real-time acoustic simulation results for positions not included in the existing training data (i.e., positions at which calculation-based simulation has not been performed) using the artificial intelligence model. Through this, the medical staff can see simulation images with the movement of the ultrasonic transducer in real-time, thereby achieving faster and more precise treatment.

While the present disclosure has been hereinabove described with reference to the embodiments, it will be apparent to those having ordinary skill in the corresponding technical field that various modifications and changes may be made to the present disclosure without departing from the spirit and scope of the present disclosure defined in the appended claims.