Patent Description:
Artificial neural networks (NNs) are revolutionizing the field of computer vision. The top-ranking algorithms in various visual object recognition challenges, including ImageNet, Microsoft COCO, and Pascal VOC, are all based on NNs. The article "<NPL>, as closest state of the art. Therein, the proposed active learning procedure is based on the uncertainty sampling strategy and a deep neural network. Stacked auto-encoders trained on redundant special and spectral features and a few label training samples are used to initialize a deep neural network. Uncertainty for a given sample is measured by the difference between the largest two class outputs of the neural network. A batch of samples with most uncertainty will be selected after label query and added into the training set. Then, the neural network is retrained. This active batch selection is iterated until the budget, that is, the upper limit of label queries, is reached. Therein, the selection of training samples to be added to the training set is based only on the uncertainty, which is measured by the difference between the largest two class outputs of the neural network. So, the uncertainty criterion is based on the top layer outputs of the neural network, a measure similar to the margin motivation behind multiclass/level uncertainty as performed by current support vector machines. In the Article "<NPL>, an active dictionary learning method is introduced in order to arrive at a proper dictionary for sparse representation. Therein, classification error and reconstruction error are considered as the active learning criteria in selection of the atoms for dictionary construction. In the proposed method, the reconstruction error and the sparse representation based classification error are used to select the most informative data in the training data.

In the visual object recognition using the NNs, the large scale image datasets are used for training the NNs to obtain good performance. However, annotating large-scale image datasets is an expensive and tedious task, requiring people to spend a large number of hours analyzing image content in a dataset because the subset of important images in the unlabeled dataset are selected and labeled by the human annotations.

Accordingly, there is need to achieve better performance with less annotation processes and, hence, less annotation budgets.

Some embodiments of the invention are based on recognition that an active learning using an uncertainty measure of features of input signals and reconstruction of the signals from the features provides less annotation processes with improving the accuracy of classifications of signals.

Accordingly, one embodiment discloses a computer-implemented method for training a neural network according to claim <NUM>.

Another embodiment discloses an active learning system for training a neural network according to claim <NUM>.

Accordingly, one embodiment discloses a non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations for training a neural network according to claim <NUM>.

In some embodiments, the use of an artificial neural network that determines an uncertainty measure may reduce central processing unit (CPU) usage, power consumption, and/or network bandwidth usage, which is advantageous for improving the functioning of a computer.

Although in this specification, reference is sometimes made to "images", the described embodiments can only relate to the invention when the images are electroencephalogram (EEG) signals or electrocardiogram (ECG) signals as required by the appended independent claims.

An active learning system includes a human machine interface, a storage device including neural networks, a memory, a network interface controller connectable with a network being outside the system. The active learning system further includes an imaging interface connectable with an imaging device, a processor configured to connect to the human machine interface, the storage device, the memory, the network interface controller and the imaging interface, wherein the processor executes instructions for classifying an electroencephalogram (EEG) signal or an electrocardiogram (ECG) signal. using the neural networks stored in the storage device, in which the neural networks perform steps of determining features of a signal using the neural network, determining an uncertainty measure of the features for classifying the signal, reconstructing the signal from the features using a decoder neural network to produce a reconstructed signal, comparing the reconstructed signal with the signal to produce a reconstruction error, combining the uncertainty measure with the reconstruction error to produce a rank of the signal for a necessity of a manual labeling, labeling the signal according to the rank to produce the labeled signal, and training the neural network and the decoder neural network using the labeled signal.

<FIG> shows an active learning system <NUM> in accordance with some embodiments of the invention. An initial setting of the active learning system <NUM> includes a neural network <NUM> initialized with random parameters, an initial set of labeled training images <NUM>, a trainer <NUM>, a set of unlabeled images <NUM>. In this case, the neural network <NUM> is a user defined neural network.

The active learning system <NUM> attempts to efficiently query the unlabeled images for performing annotations through the basic workflow shown in <FIG>. Based on the neural network (NN) <NUM> with randomly initialized parameters, the trainer <NUM> updates network parameters by fitting the NN <NUM> to the initial labeled training dataset of images <NUM>. As a result, a trained NN <NUM> with the updated network parameters is used to rank the importance of images in an unlabeled dataset <NUM>. The unlabeled images <NUM> are sorted according to importance scores <NUM> obtained from a ranking result performed by the trained NN <NUM>. The K most important images <NUM> are stored into a labeling storage in a memory (not shown in the figure) associated to a labeling interface <NUM>. In response to data inputs made by an operator (or annotator), the labeling interface <NUM> generates annotated images <NUM> having the ground truth labels. These annotated images <NUM> are then added to the initial labeled training dataset <NUM> to form a new training dataset <NUM>. The trainer <NUM> then retrains the network <NUM> by fitting the new training dataset of images <NUM> and obtains updated neural network parameters <NUM>. This procedure is iterative. The updated neural network parameters <NUM> are used to rank the importance of the rest of the unlabeled images103, and the K most important images <NUM> are sent to the labeling interface <NUM>. Usually, this procedure is repeated several times until a predetermined preferred performance is achieved or the budget for annotations is empty.

Further, a method for training a neural network uses a processor in communication with a memory, and the method includes steps of determining features of a signal using the neural network, determining an uncertainty measure of the features for classifying the signal, reconstructing the signal from the features using a decoder neural network to produce a reconstructed signal, comparing the reconstructed signal with the signal to produce a reconstruction error, combining the uncertainty measure with the reconstruction error to produce a rank of the signal for a necessity of a manual labeling, labeling the signal to produce the labeled signal, and training the neural network and the decoder neural network using the labeled signal. The labeling includes labeling the signal using the neural network if the rank does not indicate the necessity of the manual labeling process, and further the labeling includes transmitting a labeling request to an annotation device if the rank indicates the necessity of the manual labeling process.

Further, the determining features may be performed by using an encoder neural network. In this case, the encoder neural network can perform feature analysis of given signals. The signal is an electroencephalogram (EEG) signal or and electrocardiogram (ECG) signal. Accordingly, some embodiments of the invention can be applied to provide specific signals for assisting a diagnosis of medical doctors.

<FIG> is a flowchart of an active learning system for training neural network.

The active learning system <NUM> attempts to efficiently query the unlabeled images for the annotation through a process flow shown in the figure. The process flow includes the following stages:.

As shown in <FIG>, the active learning algorithms of the active learning system <NUM> attempt to efficiently query images for labeling images. An initialization model is trained on an initial for small labeled training set. Based on the current model, which is the latest trained model, the active learning system <NUM> tries to find the most informative unlabeled images to be annotated. A subset of the informative images are labeled and added to the training set for the next round of training. This training process is iteratively performed, and the active learning system <NUM> carefully adds more labeled images for gradually increasing the accuracy performance of the model on the test dataset. By the very nature, the algorithms of the active learning system <NUM> usually work much better than the standard approach for training, because the standard approach simply selects the samples at random for labeling.

Although a term "image" is used in the specification, another "signal" is used in the active learning system <NUM>:
The active learning system processes an electroencephalogram (EEG) signal or an electrocardiogram (ECG) signal. Instead of the images, the EEG or ECG signals are used to train the neural network and decoder neural network in the active learning system <NUM>.

Then the trained active learning system <NUM> is applied to determine or judge abnormality with respect to an input signal, which can be a useful assistance for medical diagnosis of relevant symptoms.

<FIG> shows a block diagram of process steps to be performed based on some embodiments of the invention. An input signal is fed into the active learning system10, an encoder neural network of the active learning system <NUM> determines features of the input signal in step SS1 and stores the features into a working memory (not shown). Further, an uncertainty measure is determined by a trained neural network <NUM> of the active learning system <NUM> in step SS2 and a result of the uncertainty measure is stored in the working memory. The features determined in SS1 is reconstructed by a decoder NN in step SS3 and a reconstructed signal is stored in the working memory. In step SS4, the reconstructed signal is fed from the working memory and compared with the input signal to compute a reconstruction error. The reconstruction error is stored in the working memory and fed to step SS5. In step SS5, the uncertain measure is read from the working memory and combined with the reconstruction error. In step SS6, the input signal is labeled according to a ranking score and the labeled signal is used in step SS7 for training the neural networks in the active learning system <NUM>.

<FIG> shows a block diagram indicating an active learning process <NUM> and a convolutional neural network (CNN) training process <NUM>, both of which are performed in the active learning system <NUM>. Upon an identical input signal <NUM> (or input images <NUM>), the active learning process <NUM> feeds the input signal <NUM> to a convolutional neural network (CNN) <NUM> and the CNN <NUM> extracts features <NUM> from the input signal <NUM>. Further, the active learning process <NUM> computes an uncertainty measure <NUM> from the features <NUM> and provides a score <NUM> based on the uncertainty measure <NUM>.

In the CNN training process <NUM>, the input signal <NUM> is fed to the CNN <NUM> and the CNN <NUM> extracts the features <NUM> from the input signal <NUM>. Then a CNN decoder <NUM> reconstructs a signal <NUM> from the features <NUM> to compare with the input signal <NUM>. By comparing the input signal <NUM> and the reconstructed signal <NUM>, the CNN training process <NUM> computes or generates a reconstruction error <NUM>. The active learning system <NUM> combines the reconstruction error <NUM> and the uncertainty measure <NUM>, and ranks the input signal <NUM> by a score <NUM>.

When the score <NUM> is higher than a predetermined threshold, the input signal <NUM> is fed to a labeling interface (not shown) that allows an operator to annotate the input signal <NUM> according to one of predetermined classified labels, which is indicated as Human labeling process <NUM>. The process steps performed in the active learning process <NUM> and the CNN training process <NUM> described above are illustrated in <FIG>, which shows key process steps performed in the active learning system <NUM>.

The rank is defined based on an addition of an entropy function and the reconstruction error.

<FIG> shows a block diagram of process steps for ranking the importance of unlabeled images in an active learning system according to some embodiments of the invention. When an input image <NUM> is provided to a front end of the NN <NUM> in step <NUM>, the trained NN <NUM> generates features <NUM> and outputs a classification result via a softmax output layer <NUM>. The classification result is used for calculating the importance score <NUM> of the input signal through uncertainty measure <NUM> based on the Rényi entropy.

The trained NN <NUM> is used for extracting the features <NUM> for each of the images in the unlabeled dataset <NUM> and also for computing classifications by the softmax output layer <NUM>. The classification result obtained by the softmax output layer <NUM> is a probability vector of dimension D where the dimension D is the number of object classes. Denoting the input image by x and the classification result computed by the softmax output layer <NUM> indicating a probability vector by p, each dimension of the probability vector p represents the probability that the input image <NUM> belongs to a specific class. The sum of the components of p is equal to one. The uncertainty of the class of the input image can then be measured in the step of the uncertain measure <NUM> by an entropy function H(x). When the entropy H(x) is computed based on the Shannon entropy, the uncertainty of the class of the input image is given by <MAT>.

In an uncertainty method, the uncertainty measure can be used as the importance score of the unlabeled image <NUM>. Further, other entropy measures defined in the Renyi entropy category can be used for the uncertainty computation. For instance, the entropy function H(x) may be Collision entropy, <MAT>.

Further, entropy based methods may be defined by <MAT> for obtaining an estimate of uncertainty, and an experimental result is shown in <FIG>.

Since the uncertainty method is a universal active learning method, it can be used in conjunction with various classifiers (SVMs, Gaussian processes, or neural networks) as long as the vector representing the class probability can be derived from each input image. In this case, the uncertainty method does not utilize the property of the classifier and reaches sub-optimal performance.

In accordance with some embodiments, an approach to improve the uncertainty method by utilizing the property of neural network computation is described in the following. It is established that a neural network computes a hierarchy of feature representation as processing an input image. The completeness of the feature representation can be used to judge how well the neural network models the input image. In order to quantify the completeness of the feature representation, an autoencoder neural network can be used.

<FIG> shows a block diagram of an autoencoder neural network <NUM> according to some embodiments of the invention. The autoencoder neural network <NUM> includes an encoder neural network <NUM>, a decoder neural network <NUM>, and a softmax output layer <NUM>.

When an input image <NUM> is provided, the autoencoder NN <NUM> outputs classification results <NUM> from the features <NUM> extracted by the encoder neural network <NUM>. Further, the features <NUM> are transmitted to the decoder neural network <NUM>. The decoder neural network <NUM> generates a reconstructed image <NUM> from the features <NUM> extracted by the encoder NN <NUM>. In some cases, the encoder NN <NUM> may be referred to as a first sub-network #<NUM>, and the decoder neural network <NUM> may be referred to as a second sub-network #<NUM>. The first sub-network <NUM> extracts the features <NUM> from the input image <NUM>. The extracted features <NUM> are fed into the softmax output layer <NUM> that outputs classification results. In this case, the extracted features <NUM> are also fed into the second sub-network #<NUM>. The second sub-network #<NUM> generates a reconstructed image <NUM> from the features <NUM> and outputs the reconstruction image.

In some embodiments, a reconstruction error is defined based on the Euclidean distance between an input image (or input signal) and a reconstructed image (or reconstructed signal).

Further, the reconstructed image <NUM> is compared to the input image <NUM> based on the Euclidean distance measurement. The Euclidean distance between the input image <NUM> and the reconstructed image <NUM> can be used for quantifying the completeness of the feature representation. When letting x be the vector representation of the input image and y be the vector representation of the reconstructed image, the reconstruction error measure R(x) is defined by the Euclidean distance as follows.

The Euclidean distance indicates how the input image is well represented by the feature representation. When a reconstruction error R(x) is small, it indicates that the neural network models the input image well. However, when the reconstruction error R(x) is large, then it indicates that the neural network does not model the input image well. In some embodiments, including the input image in training improves the representation power (accuracy) of the autoencoder NN <NUM>.

For ranking the importance of an input image, the following formula can be used, <MAT> where α and β are non-negative weighting parameters.

<FIG> shows a block diagram indicating an integrated design of sub-networks #<NUM> and #<NUM> used in the uncertainty measure based an active learning system <NUM> according to some embodiments of the invention. The block diagram shows data process steps used in a method for ranking the importance of unlabeled images in the active learning system <NUM>. The active learning system <NUM> includes an encoder neural network <NUM> (first sub-network #<NUM>), a softmax output layer <NUM>, a ranking layer <NUM>, a decoder neural network (second sub-network #<NUM>).

When the input image <NUM> is provided to the active learning system <NUM>, the encoder NN <NUM> generates the features <NUM> from the input image <NUM>. The features <NUM> can be used for generating a classification result via the Softmax output layer <NUM>. The classification result is fed to the ranking layer <NUM>. Further, the features <NUM> is fed to the decoder NN <NUM> and used to generate a reconstructed image <NUM> by using the decoder NN <NUM>. The reconstructed image <NUM> is fed to the ranking layer <NUM>. At the ranking layer <NUM>, the classification result and the reconstructed image are used to compute the importance score <NUM> with respect to an unlabeled image of the input image <NUM>.

The importance score <NUM> of the unlabeled image can be calculated from the classification output <NUM> and the reconstructed image <NUM> by using the ranking layer <NUM> in the calculation step. After obtaining the importance score <NUM> regarding the unlabeled image, the active learning system outputs the importance score <NUM> as an output.

<FIG> shows a block diagram of an active learning system <NUM> according to some embodiments of the invention. The active learning system <NUM> includes a human machine interface (HMI) <NUM> connectable with a keyboard <NUM> and a pointing device/medium <NUM>, a processor <NUM>, a storage device <NUM>, a memory <NUM>, a network interface controller <NUM> (NIC) connectable with a network <NUM> including local area networks and internet network, a display interface <NUM>, an imaging interface <NUM> connectable with an imaging device <NUM>, a printer interface <NUM> connectable with a printing device <NUM>. The processor <NUM> may include one or more than one central processing unit (CPU). The active learning system <NUM> can receive electric text/imaging documents <NUM> via the network <NUM> connected to the NIC <NUM>. The active learning system <NUM> can receive annotation data from the annotation device <NUM> via the HMI <NUM>. Further, the annotation device <NUM> includes a display screen, and the display screen of the annotation device <NUM> is configured to display the labeling interface <NUM> that allows the operator to perform labeling process of unlabeled images stored in the memory <NUM> by showing the unlabeled image in the display region <NUM> with the selection area <NUM> having predetermined annotation boxes and predetermined labeling candidates to be selected.

The storage device <NUM> includes original images <NUM>, a filter system module <NUM>, and a neural network <NUM>. The processor <NUM> loads the code of the neural network <NUM> in the storage <NUM> to the memory <NUM> and executes the instructions of the code for implementing the active learning. Further, the pointing device/medium <NUM> may include modules that read programs stored on a computer readable recording medium.

<FIG> shows an example of the labeling interface <NUM> according to some embodiments of the invention. The labeling interface <NUM> includes a display region <NUM> and a selection area <NUM>. The labeling interface <NUM> can be installed in the annotation device <NUM>, which indicates the labeling interface <NUM> on a display of the annotation device <NUM>. In some cases, the labeling interface <NUM> can be installed an input/output interface (not shown in the figure) connectable to the human machine interface (HMI) <NUM> via the network <NUM>. When the labeling interface <NUM> receives an unlabeled image of the K most important unlabeled images <NUM> in step S6 of <FIG>, the labeling interface <NUM> shows the unlabeled image on the display region <NUM>. The selection area <NUM> indicates predetermined candidates for labeling the unlabeled image shown on the display region <NUM>. The labeling interface <NUM> allows an operator to assign one of selectable annotations indicated in the selection area <NUM> with respect to the unlabeled image shown in the display region <NUM>. In <FIG>, the selection area <NUM> provides selection boxes with predetermined labeling candidates: Dog, Cat, Car, and Plane. As an example, <FIG> shows an unlabeled image indicating a cat image <NUM> displayed in the display region <NUM>. In this case, the annotation box of Cat is checked by the operator (annotator) in response to the cat image shown in the selection area <NUM>. The labeling interface <NUM> is configured to load and show unlabeled images stored the labeling storage in the memory according to the operations by the operator. The images labeled by the labeling interface <NUM> are stored into a new training image storage area in the memory in step S3 as newly labeled training images <NUM> as seen in <FIG>.

<FIG> shows experimental results of image classifications using the active learning methods on a convolutional neural network (CNN) for comparison, and the uncertainty method based on a CANN.

For comparison, the following convolutional neural network (CNN) was used for the experiments in the MNIST dataset: (<NUM>)5c-2p-(<NUM>)5c-2p-500fc-r-10fc, where "(<NUM>)5c" denotes a convolutional layer of <NUM> neurons with a kernel size <NUM>, "2p" denotes a <NUM>×<NUM> pooling, "r" denotes rectified-linear units (ReLU), and "500fc" denotes a fully connected layer with <NUM> nodes. One softmax loss layer is added to the classification output "10fc" for the backpropagation. For the convolutional autoencoder neural network (CANN) part, the structure from the deconvolutional network is adapted. For the CIFAR10 dataset: "(<NUM>)3c-2p-r-(<NUM>)3c-r-2p-(<NUM>)3c-r-2p-200fc-10fc". For the CANN part, the structure is the same as mentioned in MNIST settings.

In <FIG>, the dataset "Uncertain. " indicates data obtained by the uncertainty measure and reconstruction method according to an embodiment of the invention. The methods other than the uncertainty method shown in <FIG> are obtained by using a CNN instead of the structure with an autoencoder. Further, "RDM" indicates random method, "EMC" indicates an expected model change method, "UNC" indicates an uncertainty method without reconstruction, "DW" indicates a density weighted method, and "FF" indicates a farthest first method. In both MNIST setting and CIFAR10 setting, the uncertainty measure & reconstruction method in accordance with the embodiment of the invention shows superior performance compared to the other methods. This indicates one of advantages of the active learning system in accordance with some embodiments of the invention.

The advantage is reducing the number of annotated data, as discussed above, the artificial neural network according to some embodiments of the invention can provide less annotation processes with improving the classification accuracy, the use of artificial neural network that determines an uncertainty measure may reduce central processing unit (CPU) usage, power consumption, and/or network bandwidth usage, which is advantageous for improving the functioning of a computer.

The above-described embodiments of the present invention can be implemented in any of numerous ways. For example, the embodiments may be implemented using hardware, software or a combination thereof. When implemented in software, the software code can be executed on any suitable processor or collection of processors, whether provided in a single computer or distributed among multiple computers. Such processors may be implemented as integrated circuits, with one or more processors in an integrated circuit component. Though, a processor may be implemented using circuitry in any suitable format. The processor can be connected to memory, transceiver, and input/output interfaces as known in the art.

Also, the various methods or processes outlined herein may be coded as software that is executable on one or more processors that employ any one of a variety of operating systems or platforms. Alternatively, or additionally, the invention may be embodied as a computer readable medium other than a computer-readable storage medium, such as signals.

The terms "program" or "software" are used herein in a generic sense to refer to any type of computer code or set of computer-executable instructions that can be employed to program a computer or other processor to implement various aspects of the present invention as discussed above.

Use of ordinal terms such as "first," "second," in the claims to modify a claim element does not by itself connote any priority, precedence, or order of one claim element over another or the temporal order in which acts of a method are performed, but are used merely as labels to distinguish one claim element having a certain name from another element having a same name (but for use of the ordinal term) to distinguish the claim elements.

Claim 1:
A computer-implemented method for training a neural network (<NUM>) to determine abnormality in a signal, the signal being an electrocardiogram, ECG, signal or an electroencephalogram, EEG, signal using a processor in communication with a memory, comprising:
determining (SS1) features (<NUM>) of the signal (<NUM>) using the neural network (<NUM>);
determining (SS2) an entropy function as an uncertainty measure (<NUM>) of the features (<NUM>) for classifying the signal (<NUM>);
reconstructing (SS3) the signal (<NUM>) from the features (<NUM>) using a decoder neural network (<NUM>) to produce a reconstructed signal;
comparing (SS4) the reconstructed signal with the signal (<NUM>) to produce a reconstruction error (<NUM>);
combining (SS5) the uncertainty measure (<NUM>) with the reconstruction error (<NUM>) to produce a rank of the signal for a necessity of a manual labeling process, wherein the rank is defined based on an addition of the entropy function and the reconstruction error;
labeling (SS6) the signal using the neural network (<NUM>) to produce a labeled signal if the rank does not indicate the necessity of the manual labeling process;
transmitting a labeling request to an annotation device if the rank indicates the necessity of the manual labeling process; and
training (SS7) the neural network (<NUM>) and the decoder neural network (<NUM>) using the labeled signal.