# Anomaly Detection for IOT Systems Using Active Learning
## 4 Proposed Methodology
### Uncertainty Sampling  
A popular method in active learning for determining which data points to identify as the most useful is the uncertainty sampling algorithm. The method seeks to utilize the labeling budget as effectively as possible by considering uncertainty. The steps of the uncertainty sampling algorithm will be covered in this part, along with information on how it is implemented in code. Finding the K data samples with the highest uncertainty ratings is the first step in the uncertainty sampling algorithm. These evaluations are based on how confidently each data point was predicted by the model. The sample is thought to be more valuable for annotation the greater the uncertainty rating. The program then requests accurate labels for each of the K unsure samples and adds them to the labeled dataset D when the K uncertain samples have been discovered. Then, using the newly labeled samples, a new model is trained on the updated dataset D. Iteratively, this process goes on until the labeling budget is depleted or the desired target performance is attained. The main goal of the uncertainty sampling algorithm is to choose data points that will maximize the expected information gain. The algorithm attempts to decrease the uncertainty of the model's predictions and increase the accuracy of those predictions by actively seeking the most uncertain data. Through a series of iterations that involve selective annotation and model retraining, the performance of the model with sparsely labeled data is gradually improved. The uncertainty sampling algorithm uses the idea of entropy to quantify uncertainty in its implementation. A probability distribution's entropy measures the degree of randomness or uncertainty it contains. In order to lower the total uncertainty of the model's predictions, the algorithm chooses data points. Calculating the uncertainty score for each data point is part of the code for the uncertainty sampling algorithm. The estimated probabilities of the model for each class label can be used to obtain this score. The K samples chosen for annotation are those with the greatest scores for ambiguity. The labeled dataset is used to train the model, and the procedure is repeated until either the required performance level is reached or the budget is used up. The uncertainty sampling algorithm, which chooses data points based on their expected information gain, is an efficient method for active learning. The approach seeks to increase classification accuracy with less labeled data by iteratively annotating the most ambiguous examples and retraining the model. The method effectively makes use of the labeling budget that is available by incorporating the idea of entropy to improve the performance of the model. Define the function of the uncertainty score:  
\[\text{US (x) = 1 - P (max|x,D)} \tag{3}\]  
Given the input x and the current dataset D, y_max is the label with the highest probability. The chance that the label y_max will appear given the input x and the current dataset D is known as P (y_max|x,D). Select the K samples with the highest ratings for uncertainty. \[\text{Sk}=\{\text{x\_i}\}\text{i}=1\text{k}\text{ where x\_i}=\text{argmax x \in U US(x)} \tag{4}\]  
where U is the collection of data points without labels. Ask for the accurate label for each sample in S_k, then include them in the labeled dataset D.  
Train a new model on the revised dataset D, then continue the procedure until the budget is used up or the target performance is attained. The concept of uncertainty sampling, which evaluates the uncertainty of a model using many measurements, serves as the foundational mathematical principle behind active learning. Entropy is a statistic that quantifies the amount of information required to classify a given piece of data accurately. Entropy-based uncertainty sampling selects the data points close to the decision boundary or those for which the model exhibits the least certainty in their classification.