# Anomaly Detection for IOT Systems Using Active Learning
## 4 Proposed Methodology
### The Foundational Mathematical Equations for Active Learning  
Active learning is a machine learning methodology that involves identifying and selecting data points that possess significant informational value for expert labeling. The primary objective of active learning is to minimize the quantity of labeled data required to attain a specific performance threshold.  
On the other hand, uncertainty sampling draws upon the principles of information theory, while the notion of entropy forms the foundational mathematical framework for active learning. The measure of entropy in a probability distribution signifies the degree of randomness or uncertainty present within it. In the context of active learning, the selection of data points for labeling is determined by assessing the entropy of a model's output to identify the most informative points.  
Let us define these terms:  
* X is a collection of all potential input data items.
* Y is a collection of all potential output labels.
* D is the labeled training set of data.
* U is the group of all unlabeled data points.
* H is a collection of all conceivable hypotheses (models).
* The primary goal of active learning is to select a subset of the universal set U, denoted as the query set Q, to obtain expert annotations. The approach utilizes the entropy of the model's output to ascertain the anticipated information gain for each data point within the set Q.  
For a particular data point x, the model's output entropy is defined as  
\[H\left(Y|x,D\right)=-\sum\left(y|x,\ D\right)\log_{p}(y|x,\ D) \tag{1}\]  
where, given the input data point \(x\) and labeled data \(D\), \(p\left(y|x,\ D\right)\) is the posterior probability distribution across the output labels.  
For a data point x, the anticipated information gain is specified as follows:  
\[IG\left(x,\ D\right)=H(Y|D)-E[H(\left(Y|x,\ D\right))] \tag{2}\]  
where \(E\left[H\left(Y|x,\ D\right)\right]\) is the anticipated entropy of the output labels for \(x\) given the present model and the labeled data, and \(H(Y|D)\) is the entropy of the output labels for the entire dataset.  
The algorithm selects the data points that exhibit the highest anticipated information gain to augment the labeled dataset D. The process is iterated until the desired level of performance is achieved, at which point the model undergoes retraining using a newly labeled dataset. To achieve a specific group level of performance with minimal labeled data, active learning utilizes the concept of entropy to select the most valuable data points for annotation. Algorithm 1 illustrates the procedural phases of the uncertainty sampling technique.  
```
Input: U, D, \(\theta\)- initial pool, training data, classifier, and B-budget Output: U, D, \(\theta\)- update pool, training data, classifier
1 initialize b = 0;
2 while b < B do
3 foreach \(x\in U\)do
4 compute \(s(\theta,x)\)
5 query the label of the optimal instance \(x^{*}\) with respect to \(s(\theta,x)\)
5 D = D \(U\) \(\{x^{*},y^{*}\}\);
6 U = U \(\{x^{*},y^{*}\}\);
7 train \(\theta\) from D;
8 b = b + 1; Return \(U,D,\theta\);
```  
**Algorithm 1** Uncertainty Sampling