## 2 Literature Review
### Machine Learning Applications in Contingency Estimation
\begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**Algorithms** & **Theoretical Underpinnings** & **Strengths** \\ \hline Linear regression & Linear regression assumes a linear relationship between the input variables and the target \(\hat{y}=w_{0}+\frac{n}{\sum_{i=1}^{n}w_{i}x_{i}}\) & \\  & where \(\hat{y}\) is the predicted target, \(x_{i}\) represents the input variables, \(w_{i}\) are the coefficients (weights) that quantify the influence of each input on the target, and \(w_{0}\) is the intercept. & Simplicity, interpretability, and computational efficiency \\ Linear regression & The optimal coefficients are achieved by minimizing the ordinary least squares [11]: & Simplicity, interpretability, and computational efficiency \\  & \(L=\frac{m}{\sum_{j=1}^{n}(y_{j}-\hat{y}_{j})^{2}}\), & \\  & where \(y_{j}\) is the actual value, \(\hat{y}_{j}\) is the predicted value, and \(m\) is the number of observations. & \\  & The coefficients in the linear regression model quantify the influence of each input variable on the target. & \\ \hline SVR & Similar to linear regression, SVR fits the function \(f(x)=\mathbf{W^{T}X}+\mathbf{b}\) while allowing a margin of tolerance \(\varepsilon\). To handle non-linear relations, it performs the kernel trick, which maps data to a higher-dimensional space where linear separation is achievable. Common kernel options include linear \(0\), polynomial, and radial basis function [27]. & Capability to address outliers and model non-linear relationships \\ \hline Decision trees & Decision trees split data into branches based on features that best reduce impurity. Impurity is a measure of the homogeneity of the data within a node, aiming to make each split as “pure” as possible. It can be calculated using metrics like Gini impurity and entropy for classification and variance for regression models [25]. Specifically, Gini impurity is calculated as: \(l_{c}(D)=1-\sum\limits_{k=1}^{K}p_{k}^{2}\), where \(p_{k}\) is the proportion of class \(k\) in the node \(D\); and entropy is measured as: \(l_{M}(D)=-\sum\limits_{k=1}^{K}p_{k}\log_{2}p_{k}\). & \\ Random forest & Random forest improves predictive accuracy by combining multiple decision trees. Each tree \(f_{h}(\mathrm{X})\) is trained on a random subset of data and features, reducing variance and overfitting [26]. The final prediction is the average outcome across all trees: \(\hat{y}=\frac{1}{\sum\limits_{k=1}^{B}f_{h}(\mathrm{X})}\), where \(B\) is the total number of trees. & Robustness to overfitting \\ \hline Gradient boosting & Gradient boosting builds an ensemble of decision trees sequentially, with each new tree \(f_{m}(\mathrm{X})\) focused on reducing the errors of the previous one [36]. At each step \(m\), the model updates predictions as: \(g_{j}^{(m)}=g_{j}^{(m-1)}+\eta f_{m}(\mathrm{X}_{j})\), where \(\eta\) is the learning rate controlling the contribution of each tree. & Promising predictive accuracy through interactive error correction \\  & The final model combines these sequentially trained trees, each contributing based on its ability to minimize prior errors. & \\ \hline XGBost & XGBoost employs regularization and optimization techniques to enhance model performance. Regularization, which prevents overfitting by penalizing model complexity; includes two techniques: L1 and L2 [31]. L1 regularization applies a penalty based on the absolute magnitude of coefficients, given by \(\alpha\frac{1}{\gamma}|w_{j}|\), while L2 uses the squared magnitude, i.e., \(\frac{1}{\gamma}\lambda\frac{\
abla}{\gamma}w_{j}^{2}\), where \(w_{j}\) are model parameters, and \(\alpha\) and \(\lambda\) control the strength of L1 and L2 regularization, respectively. XGBoost also uses optimization techniques, like second-order approximation and shrinkage. & High predictive accuracy through optimization \\ \hline ANN & An ANN consists of interconnected neuron layers: input, hidden, and output. Each neuron in layer \(l\) receives weighted inputs from the previous layer \(l-1\): \(z_{l}^{(l)}=\sum\limits_{j}w_{j}^{(l)}a_{j}^{(l-1)}+b_{j}^{(l)}\), where \(w_{j}^{(l)}\) and \(b_{j}^{(l)}\) refer to the weight and bias, respectively. & Ability to model complex relationships \\  & The activation function \(\phi\) introduces non-linearity: & \\  & \(a_{j}^{(l)}=\phi(z_{i}^{(l)})\). During training the ANN adjusts these weights using backpropagation which calculates the gradient of the loss function (a measure of prediction error) with respect to each weight [28]. & \\ \hline \hline \end{tabular}
\end{table}