# Weibull-Open-World (WOW) Multi-Type Novelty Detection in CartPole3D
## 3 Method
### EVT-Based Probability of Per-Instance Novelty and Overall Novelty Detection
#### 3.2.1 WOW Novelty Detection: EVT Per-Step Combined  
When using the dissimilarity-based theory, multiple sources of uncertainty impact how to set the novelty detection threshold. The first is the normal uncertainty caused by random variations in the measure. But the second source of uncertainty would be associated with any model assumptions (e.g., if one assumes Gaussian distribution on values that may or may not hold). The important observation of this section is that we remove the second source of uncertainty by using Extreme Value Theory (EVT) because we do not need to assume much about the distribution of dissimilarity in the normal world. In the [4] framework, they simply declare thresholds for dissimilarity \(\delta_{w}\delta_{o}\), without providing any meaningful discussion on how they might be defined/estimated. Given the inherent variations and noise in real problems and the undefined scaling issues, defining these to account for uncertainty in the actual task/worlds is critical. In this section, we propose using EVT to define meaningful thresholds, including the agent threshold \(\delta_{a}\). There are two primary Extreme Value theorems, and we build from the Fisher-Tippet Theorem, also known as the statistical EVT of the first type. Just as the Central Limit Theorem dictates that the random variables generated from certain stochastic processes follow Gaussian distributions, EVT dictates that given a well-behaved initial distribution of values (e.g., a distribution that is continuous and has an inverse), the distribution of the maximum (minimum) values can assume only limited forms. We restate the core theorem here for reader convenience, and one can find proofs in many books, including in [9]. **Theorem 1** (Fisher-Tippet Theorem). : _Let \((v_{1},v_{2},\ldots)\) be a sequence of i.i.d samples. Let \(\zeta_{n}=\max\{v_{1},\ldots,v_{n}\}\). If a sequence of pairs of real numbers \((a_{n},b_{n})\) exists such that each \(a_{n}>0\) and \(\lim_{z\rightarrow\infty}P(\frac{b_{n}-b_{n}}{a_{n}}\leq z)=F(z)\) then if \(F\) is a non-degenerate distribution function, it belongs to the [PERSON], the Fredet, or the Reversed-Weibull family._Assuming dissimilarity, like distance, is bounded from below by 0, of the three distributions only the Reversed-Weibull can apply for maximum (and hence Weibull for minima). This EVT, and in particular the Weibull-based fitting, is widely used in many fields [9], such as manufacturing (e.g., estimating time to failure), natural sciences (e.g., estimating 100- or 500-year flood levels), and finance (e.g., portfolio risks). EVT has recently been (re)introduced and applied in recognition, machine learning, and computer vision [8; 10; 11; 12] where it is often used in open-set or open-world recognition tasks. We note that some of the fields we will model are not bounded apriori, e.g., maximum velocity is not easily bounded. Such fields might be better modeled with a different type-1 EVT distribution (e.g., Freshet or Gumbel), or may even be better modeled with a peak-over-threshold, i.e., EVT type 2 [13; 14], approaches. However, for simplicity we have used Weibull modeling for all fields and found it to be sufficiently effective and leave the search for better per-field models to future work. To apply the Weibull to determine a novelty detection threshold we consider some dissimilarity measures and collect values of this measure of a large number of trials in the normal world. We can then use the largest or smallest of these scores, depending on the variable type, and fit a three-parameter Weibull distribution to them:  
\[\text{W}(x;\mu,\sigma,\xi)= \begin{cases}\frac{\xi}{\sigma}\left(\frac{x-\mu}{ \sigma}\right)^{\xi-1}e^{-(\frac{x-\mu}{\sigma})^{\xi}}&x<\mu-\frac{\sigma}{ \xi}\\ 0&x\leq\mu\end{cases} \]  
where \(\mu\in\mathbb{R}\), \(\sigma\in\mathbb{R}^{+}\), and \(\xi\in\mathbb{R}^{-}\) are locations, scale, and shape parameters. Multiple libraries can compute the three-parameter Weibull, including SciPy, used for this paper.