# A Mixed Clustering Approach for Real-Time Anomaly Detection
## 3 Problem Definitions  
In this section, we will navigate some significant terms, definitions, and formulae to be used in the proposed algorithm. Since most of the real-life datasets are hybrids, and the \(k\)-means algorithm uses the distance between the object and the cluster, typical distance formulae do not work. Therefore, it is required to formulate a general distance function which can be appropriate for numeric, categorical, or hybrid attributes. The formulae are given below. **Definition 1**. : _Distance in Categorical Attributes._  
_In [6; 14], the authors have formulated a distance function for categorical attributes as follows. If the data instances set have categorical attributes \(A_{1}\), \(A_{2}\),... \(A_{d}\). The domain (\(A_{i}\); \(i\) = 1,2,... \(d\)) = {\(a_{11}\), \(a_{12}\),... \(a_{im}\)} comprises-finite, unordered possible values that can be taken by each attribute \(A_{i}\), such that for any \(a\), \(b\)\(\in\)\(dom(A_{i})\), either \(a\) = \(b\) or \(a\) \(\
eq\)\(b\). Any data instance \(x_{i}\) is a vector (\(x_{i1}\), \(x_{i2}\),... \(x_{id}\)), where \(x_{ip}\)\(\in\)\(dom(A_{p})\), \(p\) = 1,2,... \(d\). The distance \(d(x_{i}\), \(C_{j})\) between data instance \(x_{i}\) and cluster \(C_{j}\), \(i\) = 1, 2,... \(n\) and \(j\) = 1, 2,... \(k\). as proposed in [6; 14] is given by_  
\[d(x_{i},C_{j})=\sum_{p=1}^{d}w_{p}d(x_{ip},C_{j})\,\ \text{where}\ \sum_{p=1}^{d}w_{p}=1 \tag{1}\]  
_Here \(w_{i}\), the weight factor associated with each attribute, describes the importance of the attribute which controls the contribution attribute-cluster distance to data instance-cluster distance. The attribute-cluster distance between \(x_{ip}\) and \(C_{j}\) is proposed as follows._  
\[d(x_{ip}\,,C_{j})=\frac{\left|C_{j}(A_{p}=\ x_{ip})\right|}{\left|C_{j}(A_{p} \
eq\ \varphi)\right|} \tag{2}\]  
_Obviously \(d(x_{ip}\), \(C_{j})\)\(\in\) [0, 1] means \(d(x_{ip}\), \(C_{j})\) = 1 only if all the data instance in \(C_{j}\) for which \(A_{p}\) = \(x_{ip}\) and \(d(x_{ip}\), \(C_{j})\) = 0 only if no data instance in \(C_{j}\) for which \(A_{p}\) = \(x_{ip}\). With the help of Equations (1) and (2) becomes_  
\[d(x_{i},C_{j})=\sum_{p=1}^{d}w_{p}d(x_{ip},C_{j})=\sum_{p=1}^{d}w_{p}\frac{ \left|C_{j}(A_{p}=x_{ip})\right|}{\left|C_{j}(A_{p}\
eq\ \varphi)\right|} \tag{3}\]  
_where \(d(x_{i}\), \(C_{j})\)\(\in\) [0, 1], \(i\) = 1, 2,... \(n\) and \(j\) = 1, 2,... \(k\)._  
**Definition 2**. : _Calculating the Weight of an Attribute._  
_In [6], the formula to calculate the weights of attributes is given as follows. The importance \(I_{A}\) of an attribute is quantified by the entropy metric as follows._  
\[I_{A}=-\int\rho(x(A))\log(\rho(x(A)))dx(x) \tag{4}\]  
_where \(x(A)\) is the value of the attribute \(A\), and \(\rho(x(A))\) is the distribution function of data along \(A\) dimension. As the values of categorical attributes are discrete and independent, then an attribute'sprobability is computed by counting the frequency of the attribute value. Accordingly, any categorical attribute \(A_{p}\)'s (\(p\in\{1,2,\ldots,d\}\)) importance can be evaluated by the formula._  
\[I_{A_{p}}=-\sum\limits_{t=1}^{m_{p}}\rho(a_{pt})\log(\rho(a_{pt})) \tag{5}\]  
_And_  
\[\rho\big{(}a_{pt}\big{)}=\frac{\big{|}D\big{(}A_{p}=a_{pt}\big{)}\big{|}}{\big{|} D\big{(}A_{p}\
eq\varphi\big{)}\big{|}}\]  
_where \(a_{p}\in_{t}\)dom(\(A_{p}\)), \(m_{p}\) is the \(A_{p}\)'s total number of possible values, and \(D\) is the whole dataset. From Equation (5) it is concluded that an attribute's importance is directly proportional to the number of different values of the categorical attribute. Although, practically, an attribute with immensely diverse values contributes minimum to the cluster. Hence, Equation (5) can further be modified as_  
\[I_{A_{p}}=-\frac{1}{m_{p}}\sum\limits_{t=1}^{m_{p}}\rho(a_{pt})\log(\rho(a_{pt})) \tag{6}\]  
_Therefore, we can quantify the importance of an attribute using its average entropy over each attribute value.