# Time Aware F-Score for Cybersecurity Early Detection Evaluation
## 3 Materials and Methods
### Methods  
The early detection problem presents certain characteristics that must be taken into account, and it must be formally described in order to define proper methods. Although a thorough description of the formal definition of the early detection problem can be found in [3], a summary is presented next for ease of interpretation. To do so, we define \(E=\{e_{1},e_{2},\ldots,e_{|E|}\}\) as the set of entities susceptible of being classified, in this case \(|E|\) describes the amount of entities. Each one of them (\(e\)) is composed from a series of items (\(I_{e}\)), with a label (\(I_{e}\)) for the class of the entity. This could represent either that the entity is anomalous or not, \(I_{e}=true\) and \(I_{e}=false\) respectively. It must be added that although this represents a binary classification task, early detection systems could provide a third value showing that the decision has not been taken yet (i.e., a delay).  
The set of items for a particular entity is expected to change over time and is represented by \(I_{e}=(<I_{1}^{e},I_{1}^{e}>,<I_{2}^{e},I_{2}^{e}>,\ldots,<I_{n}^{e},I_{n}^{e}>)\), being \(<I_{k}^{e},I_{k}^{e}>,k\in[1,n]\) the tuple that represents, for each entity \(e\), its k-th item \(I_{k}^{e}\), and the timestamp associated with the particular item \(I_{k}^{e}\) is denoted as \(I_{k}^{e}\).  
Also, it is important to notice that the following statement must be true, as items must be time ordered:  
\[\forall<I_{k}^{e},I_{k}^{e}>\ :\ t_{k}^{e}\ before\ t_{k+1}^{e}\]  
An item, \(I_{k}^{e}\), is characterized by a feature vector, and, particularly, it can be inferred that all items linked to an entity, \(I_{k}^{e},k\in[1,n]\), share the same feature vector. Those attribute values will be expected to change over time.  
\[I_{k}^{e}=\left[f_{k_{1}}^{e},f_{k_{2}}^{e},\ldots,f_{k_{m}}^{e}\right],k\in[ 1,n]\]  
Due to the independence of entities, the sequence of items \(I_{e}\) for each entity \(e\in E\) may exhibit varying lengths, denoted as \(n\). It is important to emphasize that the number of features, \(m\), remains consistent across all items.  
In this kind of problem, for a given entity \(e\), the goal is to identify any anomalous behaviour while examining the fewest number of items from \(I_{e}\) possible.  
The objective function will be defined as \(f(I_{e},I_{e}\times[1..n])\rightarrow\{0,1,2\}\). If an entity \(e\) is deemed anomalous following the analysis of \(i_{1}\) to \(I_{k}\) items, this function will output \(1\) (i.e., _positive_). If an entity \(e\) is determined to be normal (i.e., non-anomalous or _negative_) after processing an amount of \(k\) items and the preceding ones, then \(f(I_{e},I_{e},k)=0\). Lastly, \(f(I_{e},I_{e},k)=2\) represents that no definitive decision can be made regarding \(e\) entity after processing an amount \(k\) of items, indicating the need for further processing (i.e., _delay_). As a result, when \(f(I_{e},I_{e},k)\) yields outputs \(0\) and \(1\), the processing of items \(I_{k+1},\ \ldots,I_{n}\) is unnecessary. Conversely, if the output is \(2\), it necessitates the processing of additional items \(I_{k+1},\ \ldots,I_{n}\) until a conclusive output is obtained or the sequence of items is exhausted.  
Among the different existent evaluation methods (e.g., batch, streaming, time-based, etc.) we choose streaming evaluation as it provides the best representation for each individual entity.  
In this case, \(I_{e}\), which represents the sequence formed by individual items, is treated individually and following the original sequence. As a result, for each entity \(e\), the function \(f(l_{e},l_{e},k)\), \(k\in[1,n]\), must be executed until a conclusive output is achieved or until the maximum number of \(items\) (\(n\)) is reached. Similar to previous scenarios, when item \(k\), \(I_{k}\), is being processed, the model can incorporate all preceding items, \(I_{1}\),..., \(I_{k-1}\), as illustrated in Figure 1.