# Process-Oriented Stream Classification Pipeline: A Literature Review
## 5 Stream Classification Algorithms and Architectures
### Specific Classification Problems  
Although classification scenarios are traditionally often introduced as binary decision problems, we frequently encounter more than two classes and/or labels, both in practice and in theoretical application scenarios [40]. More precisely, a distinction has to be made between multi-class and multi-label problems.  
In the first case, an observation \(\vec{x}\in\mathcal{X}\) is classified into an \(l\)-dimensional space \(\mathcal{Y}=\{C_{1},\ldots,C_{l}\}\) of \(l\geq 3\) classes. A common way to deal with multi-class problems is the one-vs-all principle, which is based on \(l\) binary classification models [149] (one model per class). Each of them is generated using labeled instances of the respective class of interest on the one hand (1) and the instances of all remaining classes on the other (0). It should be noted that nowadays, various classification algorithms--e.g., neighborhood-based methods, or trees--can deal with more than two classes by nature.  
In the case of multi-label problems, instances will not only be assigned one but several classes simultaneously. That is, the feature vector of the \(i\)-th observation remains a \(p\)-dimensional vector \(\vec{x}_{i}=(x_{i1},\ldots,x_{ip})\in\mathcal{X}\). However, extending the previous notation, the corresponding outcome will no longer be a single class label \(y_{i}\in\mathcal{Y}\), but instead a \(k\)-dimensional vector \(\vec{y}_{i}=(y_{i1},\ldots,y_{ik})\in(\mathcal{Y}_{1}\times\ldots\times \mathcal{Y}_{k})\), where each element \(y_{ij}\in\mathcal{Y}_{j}\), \(j=1,\ldots,k\), maps to its own finite set \(\mathcal{Y}_{j}\) of \(l_{j}\) class labels.  
The following approaches are common when dealing with Multi-Label problems: First, multi-label problems can be separated into several binary problems (called binary relevance), meta labels of label combinations (called label power sets), and a set of labelpairs (called pairwise). These approaches are referred to as problem transformation. The appropriate strategy depends on the data basis. Binary relevance is an easy-to-understand method, which is often implemented in the context of ensemble models. A disadvantage of this method is that no label correlations are considered here, i.e., each label is assigned independent of all other labels. On the other hand, label correlations are considered within the label powerset approach. Here, however, the focus is on essential interactions to counteract the sparsity of data points and achieve a meaningful classification, e.g., by selecting important label correlations [150]. The pairwise approach is rarely used in online settings due to its high computational complexity in large-scale multi-label problems.  
Another possibility for dealing with non-binary problems is extending the classification algorithms (such as decision trees). One example is the extension of the Hoeffding Tree algorithm [151]. Here, leaf nodes exist for every necessary label combination in the label powerset.  
For a comprehensive overview of multi-label stream classification, we refer to the survey paper by [40]. They discuss a wide range of multi-label stream classification algorithms, their ability to adapt to changes in the data stream, and standard multi-label datasets. As an evaluation criterion, the authors analyzed the algorithm performance in terms of the F1 metric and the running time. By benchmarking multiple algorithms on different real-world and synthetic data sets, they give a good overview of the strengths and weaknesses of the individual approaches. The authors conclude that a Multi-Label ensemble with Adaptive Windowing (MLAW), proposed by, shows the best classification performance and efficiency results. The authors conclude that there is room for further research on handling and detecting different types of concept drift, imbalanced classes, and temporal dependencies of labels.