## 2 Materials and Methods
### Clustering  
STBs send event logs to the data collector when plugged in. Data can be sent on standby and during content playback. It is necessary to use a continuous approach to assess the QoE level continuously. The stream clustering algorithms might be a solution to that issue. The incoming data stream can be shown as \(DS=\{x_{1},\,x_{2},\,x_{3},\,\ldots,\,x_{N}\}\), where the \(x_{i}\) is the single event and \(n\) goes to infinity [29]. Each plugged-in STB is the source of the \(x_{i}\) data instances. Data stream algorithms can use a single-phase approach, but most use a two-phase approach [30]. In the first phase, the synopsis of the stream is calculated and updated when a new entry appears. The second phase is used to cluster the data on the ground of a calculated summary.  
Five clustering algorithms were considered during the experiments (Table 3). The evoStream [31], DBSREAM [32], as well as D-Stream [33] were tested. Those algorithms are among the most popular stream clustering methods. The evoStream implements the evolutionary algorithm to optimize the macro cluster assignment. evoStream utilizes the idle time between new stream objects' appearance to build and refine macro clusters incrementally. The idle time optimization might be an effective solution for the data stream that comes from the TV system due to the variability of data arrival. The heuristic approach in the offline phase might be a significant asset regarding the QoE assessment in grade optimization. The DBSream is a micro-cluster-based algorithm that can utilize the shared density mechanism for reclustering. It has been shown that DBSream can outperform other stream clustering algorithms in most settings [34]. The D-Stream is a density-based algorithm that uses the grid structure to associate incoming data instances with grid cells. The D-Stream can adjust the returned groups in real time and capture the changing streaming character due to the usage of the density decaying technique.  
\begin{table}
\begin{tabular}{c c} \hline
**Algorithm** & **Duration [s]** \\ \hline DBSREAM/Hierarchical & 338 \\ DBSREAM/K-means & 195 \\ evoStream & 1837 \\ D-Stream/Hierarchical & 252 \\ D-Stream/K-means & 186 \\ \hline \end{tabular}
\end{table}
Table 3: Time comparison of chosen stream clustering algorithms.  
Figure 7: Pearson correlation of input parameters for the entire dataset.  
All of them returned comparable output. The final choice was based on the processing time, which may be a crucial metric for clustering rapidly emerging data instances from the production TV system. The evoStream algorithm was the slowest during the clustering of the dataset, split into 32-instance batches (Table 3). The K-means [35] in the second phase returned a similar output to DBSream [32] and DStream [33] in the first phase. The K-means overcame the hierarchical algorithm, but the DStream and hierarchical algorithm were eventually chosen. The output of K-means was difficult to map into the ACR scale, as the K-means can return the other cluster assignment in every clustering process due to initial algorithm points. The agglomerative algorithm will always return the same clusters for the given dataset and thus can be considered sufficient during the current research. The selection of the most efficient stream clustering algorithm for QoE assessment for production TV systems may be the subject of further research and will not be considered in this article. The continuous TV system development that results in enormously increased data stream size has to be deliberated, as well as the idle times between the appearance of new data points.