# QoE Modeling on Split Features with Distributed Deep Learning
## 5 Distributed Learning Approaches on Split Feature Scenarios
### Distributed Ensemble Learning (DEL)  
Our approach consists of both ML and domain expertise; hence we reveal the composite nature of a QoE model that is built on well-defined generic and specific features. We suggest dividing the QoE model into two parts, a _generic base model_ that is universally applicable, and a _specific local model_ that is specific and dependent on local context. Thereby, we relax the requirements towards the goal of achieving one superior customized QoE model. We hypothesize that, when a generic base model is used, a QoE model customization at the target domain is needed to improve the accuracy further, since the generic model in the source domain lacks additional features that are strongly representative for the target domain. Customization of the model in the target domain is done via its ensemble with the pre-trained source domain model. The \(\mathit{DEL}\) process does not necessitate these two models being of the same algorithm type, hence making the process _agnostic_ to the involved ML algorithms, which provides the freedom of choice of algorithms that fit best to global and local domains, respectively. Thus, we present a baseline algorithm-agnostic \(\mathit{DEL}\). In classical QoE studies, there are (rather strict) assumptions about which context and content parameters matter. Hence, an additional process of distinguishing the generic features from the specific ones needs to be developed. In QoE studies, there is a wide variety of content being shown to users for assessments, where the dataset includes some features that are only specific to that group. Since a consensus of running large scale QoE experiments exactly on the same or similar video content is not feasible (if ever possible), it could be a good practice to distinguish those features from the training of the generic base model. This way, the generic base model would be trained only with generic features that yield similar effects on the MOS values that are ideally universally applicable. The procedure consists of multiple steps as described in Figure 5. The assumption is that there exists a generic base model, which is pre-trained with a rather large dataset viaparticipation of preferably multiple research entities (G1 at source domain) using generic features as shown in Step 1. In Step 2, a new research group (G0 at a target domain) trains a separate ML model with any choice of feature set locally unique to the collected data samples in the target domain. In Step 3, the research group (G0 at target domain), receives the pre-trained generic model together with the metadata containing information about the set of generic features. Next, the locally trained specific model and the transferred generic model outputs are ensembled during inference. The _ensemble_ is a weighted average of the predictions of the base model and the local model on the test set as given in Equation (2). \[y^{\prime}_{\text{test},G0}=W_{0}M_{0,[S0,S1]}(X_{\text{test},G0})+W_{1}M_{1,[ S1]}(X_{\text{test},G0}) \tag{2}\]  
\(M_{i,j}\) is the trained model with the training set of group \(i\), using the feature set \(j\). For the transferred \(M_{1}\), the feature set consists of _generic features_ (GF), while the model \(M_{0}\) can be trained on both GF and _specific features_ (SF). The model weights \(W_{i}\) for group \(i\) are decimal numbers within the range of 0 and 1, which add up to 1. \(y^{\prime}_{\text{test},\text{G0}}\) is the weighted average of the two trained models' inferred outputs on the test set of G0, \(X_{\text{test},G0}\). In _DEL_ there were 32, 64, and 2 neurons in the first, second, and final layers, respectively. The non-linear activation functions were identical, i.e., ReLu on all hidden layers, and Sigmoid on the last layer. We trained the models until the saturation points of F1-score values. All layers were located on the same group (domain), where they are trained.