# QoE Modeling on Split Features with Distributed Deep Learning
### Distributed Learning in QoE  
As of today, two non-distributed learning techniques are most common in QoE modeling, (e.g., in crowd-sourcing based QoE assessment techniques, QoE databases that require special access): _Isolated_ and _Fully Centralized_, and we use the two as reference scenarios. The most conventional technique for QoE model training today is _Isolated_, where every entity trains individual decoupled machine learning models on the locally collected inherently decentralized datasets. We also use a _Fully Centralized_ learning technique as a reference as this is an existing way of QoE model training in the community in the cases when moving data in between entities is not an issue. In a _Fully Centralized_ reference scenario, all training datasets are collected into one physical storage where the training of all dataset is performed. Opposed to this, a local dataset is visible only to its owner (where the data originates from) and the single computation node that collects the dataset trains the model. The comparison between conventional Fully Centralized and Isolated machine learning model training methodologies in the scope of QoE is given in Figure 1.  
Distributed learning techniques can be split into two major categories: _model-parallel_ and _data-parallel_ based on the state-of-the-art distributed learning survey [8]. Accordingly, we position our article as stated in bold in Figure 2. In the _model-parallel_ category, datasets can be shared from where they are originally produced/collected, and multiple models with the exact same type and architecture are used in the training. The main goal is to utilize more resources in parallel for faster computation and training. In this scenario, the datasets are completely or partially accessible by other computation nodes, by means of redistributing and reshuffling of datasets among those nodes. All collaborating nodes can communicate and share parameters and some data with each other. In the _data-parallel_ category, both models and datasets are not necessarily identical, and datasets cannot be shared from where they are originally produced and collected. The main goal here is to reduce the privacy leakage of a raw dataset by avoiding moving raw data, making this solution an enabler for combining the individual learned knowledge from individual local models and from inherently decentralized datasets. In this scenario, the individual workers can share only trained model parameters in between each other in a fully- or semi-decentralized way, as in the case of gossip learning [9]. In case any direct communication in between workers is not allowed, two main groups can be presented: (i) Horizontal Federated Learning (_iFL_), in which the models need to train on the exact same feature set. (ii) Distributed Ensemble Learning (_DEL_) and Vertical Federated Learning (_vFL_), in which feature sets are not necessarily identical. Existing _hFL_ techniques [10; 11] enable model training without moving datasets in between different computation nodes by means of sending an aggregation of model parameters. Yet, _hFL_ necessitates all computation nodes to have the same observation attributes or, more specifically, ML feature sets. In the cases where the feature sets are different in between collaborating computation nodes of research entities due to a distributed heterogeneous sensor environment, other decentralized learning techniques that permit collaborative training with distributed unique attributes in each computation node are necessary to be considered. Therefore, in this article, we mainly study two distributed _data-parallel_-based learning techniques for datasets that are potentially and inherently decentralized in feature space: (i) a baseline _DEL_ technique; (ii) a Distributed Split Learning (SplitNN)-based _vFL_ technique [12; 13] with partial neural networks. A combination of predictions of different independent trained models, is known to perform better than each model alone [14].