# QoE Modeling on Split Features with Distributed Deep Learning
## 2 Related Work  
Decentralized and distributed learning techniques such as ensemble learning, transfer learning, and federated learning are well-known [17], and we foresee the importance of these methods in future generation mobile network architectures, especially since we see these techniques as enablers for energy-efficient QoE modeling while preserving privacy. Yet, there exists very little work in the literature that leverage decentralized learning technologies such as federated learning in QoE [11]. * Horizontal Federated Learning (_hFL_): In horizontal federated learning [10], models are trained collaboratively by combining different models which have been trained locally on the same feature set, to a single model at a _master_ node. When such a model is shared back to the individuals that contributed to the training process, immediate benefits can be detected for those that have similar data distribution representing similar underlying conditions and context. In [18], feature selection is studied within Neural Network-based Federated Learning, where model parameters are divided into private and federated parameters. Only the federated parameters are shared and aggregated during federated learning. It is shown that this model customization approach significantly improves the model performance while not sharing raw data in between. * Distributed Ensemble Learning (_DEL_): In [19], the authors used transfer learning to estimate the labels of an unlabeled dataset where the labels represent user emotions. In [11], a Round Robin-based learning technique (inspired by _Baidu-AllReduce_[20]) is presented within the scope of web QoE. This approach, similar to _hFL_, requires the distributed domains to have at least a subset of the features (if not all) to be the same. * Vertical Federated Learning (_vFL_): Vertical federated learning enables collaborating nodes to jointly train a machine learning model with fully orthogonal feature sets. Most relevant _vFL_ implementations in the literature are: PaddleFL [21], PySyft [16], FATE [22], and FedML [23]. Although all of the above solutions indicate great progress, PaddleFL, FATE, and PySyft implementations were inadequate for our needs due to their high level abstraction. The closest implementation among them all is FedML;however, we have not yet performed a complete comparison with that work and ours as it was very recently published. * Privacy-preserving communication: It is shown in the literature [24] that sending model weights that emerged from training, instead of sending the actual raw dataset, may still leak sensitive information about the ground truth used to train the model. Therefore, there exist techniques to further protect privacy on the shared weights of the models. Techniques such as differential privacy [25] and secure aggregation [26] can be utilized for sharing private information for the purposes of training ML models without revealing the original dataset and also concealing the identity of the dataset's origin. In \(DEL\), only the generic model (architecture and internal representation) is shared, which is already known and as such does not need to be protected. In the scope of this study, we consider these techniques as complementary since they can be applied to _hFL_ or _vFL_ without affecting the inner workings of each approach. * Compression: In this study, we make use of lossless _Elias_ encoding [27] and data compression [28] techniques not only to enable secure communication, but also to reduce communication cost. * QoE modeling with ML: ML techniques have been studied previously in QoE modeling [29]. Furthermore, a mix of conventional and a ML based QoE modeling has been implemented and demonstrated previously in [30]. All existing QoE models are trained via _Fully Centralized_ manner. Our proposed vFL in this article, can be considered as a good candidate solution to ensemble local models at the workers without predefined model weight thresholds, and without necessitating the NN models to be of the same architecture and feature space.