# QoE Modeling on Split Features with Distributed Deep Learning
## 7 Conclusions and Outlook  
In this article, we present two distributed machine learning techniques, \(DEL\) and \(vFL\), which can be used for collaborative model development on decentralized datasets with different feature sets. \(vFL\) in particular enables training machine learning models collaboratively between QoE research entities, which potentially benefits all collaborating entities mutually, even if the research entities have different feature sets. We first reveal the composite nature of a QoE model, by decoupling the specific QoE factors from the generic ones, using both domain-expertise and data-driven approaches. We demonstrate the knowledge sharing via model transfer followed by ensemble, \(DEL\). We show that this process helps to customize the local model with the help from the generic model. This is beneficial in the case when there is no sufficient local training dataset at the target domain. In addition, due to the nature of the ensemble method, the models that participate in the training can be of any algorithm, hence they can be freely selected. We show that, by using \(DEL\), a small QoE dataset with specific features at the target domain can benefit (up to 12% improvement in estimation accuracy) further from a generic model (received from source domain). We primarily present \(vFL\) in the scope of QoE modeling, as a suitable technique to train machine learning models on split features in a data-parallel distributed setting where there exists no direct communication link in between the collaborating entities. We present that \(vFL\) results are on-par with the _Fully Centralized_ setting. Experiments indicated that \(vFL\) can benefit the local nodes (on average 26%) especially when the features are split evenly such that all split nodes have equally weak indicative features for the model estimation. Moreover, we presented different ML techniques, thanks to our low level implementation, for reducing the network footprint and the training time, which would help in minimizing the energy consumption during \(vFL\) training. In cases when the target group model lacks a large size training dataset and has at least a subset of generic features as in the source group, \(DEL\) approach is suitable given that the target domain model can benefit from a model trained on a larger dataset with generic features on the source domain. From a privacy perspective, data exchanged in \(vFL\), is bounded by the cut-layer. Consequently this reduces the attack surface as opposed to \(hFL\) where the entire model is shared in every iteration from each worker. At the same time, it introduces a critical point in the architecture that can be further reinforced through the use of known techniques such as Secure Multiparty Computation (MPC) or oblivious protocols. Overall, we recommend the presented \(vFL\) approach due to its capability to \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & _Isolated_ & _Fully Centralized_ & _DEL_ & \(vFL\) \\ \hline Data share & \(\times\) & ✓ & \(\times\) & \(\times\) \\ \hline Model share & \(\times\) & \(\times\) & ✓ All-layers & ✓ Cut-layer \\ \hline Sample dep. & \(\times\) & ✓ & \(\times\) & ✓ Synch. required \\ \hline Feature dep. & \(\times\) & ✓ & ✓ Subset & \(\times\) \\ \hline Jointly train. & \(\times\) & ✓ & \(\times\) & ✓ \\ \hline Aggr. & \(\times\) & \(\times\) & Average & Concatenate \& Split \\ \hline Model agnos. & ✓ & ✓ & ✓ & \(\times\) Neural Net. \\ \hline Worker contrib. & \multirow{2}{*}{N/A} & \multirow{2}{*}{N/A} & \multirow{2}{*}{Manual} & \multirow{2}{*}{Automatic} \\ adjustment & & & & \\ \hline Worker & \multirow{2}{*}{0} & \(\sum_{f}\sum_{s}v_{f,s}\) & \(\sum_{l}\sum_{n}v_{l,n}\) & \(2\sum_{r}\sum_{s}\sum_{c}v_{r,s,c}\) \\ \hline Master & \multirow{2}{*}{0} & \(\sum_{w}\sum_{f}\sum_{s}v_{w,f,s}\) & \(\sum_{w}\sum_{l}\sum_{n}v_{w,l,n}\) & \(2\sum_{w}\sum_{r}\sum_{s}\sum_{c}v_{w,r,s,c}\) \\ \hline Accuracy & Poor & Good & Good & Good \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparison of different training methodologies.