# AutoFL: Towards AutoML in a Federated Learning Context
## 3 Analysis of a Contemporary AutoML Framework
While auto-sklearn is a powerful framework, it is not immediately suitable for automated machine learning in a federated learning context. Here is an overview of some of the more practical challenges:  
1. Auto-sklearn supports parallel computation and evaluation of ML models via the Dask.distributed framework ([[https://distributed.dask.org](https://distributed.dask.org)]([https://distributed.dask.org](https://distributed.dask.org)), accessed on 15 May 2023). Furthermore, parallelizing auto-sklearn across multiple machines is technically feasible (by configuring a Dask scheduler, a client, and multiple workers), though not as straightforward as a single-machine deployment. However, the nature of the distributed computing does not correspond with a federated learning context where no data but only model updates are shared with a centralized coordinating server. 2. Auto-sklearn leverages the scikit-learn [39] library, which specializes in classical machine learning pipelines. It has support for simple neural networks, such as a multi-layer perceptron (MLP) classifier, but not for the construction of more sophisticated neural networks for which specialized libraries, such as TensorFlow and PyTorch, exist. Whether or not to explore both classical ML models and deep learning models is a decision to be made by the MLOps engineer or data scientist, but ideally, the opportunity should be granted by the federated AutoML framework. 3. The scikit-learn library implements a multitude of classifiers and regressors for which implementing a federated equivalent is not trivial. For example, the iterative approach of federated learning works well for logistic regression (LR) and multi-layer perceptron (MLP) but not for support vector classification (SVC). Indeed, for LR model updates, the internal coefficients of the model can be easily merged with those of other model updates to construct an aggregated model, for example, through federated averaging. For other methods, such as SVC, the implementation does not allow direct access to these coefficients or to construct a new model via the aggregated coefficients. 4. As depicted in Figure 1, auto-sklearn not only selects ML models and optimizes their hyperparameters, but it also implements many techniques for automated data and feature pre-processing. In federated learning, it is not trivial to implement these. Regarding data pre-processing, one-hot encoding only works effectively if all clients in the federation have the same categorical values and use the same process to compute the derived features. However, this scenario is rather unlikely. The rescaling of numeric features may be a bit easier to compute collaboratively, but it assumes that at least some meta-features (e.g., min and max values) are being shared by the clients with the coordinating server. For feature pre-processing, techniques including dimensionality reduction are far less trivial to realize without centralizing the training data. 5. Auto-sklearn constructs an ensemble model that is optimized for a particular error metric (e.g., f1, see line 31 in Listing 1 while computing other metrics after constructing the ensemble (see line 32 in the same figure). However, in a federated learning context, there are multiple trade-offs, including the resource usage (CPU, memory, network) for the clients in the federation as well the coordinating server. While it is possible to implement a custom error metric, the auto-sklearn framework can only be extended by following the provided APIs, and those are only data-oriented. As such, auto-sklearn does not support multi-objective optimization out of the box. 6. In a federated learning context, there are many more hyperparameters for collaboratively learning an ML model.