# AutoFL: Towards AutoML in a Federated Learning Context
## 4 Design and Implementation of the AutoFL Framework
### Methodology
#### 4.2.5 Bayesian Optimization  
Similar to auto-sklearn, our framework also adopts a Bayesian optimization approach to find the best model feasible through federated learning. However, contrary to the single optimization objective approach of auto-sklearn, our automated federated learning framework considers multiple optimization objectives in parallel, as outlined earlier. Additionally, the list of hyperparameters to be optimized is further extended with some federated learning-specific ones, such as the number of local training rounds and the number of participating clients in each training round. Under the hood, our framework uses the same optimization framework as auto-sklearn, namely SMAC3 [40]. This makes it fairly trivial to configure the hyperparameter search space for different kinds of ML models and their respective hyperparameters, as illustrated in Listing 2. The code example is a reduced version of the actual implementation that configures several more ML models and hyperparameters, as well as hyperparameters specifically for federated learning. In addition, the example only implements a single optimization objective. ```
1 cs=ConfigurationSpace()
2 model_type=Categoricalhyperparameter('model',['logistic_regression','mlp','random_forest'],default_value='logistic_regression')
3
4 cs.add_hyperparameter(model_type)
5
6################################################
7
8 hidden_layer_depth=UniformIntegerHyperparameter(name=\"hidden_layer_depth\",Lower=1,upper=3,default_value=1)
9 num.nodes_per_layer=UniformIntegerHyperparameter(name=num.nodes_per_layer\",lower=16,upper=264,default_value=32,log=True)
11
12 activation=Categoricalhyperparameter(\"activation\",choices=['tanh\",'relu\"],default_value='relu\")
13 alpha=UniformFloathyperparameter(\"alpha\",lower=1e-7,operoper=1e-1,default_value=1e-4,log=True)
14 learning_rate_init=UniformFloathyperparameter(\"learning_rate_init\",lower=1e-4,upper=0.5,default_value=1e-3,log=True)
15
16 early_stopping=CategoricalHyperparameter(\"early_stopping\",choices=[True,False],default_value=True)
17 max_depth=UniformIntegerHyperparameter(name=\"max_depth\",lower=5,upper=25,default_value=15,log=True)
18 max_features=UniformIntegerHyperparameter(name=\"max_features\",lower=1,upper=3,default_value=2)
19n.estimators=UniformIntegerHyperparameter(name=\"n.estimators\",lower=5,upper=25,default_value=15)
20
21 cs.add_hyperparameters([
22 hidden_layer_depth,num_nodes_per_layer,activation,alpha,learning_rate_init,early_stopping,max_depth,max_features,n_estimators)
23
24 nz.depth,max_features,n_estimators
25]
26
27
28 cs.add_conditions[[
30 InCondition(child=hidden_layer_depth,parent=model_type,values=['mlp']),
31 InCondition(child=num.nodes_per_layer,parent=model_type,values=['mlp']),
32 InCondition(child=activation,parent=model_type,values=['mlp']),
33 InCondition(child=alpha,parent=model_type,values=['mlp']),
34 InCondition(child=learning_rate_init,parent=model_type,values=['mlp']),
35 InCondition(child=early_stopping,parent=model_type,values=['mlp']),
36 InCondition(child=max_depth,parent=model_type,values=['random_forest']),
37 InCondition(child=max_features,parent=model_type,values=['random_forest']),
38 InCondition(child=n_estimators,parent=model_type,values=['random_forest']),
39]]]
40
41#Scenarioobject specifying the optimization environment
42 scenario=Scenario('run.obj':'quality',\"funcount:limit':funcount,\"cs':cs,\"deterministic':'true'
43])
44 smac=SMAC4 BB(scenarioScenario,te_runner=test_model_config)
45 incumbent=smac.optimize()
```  
**Listing 2** Simplified code example for the configuration of the hyperparameter search space of a subset of ML classifiers using the SMAC3 optimization framework.