Package: openfl-x
OpenFL-x - OpenFederatedLearning-extendedOpenFederatedLearning-extended (OpenFL-x) is anopen-source extensionofIntel® OpenFL1.4 supportingfederated bagging and boosting of any ML model. The software is entirely Python-based and comes with extensive examples, as described below, exploitingSciKit-Learnmodels. It has been successfully tested on x86_64, ARM and RISC-V platforms.InstallationIt is highly recommended to create avirtual environmentprior to installing OpenFL-x (with bothvenvorconda) to avoid compatibility issues with existing Python software already installed on the system.
Furthermore, since OpenFL-x is an extended version of OpenFL, it integrates all the features of the base framework plus the federated bagging and boosting functionalities: it is then suggested tonot install both software in the same virtual environmentsince this can lead to issues in the working of both software.You can get the last version of OpenFL-x frompypi:pip install openfl-xor, alternatively, you can clone this repository and runpip install:git clone https://github.com/alpha-unito/OpenFL-extended.git
cd OpenFL-extended
pip install .If this procedure completes successfully, you now have access to all the base features of OpenFL, plus the distributed bagging and boosting functionalities. Enjoy!Getting StartedThe quickest way to test OpenFL-x is to execute the examples available in theopenfl-tutorials/boosting-examplesfolder. There are many of them available to be run out of the box, each employing a different dataset and a different number of participants in the federation:NameAlgorithmDataset# Train samples# Test samples# labels# Features# EnvoyML modelAdaBoostF_AdultFederated boostingAdult30,13215,06021410DecisionTreeClassifierAdaBoostF_IrisFederated boostingIris3515342DecisionTreeClassifierAdaBoostF_forestcoverFederated boostingForestCover250,000245,14125410DecisionTreeClassifierAdaBoostF_krvskpFederated boostingKrvsKp2,55763923610DecisionTreeClassifierAdaBoostF_letterFederated boostingLetter16,0004,000261610DecisionTreeClassifierAdaBoostF_pendigitsFederated boostingPendigits7,4943,498101610DecisionTreeClassifierAdaBoostF_satFederated boostingSat4,4352,00083610DecisionTreeClassifierAdaBoostF_segmentationFederated boostingSegmentation2092,09971910DecisionTreeClassifierAdaBoostF_spliceFederated boostingSplice2,55263836110DecisionTreeClassifierAdaBoostF_vowelFederated boostingVowel792198112710DecisionTreeClassifierAdaBoostF_vehicleFederated boostingVehicle67716941810DecisionTreeClassifierRandomForest_IrisFederated baggingIris3515342DecisionTreeClassifierThe user can customise each example by changing the data distribution across the envoys, the number of envoys itself, the ML model used as weak learner, and the federation's aggregation algorithm. More information on how to run and personalise the examples are available in theboosting-examplesfolder.Example results of the execution of these tests and their variations are freely available onWandB. During the experimentation, many SciKit-Learn classifiers have been used as weak learners, such asExtremelyRandomizedTree,RidgeLinearRegression,MultiLayerPerceptron,KNearestNeighbors,GaussianNaiveBayes, andDecisionTree.Aggregation algorithmsOpenFL-x offers the possibility to experiment with basicfederated baggingand the federated boosting method calledAdaBoost.Fdeveloped by Polato et al. [1]. This work proposes two other federated versions of AdaBoost, namely DistBoost.F and PreWeak.F. However, AdaBoost.F has been selected as it provided the best experimental performances to the other two approaches.Concisely, AdaBoost.F createsiterativelyan AdaBoost model selecting the best-performing weak learner during each federated round. It goes like this:The aggregator receives the dataset size N from each collaborator and sends
them an initial version of the weak hypothesis.The aggregator receives the weak hypothesis hi from each collaborator and
broadcasts the entire hypothesis space to every collaborator.The errors ε committed by the global weak hypothesis on the local data are
calculated by each client and sent to the aggregator.The aggregator exploits the error information to select the best weak hypothesis c, adds it to the global strong hypothesis and sends the calculated
AdaBoost coefficient α to the collaboratorsMore details can be found in the original paper reported below.[1]Polato, Mirko, Roberto Esposito, and Marco Aldinucci. "Boosting the federation: Cross-silo federated learning without gradient descent." 2022 International Joint Conference on Neural Networks (IJCNN). IEEE, 2022.Performance optimisationsTogether with the new aggregation algorithm made available, OpenFL-x also introducesmany other little optimisationsto the original OpenFL coda, all aiming to reduce the execution time and improve the computational performance of the code:we empirically optimised the buffer sizes used by gRPC to accommodate larger models and avoid resizing operations (∼1.5% execution time improvement over the base implementation);we employed theCloudpickleserialisation framework over other options available (likedillandpickle) (∼2.6% execution time improvement over the base implementation);we modified the TensorDB to store only the essential information of the last two federation rounds (∼14.4% execution time improvement over the base implementation);we have lowered the few sleeps present in the code from 10 seconds to 0.01 seconds, which according to our experimentation on cluster systems, is the lowest value still yielding improvement (∼48.2% execution time improvement over the base implementation).These minor optimisations achieved an overall 5.46x speedup
over the base software performance. This performance improvement makes the execution of OpenFL-x and the choice of lightweight weak learners possible on a wide range of computing platforms, even low-power ones.PublicationThis work is currently under review at the29th International European Conference on Parallel and Distributed Computing.
The paper's citation and link will be provided as soon as they become available.A pre-print version of this software's paper is available onarXiv.ContactsThis software is developed and maintained byGianluca Mittone(gianluca.mittone@unito.it).