Package: fedsim
FedSimFedSim is a comprehensive and flexible Federated Learning Simulator! It aims to provide the researchers with an easy to develope/maintain simulator for Federated Learning.
See documentation athere!InstallationpipinstallfedsimThat’s it! You are all set!Design ArchitectureCLIMinimal exampleFedsim provides powerful cli tools that allow you to focus on designing what is truly important.
Simply enter the following command to begin federatively training a model.fedsim-clifed-learnThe “MNIST” dataset is partitioned on 500 clients by default, and the FedAvg algorithm is used to train a minimal model with two fully connected layers.
A text file is made that descibes the configuration for the experiment and a summary of results when it is finished. Additionally, a tensorboard log file is made to monitor the scores/metrics of the training.
The directory that these files are stored is (reconfigurable and is) displayed while the experiment is running.Hooking scores to cli toolsIn case you are interested in a certain metric you can make a query for it in your command.
For example, lets assume we would like to test and report:
* the accuracy score of the global model on global test dataset both every 21 rounds and every 43 rounds.
* the average accuracy score of the local models every 15 rounds.
Here’s how we modify the above command:fedsim-clifed-learn\--global-scoreAccuracyscore_name:acc21split:testlog_freq:21\--global-scoreAccuracyscore_name:acc43split:testlog_freq:43\--local-scoreAccuracysplit:trainlog_freq:15CheckFedsim Scores Pagefor the list of all other scores like Accyracy or define your custom score.Changing the DataData partitioning and retrieval is controlled by aDataManagerobject. This object could be controlled through-dor–data-managerflag in most cli commands.
In the following we modify the arguments of the defaultDataManagersuch thatCIFAR100is partitioned over 1000 clients.fedsim-clifed-learn\--data-mangerBasicDataManagerdataset:cifar100num_partitions:1000\--num-clients1000\--modelSimpleCNN2num_classes:100\--global-scoreAccuracysplit:testlog_freq:15Notice that we also changed the model from default toSimpleCNN2which by default takes 3 input channels.
You can learn about existing data managers atdata manager documentationand Custom data managers atthis guide to make Custom data managers.NoteArguments of the constructor of any component (rectangular boxes in the image of design architecture) could be given inarg:valueformat following its name (orpathif a local file is provided).
Among these component, the algorithm is special, in that the arguments are controlled internally. The only arguments of the algorithm object that could be directly controlled in your commands is the algorithm specific ones (mostly hyper-parameters).
Examples:fedsim-clifed-learn--algorithmAdaBestmu:0.01beta:0.6...Feed CLI with Customized ComponentsThe cli tool can take a locally defined component by ingesting its path.
For example, to automatically include your custom algorithm by the a command of the cli tool, you can place your class in a python file and pass the path of the file to-aor–algorithmoption (without .py) followed by colon and name of the algorithm definition (class or method).
For instance, if you have algorithmCustomFLAlgorithmstored in afoo/bar/my_custom_alg.py, you can pass–algorithm foo/bar/my_custom_alg:CustomFLAlgorithm.fedsim-clifed-learn--algorithmfoo/bar/my_custom_alg_file:CustomFLAlgorithmmu:0.01...The same is possible for any other component, for instance for a Custom model:fedsim-clifed-learn--modelfoo/bar/my_model_file:CustomModelnum_classes:1000...More about cli commandsFor help with cli checkfedsim-cli documentationor read the output of the following commands:fedsim-cli--helpfedsim-clifed-learn--helpfedsim-clifed-tune--helpPython APIFedsim is shipped with some of the most well-known Federated Learning algorithms included. However, you will most likely need to quickly develop and test your custom algorithm, model, data manager, or score class.
Fedsim has been designed in such a way that doing all of these things takes almost no time and effort. Let’s start by learning how to import and use Fedsim, and then we’ll go over how to easily modify existing modules and classes to your liking.
Check the following basic example:fromlogallimportTensorboardLoggerfromfedsim.distributed.centralized.trainingimportFedAvgfromfedsim.distributed.data_managementimportBasicDataManagerfromfedsim.modelsimportSimpleCNN2fromfedsim.lossesimportCrossEntropyLossfromfedsim.scoresimportAccuracyn_clients=1000dm=BasicDataManager("./data","cifar100",n_clients)sw=TensorboardLogger(path=None)alg=FedAvg(data_manager=dm,num_clients=n_clients,sample_scheme="uniform",sample_rate=0.01,model_def=partial(SimpleCNN2,num_channels=3),epochs=5,criterion_def=partial(CrossEntropyLoss,log_freq=100),batch_size=32,metric_logger=sw,device="cuda",)alg.hook_local_score(partial(Accuracy,log_freq=50),split='train,score_name="accuracy",)alg.hook_global_score(partial(Accuracy,log_freq=40),split='test,score_name="accuracy",)report_summary=alg.train(rounds=50)Side NotesDo not use double underscores (__) in argument names of your customized classes.0.9.0 (2022-09-21)make all user methods in algorithms static (no self argument).better encapsulation.expand definition of storage for read and write protection.0.8.3 (2022-09-18)add option for valid split on global data in basic dta manager.0.8.2 (2022-09-12)changed image paths to links from github in readme0.8.1 (2022-09-12)fix a minor bug with a link in readme0.8.0 (2022-09-12)some major revision to documentationsome enhancement to FedProx compatibility with v0.7+0.7.0 (2022-09-10)algorithms got more secure with local storageredefined model architecturesfixed bug in default step closure’made random seed more consistent0.6.2 (2022-08-31)fixed some errors in docstring of central FL algorithmsadd sample balance param to to identifiers of data manager0.6.1 (2022-08-17)fixed bug inpartition_global_dataofBasicDataManagersome changes in default values for better log storage and aggregation0.6.0 (2022-08-16)changed the name of cli directoryadded cli testsadded support for pytorch original lr schedulersimproved docsadded version option to fedsim-cli0.5.0 (2022-08-15)completed lr schedulers and generalized them for all levelschanged some argument names and default values0.4.1 (2022-08-12)fixed bugs with mismatched loss_fn argument name in cli commandschanged alleval_freqarguemnts to unifiedlog_req0.4.0 (2022-08-12)changed the structure of scores and lossesmade it possible to hook multiple local and global scores0.3.1 (2022-08-09)added advanced learning rate schedulersproperly tested r2r lr scheduler0.3.0 (2022-08-09)added fine-tuning to cli,fed-tunecleaner climade optimizers and schedulers user definableimproved logging0.2.0 (2022-08-01)cleaned the API reference in docschanged cli name tofedsim-cliimproved documentationimproved importingchanged the way custom objects are passed to cli0.1.4 (2022-07-23)changed FLAlgorithm to CentralFLAlgorithm for more clearityset default device to cuda if available otherwise to cpu in fed-learn clifix wrong superclass names in demofix the confusion withsave_dirandsave_pathin DataManager classes0.1.3 (2022-07-08)the documentation is redesigned and mostly automated.documentation now is available athttps://fesim.varnio.comadded code of coduct from github tempalate0.1.2 (2022-07-05)changed ownership of repo from fedsim-dev to varnio0.1.1 (2022-06-22)added fedsim.scores which wraps torch loss functions and sklearn scoresmoved reporting mechanism of distributed algorithm for supporting auto monitoradded AppendixAggregator which is used to hold metric scores and report final resultsapply a patch for wrong pypi supported python versions0.1.0 (2022-06-21)First major pre-release.The package is restructureddocs is updated and checked to pass through tox steps0.0.4 (2022-06-14)Fourth release on PyPI.