Package: xain-fl
Disclaimer: This is work-in-progress and not production-ready, expect errors to occur! Use at your own risk! Do not use for any security related issues!XAINThe XAIN project is building a privacy layer for machine learning so that AI projects can meet compliance such as
GDPR and CCPA. The approach relies on Federated Learning as enabling technology that allows production AI
applications to be fully privacy compliant.Federated Learning also enables different use-cases that are not strictly privacy related such as connecting data
lakes, reaching higher model performance in unbalanced datasets and utilising AI models on the edge.This repository contains the source code for running the Coordinator. The Coordinator is the component of Federated
Learning that selects the Participants for training and aggregates the models using federated averaging.The Participants run in a separate environment than the Coordinator and connect to it using an SDK. You can findherethe source code for it.Quick StartXAIN requiresPython 3.6.4+. To install thexain-flpackage just run:$python-mpipinstallxain-flInstall from sourceClone this repository:gitclonehttps://github.com/xainag/xain-fl.gitInstall this project with thedevprofile (NOTE: it is
recommended to install the project in a virtual environment):cdxain-fl
pipinstall-e'.[dev]'Verify the installation by running the testspytestBuilding the DocumentationThe project documentation resides underdocs/. To build the documentation
run:$cddocs/
$makedocsThe generated documentation will be underdocs/_build/html/. You can open the
root of the documentation by openingdocs/_build/html/index.htmlon your
favorite browser or simply run the command:$makeshowRunning the Coordinator locallyTo run the Coordinator on your local machine, you can use theexample-config.tomlfile:# If you have installed the xain_fl package,# the `coordinator` command should be directly availablecoordinator--configconfigs/example-config.toml# otherwise the coordinator can be started by executing the# `xain_fl` package:pythonxain_fl--configconfigs/example-config.tomlRun the Coordinator from a Docker imageThere are two docker-compose files, one for development and one for release.Development imageTo run the coordinator's development image, first build the Docker image:$dockerbuild-txain-fl-dev-fDockerfile.dev.Then run the image, mounting the directory as a Docker volume:$dockerrun-v$(pwd):/app-v'/app/xain_fl.egg-info'xain-fl-devcoordinatorThe command above uses a default configuration but you can also use a
custom config file:For instance, if you have a./custom_config.tomlfile that you'd
like to use, you can mount it in the container and run the coordinator
with:dockerrun\-v$(pwd)/custom_config.toml:/custom_config.toml\-v$(pwd):/app\-v'/app/xain_fl.egg-info'\xain-fl-dev\coordinator--config/custom_config.tomlRelease imageTo run the coordinator's release image, first build it:$dockerbuild-txain-fl.And then run it (this example assumes you'll want to use the default port):$dockerrun-p50051:50051xain-flDocker-composeThe coordinator needs a storage service that provides an AWS S3
API. For development, we useminio. We providedocker-composefiles that start coordinator container along with aminiocontainer,
and pre-populate the appropriate storage buckets.DevelopmentTo start both the coordinator and theminioservice use:docker-compose-fdocker-compose-dev.ymlupIt is also possible to only start the storage service:docker-compose-fdocker-compose-dev.ymlupminio-devinitial-bucketsRelease$docker-composeupRelated Papers and ArticlesAn introduction to XAIN’s GDPR-compliance Layer for Machine LearningCommunication-Efficient Learning of Deep Networks from Decentralized DataAnalyzing Federated Learning through an Adversarial LensTowards Federated Learning at Scale: System Design