Package: delta-task
deltaDelta is a lib for writing privacy computing task executed ondelta framework, uses Pytorch as it backend.
It helps users focus on implementing computing models and logic, no need to worry about the data privacy (Delta framework do it for you).installFirst install pytorchusing pip:pipinstall-Udelta-taskinstall from source code:git clone this repopipinstall.exampleexample.pyusageNow lib delta supports horizontal federated learning task.how to write a TaskIn delta, you can define a horizontal federated learning task by inherit the base classdelta.task.HorizontalTaskHorizontalTaskis a abstract class, inspite of the constructor__init__, you have to implement the following four methods:train(self, dataloader: Iterable)get_params(self) -> List[torch.Tensor]validate(self, dataloader: Iterable) -> Dict[str, float]preprocess(self, x, y=None)initdef__init__(self,name:str,dataset:str,max_rounds:int,validate_interval:int=1,validate_frac:float=0.1):When inheritHorizontalTask, you must call the constructor of base class bysuper().__init__params:paramtypedescriptionnamestringthe task namedatasetstringthe dataset which the task usesmax_roundsintmax execution rounds of taskvalidate_intervalintvalidation every {validate_interval} roundsvalidate_fracfloatvalidation dataset percentage, should be in [0, 1)traindeftrain(self,dataloader:Iterable)You can implement model training logic in this function, a traditional implementation is like this:deftrain(self,dataloader:Iterable):x,y=batch# forwardingy_pred=self.model(x)# loss calculationloss=self.loss_func(y_pred,y)# backwardingself.optimizer.zero_grad()loss.backward()self.optimizer.step()params:paramtypedescriptiondataloaderIterablethe dataloader of training datasetget_paramsdefget_params(self)->List[torch.Tensor]Get trainable model parameters. Delta framework will secure aggregate theses parameters from different nodes to get the global model parameters.A traditional implementation is like this:defget_params(self)->List[torch.Tensor]:returnlist(self.model.parameters())returns:typedescriptionList[torch.Tensor]parameters listvalidatedefvalidate(self,dataloader:Iterable)->Dict[str,float]You can implement the validation logic in this function.
The validation result is aDict[str, float], whose key is the validation metric name, value is the vaidation metric value.
Delta framework will secure aggregate nodes' validation result to get the global validation result.A traditional implementation is like this:defvalidate(self,dataloader:Iterable)->Dict[str,float]:total_loss=0count=0ys=[]y_s=[]forbatchindataloader:x,y=batchy_pred=self.model(x)loss=self.loss_func(y_pred,y)total_loss+=loss.item()count+=1y_=torch.argmax(y_pred,dim=1)y_s.extend(y_.tolist())ys.extend(y.tolist())avg_loss=total_loss/counttp=len([1foriinrange(len(ys))ifys[i]==y_s[i]])precision=tp/len(ys)return{"loss":avg_loss,"precision":precision}In the above code,validatefunction compute the average loss and precision on the input validation dataset.params:paramtypedescriptiondataloaderIterablethe dataloader of validation datasetpreprocessdefpreprocess(self,x,y=None)Preprocess function preprocess each data item before training and validation.The input parameterxis the data item,yis the corresponding data label if the dataset has a label.The type ofxdepends on the dataset. It can be anp.ndarray, atorch.Tensor, apd.DataFrameor aImage.Image.
In the preprocess function, you should convert thexto atorch.Tensor.The type ofyis string (if hasy), means the label name, and you should convert it to atorch.Tensorcompatible with your loss function.The preprocess function should return twotorch.Tensorifxandyall exist, or only return onetorch.Tensor.(optional) dataloader_configdefdataloader_config(self)->Union[Dict[str,Any],Tuple[Dict[str,Any],Dict[str,Any]]]:You can specify dataloader arguments in this function.This function can return one or two dictionary as the kwargs passed to dataloader constructor.
If it returns only one dictionary, then the result will be passed to both training dataloader and validation dataloader.
If it returns two dictionary, then the first one will be passed to training dataloader, the second one will be passed to validation dataloader.The default implementation is as the follows:defdataloader_config(self,)->Union[Dict[str,Any],Tuple[Dict[str,Any],Dict[str,Any]]]:return{"shuffle":True,"batch_size":64,"drop_last":True}(optional) algorithmdefalgorithm(self)->HorizontalAlgorithm:You can specify secure aggregation algorithm in this function.The algorithm you can use now isdelta.algorithm.horizontal.FedAvganddelta.algorithm.horizontal.FaultTolerantFedAvg, which are subclasses ofdelta.algorithm.horizontal.HorizontalAlgorithm.The difference betweenFedAvgandFaultTolerantFedAvgis thatFaultTolerantFedAvgcan tolerant some nodes being offline during secure aggregation, butFedAvgcan't.Parameters ofFedAvgandFaultTolerantFedAvgare the same.params:paramtypedescriptionmerge_interval_iterintaggregate every {merge_interval_iter} itermerge_interval_epochintaggregate every {merge_interval_epoch} round. This param is mutually exclusive with merge_interval_iter, only one of them can be greater than 0min_clientsintminimal node count in a roundmax_clientsintmaximal node count in a roundwait_timeoutOptional[float]timeout in seconds for task computation in a round, default value is 60 secondconnection_timeoutOptional[float]timeout in seconds for each stage of the secure aggregation, default value is 60 secondprecisionintprecision of the result. result will be rounded by this precisioncurveCURVE_TYPEelliptic curve used for asymmetric encryption in the secure aggregation, default value is "secp256k1"dataset formatDelta framework now supports four kinds of dataset format:Numpy array, with file extension.npy,.npzTorch tensor, with file extension.ptTable file, with file extension.csv,.tsv,.xls,.xlsx. Table file will be loaded bypandasImage file. Image file will be loaded byPILA dataset can be a single file or a directory. When creating a task, you can refer the dataset by its name.When dataset is a single file, the first dimension of the data is the data sample, and each data sample has no label.When dataset is a directory, there are two acceptable directory structure.
The first one is that data files are placed in the directory. Each file represents a data sample and each data sample has no label.
The second one is that directory contains several sub directories, and each sub directory represents a class of data. The sub directory name is the class name.
Data files are placed in one sub directory according to its class.