Package: enscale
The project is currently under heavy development, and focusing onPyTorchand therecommendationscenario.AboutEnscaleis aninstant distributed computinglibrary based on theRay TrainandRay Data, which is scalable, efficient, and easy-to-use. It accelerates the development of any ML/DL training workload, on any cloud or local, at any parallelism size.GoalsLaunch any interactive ML/DL workloadsinstantlyon your laptop or to any cloudScaleyour own single-machine neural network modules to a nativedistributedmannerApplyheterogeneousarchitectureData scientist-friendlyAPISparseanddense featuredefinitionsNon-GoalsNot a neural network library, there are only some benchmark modules provided.Getting StartedPrerequisitesPython version >=3.7.1Packagesrequests >=2.26.0ray >=1.9.2torch >=1.9.1pandas >=1.3.5pyarrow >=6.0.1InstallationUsing Pre-compiled Wheels# CPU versionpipinstallenscaleFrom Sourcegitclonehttps://github.com/ryantd/enscalecdenscale
pipinstall-e.Runtime environmentThe library can launchlocallyor on anycloud providerwith Ray set up.If you want to launch on the cloud, go through thisdocto set up your Ray Cluster. And then you can useenviron_validate(n_cpus=N, cluster_endpoint="ray://<head_node_host>:<port>")to connect your cluster.Or just useenviron_validate(n_cpus=N)to have a local experience.You can add more nativeray.initarguments, just put them intoenviron_validatecall. Likeenviron_validate(n_cpus=N, ignore_reinit_error=True)to make Ray suppresses errors from callingray.init()a second time.Lightning exampleSee more hands-on and advanced exampleshere, like heterogeneous support and sparsity definition.The following example requiressklearnto be installed. Andtqdmis optional, which enables progress reporting.importtorchimporttorch.nnasnnfromsklearn.metricsimportroc_auc_scorefromenscale.utilimportpprint_results,load_benchmark_datasetfromenscale.model.ctrimportDeepFMfromenscaleimportNeuralNetTrainer,environ_validateN_WORKERS=2N_DATA_PROCESSOR=1# ray environment setupenviron_validate(n_cpus=N_DATA_PROCESSOR+N_WORKERS)# load dataset and sparsity definition pre-defineddatasets,feature_defs,dataset_options=load_benchmark_dataset(# set your own dataset by `data_path="criteo_mini.txt"`separate_valid_dataset=False)# trainer setuptrainer=NeuralNetTrainer(# module and dataset configsmodule=DeepFM,# your own nn.Module or built in modulesmodule_params={"dense_feature_defs":feature_defs["dense"],"sparse_feature_defs":feature_defs["sparse"],},dataset=datasets,dataset_options=dataset_options,# trainer configsepochs=5,batch_size=512,loss_fn=nn.BCELoss(),optimizer=torch.optim.Adam,metric_fns=[roc_auc_score],# logger callbackscallbacks=["json"],# computation abstract on distributednum_workers=N_WORKERS,)# run and print resultsresults=trainer.run()pprint_results(results)ArchitectureRoadmapHeterogeneous Strategy on Distributed TrainingSync Parameter ServerAync Parameter ServerHybird Phase 1: use sync or async for the dense or sparse component as you like, under homogeneous architectureHybird Phase 2: you can choose async PS for the sparse component, and sync Ring Allreduce (like PyTorch's DDP) for the dense componentFramework SupportPyTorch: no specific plan to support other frameworksAdvanced Parallel MechanismHeavy integratedtorchrecAccelerator SupportGPU: complete inspection requiredReferenceRay and Ray Train: Ray Train is a lightweight library for distributed deep learning, allowing you to scale up and speed up training for your deep learning models. Docshere.DeepCTR-Torch: Easy-to-use, modular and extendible package of deep-learning based CTR models.LicenseEnscale is MIT licensed, as found in theLICENSEfile.