Package: ablator
🚀 ABLATOR is aDISTRIBUTED EXECUTION FRAMEWORKdesigned to enhance ablation studies in complex machine learning models. It automates the process of configuration and conducts multiple experiments in parallel.What are Ablation Studies?It involves removing specific parts of a neural network architecture or changing different aspects of the training process to examine their contributions to the model's performance.Why ABLATOR?As machine learning models grow in complexity, the number of components that need to be ablated also increases. This consequently expands the search space of possible configurations, requiring an efficient approach to horizontally scale multiple parallel experimental trials. ABLATOR is a tool that aids in the horizontal scaling of experimental trials.Instead of manually configuring and conducting multiple experiments with various hyperparameter settings, ABLATOR automates this process. It initializes experiments based on different hyperparameter configurations, tracks the state of each experiment, and provides experiment persistence on the cloud.Key FeaturesIt is a tool that simplifies the process of prototyping of models.It streamlines model experimentation and evaluation.It offers a flexible configuration system.It facilitates result interpretation through visualization."Auto-Trainer" feature reduces redundant coding tasks.ABLATOR vs. Without ABLATORLeft, ABLATOR efficiently conducts multiple trials in parallel based and log the experiment results.Right,manually, one would need to run trials sequentially, demanding more effort and independent analysis.InstallFor MacOS and Linux systems directly install via pip.pip install ablatorIf you are using Windows, you will need to install WSL using the official from Microsoft.WSL is a Linux subsystem and for ABLATOR purposes is identical to using Linux.Basic Concepts1. Create your Configurationfromtorchimportnnimporttorchfromablatorimport(ModelConfig,ModelWrapper,OptimizerConfig,TrainConfig,configclass,Literal,ParallelTrainer,SearchSpace,)fromablator.config.mpimportParallelConfig@configclassclassTrainConfig(TrainConfig):dataset:str="random"dataset_size:int@configclassclassModelConfig(ModelConfig):layer:Literal["layer_a","layer_b"]="layer_a"@configclassclassParallelConfig(ParallelConfig):model_config:ModelConfigtrain_config:TrainConfigconfig=ParallelConfig(experiment_dir="ablator-exp",train_config=TrainConfig(batch_size=128,epochs=2,dataset_size=100,optimizer_config=OptimizerConfig(name="sgd",arguments={"lr":0.1}),scheduler_config=None,),model_config=ModelConfig(),device="cpu",search_space={"model_config.layer":SearchSpace(categorical_values=["layer_a","layer_b"])},total_trials=2,)2. Define your ModelclassSimpleModel(nn.Module):def__init__(self,config:ModelConfig)->None:super().__init__()ifconfig.layer=="layer_a":self.param=nn.Parameter(torch.ones(100,1))else:self.param=nn.Parameter(torch.randn(200,1))defforward(self,x:torch.Tensor):x=self.paramreturn{"preds":x},x.sum().abs()classSimpleWrapper(ModelWrapper):defmake_dataloader_train(self,run_config:ParallelConfig):dl=[torch.rand(100)foriinrange(run_config.train_config.dataset_size)]returndldefmake_dataloader_val(self,run_config:ParallelConfig):dl=[torch.rand(100)foriinrange(run_config.train_config.dataset_size)]returndl3. Launch 🚀mywrapper=SimpleWrapper(SimpleModel)withParallelTrainer(mywrapper,config)asablator:ablator.launch(".")Learn More about ABLATOR ModulesConfiguration ModuleTraining ModuleExperiment and Metrics ModuleAnalysis ModuleTutorialsExplore a variety of tutorials and examples on how to utilize ABLATOR.
Ready to dive in? 👉Ablation TutorialsContribution GuidelinesABLATOR is open source, and we value contributions from our community!
Check out ourDevelopment Guidefor details on our development process and insights into the internals of the ABLATOR library.For any bugs or feature requests related to ABLATOR, please visit our GitHub Issues or reach out to slackAblator CommunityPlatformPurposeSupport LevelGitHub IssuesTo report issues or suggest new features.ABLATOR TeamSlackTo collaborate with fellow ABLATOR users.CommunityDiscordTo inquire about ABLATOR usage and collaborate with other ABLATOR enthusiasts.CommunityTwitterFor staying up-to-date on new features of Ablator.ABLATOR TeamReferences@inproceedings{fostiropoulos2023ablator,
  title={ABLATOR: Robust Horizontal-Scaling of Machine Learning Ablation Experiments},
  author={Fostiropoulos, Iordanis and Itti, Laurent},
  booktitle={AutoML Conference 2023 (ABCD Track)},
  year={2023}}