You are viewing v0.15.0 version.
A newer version
v0.29.3 is available.
Example Zoo
Below contains a non-exhuastive list of tutorials and scripts showcasing Accelerate
Official Accelerate Examples:
Basic Examples
These examples showcase the base features of Accelerate and are a great starting point
- Barebones NLP example
- Barebones distributed NLP example in a Jupyter Notebook
- Barebones computer vision example
- Barebones distributed computer vision example in a Jupyter Notebook
- Using Accelerate in Kaggle
Feature Specific Examples
These examples showcase specific features that the Accelerate framework offers
- Automatic memory-aware gradient accumulation
- Checkpointing states
- Cross validation
- DeepSpeed
- Fully Sharded Data Parallelism
- Gradient accumulation
- Memory-aware batch size finder
- Metric Computation
- Using Trackers
- Using Megatron-LM
Full Examples
These examples showcase every feature in Accelerate at once that was shown in “Feature Specific Examples”
- Complete NLP example
- Complete computer vision example
- Causal language model fine-tuning example
- Masked language model fine-tuning example
- Speech pretraining example
- Translation fine-tuning example
- Text classification fine-tuning example
- Semantic segmentation fine-tuning example
- Question answering fine-tuning example
- Beam search question answering fine-tuning example
- Multiple choice question answering fine-tuning example
- Named entity recognition fine-tuning example
- Image classification fine-tuning example
- Summarization fine-tuning example
- End-to-end examples on how to use AWS SageMaker integration of Accelerate
- Megatron-LM examples for various NLp tasks
Integration Examples
These are tutorials from libraries that integrate with 🤗 Accelerate:
Catalyst
DALLE2-pytorch
🤗 diffusers
fastai
- Distributed training from Jupyter Notebooks with fastai
- Basic distributed training examples with fastai