Package: fastxtend
fastxtendTrain fastai models faster (and other useful tools)Train fastai models faster with fastxtend’sfused
optimizers,Progressive
Resizingcallback, integratedFFCV
DataLoader,
and integratedPyTorch
Compilesupport.Feature overviewTrain Models FasterDrop infused
optimizers,
which are 21 to 293 percent faster then fastai native optimizers.Up to 75% optimizer memory savings with integratedbitsandbytes8-bit
optimizers.Increase GPU throughput and decrease training time with theProgressive
Resizingcallback.Use the highly optimizedFFCV
DataLoader,
fully integrated with fastai.Integrated support fortorch.compilevia theCompilecallbacks.General FeaturesFused implementations of modern optimizers, such asAdan,Lion, &StableAdam.Hugging FaceTransformers
compatibilitywith fastaiFlexiblemetricswhich can log on train, valid, or both. Backwards compatible with
fastai metrics.Easily usemultiple
lossesand log
each individual loss on train and valid.Multiple
profilersfor profiling training and identifying bottlenecks.A fastExponential Moving
Averagecallback for smoother training.VisionApplyMixUp,CutMix,
or Augmentations at once withCutMixUporCutMixUpAugment.Additionalimage
augmentations.Support for running fastaibatch transforms on
CPU.MoreattentionandpoolingmodulesA flexible implementation of fastai’sXResNet.Check out the documentation for additional splitters, callbacks,
schedulers, utilities, and more.Documentationhttps://fastxtend.benjaminwarner.devInstallfastxtend is avalible on pypi:pipinstallfastxtendfastxtend can be installed with task-specific dependencies forvision,ffcv,text,audio, orall:pipinstall"fastxtend[all]"To easily install most prerequisites for all fastxtend features, useCondaorMiniconda:condacreate-nfastxtendpython=3.11"pytorch>=2.1"torchvisiontorchaudio\pytorch-cuda=12.1fastainbdevpkg-configlibjpeg-turboopencvtqdmpsutil\terminaltablesnumpy"numba>=0.57"librosatimmkorniarichtyperwandb\"transformers>=4.34""tokenizers>=0.14""datasets>=2.14"ipykernelipywidgets\"matplotlib<3.8"-cpytorch-cnvidia-cfastai-chuggingface-cconda-forge

condaactivatefastxtend

pipinstall"fastxtend[all]"replacingpytorch-cuda=12.1with your preferredsupported version of
Cuda.To create an editable development install:gitclonehttps://github.com/warner-benjamin/fastxtend.gitcdfastxtend
pipinstall-e".[dev]"UsageLike fastai, fastxtend provides safe wildcard imports using python’s__all__.fromfastai.vision.allimport*fromfastxtend.vision.allimport*fromfastxtend.ffcv.allimport*In general, import fastxtend after all fastai imports, as fastxtend
modifies fastai. Any method modified by fastxtend is backwards
compatible with the original fastai code.ExamplesUse a fused ForEach optimizer:Learner(...,opt_func=adam(foreach=True))Or a bitsandbytes 8-bit optimizer:Learner(...,opt_func=adam(eightbit=True))Speed up image training using Progressive Resizing:Learner(...cbs=ProgressiveResize())Log an accuracy metric on the training set as a smoothed metric and
validation set like normal:Learner(...,metrics=[Accuracy(log_metric=LogMetric.Train,metric_type=MetricType.Smooth),Accuracy()])Log multiple losses as individual metrics on train and valid:mloss=MultiLoss(loss_funcs=[nn.MSELoss,nn.L1Loss],weights=[1,3.5],loss_names=['mse_loss','l1_loss'])Learner(...,loss_func=mloss,metrics=RMSE(),cbs=MultiLossCallback)Compile a model withtorch.compile:fromfastxtend.callbackimportcompilerlearn=Learner(...).compile()Profile a fastai training loop:fromfastxtend.callbackimportsimpleprofilerlearn=Learner(...).profile()learn.fit_one_cycle(2,3e-3)BenchmarkTo run the benchmark on your own machine, see theexample
scriptsfor details on how to replicate.