Package: pytorch-forecasting
Documentation|Tutorials|Release NotesPyTorch Forecastingis a PyTorch-based package for forecasting time series with state-of-the-art network architectures. It provides a high-level API for training networks on pandas data frames and leveragesPyTorch Lightningfor scalable training on (multiple) GPUs, CPUs and for automatic logging.Our article onTowards Data Scienceintroduces the package and provides background information.PyTorch Forecasting aims to ease state-of-the-art timeseries forecasting with neural networks for real-world cases and research alike. The goal is to provide a high-level API with maximum flexibility for professionals and reasonable defaults for beginners.
Specifically, the package providesA timeseries dataset class which abstracts handling variable transformations, missing values,
randomized subsampling, multiple history lengths, etc.A base model class which provides basic training of timeseries models along with logging in tensorboard
and generic visualizations such actual vs predictions and dependency plotsMultiple neural network architectures for timeseries forecasting that have been enhanced
for real-world deployment and come with in-built interpretation capabilitiesMulti-horizon timeseries metricsHyperparameter tuning withoptunaThe package is built onpytorch-lightningto allow training on CPUs, single and multiple GPUs out-of-the-box.InstallationIf you are working on windows, you need to first install PyTorch withpip install torch -f https://download.pytorch.org/whl/torch_stable.html.Otherwise, you can proceed withpip install pytorch-forecastingAlternatively, you can install the package via condaconda install pytorch-forecasting pytorch -c pytorch>=1.7 -c conda-forgePyTorch Forecasting is now installed from the conda-forge channel while PyTorch is install from the pytorch channel.To use the MQF2 loss (multivariate quantile loss), also installpip install pytorch-forecasting[mqf2]DocumentationVisithttps://pytorch-forecasting.readthedocs.ioto read the
documentation with detailed tutorials.Available modelsThe documentation provides acomparison of available models.Temporal Fusion Transformers for Interpretable Multi-horizon Time Series Forecastingwhich outperforms DeepAR by Amazon by 36-69% in benchmarksN-BEATS: Neural basis expansion analysis for interpretable time series forecastingwhich has (if used as ensemble) outperformed all other methods including ensembles of traditional statical
methods in the M4 competition. The M4 competition is arguably the most important benchmark for univariate time series forecasting.N-HiTS: Neural Hierarchical Interpolation for Time Series Forecastingwhich supports covariates and has consistently beaten N-BEATS. It is also particularly well-suited for long-horizon forecasting.DeepAR: Probabilistic forecasting with autoregressive recurrent networkswhich is the one of the most popular forecasting algorithms and is often used as a baselineSimple standard networks for baselining: LSTM and GRU networks as well as a MLP on the decoderA baseline model that always predicts the latest known valueTo implement new models or other custom components, see theHow to implement new models tutorial. It covers basic as well as advanced architectures.Usage exampleNetworks can be trained with thePyTorch Lighning Traineronpandas Dataframeswhich are first converted to aTimeSeriesDataSet.# imports for trainingimportlightning.pytorchasplfromlightning.pytorch.loggersimportTensorBoardLoggerfromlightning.pytorch.callbacksimportEarlyStopping,LearningRateMonitor# import dataset, network to train and metric to optimizefrompytorch_forecastingimportTimeSeriesDataSet,TemporalFusionTransformer,QuantileLossfromlightning.pytorch.tunerimportTuner# load data: this is pandas dataframe with at least a column for# * the target (what you want to predict)# * the timeseries ID (which should be a unique string to identify each timeseries)# * the time of the observation (which should be a monotonically increasing integer)data=...# define the dataset, i.e. add metadata to pandas dataframe for the model to understand itmax_encoder_length=36max_prediction_length=6training_cutoff="YYYY-MM-DD"# day for cutofftraining=TimeSeriesDataSet(data[lambdax:x.date<=training_cutoff],time_idx=...,# column name of time of observationtarget=...,# column name of target to predictgroup_ids=[...],# column name(s) for timeseries IDsmax_encoder_length=max_encoder_length,# how much history to usemax_prediction_length=max_prediction_length,# how far to predict into future# covariates static for a timeseries IDstatic_categoricals=[...],static_reals=[...],# covariates known and unknown in the future to inform predictiontime_varying_known_categoricals=[...],time_varying_known_reals=[...],time_varying_unknown_categoricals=[...],time_varying_unknown_reals=[...],)# create validation dataset using the same normalization techniques as for the training datasetvalidation=TimeSeriesDataSet.from_dataset(training,data,min_prediction_idx=training.index.time.max()+1,stop_randomization=True)# convert datasets to dataloaders for trainingbatch_size=128train_dataloader=training.to_dataloader(train=True,batch_size=batch_size,num_workers=2)val_dataloader=validation.to_dataloader(train=False,batch_size=batch_size,num_workers=2)# create PyTorch Lighning Trainer with early stoppingearly_stop_callback=EarlyStopping(monitor="val_loss",min_delta=1e-4,patience=1,verbose=False,mode="min")lr_logger=LearningRateMonitor()trainer=pl.Trainer(max_epochs=100,accelerator="auto",# run on CPU, if on multiple GPUs, use strategy="ddp"gradient_clip_val=0.1,limit_train_batches=30,# 30 batches per epochcallbacks=[lr_logger,early_stop_callback],logger=TensorBoardLogger("lightning_logs"))# define network to train - the architecture is mostly inferred from the dataset, so that only a few hyperparameters have to be set by the usertft=TemporalFusionTransformer.from_dataset(# datasettraining,# architecture hyperparametershidden_size=32,attention_head_size=1,dropout=0.1,hidden_continuous_size=16,# loss metric to optimizeloss=QuantileLoss(),# logging frequencylog_interval=2,# optimizer parameterslearning_rate=0.03,reduce_on_plateau_patience=4)print(f"Number of parameters in network:{tft.size()/1e3:.1f}k")# find the optimal learning rateres=Tuner(trainer).lr_find(tft,train_dataloaders=train_dataloader,val_dataloaders=val_dataloader,early_stop_threshold=1000.0,max_lr=0.3,)# and plot the result - always visually confirm that the suggested learning rate makes senseprint(f"suggested learning rate:{res.suggestion()}")fig=res.plot(show=True,suggest=True)fig.show()# fit the model on the data - redefine the model with the correct learning rate if necessarytrainer.fit(tft,train_dataloaders=train_dataloader,val_dataloaders=val_dataloader,)