Package: pyaf
PyAF (Python Automatic Forecasting)PyAF is an Open Source Python library for Automatic Forecasting built on top of
popular data science python modules: NumPy, SciPy, Pandas and scikit-learn.PyAF works as an automated process for predicting future values of a signal
using a machine learning approach. It provides a set of features that is
comparable to some popular commercial automatic forecasting products.PyAF has been developed, tested and benchmarked using apython 3.xversion.PyAF is distributed under the3-Clause BSD license.Demoimportnumpyasnp,pandasaspdimportpyaf.ForecastEngineasautofif__name__=='__main__':# generate a daily signal covering one year 2016 in a pandas dataframeN=360df_train=pd.DataFrame({"Date":pd.date_range(start="2016-01-25",periods=N,freq='D'),"Signal":(np.arange(N)//40+np.arange(N)%21+np.random.randn(N))})# create a forecast engine, the main object handling all the operationslEngine=autof.cForecastEngine()# get the best time series model for predicting one weeklEngine.train(iInputDS=df_train,iTime='Date',iSignal='Signal',iHorizon=7);lEngine.getModelInfo()# => relative error 7% (MAPE)# predict one weekdf_forecast=lEngine.forecast(iInputDS=df_train,iHorizon=7)# list the columns of the forecast datasetprint(df_forecast.columns)# print the real forecasts# Future dates : ['2017-01-19T00:00:00.000000000' '2017-01-20T00:00:00.000000000' '2017-01-21T00:00:00.000000000' '2017-01-22T00:00:00.000000000' '2017-01-23T00:00:00.000000000' '2017-01-24T00:00:00.000000000' '2017-01-25T00:00:00.000000000']print(df_forecast['Date'].tail(7).values)# signal forecast : [ 9.74934646  10.04419761  12.15136455  12.20369717  14.09607727 15.68086323  16.22296559]print(df_forecast['Signal_Forecast'].tail(7).values)also availabe as a jupyter notebookFeaturesPyAF allows forecasting a time series (or a signal) for future values in a fully automated
way. To build forecasts, PyAF allows using time information (by identifyinglong-termevolution andperiodic patterns), analyzes thepast of the signal, exploitsexogenous data(user-provided time series that may be correlated with the signal) as well as thehierarchical structureof the signal (by aggregating spatial components forecasts, for example).PyAFusesPandasas a data access layer. It consumes data coming from a pandas data-
frame (with time and signal columns), builds a time series model, and outputs
the forecasts in a pandas data-frame. Pandas is an excellent data access layer,
it allows reading/writing a huge set of file formats, accessing various data
sources (databases) and has an extensive set of algorithms to handle data-
frames (aggregation, statistics, linear algebra, plotting, etc.).PyAF statistical time series models are built/estimated/trained usingscikit-learn.The following features are available :Training a modelto forecast a time series (given in a pandas data-frame
with time and signal columns).PyAF uses amachine learning approach(the signal is cut into estimation
and validation parts, respectively, 80% and 20% of the signal).Atime-series cross-validationcan also be used.Forecasting a time series model on a givenhorizon(forecast result is
also a pandas data-frame) and providingprediction/confidence intervalsfor
the forecasts.Generic training featuresSignal decompositionas the sum of a trend, periodic and AR components.PyAF works as a competition between acomprehensive set of possible signal
transformations and linear decompositions. For each transformed
signal, a set of possible trends, periodic components and AR models is
generated and all the possible combinations are estimated. The best
decomposition in terms of performance is kept to forecast the signal (the
performance is computed on a part of the signal that was not used for the
estimation).Signal transformationis supported beforesignal decompositions. Four
transformations are supported by default. Other transformations are
available (Box-Cox, etc.).All models are estimated usingstandard procedures and state-of-the-art
time series modeling. For example, trend regressions and AR/ARX models
are estimated using scikit-learn linear regression models.Standard performance measures are used (L1, RMSE, MAPE, MedAE, LnQ, etc.)PyAF analyzes thetime variableand infers the frequency from the data.Natural time frequencies are supported: Minute, Hour, Day, Week and Month.Strange frequencies like every 3.2 days or every 17 minutes are supported if data are recorded accordingly (every other Monday => two weeks frequency).The frequency is computed as the mean duration between consecutive observations by default (as apandas DateOffset).The frequency is used to generate values for future dates automatically.PyAF does its best when dates are not regularly observed. Time frequency is approximate in this case.Real/Integer valued (fake) dates are also supported and handled in a similar way.Exogenous Data SupportExogenous data can be provided to improve the forecasts. These are
expected to bestored in an external data-frame(this data-frame will be
merged with the training data-frame).Exogenous data are integrated into the modeling process through theirpast values(ARX models).Exogenous variables can be of any type(numeric, string, date or
object).Exogenous variables aredummifiedfor the non-numeric types, andstandardizedfor the numeric types.PyAF implementsHierarchical Forecasting. It follows the excellent approach used inRob J
Hyndman and George Athanasopoulos book. Thanks @robjhyndmanHierarchiesandgrouped time seriesare supported.Bottom-Up,Top-Down(using proportions),Middle-OutandOptimal Combinationsare
implemented.The modeling process iscustomizableand has a huge set ofoptions. The
default values of these options should however be OK to produce a reasonable quality model in a limited amount of time (a few minutes).These options give access to a full set ofsignal transformationsandAR-like modelsthat are not enabled by default.Gives rise to Logit, Fisher transformations as well as XGBoost, Support Vector Regressions and Croston intermittent models, LGBM, among others.By default, PyAF uses afast modethat activates many popular models. It is also possible to activate aslow mode, in which PyAF explores all possible models.Specific models and features can be customized.Abenchmarking processis in place (using M1, M2, M3 competitions, NN3,
NN5 forecasting competitions).This process will be used to control the quality of modeling changes introduced in the future versions of PyAF. Arelated  github issueis created.Benchmarks data/reports are saved in a separategithub repository.Samplebenchmark reportwith 1001 datasets from the M1 Forecasting Competition.Basicplottingfunctions using matplotlib with standard time series and
forecasts plots.Software QualityHighlightsAnobject-orientedapproach is used for the system design. Separation of
concerns is the key factor here.Fully written in Pythonwith NumPy, SciPy, Pandas and scikit-learn
objects. Tries to becolumn-basedeverywhere for performance reasons (respecting some modeling time and memory constraints).Internally using a fit/predict pattern, inspired by scikit-learn, to estimate/forecast the different signal components (trends, cycles and AR models).Atest-driven approach (TDD)is used. Test scripts are available in thetestsdirectory, one directory for each feature.TDDimplies that even the most recent features have some sample scripts in thisdirectory. Want to know how to use cross-validation with PyAF? Here aresome scripts.Somejupyter notebooksare available for demo purposes with standard time series and forecasts plots.Verysimple APIfor training and forecasting.A basicRESTful Web Service(Flask) is available.This service allows building a time series model, forecasting future data and some standard plots by providing a minimal specification of the signal in the JSON request body (at least a link to a csv file containing the data).Seethis docand therelated github issuefor more details.PyAF is a work in progress. The set of features is evolving. Your feature
requests, comments, help, hints are very welcome.InstallationPyAF has been developed, tested and used on a python 3.x version.It can be installed from PyPI for the latest official release:pip install pyafThe development version is also available by executing:pip install scipy pandas scikit-learn matplotlib pydot xgboost statsmodels
pip install --upgrade git+git://github.com/antoinecarme/pyaf.gitDevelopmentCode contributions are welcome. Bug reports, request for new features and
documentation, tests are welcome. Please use the GitHub platform for these tasks.You can check the latest sources of PyAF from GitHub with the command::git clone http://github.com/antoinecarme/pyaf.gitProject historyThis project was started in summer 2016 as a POC to check the feasibility of an
automatic forecasting tool based only on Python available data science software
(NumPy, SciPy, Pandas, scikit-learn, etc.).See theAUTHORS.rstfile for a complete list of contributors.Help and SupportPyAF is currently maintained by the original developer. PyAF support will be
provided when possible and even if you are not creating an issue, you are encouraged to followthese guidelines.Bug reports, improvement requests, documentation, hints and test scripts are
welcome. Please use the GitHub platform for these tasks.Please don't ask too much about new features. PyAF is only about forecasting (the last F). To keep PyAF design simple and flexible, we avoidFeature Creep.For your commercial forecasting projects, please consider using the services of a forecasting expert near you (be it an R or a Python expert).DocumentationAnintroductory notebookto the time series forecasting with PyAF is available here. It contains some real-world examples and use cases.A specific notebook describing the use of exogenous data isavailable here.Notebooks describing an example of hierarchical forecasting models are available forSignal Hierarchiesand forGrouped Signals.The python code is not yet fully documented. This is a top priority (TODO).CommunicationComments, appreciations, remarks, etc .... are welcome. Your feedback is
welcome if you use this library in a project or a publication.