nowcast-lstm-dhopp1 0.0.2
Code for running LSTM neural networks on economic data for nowcasting
Requires Python: >=3.6

# nowcast_lstm
**Installation**: `pip install nowcast-lstm-dhopp1`
<br><br>
[LSTM neural networks](https://en.wikipedia.org/wiki/Long_short-term_memory) have been used for nowcasting [before](https://papers.nips.cc/paper/2015/file/07563a3fe3bbe7e3ba84431ad9d055af-Paper.pdf), combining the strengths of artificial neural networks with a temporal aspect. However their use in nowcasting economic indicators remains limited, no doubt in part due to the difficulty of obtaining results in existing deep learning frameworks. This library seeks to streamline the process of obtaining results in the hopes of expanding the domains to which LSTM can be applied.

While neural networks are flexible and this framework may be able to get sensible results on levels, the model architecture was developed to nowcast growth rates of economic indicators. As such training inputs should ideally be stationary and seasonally adjusted. One of the model's strengths is its ability to handle mixed frequency data and missing values, an ability predicated on growth rates and the ability to fill missing values with 0s. 

Further explanation of the background problem can be found in [this UNCTAD research paper](https://unctad.org/system/files/official-document/ser-rp-2018d9_en.pdf). Further explanation and results to be published in an upcoming [UNCTAD](https://unctad.org/) research paper.

## Performance vs Dynamic Factor Models
The graphic below shows the LSTM's performance relative to the current nowcasting standard, DFMs. The data was obtained in the following manner:
- three target variables were chosen: global exports in services, merchandise trade value, and merchandise trade volume
- out of a pool of 100 potential explanatory variables/economic series, a random sample was chosen 500 times for each target variable
- the LSTM and DFM were trained on a training sample, then their performance metrics (MAE and RMSE) were calculated on a test sample. 
- LSTM's performance metrics as a percentage of the DFM's were then calculated and plotted in the below distribution plots.i
- a distribution to the left of the vertical line means on average an LSTM performance metric better than the DFM's, e.g. a distribution centered around 50% means on average the LSTM's error was only 50% that of the DFM's.
- vertical faceting refers to different vintages, how the trained models were able to perform on older data, i.e. with less recent data published. 
![text](performance.jpg)

## Quick usage
The main object and functionality of the library comes from the `LSTM` object. Given `data` = a pandas DataFrame of monthly data + a quarterly target series to run the model on, usage is as follows:
```python
from nowcast_lstm.LSTM import LSTM

model = LSTM(data, "target_col_name", n_timesteps=12) # default parameters with 12 timestep history

model.X # array of the transformed training dataset
model.y # array of the target values

model.mv_lstm # list of trained PyTorch network(s)
model.train_loss # list of training losses for the network(s)

model.train()
model.predict(model.X) # predictions on the training set
# predicting on a testset/completely new data
model.predict(
	LSTM(new_data, "target_col_name", n_timesteps, drop_missing_ys=False).X # drop missing ys = false to keep observations where actual target isn't available yet
)

# to generate a synthetic data vintage of data
model.gen_ragged_X(pub_lags, lag)

```

## LSTM parameters
- `data`: `pandas DataFrame` of the data to train the model on. Should contain a target column. Any non-numeric columns will be dropped. It should be in the most frequent period of the data. E.g. if I have three monthly variables, two quarterly variables, and a quarterly series, the rows of the dataframe should be months, with the quarterly values appearing every three months (whether Q1 = Jan 1 or Mar 1 depends on the series, but generally the quarterly value should come at the end of the quarter, i.e. Mar 1), with NAs or 0s in between. The same logic applies for yearly variables.
- `target_variable`: a `string`, the name of the target column in the dataframe.
- `n_timesteps`: an `int`, corresponding to the "memory" of the network, i.e. the target value depends on the x past values of the independent variables. For example, if the data is monthly, `n_timesteps=12` means that the estimated target value is based on the previous years' worth of data, 24 is the last two years', etc. This is a hyper parameter that can be evaluated.
- `drop_missing_ys`: a `boolean` dictating whether observations with a missing target (y) should be dropped from the model input datasets created. This should be `True` for initial instantiation and generating the training dataset, since missing ys are set to 0 and shouldn't be considered when training. However, it should be set to `False` when predicting on a test or new dataset, to ensure that predictions on time periods where the target isn't available yet are still produced.
- `n_models`: `int` of the number of networks to train and predict on. Because neural networks are inherently stochastic, it can be useful to train multiple networks with the same hyper parameters and take the average of their outputs as the model's prediction, to smooth output.
- `train_episodes`: `int` of the number of training episodes/epochs. A short discussion of the topic can be found [here](https://machinelearningmastery.com/difference-between-a-batch-and-an-epoch/).
- `batch_size`: `int` of the number of observations per batch. Discussed [here](https://machinelearningmastery.com/difference-between-a-batch-and-an-epoch/)
- `lr`: `float` of the learning rate of network. Discussed [here](https://machinelearningmastery.com/understand-the-dynamics-of-learning-rate-on-deep-learning-neural-networks/).
- `decay`: `float` of the rate of decay of the learning rate. Also discussed [here](https://machinelearningmastery.com/understand-the-dynamics-of-learning-rate-on-deep-learning-neural-networks/). Set to `0` for no decay.
- `n_hidden`: `int` of the number of hidden states in the LSTM network. Discussed [here](https://machinelearningmastery.com/stacked-long-short-term-memory-networks/).
- `n_layers`: `int` of the number of LSTM layers to include in the network. Also discussed [here](https://machinelearningmastery.com/stacked-long-short-term-memory-networks/).
- `dropout`: `float` of the proportion of layers to drop in between LSTM layers. Discussed [here](https://machinelearningmastery.com/dropout-for-regularizing-deep-neural-networks/).
- `criterion`: `PyTorch loss function`. Discussed [here](https://machinelearningmastery.com/loss-and-loss-functions-for-training-deep-learning-neural-networks/), list of available options in PyTorch [here](https://pytorch.org/docs/stable/nn.html#loss-functions).
- `optimizer`: `PyTorch optimizer`. Discussed [here](https://towardsdatascience.com/optimizers-for-training-neural-network-59450d71caf6), list of available options in PyTorch [here](https://pytorch.org/docs/stable/optim.html)

## LSTM outputs
Assuming a model has been instantiated and trained with `model = LSTM(...)`:
- `model.train()`: trains the network. Set `quiet=True` to suppress printing of losses per epoch during training.
- `model.X`: transformed data in the format the model was/will actually be trained on. A `numpy array` of dimensions `n observations x n timesteps x n features`.
- `model.y`: one-dimensional list target values the model was/will be trained on.
- `model.predict(model.X)`: returns a one-dimensional list of the predicted values on the training set.
- `LSTM(new_data, target_col, n_timesteps, drop_missing_ys=False).X`: this transforms a new dataset into the format required to run predictions on.
- `model.predict(new_X)`: generate predictions on a new dataset. Can be nested with above to generate predictions like `model.predict(LSTM(new_data, target_col, n_timesteps, drop_missing_ys=False).X)`
- `model.mv_lstm`: a `list` of length `n_models` containing the PyTorch networks. 
- `model.train_loss`: a `list` of length `n_models` containing the training losses of each of the trained networks.
- `model.gen_ragged_X(pub_lags, lag)`: a `numpy array` of a dataset with the same dimensions as `model.X`, but with some values filled as missing. This is especially useful as a testing mechanism, to generate datasets to see how a trained model would have performed at different synthetic vintages or periods of time in the past. `pub_lags` should be a list of ints (in the same order as the columns of the original data) of length `n_features` (i.e. excluding the target variable) dictating the normal publication lag of each of the variables. `lag` is an int of how many periods back we want to simulate being. E.g. if the data is as of 1 June, `lag = -2` will fill with missings values that wouldn't have been availabe yet as of 1 April, given the publication lags provided in `pub_lags`. `lag = 2` would simulate being in 1 August, when June numbers for features with a publication lag of 2 months would be published.