Repository: awslabs/gluonts

Issue:
DeepStateEstimator with add_trend=True throws "module 'mxnet.symbol' has no attribute 'array'"
## Description
DeepStateEstimator with add_trend=True throws the exception below. Setting add_trend=False (the default) does not throw the exception.

## To Reproduce

See attached modified **benchmark_m4.py.txt**

[benchmark_m4.py.txt](https://github.com/awslabs/gluon-ts/files/3796950/benchmark_m4.py.txt)

## Error Message
```
Traceback (most recent call last):
  File "./benchmark_m4.py", line 84, in <module>
    results.append(evaluate(dataset_name, estimator))
  File "./benchmark_m4.py", line 60, in evaluate
    predictor = estimator.train(dataset.train)
  File "/home/repro/.local/lib/python3.6/site-packages/gluonts-0.3.4.dev78+g8642509-py3.6.egg/gluonts/model/estimator.py", line 201, in train
    return self.train_model(training_data).predictor
  File "/home/repro/.local/lib/python3.6/site-packages/gluonts-0.3.4.dev78+g8642509-py3.6.egg/gluonts/model/estimator.py", line 187, in train_model
    train_iter=training_data_loader,
  File "/home/repro/.local/lib/python3.6/site-packages/gluonts-0.3.4.dev78+g8642509-py3.6.egg/gluonts/trainer/_base.py", line 246, in __call__
    output = net(*inputs)
  File "/home/repro/.local/lib/python3.6/site-packages/mxnet/gluon/block.py", line 540, in __call__
    out = self.forward(*args)
  File "/home/repro/.local/lib/python3.6/site-packages/mxnet/gluon/block.py", line 907, in forward
    return self._call_cached_op(x, *args)
  File "/home/repro/.local/lib/python3.6/site-packages/mxnet/gluon/block.py", line 797, in _call_cached_op
    self._build_cache(*args)
  File "/home/repro/.local/lib/python3.6/site-packages/mxnet/gluon/block.py", line 749, in _build_cache
    data, out = self._get_graph(*args)
  File "/home/repro/.local/lib/python3.6/site-packages/mxnet/gluon/block.py", line 741, in _get_graph
    out = self.hybrid_forward(symbol, *grouped_inputs, **params)  # pylint: disable=no-value-for-parameter
  File "/home/repro/.local/lib/python3.6/site-packages/gluonts-0.3.4.dev78+g8642509-py3.6.egg/gluonts/model/deepstate/_network.py", line 171, in hybrid_forward
    length=self.past_length,
  File "/home/repro/.local/lib/python3.6/site-packages/gluonts-0.3.4.dev78+g8642509-py3.6.egg/gluonts/model/deepstate/_network.py", line 129, in compute_lds
    seasonal_indicators
  File "/home/repro/.local/lib/python3.6/site-packages/gluonts-0.3.4.dev78+g8642509-py3.6.egg/gluonts/model/deepstate/issm.py", line 341, in get_issm_coeff
    self.nonseasonal_issm.get_issm_coeff(seasonal_indicators),
  File "/home/repro/.local/lib/python3.6/site-packages/gluonts-0.3.4.dev78+g8642509-py3.6.egg/gluonts/model/deepstate/issm.py", line 131, in get_issm_coeff
    self.transition_coeff(seasonal_indicators),
  File "/home/repro/.local/lib/python3.6/site-packages/gluonts-0.3.4.dev78+g8642509-py3.6.egg/gluonts/model/deepstate/issm.py", line 205, in transition_coeff
    F.array([[1, 1], [0, 1]])
AttributeError: module 'mxnet.symbol' has no attribute 'array'
```

## Environment
- Operating system:
4.15.0-65-generic #74-Ubuntu SMP Tue Sep 17 17:06:04 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

- Python version:

Python 3.6.8

- GluonTS version:

Name: gluonts
Version: 0.3.4.dev78+g8642509
Summary: GluonTS is a Python toolkit for probabilistic time series modeling, built around MXNet.
Home-page: https://github.com/awslabs/gluon-ts
Author: Amazon
Author-email: gluon-ts-dev@amazon.com
License: Apache License 2.0
Location: /home/repro/.local/lib/python3.6/site-packages/gluonts-0.3.4.dev78+g8642509-py3.6.egg
Requires: boto3, holidays, matplotlib, mxnet-cu92mkl, numpy, pandas, pydantic, tqdm, ujson

Name: pandas
Version: 0.25.2
Summary: Powerful data structures for data analysis, time series, and statistics
Home-page: http://pandas.pydata.org
Author: None
Author-email: None
License: BSD
Location: /home/repro/.local/lib/python3.6/site-packages/pandas-0.25.2-py3.6-linux-x86_64.egg
Requires: numpy, python-dateutil, pytz

Name: numpy
Version: 1.14.6
Summary: NumPy: array processing for numbers, strings, records, and objects.
Home-page: http://www.numpy.org
Author: Travis E. Oliphant et al.
Author-email: None
License: BSD
Location: /home/repro/.local/lib/python3.6/site-packages
Requires: 

Name: mxnet
Version: 1.4.1
Summary: MXNet is an ultra-scalable deep learning framework. This version uses openblas.
Home-page: https://github.com/apache/incubator-mxnet
Author: UNKNOWN
Author-email: UNKNOWN
License: Apache 2.0
Location: /home/repro/.local/lib/python3.6/site-packages



Discussion:
Currently trend model does not work in symbolic mode. This needs a fix [here](https://github.com/awslabs/gluon-ts/blob/85ea2185c123dddf19db46ce48c354f10d66d68f/src/gluonts/model/deepstate/issm.py#L204).

Meanwhile, can you try disabling `hybridize` mode like this:

```
estimators = [
    partial(
        DeepStateEstimator,
        add_trend=True,
        trainer=Trainer(
            epochs=epochs, num_batches_per_epoch=num_batches_per_epoch, hybridize=False
        ),
    ),
]

```
Cheers! Will try when I have an opportunity.
@gjmulder Have you found any substantial difference in accuracy for specific types of time series when running with `add_trend=True`?
@andmib I've been focusing on DeepAR due to the reported GPU issue with DeepState. Now that someone has confirmed the GPU fix works I'll switch back to DeepState - when I have time. Prof wants to publish a paper on my results to date with DeepAR, so I'm busily confirming them.

Resolving patch:
diff --git a/src/gluonts/model/deepstate/issm.py b/src/gluonts/model/deepstate/issm.py
--- a/src/gluonts/model/deepstate/issm.py
+++ b/src/gluonts/model/deepstate/issm.py
@@ -201,8 +201,7 @@ def transition_coeff(
         F = getF(seasonal_indicators)
 
         _transition_coeff = (
-            # TODO: this won't work if F == mx.sym
-            F.array([[1, 1], [0, 1]])
+            (F.diag(F.ones(shape=(2,)), k=0) + F.diag(F.ones(shape=(1,)), k=1))
             .expand_dims(axis=0)
             .expand_dims(axis=0)
         )


Test patch:
diff --git a/test/model/test_deepstate_smoke.py b/test/model/test_deepstate_smoke.py
--- a/test/model/test_deepstate_smoke.py
+++ b/test/model/test_deepstate_smoke.py
@@ -26,8 +26,11 @@
 common_estimator_hps = dict(
     freq="D",
     prediction_length=3,
-    trainer=Trainer(epochs=3, num_batches_per_epoch=2, batch_size=1),
+    trainer=Trainer(
+        epochs=3, num_batches_per_epoch=2, batch_size=1, hybridize=True
+    ),
     past_length=10,
+    add_trend=True,
 )