Title: TreNet: Hybrid Neural Networks for Learning the Local Trend in Time Series
Authors: Tao Lin
Tian Guo
Karl Aberer
Venue: ICLR 2017

Abstract: Local trends of time series characterize the intermediate upward and downward patterns of time series. Learning and forecasting the local trend in time series data play an important role in many real applications, ranging from investing in the stock market, resource allocation in data centers and load schedule in smart grid. Inspired by the recent successes of neural networks, in this paper we propose TreNet, a novel end-to-end hybrid neural network that predicts the local trend of time series based on local and global contextual features. TreNet leverages convolutional neural networks (CNNs) to extract salient features from local raw data of time series. Meanwhile, considering long-range dependencies existing in the sequence of historical local trends, TreNet uses a long-short term memory recurrent neural network (LSTM) to capture such dependency. Furthermore, for predicting the local trend, a feature fusion layer is designed in TreNet to learn joint representation from the features captured by CNN and LSTM. Our proposed TreNet demonstrates its effectiveness by outperforming conventional CNN, LSTM, HMM method and various kernel based baselines on real datasets.


Review discussion:
Dear Reviewers,
We upload a new version of the paper and list the content updated as follows:
1. Refine Figure 1 and Figure 2.
2. In the experiment section, update some experiment results and the result discussion.
3. In the appendix section, add data pre-processing subsection to explain the local trend extraction from raw time series data. Add the result discussion.
Thanks!
This is very nice work but the local trend features seem quite ad-hoc to me. The segmentation seems to depend on the time scale and on the variance one tolerates within each segment, and Figure 1 looks unconvincing, as many other segmentations could have been used.
It seems that one simpler architecture, relying on a convnet and an LSTM, has been overlooked: connect the convnet outputs directly to the LSTM, without local trend extraction. The LSTM could have several layers to handle hierarchies. The convnet would extract local trends (at a fixed scale) and the LSTM would have to learn to integrate them to extract long-term trends end-to-end. Have you considered such an architecture?
Dear Reviewer,
Thanks for your insightful comments! Please find the reply inline below.
Comment 1: "This is very nice work but the local trend features seem quite ad-hoc to me. The segmentation seems to depend on the time scale and on the variance one tolerates within each segment, and Figure 1 looks unconvincing, as many other segmentation could have been used."
Reply: Yes, the error bound of the segmentation process affects the extracted local trends. The error bound determines the approximation quality of the segment w.r.t. the raw data covered by it.
Specifically, for time series, lower error bound (i.e., high approximation quality) would lead to short local trends in terms of duration, thereby yielding more local trends from the time series.
On the other hand, higher error bound gives rise to less local trends with long duration. For more details, please refer to the Appendix section.
Depending on applications, users can tune the local trend extraction to address different requirements.
For instance, if users care more about the approximation qualify and data variance within the trend, then the error bound can be set at a low value and the neural network is trained to learn relatively short-term local trend.
On the other hand, if users are more tolerant of data variance within local trend, error bound can be chosen as a higher value.
Comment 2: "It seems that one simpler architecture, relying on a ConvNet and an LSTM, has been overlooked: connect the ConvNet outputs directly to the LSTM, without local trend extraction. The LSTM could have several layers to handle hierarchies. The ConvNet would extract local trends (at a fixed scale) and the LSTM would have to learn to integrate them to extract long-term trends end-to-end. Have you considered such an architecture?"
Reply: We agree with you. It is indeed an alternative architecture purely built on top of raw data. Actually, we considered this option in the beginning.
Then we thought there could be a way to utilize both the raw data and extracted trend information, which led to the approach presented in the paper.
We will also evaluate this cascade architecture of ConvNet and LSTM, and update the results and comparisons as soon as possible.
Thank you again for your comments!
Revision of the review:
The authors did a commendable job of including additional references and baseline experiments.
This paper presents a hybrid architecture for time series prediction, focusing on the slope and duration of linear trends. The architecture consists of combining a 1D convnet for local time series and an LSTM for time series of trend descriptors. The convnet and LSTM features are combined into an MLP for predicting either the slope or the duration of the next trend in a 1D time series. The method is evaluated on 3 small datasets.
Summary:
This paper, while relative well written and presenting an interesting approach, has several methodology flaws, that should be handled by new experiments.
Pros:
The idea of extracting upward or downward trends from time series - although these should, ideally be learned, not rely on an ad-hoc technique, given that this is a submission for ICLR.
Methodology:
* In section 3, what do you mean by predicting “either [the duration] $\hat l_t$ or [slope] $\hat s_t$” of the trend? Predictions are valid only if those two predictions are done jointly. The two losses should be combined during training.
* In the entire paper, the trend slope and duration need to be predicted jointly. Predicting a time series without specifying the horizon of the prediction is meaningless. If the duration of the trends is short, the time series could go up or down alternatively; if the duration of the trend is long, the slope might be close to zero. Predictions at specific horizons are needed.
* In general, time series prediction for such applications as trading and load forecasting is pointless if no decision is made. A trading strategy would be radically different for short-term and noisy oscillations or from long-term, stable upward or downward trend. An actual evaluation in terms of trading profit/loss should be added for each of the baselines, including the naïve baselines.
* As mentioned earlier in the pre-review questions, an important baseline is missing: feeding the local time series to the convnet and connecting the convnet directly to the LSTM, without ad-hoc trend extraction.
* The convnet -> LSTM architecture would need an analysis of the convnet filters and trend prediction representation.
* Trend prediction/segmentation by the convnet could be an extra supervised loss.
* The detailed analysis of the trend extraction technique is missing.
* In section 5, the SVM baselines have local trend and local time series vectors concatenated. Why isn’t the same approach used for LSTM baselines (as a multivariate input) and why the convnet operates only on local
* An important, “naïve” baseline is missing: next local trend slope and duration = previous local trend slope and duration.
Missing references:
The related work section is very partial and omits important work in hybrid convnet + LSTM architectures, in particular:
Vinyals, Oriol, Toshev, Alexander, Bengio, Samy, and Erhan, Dumitru. Show and tell: A neural image caption generator. CoRR, abs/1411.4555, 2014.
Donahue, Jeff, Hendricks, Lisa Anne, Guadarrama, Sergio, Rohrbach, Marcus, Venugopalan, Subhashini, Saenko, Kate, and Darrell, Trevor. Long-term recurrent convolutional networks for visual recognition and description. CoRR, abs/1411.4389, 2014.
Karpathy, Andrej, Toderici, George, Shetty, Sanketh, Leung, Thomas, Sukthankar, Rahul, and Fei-Fei, Li. Large-scale video classification with convolutional neural networks. In CVPR, 2014.
The organization of the paper needs improvement:
* Section 3 does not explain the selection of the maximal tolerable variance in each trend segment. The appendix should be moved to the core part of the paper.
* Section 4 is unnecessarily long and gives well known details and equations about convnets and LSTMs. The only variation from standard algorithm descriptions is that $l_k$ $s_k$ are concatenated. The feature fusion layer can be expressed by a simple MLP on the concatenation of R(T(l)) and C(L(t)). Details could be moved to the appendix.
Additional questions:
*In section 5, how many datapoints are there in each dataset? Listing only the number of local trends is uninformative.
Typos:
* p. 5, top “duration and slop”
Dear Reviewer,
Thanks for the detailed review! Please find the reply inline below.
First of all, we would like to clarify the idea of TreNet. In time series, local trends are consecutive and form a sequence. The problem we aim to resolve is to predict the subsequent trend given previous trends and local raw data. There are two types of dependencies behind the data, the dependency between features and the target, namely local raw data and the subsequent trend and the sequential dependency within the target variable itself, i.e. the sequence of historical trends (refer to the fourth paragraph and Fig.1 in the introduction section). The sequence of historical trends carries the information about long-term trend evolving and local raw data delivers information about abrupt changing behavior of the trend evolution. The idea of TreNet is to capture both two types of dependency and it shares something in common with [1].
[1] Wang J, Yang Y, Mao J, et al. CNN-RNN: A Unified Framework for Multi-label Image Classification. CVPR, 2016.
Comment:
* In section 3, what do you mean by predicting "either [the duration] $\hat l_t$ or [slope] $\hat s_t$" of the trend? Predictions are valid only if those two predictions are done jointly. The two losses should be combined during training.
* In the entire paper, the trend slope and duration need to be predicted jointly. Predicting a time series without specifying the horizon of the prediction is meaningless. If the duration of the trends is short, the time series could go up or down alternatively; if the duration of the trend is long, the slope might be close to zero. Predictions at specific horizons are needed.
Reply: During the experiments, all the approaches are trained to learn and predict the duration and slope jointly. We will refine the presentation to make this point clearer.
Comment: In general, time series prediction for such applications as trading and load forecasting is pointless if no decision is made. A trading strategy would be radically different for short-term and noisy oscillations or from long-term, stable upward or downward trend. An actual evaluation in terms of trading profit/loss should be added for each of the baselines, including the naïve baselines.
Reply: It is a good idea to evaluate the approaches by employing the decision making on top of the predicted trends.
Comment:
* As mentioned earlier in the pre-review questions, an important baseline is missing: feeding the local time series to the convnet and connecting the convnet directly to the LSTM, without ad-hoc trend extraction.
* The convnet -> LSTM architecture would need an analysis of the convnet filters and trend prediction representation.
* Trend prediction/segmentation by the convnet could be an extra supervised loss.
Reply: This baseline is being evaluated and we will update the results as soon as possible.
Comment: The detailed analysis of the trend extraction technique is missing.
Reply: We will provide some details about the trend extraction technique.
Comment: In section 5, the SVM baselines have local trend and local time series vectors concatenated. Why isn’t the same approach used for LSTM baselines (as a multivariate input) and why the convnet operates only on local
Reply: It is nonsense to feed the concatenated sequence of trends and raw data into LSTM since they are different types of data with the different time scale and are not synchronized, and therefore cannot be processed as one (multivariate) sequence for LSTM to learn.
Likewise, it does not apply to CNN as well.
Meanwhile, this is because we aim to demonstrate that only training neural networks on either historical trend sequence or raw data points lead to inferior results compared with the hybrid network utilizing both.
Comment: An important, "naïve" baseline is missing: next local trend slope and duration = previous local trend slope and duration.
Reply: This baseline is naive. A simple counter example is that for a quickly increasing time series, the next trend is obviously different from the previous one. We will evaluate this approach to demonstrate its inferiority.
Comment: Missing references: The related work section is very partial and omits important work in hybrid convnet + LSTM architectures, in particular: [3][5][6].
Reply: We will add these reference. Due to the page limit, we only refer some more recent papers in the submission.
The cited reference [2] in the submission evaluates the proposed network architecture with that in [3]. The cited reference [4] in the submission extends the idea in [5].
[2] Junhua Mao, Wei Xu, Yi Yang, Jiang Wang, Zhiheng Huang, and Alan Yuille. Deep captioning with multimodal recurrent neural networks (m-rnn). arXiv preprint arXiv:1412.6632, 2015.
[3] Vinyals, Oriol, Toshev, Alexander, Bengio, Samy, and Erhan, Dumitru. Show and tell: A neural image caption generator. CoRR.
[4] Nicolas Ballas, Li Yao, Chris Pal, and Aaron Courville. Delving deeper into convolutional networks for learning video representations. arXiv preprint arXiv:1511.06432, 2015.
[5] Donahue, Jeff, Hendricks, Lisa Anne, Guadarrama, Sergio, Rohrbach, Marcus, Venugopalan, Subhashini, Saenko, Kate, and Darrell, Trevor. Long-term recurrent convolutional networks for visual recognition and description. CoRR, abs/1411.4389, 2014.
[6] Karpathy, Andrej, Toderici, George, Shetty, Sanketh, Leung, Thomas, Sukthankar, Rahul, and Fei-Fei, Li. Large-scale video classification with convolutional neural networks. In CVPR, 2014.
Comment: The organization of the paper needs improvement:
* Section 3 does not explain the selection of the maximal tolerable variance in each trend segment. The appendix should be moved to the core part of the paper.
* Section 4 is unnecessarily long and gives well known details and equations about convnets and LSTMs. The only variation from standard algorithm descriptions is that $l_k$ $s_k$ are concatenated. The feature fusion layer can be expressed by a simple MLP on the concatenation of R(T(l)) and C(L(t)). Details could be moved to the appendix.
Reply: We will adjust the organization of the paper and shrink section 4.
Comment: Additional questions:
* In section 5, how many datapoints are there in each dataset? Listing only the number of local trends is uninformative.
* Typos: p. 5, top “duration and slop”
Reply: We will add the details about datasets and correct the typos.
Thanks again for your review!
1) Summary
This paper proposes an end-to-end hybrid architecture to predict the local linear trends of time series. A temporal convnet on raw data extracts short-term features. In parallel, long term representations are learned via a LSTM on piecewise linear approximations of the time series. Both representations are combined using a MLP with one hidden layer (in two parts, one for each stream), and the entire architecture is trained end-to-end by minimizing (using Adam) the (l2-regularized) euclidean loss w.r.t. ground truth local trend durations and slopes.
2) Contributions
+ Interesting end-to-end architecture decoupling short-term and long-term representation learning in two separate streams in the first part of the architecture.
+ Comparison to deep and shallow baselines.
3) Suggestions for improvement
Add a LRCN baseline and discussion:
The benefits of decoupling short-term and long-term representation learning need to be assessed by comparing to the popular "long-term recurrent convolutional network" (LRCN) of Donahue et al (https://arxiv.org/abs/1411.4389). This approach stacks a LSTM on top of CNN features and is typically used on time series of video frames for tasks that are more general than local linear trend prediction.
Furthermore, LRCN does not require the hand-crafted preprocessing of time series to extract piecewise linear approximations needed by the LSTM of the TreNet architecture proposed here.
Finally, LRCN might be more parameter efficient, as it does not have the fully connected fusion layers of TreNet (eq. 8).
Add more complex multivariate datasets:
The currently used 3 datasets are limited, especially compared to modern research in representation learning for time series forecasting.
For instance, and of particular interest to ICLR, I would suggest investigating future frame prediction on natural video datasets like UCF101 where CNN+LSTM are typically used albeit with a more complex loss (cf.
for instance the popular adversarial one of Mathieu et al).
Although different from the task of local linear trend prediction, it would be interesting to see how TreNet could be applied to the encoder stage of existing encoder-decoder architectures for frame prediction. It seems that decoupling short term and long term motion representation learning (for instance) could be beneficial in natural videos, as they often contain fast object motions together with slower camera ones.
Clarification about the target variables:
The authors need to clarify whether they handle separately or jointly the duration and slope. The text is ambiguous and seems to suggest training two separate models, one for slope, one for duration, which is particularly puzzling considering that predicting them jointly is in fact much easier (just two output variables instead of one), makes more sense, and is entirely feasible with the current method.
Other parts of the text can be improved too.
For instance, the authors can vastly compress the generic description of standard convnet and LSTM equations in section 4, while the preprocessing of the time series needs to appear much earlier.
4) Conclusion
Although the architecture seems promising, the current experiments are too preliminary to validate its usefulness, in particular to existing alternatives like LRCN, which are not compared to.
Dear Reviewer,
Thanks for the detailed comments! Please find the reply inline below.
First of all, we would like to clarify the idea of TreNet. In time series, local trends are consecutive and form a sequence. The problem we aim to resolve is to predict the subsequent trend given previous trends and local raw data. There are two types of dependencies behind the data, the dependency between features and the target, namely local raw data and the subsequent trend and the sequential dependency within the target variable itself, i.e. the sequence of historical trends (refer to the fourth paragraph and Fig.1 in the introduction section). The sequence of historical trends carries the information about long-term trend evolving and local raw data delivers information about abrupt changing behavior of the trend evolution. The idea of TreNet is to capture both two types of dependency and it shares something in common with [1].
[1] Wang J, Yang Y, Mao J, et al. CNN-RNN: A Unified Framework for Multi-label Image Classification. CVPR, 2016.
Comment: Add a LRCN baseline and discussion...
Reply: This type of baseline architecture is also spotted during the pre-review phase. We are evaluating this baseline and the results will be updated as soon as possible.
Comment: Add more complex multivariate datasets...
Reply: This is a pretty helpful suggestion for exploring alternative scenarios where the idea of TreNet is effective. We would like to take this work as the next step after the LRCN is evaluated.
Comment: Clarification about the target variables...
Reply: During the experiments, we trained the model for jointly slope and duration prediction. We will refine the description of the paper to make it clear.
Comment: Other parts of the text can be improved too.
For instance, the authors can vastly compress the generic description of standard ConVnet and LSTM equations in section 4, while the preprocessing of the time series needs to appear much earlier.
Reply: We agree. The generic description will be compressed and more space will be devoted to the time series preprocessing.
Thanks again for your review!
Updated review: the authors did an admirable job of responding to and incorporating reviewer feedback.
In particular, they put a lot of effort into additional experiments, even incorporating a new and much stronger baseline (the ConvNet -> LSTM baseline requested by multiple reviewers). I still have two lingering concerns previously stated -- that each model's architecture (# hidden units, etc.) should be tuned independently and that a pure time series forecasting baselines (without the trend preprocessing) should be tried. I'm going to bump up my score from a clear rejection to a borderline.
This paper is concerned with time series prediction problems for which the prediction targets include the slope and duration of upcoming local trends. This setting is of great interest in several real world problem settings (e. g. , financial markets) where decisions (e. g. , buy or sell) are often driven by local changes and trends. The primary challenge in these problems is distinguishing true changes and trends (i. e. , a downturn in share price) from noise. The authors tackle this with an interesting hybrid architecture (TreNet) with four parts: (1) preprocessing to extract trends, (2) an LSTM that accepts those trends as inputs to ostensibly capture long term dependencies, (3) a ConvNet that accepts a local window of raw data as its input at each time step, and (4) a higher "feature fusion" (i.
e. , dense) layer to combine the LSTM's and ConvNet's outputs. On three univariate time series data sets, the TreNet outperforms the competing baselines including those based on its constituent parts (LSTM + trend inputs, CNN).
Strengths:
- A very interesting problem setting that can plausibly be argued to differ from other sequential modeling problems in deep learning (e.g., video classification). This is a nice example of fairly thoughtful task-driven machine learning.
- Accepting the author's assumptions as true for the moment, the proposed architecture seems intuitive and well-designed.
Weaknesses:
- Although this is an interesting problem setting (decisions driven by trends and changes), the authors did not make a strong argument for why they formulated the machine learning task as they did. Trend targets are not provided from "on high" (by data oracle) but extracted from raw data using a deterministic algorithm.
Thus, one could just easily formulate this as plain time series forecasting problem in which we forecast the next 100 steps and then apply the trend extractor to convert those predictions into a trend. If the forecasts are accurate, so will be the extracted trends.
- The proposed architecture, while interesting, is not justified, in particular the choice to feed the extracted trends and raw data into separate LSTM and ConvNet layers that are only combined at the end by a shallow MLP. An equally straightforward but more intuitive choice would have been to feed the output of the ConvNet into the LSTM, perhaps augmented by the trend input. Without a solid rationale, this unconventional choice comes across as arbitrary.
- Following up on that point, the raw->ConvNet->LSTM and {raw->ConvNet,trends}->LSTM architectures are natural baselines for experiments.
- The paper presupposes, rather than argues, the value of the extracted trends and durations as inputs. It is not unreasonable to think that, with enough training data, a sufficiently powerful ConvNet->LSTM architecture should be able to learn to detect these trends in raw data, if they are predictive.
- Following up on that point, two other obvious baselines that were omitted: raw->LSTM and {raw->ConvNet,trends}->MLP. Basically, the authors propose a complex architecture without demonstrating the value of each part (trend extraction, LSTM, ConvNet, MLP). The baselines are unnecessarily weak.
One thing I am uncertain about in general: the validity of the practice of using the same LSTM and ConvNet architectures in both the baselines and the TreNet. This *sounds* like an apples-to-apples comparison, but in the world of hyperparameter tuning, it could in fact disadvantage either. It seems like a more thorough approach would be to optimize each architecture independently.
Regarding related work and baselines: I think it is fair to limit the scope of in-depth analysis and experiments to a set of reasonable, representative baselines, at least in a conference paper submitted to a deep learning conference.
That said, the authors ignored a large body of work on financial time series modeling using probabilistic models and related techniques. This is another way to frame the above "separate trends from noise" problem: treat the observations as noisy. One semi-recent example: J. Hernandez-Lobato, J. Lloyds, and D. Hernandez-Lobato. Gaussian process conditional copulas with applications to financial time series. NIPS 2013.
I appreciate this research direction in general, but at the moment, I believe that the work described in this manuscript is not suitable for inclusion at ICLR. My policy for interactive review is to keep an open mind and willingness to change my score, but a large revision is unlikely. I would encourage the authors to instead use their time and energy -- and reviewer feedback -- in order to prepare for a future conference deadline (e.g., ICML).
Dear Reviewer,
Thanks for the detailed review! Please find the reply inline below.
First of all, we would like to clarify the idea of TreNet. In time series, local trends are consecutive and form a sequence. The problem we aim to resolve is to predict the subsequent trend given previous trends and local raw data. There are two types of dependencies behind the data, the dependency between features and the target, namely local raw data and the subsequent trend and the sequential dependency within the target variable itself, i.e. the sequence of historical trends (refer to the fourth paragraph and Fig.1 in the introduction section). The sequence of historical trends carries the information about long-term trend evolving and local raw data delivers information about abrupt changing behavior of the trend evolution. The idea of TreNet is to capture both two types of dependency and it shares something in common with [1].
[1] Wang J, Yang Y, Mao J, et al. CNN-RNN: A Unified Framework for Multi-label Image Classification. CVPR, 2016.
Comment: Although this is an interesting problem setting (decisions driven by trends and changes), the authors did not make a strong argument for why they formulated the machine learning task as they did. Trend targets are not provided from "on high" (by data oracle) but extracted from raw data using a deterministic algorithm.
Thus, one could just easily formulate this as plain time series forecasting problem in which we forecast the next 100 steps and then apply the trend extractor to convert those predictions into a trend. If the forecasts are accurate, so will be the extracted trends.
Reply: As we mentioned in the related work section, multi-step ahead prediction of time series is non-trivial and suffers from the shortcomings for trend prediction as follows. Multi-step ahead forecasts are often realized by either recursively iterating a one-step-ahead time series model or directly by estimating a set of models each of which is respectively for individual forecast horizon. Recursive methods often present increasing prediction errors w.r.t. the horizon [2][3] and therefore it is unreliable to determine the slope and duration through such predicted values.
Regarding the method using a set of separated-models, it requires a prior specified time horizon [2], which is unknown beforehand and is one of the targets to predict in our setting.
Therefore, multi-step ahead prediction is unsuitable for the trend forecasting problem in terms of both prediction accuracy and model formulation.
[2] Taieb S B, Atiya A F. A Bias and Variance Analysis for Multistep-Ahead Time Series Forecasting[J]. IEEE transactions on neural networks and learning systems, 2016, 27(1): 62-76.
[3] Bao Y, Xiong T, Hu Z. Multi-step-ahead time series prediction using multiple-output support vector regression[J]. Neurocomputing, 2014, 129: 482-493.
Comment:
- The proposed architecture, while interesting, is not justified, in particular the choice to feed the extracted trends and raw data into separate LSTM and ConvNet layers that are only combined at the end by a shallow MLP. An equally straightforward but more intuitive choice would have been to feed the output of the ConvNet into the LSTM, perhaps augmented by the trend input. Without a solid rationale, this unconventional choice comes across as arbitrary.
- Following up on that point, the raw->ConvNet->LSTM and {raw->ConvNet,trends}->LSTM architectures are natural baselines for experiments.
Reply: We analyze the motivation of taking into account both the sequence of historical trends and local raw data for trend forecasting in the fourth paragraph of the introduction section. The idea of TreNet is to not only capture the dependency between features and the target, namely local raw data and the subsequent trend, but also the sequential dependency within the target variable itself, i.e. in the sequence of historical trends.
ConveNet-LSTM has already been spotted as another baseline during the pre-review. We are evaluating it and will update the results as soon as possible.
Comment:
- The paper presupposes, rather than argues, the value of the extracted trends and durations as inputs. It is not unreasonable to think that, with enough training data, a sufficiently powerful ConvNet->LSTM architecture should be able to learn to detect these trends in raw data, if they are predictive.
- Following up on that point, two other obvious baselines that were omitted: raw->LSTM and {raw->ConvNet,trends}->MLP. Basically, the authors propose a complex architecture without demonstrating the value of each part (trend extraction, LSTM, ConvNet, MLP). The baselines are unnecessarily weak.
Reply: ConvNet-LSTM architecture only captures the dependency of the predicted trend on the long-term trend evolving and fails to characterize the effect of local data on abrupt trend changes (Refer to Fig. 1(c) in the introduction section).
For these methods raw->LSTM and {raw->ConvNet,trends}->MLP, none of them can capture both two types of dependency (i.e., sequential dependency within the sequence of trends, the dependency of the successive trend on local data). We will evaluate them as additional baselines.
Comment: One thing I am uncertain about in general: the validity of the practice of using the same LSTM and ConvNet architectures in both the baselines and the TreNet. This *sounds* like an apples-to-apples comparison, but in the world of hyperparameter tuning, it could in fact disadvantage either. It seems like a more thorough approach would be to optimize each architecture independently.
Reply: We will update the results to do more comparison.
Comment: Regarding related work and baselines: I think it is fair to limit the scope of in-depth analysis and experiments to a set of reasonable, representative baselines, at least in a conference paper submitted to a deep learning conference.
That said, the authors ignored a large body of work on financial time series modeling using probabilistic models and related techniques. This is another way to frame the above "separate trends from noise" problem: treat the observations as noisy. One semi-recent example: J. Hernandez-Lobato, J. Lloyds, and D. Hernandez-Lobato. Gaussian process conditional copulas with applications to financial time series. NIPS 2013.
Reply: Before the submission, we surveyed the literature on probabilistic models of time series, none of them can be directly used for learning and forecasting local trends.
Comment: I appreciate this research direction in general, but at the moment, I believe that the work described in this manuscript is not suitable for inclusion at ICLR. My policy for interactive review is to keep an open mind and willingness to change my score, but a large revision is unlikely. I would encourage the authors to instead use their time and energy -- and reviewer feedback -- in order to prepare for a future conference deadline (e.g., ICML).
Reply: We will try to address all the review during the rebuttal phase.
Thanks again for the review!
Just wanted to call this to the attention of the meta-reviewer.
What is your argument in favor of manually extracting local trends and feeding them to the LSTM vs. feeding the raw data into a neural net architecture and allowing it to implicitly discover such patterns, if they are predictive? This seems like an odd choice for work being reviewed for the International Conference on *Learning* Representations.
Also, is it correct to assert that RNNs do not make a Markov assumption?
After all, the hidden state H_T at time T is a function ONLY of H_{T-1} and not any previous hidden states H_{T-2}, H_{T-3}, etc. They overcome this by using high dimensional, distributed hidden states that can encode long histories in the state itself. One *could* reasonably argue that LSTMs are not properly Markov because of the error carousel mechanism.
Dear Reviewer,
Many thanks for your review. Please find the reply inline below.
Comment: What is your argument in favor of manually extracting local trends and feeding them to the LSTM vs. feeding the raw data into a neural net architecture and allowing it to implicitly discover such patterns, if they are predictive? This seems like an odd choice for work being reviewed for the International Conference on *Learning* Representations.
Reply: Historical trends are consecutive and form a sequence. The target variable to forecast is the subsequent trend given the historical trend and local raw data. As we analyzed in the introduction section and the overview subsection of Section 4, TreNet aims to capture two type of dependency, the sequential dependence within the target variable, i.e., the sequence of historical trends before the prediction time instant and the dependency of the subsequent trend on local raw data. This is because the sequence of historical trends carries the information about long-term trend evolving and local raw data delivers information about abrupt changing behavior of the trend evolution.
Comment: Also, is it correct to assert that RNNs do not make a Markov assumption?
After all, the hidden state H_T at time T is a function ONLY of H_{T-1} and not any previous hidden states H_{T-2}, H_{T-3}, etc. They overcome this by using high dimensional, distributed hidden states that can encode long histories in the state itself. One *could* reasonably argue that LSTMs are not properly Markov because of the error carousel mechanism.
Reply: We mean that RNN is able to capture long-term sequential dependency and will refine this part.
Many thanks for your review!
Dear Reviewers,
We have uploaded a revised version of the paper, details are as follow:
* We have extended the experiment section by adding results from two new baselines suggested by the earlier review. One is the naive approach which takes the duration and slope of the last trend as the prediction for the next one. The other one is based on the cascade structure of ConvNet and LSTM in which feeds the features learned by ConvNet over time series to an LSTM and obtains the prediction from the LSTM.
* We have fixed some small typos.
We are also going to further reorganize the paper based on the comments.
Thank you for the revision and comments!
Although this is good work and the new CLSTM results are interesting and confirm this is a strong baseline (outperformed by the proposed method), I agree with the other reviewers that there is still substantial additional work to do (cf. suggestions in the reviews) and that ICML would probably be a better fit (and where your work may have more impact).
So I leave my rating unchanged.
Good luck!
Just so that the warning does not show up. Please ignore.

Meta-review:
role: PC
data: comment: I appreciate the authors putting a lot of effort into the rebuttal. But it seems that all the reviewers agree that the local trend features segmentation and computation is adhoc, and the support for accepting the paper is lukewarm.
 
 As an additional data point, I would argue that the model is not end-to-end since it doesn't address the aspect of segmentation. Incorporating that into the model would have made it much more interesting and novel.

Decision: Reject