From: ~kehinde_owoeye1
comment: Thank you for the clarification.
title: Some comments on the results

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper13/Authors
title: Some answers to your comments
comment: Hello, and thank you for your comments.

For the BCD5 method, it is usually called a 'statistical model', using only 0D information (longitude, latitude, windspeed, J-day predictor..) from past hurricanes in order to perform a regression.

For the question related to LSTM, we first have to say that in our CNN we do take time into account as we also use features from past time steps. It is not trivial to say that LSTMs would extract more time information, especially knowing that it is hard to train in general. Moreover, a standard LSTM cannot take spatial maps as input. There exists now few convolutional-LSTM studies that seem to work on other problems, even if the training/convergence might be even harder. 
Finally, 'fusing' (as we did) a CNN to an LSTM network is an interesting idea. I am just not sure that we could re-train them together as the LSTM training is very specific (one storm at a time, ect). But a simpler idea could be to just combine the results of both methods.

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper13/Authors
title: Answer to reviewer 1
comment: Thank you for your valuable comments. 

Concerning the comparison with the official OFCL models, we think that it is natural that one method alone cannot beat an ensemble of physical models, but integrating our algorithm to their ensemble could improve the overall results.

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper13/AnonReviewer1
evaluation: This paper presents a model for tracking storms and achieve 24 prediction. It uses multiple CNNs for each relevant covariate over space and time and then fuses their output into a unified prediction which also considers auto-regressive factor over last displacement. The forecasts error suggest that the CNN fusion method is better than BCD5 model (state of the art in forecasting of storms).  

1. What did you learn from reading this paper that you did not know before?

It was interesting to see CNN applied to spatial-temporal regression.

2. What was the most exciting part of the work?

It is good to see comparisons with benchmark models for storm location forecasting. However, as shown in figure 4 and admitted by the authors, there is no clear advantage over official OFCL models, which present overall smaller errors.

3. What in your opinion makes this work stand apart from other papers in this area?

This paper does present clear methodology and results including comparisons with benchmark methods.

4. Does the paper fit into the scope of the workshop?

Yes, there is clear contribution regarding spatial-temporal modelling using multiple CNNs.

5. Are you aware of relevant literature that you may want to make the authors aware of?

No, unfortunately I am not aware of Hurricane tracking literature. I think it is worth mentioning and reviewing the following papers on the Deep Learning spatial temporal modelling literature:

Xingyi Cheng†∗, Ruiqing Zhang†, Jie Zhou, Wei Xu - DeepTransport: Learning Spatial-Temporal Dependency for Traffic Condition Forecasting

Dixon, Polson and Sokolov - Deep Learning for Spatio-Temporal Modeling: Dynamic Traffic Flows and High Frequency Trading

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper13/AnonReviewer1
relevance: 4: high
impact: 2: low
novelty: 3: borderline

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper13/Authors
comment: Thank you for your comments, here are some answers to your 3 questions.

a) You are right, we did not talk about predicting shorter duration. In a preliminary work, we used a simplified fused CNN method focusing on 6h predictions. However, it is of no use for climatologists as a 6h forecast cannot be used for catastrophe planning and it was not possible to compare with current forecast methods. That is why we worked on the 24h forecast as it is a more standard forecast.

We have added an explanation in the paper introduction (see below c) ).


b) OFCL are available from 1990, but since this ensemble model is modified every year and they only provide forecasts of the version N of the model for the year N, it would be unfair for them to compare with old results (obtained with earlier, less efficient models). However, the OFCL seem to perform better than ours in the last years and we compare qualitatively our method and OFCL on a recent storm. We don't expect to perform better than an official consensus of all dynamical climate models, but more to provide a different thus interesting forecast that could be integrated in the OFCL model. However, it is true that it is interesting to discuss about the time at which the OFCL outperforms our method (even if on some years there might be very few storms from the test set to compare with).

We modified the paper accordingly : Such comparison is not possible with the OFCL as this model is modified every year and they only provide forecasts of the version N of the model for the year N. We don't know the performance of the recent models on previous years, and it would be unfair for them to compare with old results (obtained with earlier, less efficient models).
Analyzing the mean errors per year, our deep learning model performs better than the OFCL forecast until year 2010 for the Pacific basin (2005 for the Atlantic). During the years 2010, the OFCL method improved and its mean errors per year are smaller than ours. [..] If we don't expect to perform better than a current official ensemble of dynamical models, a neural network model can help the current forecast modellers by providing a complementary prediction that could be integrated in a consensus method.


c) In our preliminary work, a comparison of the error ranges with the existing learning methods was performed which predicts the 6h forecast (and outperforming these, 30km error with respect to more than 60km). However, since these other learning methods only focused on 6h or 12h forecast, a comparison was not possible in this 24h-forecast study.

 We have added to our paper a comment on this in the introduction : In a recent work [citation hidden], we showed the advantage of using a moving reference CNN model for forecasting hurricanes tracks 6 hours into the future with respect to the other learning methods (30km error with respect to more than 60km). However, a 6h-forecast is of no use for catastrophe planning and it is not possible to compare to current forecasts as the smallest standard is 24 hours.
title: Answer to reviewer 2

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper13/AnonReviewer2
evaluation: This paper uses a neural network approach to forecast the location of a hurricane 24 hours into the future. The method uses past hurricane track data as well as past and current wind velocity and pressure field data at the current hurricane location as input data. The main contribution of this paper in my opinion is the use of a fusion network to predict the hurricane location in the future. The proposed method is able to fuse field data like wind velocities and pressure data with ‘0D-features’ to obtain the final prediction. The proposed method is able to predict a time dependent phenomena, without using an LSTM, and it is able to outperform some of the existing prediction approaches. In future iterations of this work, I would like to see the following additions.
a) Predicting shorter durations into the future, i.e., 6 or 12 hours instead of ( or in addition to) 24 hours. This might be of practical importance. Maybe 24 hour prediction is the standard used in weather predictions, if so the authors should mention this.

b) Are OFCL predictions not available for past data? If they are, the results should be compared with OFCL (even if data is available only for a few years). From Fig 4, it seems OFCL is much better than BCD5 at predicting hurricane tracks and would be a better candidate for comparisons.

c) Compare the results of the proposed method with those of existing learning based methods that the authors have cited in the introduction.

1. What did you learn from reading this paper that you did not know before?
Using fusion networks to fuse field like data with “0D” features, and using that for predictions.

2. What was the most exciting part of the work?
The fusion architecture.

3. What in your opinion makes this work stand apart from other papers in this area?
The architecture of the network used.

4. Does the paper fit into the scope of the workshop?
Yes.

5. Are you aware of relevant literature that you may want to make the authors aware of?
No.

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper13/AnonReviewer2
relevance: 5: very high
impact: 4: high
novelty: 3: borderline

From: ~kehinde_owoeye1
comment: Thanks for the write up! Some comments/questions from my side:
I was just wondering, how does the BCD5 work? Is it an average of all prediction errors with respect to previous models?
Also, is there a chance that your current model fused with an LSTM is going to result in a better model?
Thanks

title: Some comments/questions on the results