From: ~Sophie_Giffard-Roisin1
comment: Your new calculations seem correct, and I hope that it will help you to improve your method in the future.

One last suggestion would be to calculate your result in terms of total distance in kilometers (without separating latitude and longitude). There is an easy formula to do so, from 2 sets lat/long points it gives the distance between them (have a look for ex. https://www.movable-type.co.uk/scripts/latlong.html).

Good luck and thanks for the answer.
title: thanks for the reply

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper12/Authors
title: Thank you for your valuable comments. Here are answers to your comments. 
comment: As you said, the distance of one degree of latitude about 111km, and for the longitude it is 111km at the equator (and shrinks to 0 at the poles).  For more accurate performance check, we calculated the verage latitude and longitude of all best tracks of our 50 typhoons, and the results are 25.59 and 130.95. (max lat: 47.9 max long: 174.1, min lat: 10.6, min long: 108.0). At the point, one degree in latitude is 110.78 km, and one degree in longitude is 100.46 km.

Further, we have measured MEAN ABSOLUTE ERROR (degree) between predicted latitude and longitude and best track for more interpretable results. MAE of latitude is 1.8 and MAE of longitude is 2.12 with the model built at epoch 200. 

Therefore, the current average error around 2 degrees would mean around 200km error (lat 1.8 *110km, long 2.12*100km), and 6-hour forecast errors are about 176km in lat and 208km in long. It looks a not good result considering that the diameter of middle size TC is around 300km ~ 500km. However, please note that our WRF dataset resolution is 30km that means one cell in the grid is 30km by 30km. We expect the error would decrease if we have a higher resolution of WRF data(e.g., 10km) with the proposed method. 

Thanks again for your comments!

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper12/Authors
title: Thank you for your valuable review. Here are answers to your comments.
comment: -Dataset split
The whole dataset included 5400 simulations. We randomly divided the dataset as training (60%), validation (20%), and test set (20%). The training set had 3248 instances, and validation and test set had 1076 respectively.

-Overffing
We did not change any other variables in the architecture except the number of kernels. We have learning curves by a different number of kernels such as 8, 16, and 24 from our experiments. Please find those in the link below. The model with eight kernels (four 3D kernels and four 2D kernels) have shown stable learning while other models haven’t. 

https://docs.google.com/document/d/1UggNWr1FouhybyIPy7clYBpT6tJ480E-xuKbUbx5k50/edit?usp=sharing

-Results
As your comments, we have measured MEAN ABSOLUTE ERROR (degree) between predicted latitude and longitude and best track for more interpretable results. MAE of latitude is 1.8 and MAE of longitude is 2.12 with the model built at epoch 200. Average latitude and longitude of all best tracks of our 50 typhoons are 25.59 and 130.95. (max lat: 47.9 max long: 174.1, min lat: 10.6, min long: 108.0) At the point, one degree in latitude is 110.78 km, and one degree in longitude is 100.46 km.

Thus, the current average error around 2 degrees would mean around 200km error (lat 1.8 *110km, long 2.12*100km), and 6-hour forecast errors are about 176km in lat and 208km in long. It looks a not good result considering that the center of TC is approximately 30km and the diameter of middle size TC is around 300km ~ 500km. However, please note that our WRF dataset resolution is 30km that means one cell in the WRF grid is 30km by 30km. We expect the error would decrease if we have a higher resolution of WRF data (e.g., 10km) with the proposed method. 

Please note that we have newly calculated MAE between predicted latitude and longitude and best track with the testset based on the model previously built (with the cost function calculating RMSE between the predicted lat. and long. and best track). We also have a plan to provide results in Km, and we think we may provide those at the workshop.

From: ~Seongchan_Kim1
comment: -Results
We have measured MEAN ABSOLUTE ERROR (degree) between predicted latitude and longitude and best track for more interpretable results. Please refer to the new results on the link below.

https://docs.google.com/document/d/1UggNWr1FouhybyIPy7clYBpT6tJ480E-xuKbUbx5k50/edit?usp=sharing

We have newly calculated MAE between predicted latitude and longitude and best track with the testset based on the model previously built (with the cost function calculating RMSE between the predicted lat and long and best track). We also have a plan to provide the results in Km, and we think we may provide those at the workshop. 

Based on the new results, MAE of latitude is 1.8 and MAE of longitude is 2.12 with the model built at epoch 200. Average latitude and longitude of all best tracks of our 50 typhoons are 25.59 and 130.95. (max lat: 47.9 max long: 174.1, min lat: 10.6, min long: 108.0) At the point(25.9, 130.95), one degree in latitude is 110.78 km, and one degree in longitude is 100.46 km. 

Thus, the current error around 2 degrees would mean around 200km (lat 1.8 *110km, long 2.12*100km), even for 6-hour forecast errors are about 176km in lat and 208km in long. It looks a not good result considering that the center of TC is approximately 30km and the diameter of middle size TC is around 300km ~ 500km. However, please note that our WRF dataset resolution is 30km (that means one cell in the grid is 30km by 30km). We expect that the error would decrease if we have a higher resolution of WRF data(e.g., 10km) with the proposed method. 

-Baseline
We are going to compare ours with an existing method utilizing TC tracking tools such as GFDL Vortex Tracker and K-tracker. GFDL Vortex Tracker and K-tracker are center detectors of TC from WRF simulation data. Those extract primary parameters (e.g., 850-250 hPa thickness, Vorticity at 850 hPa)from WRF data and find candidates of TC center, finally, determine TC center. We will evaluate the performances of those tools compared to DeepTC.

title: Thank you for your valuable review. Here are answers to your comments.

From: ~Ransalu_Senanayake1
comment: Dear authors,
     
You should now be able to see the reviews. We encourage you to engage in the discussion by commenting on the reviews. Please take this as an opportunity to clarify the issues highlighted by the reviewers. 
     
Regards,
Ransalu, Neal, Fabio, and Girish
title: Rebuttal

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper12/AnonReviewer1
evaluation: This paper uses a convolutional LSTM network trained on simulated weather data to learn the paths of cyclones. The output of the model is a latitude and longitude of the predicted path at regular time intervals. The model incorporates both 2-dimensional and 3-dimenstional simulated data through the use of 2D- and 3D-conv cells in the network.

The application is interesting as predicting the path of severe weather is an important task. The focus seems to be on the use of a rich set of simulated data rather than proposing a new method or model. It's difficult to contextualize the results (root mean square error on the predicted lat-long coordinates of the path) or evaluate them versus existing methods. Rather the paper hints at motivating the use of convLSTMs for cyclone prediction.

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper12/AnonReviewer1
relevance: 4: high
impact: 3: borderline
novelty: 2: low

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper12/AnonReviewer2
relevance: 4: high
impact: 3: borderline
novelty: 3: borderline

From: NIPS.cc/2018/Workshop/Spatiotemporal/Paper12/AnonReviewer2
evaluation: Thanks for you submission,

The idea is interesting and can be explored further using the proposed methods.

Few comments:

I think it would be interesting to provide more details on how the dataset was split.

You mentioned overfitting with regards to the kernels used. Were them the only changed in the proposed architecture? I find it a bit strange that a change like that would cause the amount of overfitting described in the paper.

Results table could have more relevance if shown as MEAN ABSOLUTE ERROR.

From: ~Sophie_Giffard-Roisin1
comment: Hi, Thank you for the work. The method seem interesting.
I have few comments on the results.

When you say : 'For reference, one degree in latitude is approximately 8 km and one degree in longitude is approximately 11 km, and the average diameter of a TC center is known to be approximately 30km.'
If you talk about the distance of one degree of latitude, then it is 111km (almost constant over the earth), and for the longitude it is 111km at the equator (and shrinks to 0 at the poles, but tropical typhoons occur not too far from the equator). So an error like yours of around 3 degrees would mean around rmse=300 km, even for 6h forecast.  https://www.thoughtco.com/degree-of-latitude-and-longitude-distance-4070616

I think it could also be interesting to view the results in terms of  mean absolute error, in order to have an idea of the mean error distance. (rmse is different from mae : sum of squares is different from square of sum. But mae is smaller so you would get slightly better results to show). Maybe some plots of the forecast results would be relevant.


title: Comments on the results