Venue: NeurIPS 2021 Track Datasets_and_Benchmarks Round2

Reviews:

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary and contributions: This dataset includes over 17 million datapoints relating to fire spread in the continental US over 2012-2017 timeframe. As wildfires pose extreme risks to human lives and livelihoods, having a coherent opensource dataset that this team presents could enables a better understanding of how fires spread can help develop improved strategies to contain their spread and mitigate losses. This particular dataset focuses in on fires as detected from the VIIRS sensors as well as vegetation height, elevation, and slope from a different dataset entitled LANDFIRE (presumably from Landsat records), and daily meteorological variables (temp, windspeed) from NOAA (via Meteosat). In a sample test via classification and regression, all models performed relatively poorly. 
strengths: Pulls in data that’s scattered across a number of different organizations. Clear/easy to read
weaknesses: Susceptibility to fire also seems like it’s a function of the underlying built structures as well as some degree of human behavior (mitigating ignition conditions). How do these factors appear in this dataset? Are they fuel? Or is that anticipated in a future version?

This might explain why the performance of the models is also somewhat low – lots of missing variables. That isn’t to say this starting point is not useful, it just seems constrained by how much may be missing/unexplained. I'm not particularly clear on the value-add here.

correctness: The dataset appears to be constructed soundly and has documentation, though is not as detailed as many other submissions I’ve evaluated (missing the Gebru dataset datasheet, for example, and doesn’t describe *what* types of weather they’re pulling in). 

Timnit Gebru, Jamie Morgenstern, Briana Vecchione, Jennifer Wortman Vaughan, Hanna
381 Wallach, Hal Daume´ III, and Kate Crawford. Datasheets for datasets. arXiv:1803.09010, 2018.
clarity: Paper is well written/easy to read.
relation to prior work: What about papers that discuss things like home ignition conditions? Here's another sample other work that speaks to this:

Calkin, D.E., Cohen, J.D., Finney, M.A. and Thompson, M.P., 2014. How risk management can prevent future wildfire disasters in the wildland-urban interface. Proceedings of the National Academy of Sciences, 111(2), pp.746-751.

additional feedback: Wildfire is extremely important and so the frame is on point. I don't understand why these are the variables that are included at the expense of others though. Are there additional people or papers who are dissatisfied with the current state of affairs so that they can cite/subtantiate this comment, which seems to be the premise of the dataset collection? "While such [physics-based ]models are widely used, prediction of fire spread can be improved by a large set of covariates"
review_title: Wildfire DB -- focus on related environmental factors -- low model performance

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: This paper presents a dataset on wildfires that integrates data from different sources which allows the inclusion of possible determinants.
It also presents a benchmark study using 3 different methods: linear regression, random forest, and neural networks. The problem is addressed both as a regression and as a classification problem.
strengths: Several tasks were done:
- the discretization of the space;
- the integration of data from different sources;
- the creation of an online interface;
- two benchmark studies for both regression and classification using 3 different algorithms 


weaknesses: - how were the hyper-parameters defined, e.g., for neural networks? This is not explained.


correctness: Good
TYPOS
line 188: this results is largely -> these results are largely
clarity: Everything is very clear.
relation to prior work: No prior work reported
additional feedback:
review_title: A large dataset on wildfires in the United States combining information of geography, fires, topography, vegetation, and weather. 

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary and contributions: In this work, an spatio-temporal dataset that models how wildfires spread as a function of relevant covariates is proposed. The data generation is based on their previous work on large-scale vector and raster analysis [Singla and Eldawy, 2020]. There are a total of 17M data points. Evaluation with baseline machine learning methods is given. 
strengths: 1. The dataset is of significantly large scale (consisting of 17M data points and covering 8M sq. kilometers).
2. Many determinants such as weather, topography, and fuel levels are provided and merged with the information about fire occurrences. 
3. Baseline methods like linear regression, random forest, and neural network are evaluated and analyzed. 
weaknesses: 1. The data generation, to a large extent, is based on their previous work on large-scale vector and raster analysis [Singla and Eldawy, 2020].
2. The problem of wildfire occurrence is not expected to draw much attention in academia. 
3. Only the very basic baselines are studied. Some state-of-the-art approaches should be included. 
correctness: The dataset is constructed in a sound way. The evaluation of baseline methods is designed in an appropriate manner. 
clarity: The paper is clearly written. 
relation to prior work: The data generation is based on their previous work on large-scale vector and raster analysis [Singla and Eldawy, 2020].
additional feedback: 1. For the evaluation with neural network, it may be useful to compare different backbone architectures. 
2. Please compare with other open-source datasets for similar problems. 
3. Will there be an official leader board maintained by the creators of the dataset?
review_title: Despite the large scale, the data generation is not novel

Meta-review: In this paper the authors present a new dataset for wildfires spread with features including weather and topography. Further, they offer relatively simple baselines to predict the spread of wildfires. While there was a wide range of perspectives from reviewers, all reviewers largely agreed with the construction of the dataset as well as the clarity and validity of the work. While there were some concerns about the breadth of study in the ML community and the need for growth of research in this area (both for even more data and more models), the paper still contributes enough for it to be valuable to the academic community. As such, and after consulting with a second AC, I recommend acceptance and also encourage the authors to expand the paper in the ways suggested by the reviewers.
Decision: Accept