Venue: NeurIPS 2021 Track Datasets_and_Benchmarks Round2

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: summary and contributions: The authors propose a new dataset for high spatial resolution land-cover mapping, a problem that is often posed as a semantic segmentation task. The dataset is fairly large compared to other datasets reported in the paper, multiple classes of objects are annotated through the images with polygonal masks, and an additional label tells whether the images belong to the urban or rural domains. The latter characteristic makes the dataset distinct compared to others, which are often collected in only one or few locations and/or multiple locations but without this information. The paper provides also an extensive benchmarking for semantic segmentation and domain adaptation tasks using state-of-the-art methods demonstrating the challenges of the dataset.
strengths: - Even though the dataset is not the best in all the different characteristics (km2 area covered, classes, image width, ...) it achieves a good compromise overall.
- Providing domain labels is also a very strong point to be able to test methods and strategies that are to be deployed later in real-world scenarios.
- Useful statistics about the data are also provided.
- Extensive benchmarking by using state-of-the-art models.

weaknesses: 1) If any, I'd ask the authors if there are no other datasets to compare to. After a quick look I've found some that could be related, and potentially included in Table 1.
- Airbus Ship
- SpaceNet
- iSAID
- AIRS https://www.airs-dataset.com/
- ...
iSAID (2019) being particularly interesting having 15 categories, 655,451 instances, 2,806 images, and image widths 800~13,000.

If not relevant, it'd be great that the authors explicitly state why they are not. Doesn't it fit the semantic segmentation task?

2) Not illustrating top performances (ideally using the same metric as in this paper, i.e. mIOU) on other datasets. This would support the "challeangability" of the proposed dataset compared to other existing ones
correctness: Yes. Both of these are true:
- The dataset is constructed in a sound way.
- Benchmarking of the dataset using state-of-the-art methods is properly designed and performed.
clarity: - Table 1 would be much easier to interpret if the years in which the differents datasets were released were included as an additional column.
- Although it is included in the Data sheet, the paper lacks some information regarding the data: (a) is the number of images captured in each of the 18 areas fairly balanced - yes, I know it form the Data sheet -, (b) is the number of images and/or instances balanced across domains?, etc. Please, revise the data sheet to see if any relevant information was not included in the paper.
- I'd have expected at least sentence explaining the multi-scale inference in MSTrTe.

# Minors

- Line 34: "using" -> "use"
- Line 142: "Figure 2(a), Our" -> "Figure 2(a), our"
- Line 153: "simple" -> "simpler"
- Figure 2b and 2c: keep the position of the "Background" bar the same for the two subfigures.
- Figure 3's caption: "for 10 areas are reported" is a bit confusing although, perhaps "for 5 urban and 5 rural areas are reported" would be clearer.
- Line 201: "Semnatic-FPN" -> "Semantic-FPN"
- Figure 5's caption: "Visual results" -> "Semantic segmentation results" and "regconize" -> "recognize".
- Line 261: actually, mP and mR are NOT in Table 7.
- Table 7: "porprotion" -> "proportion"
relation to prior work: I understand the value of the proposed dataset is its size but, even more important, the inclusion of domain labels, which makes the problem much more realistic and enables the proper evaluation of future aerial image semantic segmentation approaches. Those methods could fall short by not even thinking about the huge variability across countries, cities, urban/rural, etc. This kind of dataset did not exist, thus making it quite valuable.
additional feedback: I would gladly have decided "7: Good paper, accept" instead of "6: Marginally above acceptance threshold" if Table 1 have provided a stronger overview and comparison w.r.t. to other existing datasets. I'd also like to know why other datasets (especially, iSAID) are not considered.
review_title: Valuable HSR land-cover mapping dataset

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: This paper introduces a large-scale dataset LoveDA for semantic segmentation and domain adaptation with HSR images. Unlike existing datasets, the proposed dataset focuses on improving model transferability between urban and rural scenes. The dataset shows obvious style differences and class inconsistencies between the two kinds of scenes. The HSR images are collected from 18 complex urban and rural scenes in three different cities in China. The final dataset contains 5,927 images with a resolution of 0.3 m. Several semantic segmentation and domain adaptation models are tested on the proposed dataset.
strengths: 1. The introduced dataset focuses on the domain differences between urban and rural scenes and collected images from representative regions. This is the first multi-domain dataset for remote sensing images and shall promote the related research areas. And I agree their response, that a combined dataset is not a good choice.
2. The analysis of the dataset statistics is very detailed. Through the analysis, the differences between the two domains are very obvious, thus convinces the high demand of increasing the model generability.
3. The authors conduct a comprehensive study of baseline models on both two tasks and list both the qualitative and quantitative results.
weaknesses: 1. The fonts of the figures are still quite small.
2. Even though the dataset has been extended with images from more cities, the coverage area is still limited. These cities all lie in the middle and lower reaches of the Yangtze River. A better choice is covering more regions, each region sample fewer images.
correctness: The technical aspects of the work seem correct.
clarity: The paper is structurally well-written and easy to follow.
relation to prior work: The authors demonstrate a good comparison to other similar dataset, highlighting their strengths against others.
additional feedback: None
review_title: A new remote sensing dataset that contains semantic segmentation and domain adaptation tasks

final_score: rating: 6: Marginally above acceptance threshold
review_content: summary and contributions: This paper introduced a large-scale dataset with annotated objects from three different cities. The statistics of the dataset are described and the challenges are also explained, solid experiments of the existing methods are conducted on multiple tasks. However, some descriptions of the data is unclear to me, please see the weakness for details. I would like to change my rating according to the results of the discussion.

Their contributions include:
- 5972 aerial images with 166768 annotated objects.
- solid experiments of the existing methods on semantic segmentation and domain adaptation.
strengths: - a large number of remote sensing images with high resolution are provided.
- the challenge of the dataset is described for deep learning methods.
- the conducted experiments are sufficient, and the analysis of multi-scale architectures and strategies is very nice.
weaknesses: - the authors described that their data are collected by airborne sensors, but it is still not detailed enough to me.
- they only described the annotation process in the datasheet, no collection process is described.
- the authors claimed that they added two cities (Changzhou, Wuhan) to increase the data diversity, but they did not explain the difference among the three cities.
- I am also concerned about the privacy and security of cities. To my knowledge, the permission of the Chinese government is necessary even without geo-locations, if the authors want to use or open the high-resolution images. The authors claimed that they obtained the permission of Nanjing Bureau of Surveying and Mapping, how about Changzhou and Wuhan?
- It lacks a document about the dataset. The mask images look completely black, how to use them? And many HSR images have large black regions, they are padded for uniform sizes? So the noise is involved, the authors should describe the impact. 
- In the 27 line of Introduction, the description " these datasets are designed for single-domain" may be incorrect. [8] and [33] included diverse areas, and [33] also studied transfer learning on high-resolution remote sensing images.
- the motivation of transfer learning between rural and urban is unclear to me. The rural and urban scenes are different, but is it necessary to study transfer learning between these two scenes in practice? or is this task only designed for improving the transfer learning methods?
correctness: The data source is unclear.
clarity: Yes, but some descriptions are unclear to me.
relation to prior work: The author should clarify: the prior works are all single-domain? and no domain adaptation is studied?
additional feedback: - In 1(b) of checklist, "more effective methods need to be designed" is not the limitation of the introduced dataset.
- I think that the privacy and security problems should be described in 1(c) of checklist.
- more analysis of experiments results can be added to make contributions to the deep learning community.
review_title: The description of the data is unclear to me.

Meta-review: All reviewers agree to accept the paper. After reading the paper, AC does not see flaws to contradict the reviewers' opinions.
Decision: Accept