Venue: NeurIPS 2021 Track Datasets_and_Benchmarks Round1

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: The paper proposes a remote sensing image dataset for high spatial resolution (HSR) land-cover mapping. The dataset consists of annotated aerial images of rural and urban domains, and promotes the study of both semantic segmentation and unsupervised domain adaptation (UDA). The authors also benchmark a few semantic segmentation and UDA methods.
strengths: 1. The paper is well written and easy to follow. 

2. The challenges of large-scale semantic segmentation as well as unsupervised domain adaptation (UDA) are well explained. These are also appropriate motivating problems for the proposed dataset.

3. Review and discussion of prior datasets are comprehensive.

4. The key statistics of the dataset are clearly discussed. 

5. Many semantic segmentation methods and UDA are benchmarked.

6. Code includes scripts to reproduce results from the paper.

weaknesses: Major:

1. The images are collected in Nanjing only, which almost certainly have biased statistics specific to the geographical area. Although this is understandable given the high cost of collecting images from larger areas, and most published visual datasets are implicitly or explicitly biased in certain ways. 

2. It is unclear to me why multiscale augmentations such as MST and MSTT are only applied to HRNet. In principle most (if not all) models can benefit from multiscale augmentation.

Minor (didn’t affect my decision):

3. Fonts in the figures are too small.

4. Some wording could be improved.

5. Table captions usually sit on top of tables.

correctness: The evaluation methods are mostly appropriate with standard deviation missing. The lack of standard deviation makes table 3 and 4 untrustworthy given the small effect sizes. But these results do not affect the major contribution of the paper to begin with.
clarity: The paper is overall well written and easy to follow.
relation to prior work: The paper has detailed discussions of prior work.
additional feedback: Typo:
Line 211: P -> R
review_title: Well motivated and constructed dataset

final_score: rating: 4: Ok but not good enough - rejection
review_content: summary and contributions: This paper proposes a remote sensing land-cover dataset for promoting large-scale land-cover mapping. The dataset contains 3,338 aerial images from both urban and rural area, and the annotated objects are from seven common land-cover categories. The difference between the distributions of land-cover categories in urban and rural causes a domain gap. Models trained on this dataset is expected to work well on cross-domain adaptation settings between urban and rural images. The authors conduct experiments on both semantic segmentation and unsupervised domain adaptation models.
strengths: 1. Detailed statistical analysis is performed on the proposed dataset including class distribution analysis, object, and pixel-level analysis.
2. The experiments are thorough and the dataset is well benchmarked. Results are reported on nine semantic segmentation models and eight unsupervised domain adaptation models.

weaknesses: 1. The aerial images are only collected from several relatively small regions from one city (i.e., Nanjing). The layouts of buildings, roads, and rivers may be very similar in the same city, so I am pretty concerned about the data diversity. 
2. The class labels defined on the dataset are not appropriate. I think the pixels of "barren" and "agricultural" are not recognizable in aerial images, since an agricultural area with sparse planting may seem like a barren area. 

correctness: The dataset is constructed in a small region and thus lacks data diversity. 
clarity: The paper is well organized.
relation to prior work: This dataset allows models to perform unsupervised domain adaptation (UDA) between the urban and rural images. However, in Table 1, the LandCoverNet dataset also contains urban and rural data like LoveDA, but can not support UDA tasks? Besides, the UDA setting should not be limited to urban and rural images. Datasets like GID could be used to benchmark the domain adaptation among different cities. Therefore, the authors claim that other datasets could not be used for training UDA models is not convincing.



additional feedback: N/A
review_title: A remote sensing land-cover dataset but lack of data diversity. 

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: The paper firstly introduced the Land-cover Domain Adaptation semantic segmentation (LoveDA) dataset to promote large-scale land-cover mapping.  The proposed dataset contains 3338 aerial images with 86516 annotated objects for seven common land-cover categories. Besides, compared with the previous land-cover datasets, the dataset includes two domains, urban and rural, which may introduce more possible research topics in the task.

Meanwhile, the authors have conducted comprehensive studies using LoveDA dataset on Semantic Segmentation and Unsupervised Domain Adaptation, respectively, which provided fundamental experimental settings and results for future  researches.
strengths: 1. The dataset has multi-scale objects and complex background, besides, it contains two kinds of domains data, urban and rural.

2. The experiments are comprehensive. 

3.  The experimental results on the two tasks of semantic segmentation and unsupervised domain adaptation show that the dataset is promising and useful to relevant research.


weaknesses: 1.  The source of the data is not very clear. Does it come from a legal way? 


2. there are some typo in line-209,  L_mce should be L_ce.
correctness: The dataset and benchmarks are reasonable. 
clarity: the paper is well written and can be understood easily.
relation to prior work: Yes, compared with previous works, the most difference is that LoveDA has the data from two domains.
additional feedback: see above
review_title: LoveDA datasets is helpful to the land-cover semantic segmentation task

Meta-review: The reviews are mixed. The main concerns include the limited diversity of the dataset and whether it adds significantly on top of existing datasets in terms of domain adaptation. After considering the author responses, AC doesn't consider these two main concerns satisfactorily addressed. Although it is true that the dataset is better than previous ones in terms of resolution and granularity of labels, they are relatively orthogonal to domain adaptation. The new dataset has value, but the current form of the paper doesn't meet the bar of NeurIPS. The authors are encouraged to revise and resubmit. 
Decision: Reject