Venue: NeurIPS 2022 Track Datasets and Benchmarks

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: This paper proposes CARLANE, a 3-way sim-to-real domain adaptation benchmark for 2D lane detection, which encompasses the single-target datasets MoLane, TuLane and the multi-target dataset MuLane.
strengths: Compensates for data scarcity and encourages future research in lane detection.
weaknesses: Experiments part are limited. The algorithms covered by this paper is not well rounded. More classic algorithm should be included.
correctness: Yes.
clarity: Well written.
relation to prior work: Clearly discussed.
additional feedback: No.
review_title: Good work while some details need to be improved

final_score: rating: 6: Marginally above acceptance threshold
review_content: summary and contributions: The paper proposes a 3-way sim to real benchmark for cross-domain lane detection, including single- and multi-target UDA. The author use the simulation software CARLA simulator and a real-built 1/8th Model vehicle to collect synthetic data and real lane data, respectively. Based on the three datasets, the authors introduce the first sim-to-real domain adaptation benchmark for lane detection, CARLANE. Well Known UDA methods like DANN and ADDA are evaluated, and the authors also propose a new method, named SGPCS, which achieves state-of-the-art performance on CARLANE.
strengths: 1. Lane detection is a cutting-edge field, which has gained much attention in recent years. The vast amount of collected labeled data has greatly facilitated progress in this field. Also, a large benchmark for cross-domain lane detection can greatly promote the development of the field.
2. Extensive experimental results and visualization are provided. The code is also provided, making it easy to reproduce the result.
3. The paper is well organized and easy to understand. 

weaknesses: 1. This dataset is somewhat simple for domain adaptation, i.e. the domain shift is small. First, the source-only model performs well on the target domain data, and all three settings can approach 90% lane accuracy. Second, the DA method is not significantly improved compared to the source-only results. It seems that DA has no obvious effect. This phenomenon is more obvious in the t-sne visualization. Third, the gap between the source-only results and the target-only results is small, which makes the significance of this study very limited.
2. More explanations are needed why DANN performs worse than source only in Tab3. Why does DANN cause the phenomenon of negative transfer, is it caused by the distribution of the dataset or is it caused by the characteristics of lane detection.
3. Only sim to real adaptation is included. Benchmark for 'real to real' (e.g. model vehicle - > TuSimple) is also important.
4. The statistics of the datasets proposed by the authors and those of previous works need to be compared.
5. Some related works for cross-domain detection and self-supervised land detection methods are missing.  
correctness: I think the claims made in the submission is correct.
clarity: The paper is well organized and easy to understand. 

relation to prior work: This work proposes a 3-way UDA dataset and also proposes a UDA benchmark for the first time, which makes this work a unique contribution. Furthermore, this work has discussed the few previous UDA datasets in related work. However, the statistics of the datasets proposed by the authors and those of previous works need to be compared.
additional feedback: Please see the 'Weakness'.
review_title: A useful large-scale lane detection dataset for domain adaptation

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: The paper introduces a 3-way benchmark called CARLANE focused on simulation to real-world unsupervised domain adaptation with single- and multi-target. The paper also provides dataset tools for labeling images. The authors conduct extensive experiments and draw interesting conclusions.
strengths: 1. The paper proposes a systematic way to evaluate UDA methods on the proposed datset, which is useful for research in lane detection.
1. Data from simulation, the model vehicle, and real-world scenarios allows flexible usage to learn lane detection models.
1. The paper extends the standard single-target UDA to multi-target UDA, which is closer to real-world applications.
weaknesses: It would be good to consider different weather conditions in the 3 proposed datasets. 
correctness: Yes.
clarity: Yes, it's well-written.
relation to prior work: Yes, it's well-discussed.
additional feedback: NA
review_title: Interesting paper on a lane detection benchmark

Meta-review: metareview: The meta reviewer has read the paper, rebuttal, reviews, and the author's revision. The meta reviewer agrees with the reviewers that this lane benchmark is of interest to UDA, and the authors release the data, tool, and source codes to support their claimed contribution. The meta reviewer thus recommends acceptance.
confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept