Venue: NeurIPS 2022 Track Datasets and Benchmarks

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: summary and contributions: MBW are proposed to tackle two problems: (1) uncalibrated but synchronized videos with few views, and (2) a limited manual annotations. The main contributions lie in 3 aspects: 
1. Propose a zoo dataset with uncalibrated two synchronized views
2. Propose a semi-supervised framework by leveraging MV-NRSfM and optical flow-based algorithms
3. Study different factors of the proposed method on the proposed dataset systematically
strengths: [S1] The paper create a zoo dataset with uncalibrated two synchronized views. The mutli-view animal datasets are rare and will be useful for future research.

[S2] The paper proposes a semi-supervised framework by leveraging MV-NRSfM and optical flow-based algorithms. MV-NRSfM handles the uncalibrated few views, while optical flow-based methods employ limited annotations to generate candidates. The whole framework run iteratively to perform auto-labeling.

[S3] The experiments show that they achieve similar performance with 1% annotations to fully supervised methods. Besides, the authors ablate the proposed method on the proposed dataset systematically

weaknesses: [W1] Technical novelty is limited. The merits of handling few uncalibrated views are originated from MV-NRSfM, and the 2d landmarks are generated from off-the-shelf flow-based model. The main techinical contribution is to use them in a semi-supervised scheme.

[W2] As shown in Fig. 4(a), the performance of MBW drops as the number of views increases. What's the reason behind that?

[W3] The experiments show that the method works well when there is only 1% annotation. What’s the minimum percentage for the algorithm to work well?

[W4] What will happen if optical flow-based model performs not very well? Any requirement for the minimal performance of optical flow-based model?
correctness: Some claims might be too strong, e.g., in abstract, the paper claim that "With just a few annotations ... obtain 3D reconstructions that are impossible with other existing approaches"
clarity: Yes, the paper is overall well written and easy to understand.
relation to prior work: The authors clearly discussed how the proposed method and dataset differ from previous works
additional feedback: NA
review_title: A multi-view zoo dataset with a semi-supervised method

final_score: rating: 6: Marginally above acceptance threshold
review_content: summary and contributions: In this paper, the authors have combined a non-rigid 3D neural prior with deep flow to obtain high-fidelity landmark estimates from videos with only two or three uncalibrated, handheld cameras.  Utilizing Multi-View Non-Rigid Structure from Motion (MV-NRSfM) to more reliably estimate camera poses and 3D landmark positions from noisy 2D inputs with few cameras and leveraging deep optical flow, the paper presents a novel process. The dataset collected for this project is another contribution to the community. 
strengths: The ability to do achieve these results without calibration is worth highlighting. Calibration is a critical process and in many cases, essential. but by eliminating the need to calibrate, the proposed idea can expand to variety of applications. 
To capture the 3D reconstructions of the unoccluded objects of interest with the minimal cameras is also worth mentioning. 
The significantly low number of annotated frames needed for this method to work is impressive. 
The comparison in Table 1 is also highly appreciated as a reader. 
The approach is clearly laid out. 
The paper is well written and easy to read. 

weaknesses: The dataset being released is very small. It captures the essence of the proposed idea but seems lacking. Releasing a larger dataset with multitude of variations in addition to the mentioned ones would help further help (also to highlight the strength of the technique)
Impact of the technique on semi-occluded objects in the scene would further help. 
In Appendix D, the idea for bounding box estimation and reducing the problem to single object-single frame sub problem is good but better techniques can be used for the same. This could help in reducing the number of iterations to solve this. 
Also it would help to understand the impact of the bounding boxes not completely covering the Chimpanzee in the scene.  

correctness: Yes. 
clarity: Yes. Especially appreciated was the Experiments section. Setting up the questions before jumping into the results is helpful, and very structured. There are some grammatical errors in the paper but easy to fix. For example, Fig 5(b) title - ".. denoise the improve the.."
relation to prior work: Yes
additional feedback: Some shared in weaknesses. 
What percentage of frames annotation had to be fixed by manual intervention? Which video had a higher percentage and any insights into the reasons for the same?
Wishing the very best to the authors and thank you for your contributions.
review_title: MBW 

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: This paper proposes Multi-view Bootstrapping in the Wild (MBW), a semi-supervised pipeline that can annotate 2D and 3D landmarks of articulated objects given videos captured by two or three uncalibrated handheld cameras, where 1-2% of frames are manually annotated. Specifically, the proposed pipeline builds a neural shape prior from sparse manual annotations using Multi-View Non-Rigid Structure from Motion (MV-NRSfM), then propagates the initial labels through the entire video using a pre-trained optical flow method. The propagated 2D landmark candidates are used to iteratively retrain the MV-NRSfM as well as a 2D detector. The authors collect the “zoo dataset” and apply the proposed pipeline for 2D and 3D landmark annotation. Extensive experiments on the “zoo dataset” and the Human3.6M dataset show the reliability and scalability of the proposed annotation pipeline, i.e., given very limited number of camera views and manual annotations, the proposed pipeline can reliably annotate the 2D and 3D landmarks while rejecting (correcting) outliers.
strengths: 1. This paper proposes a feasible semi-supervised pipeline for 2D and 3D landmark annotation of articulated objects under in-the-wild setting. In contrast to complex capture systems like Panoptic Studio that can only be used in the laboratory and applied to limited objects, the proposed pipeline is more flexible, and has the potential to be applied to a wide variety of objects. Collecting large-scale dataset is possible using the proposed pipeline, as only sparse manual annotation is necessary.
2. The authors successfully apply the proposed annotation pipeline to the newly collected “zoo dataset”. This shows the effectiveness of the proposed pipeline, while providing researchers with valuable data of less studied articulated object categories.
3. Extensive experiments show the reliability of the proposed annotation pipeline.
weaknesses: 1. This paper focuses on the semi-supervised annotation pipeline rather than the dataset itself. However, the novelty of the proposed pipeline is limited, as it mainly takes advantage of MV-NRSfM and optical flow. 
2. Although the proposed pipeline outperforms baseline methods, it is not clear whether the annotation quality is sufficient to be used as ground truth. More analyses of the pipeline may be necessary, e.g., what should be done when the semi-supervised annotation quality is not good enough? Does adding more manual annotation help? If so, how many additional annotations would be meaningful? From Fig 4(a), increasing manual annotation from 1% to 2% does not seem to significantly improve the outlier rejection.
3. There is limited discussion on the “zoo dataset”. It would be nice if the authors can provide more in-depth analyses of the dataset, e.g., what is the potential usage of the “zoo dataset” given it is annotated by the semi-supervised pipeline? 
correctness: Yes. The claims made in the submission are correct. The dataset is constructed in a sound way.
clarity: Yes. The paper is well written and easy to follow.
relation to prior work: Yes. The difference of this work from previous works is clearly discussed.
additional feedback: 1. More analyses on the robustness of the semi-supervised annotation pipeline would be helpful. For example, does it always work when the camera baseline is sufficiently large (all keypoints are visible from at least one view)? If not, what would be the typical failure pattern?
2. The authors can showcase the value of the “zoo dataset” by applying it to some downstream tasks.
review_title: A scalable semi-supervised pipeline for 3D landmark annotation of articulated objects

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: Hand labeling objects in natural videos is challenging and labor-intensive. Instead of using rigid geometry and calibrated cameras, the paper proposed an approach of combining a non-rigid 3D neural prior with the deep optical flow to obtain good-quality landmark detection. The approach requires annotations from 1-2% of the video frames from only 2-3 uncalibrated handheld cameras. The authors conducted a comprehensive analysis in 2D and 3D reconstructions of the remarkable experimental results on two benchmark datasets: human and newly curated datasets.
strengths: 1. paper is well-written and well-organized with great clarity on design motivations
2. the arguments made in the paper are justified and backed up by experimental results. Comprehensive analysis is provided to better understand the design motivations and model behaviors.
3. the performance improvements in 2D and 3D results are significantly greater than in baselines
4. the idea of active learning is not new; but utilizing multi-view for bootstrapping in the wild is interesting. It could extend to other applications in 3D vision.
weaknesses: 1. it would be great if the authors can conduct experiments to demonstrate the effectiveness of the method in more unconstrained natural environments when there involve large camera motions as well. (*quantify how large camera motions would impact the results)
2. test generalization abilities when context changes (e.g. tigers move from one bridge cage to another dark cage in the zoo) and quantify how the method could quickly adapt to continuously changed domains
correctness: yes, the claims seem to be correct as far as I understand from the experimental results.
The evaluation methods and designs are appropriate and convincing
clarity: 1. model design is well motivated.
2. all the arguments made in the result section are well justified and backed up by experimental results
3. Good use of language. It is well-organized and easy to read and understand.
4. contributions are listed explicitly and well justified
5. in the related work section, all the differences in the current approach compared with previous methods are highlighted
relation to prior work: yes, the authors have explicitly identified key differences between current approaches and others in the related works. These include temporal consistency, camera calibration, 3D labels, number of annotations and wild setups.
additional feedback: see weakness section
review_title: good paper; design well justified; comprehensive analysis; accept

Meta-review: metareview: This paper presents a dataset of uncalibrated pairs/triplets of videos of zoo animals. The paper also presents a method to label this data in a self-supervised manner with minimal annotation. Reviewers appreciated the contents of this dataset and see value for follow up work. The method for labelling was also appreciated. Weaknesses of the paper have been addressed or acknowledged. I recommend accpeting this paper to the NeurIPS 2022 Datasets and Benchmarks program.
confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept