Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposes a semi-supervised approach to perform salient object detection in videos using a combination of pseudo-label prediction and adversarial noise reduction training. The authors propose a multi stage training structure where the first focuses on pseudo-label generation using an uncertainty guided approach. The second stage then uses a GAN based structure to improve uncertainty identification and pseudo labels while reducing noise in the overall pseudo label generation. The authors demonstrate the use in multiple datasets and show competitive results for semi-supervised approach. 
strengths_and_weaknesses: Strengths

- The idea of using temporal information for generating semantically matching pseudo-labels is an interesting approach. While optical flow base warping is generally used for temporal consistency, there are errors and drawbacks in optical flow based warping. However, using a non-local attention based similarity mapping removes those errors for matching features to generate pseudo labels.

- The use of uncertainty in the dual decoder module is also an interesting approach to generate positive bias to learn the pseudo labels. As the pseudo labels are noisy, having different weight bias for each pixel is essential to learn better. Uncertainty has been effectively used for such task in image-based pseudo label learning.

- The authors show the result on one of the biggest VSOD dataset (DAVIS) as well as 4 other dataset and compare with prior fully-supervised and one semi-supervised method. The authors also show in table 5 that the uncertainty based pseudo-label approach can achieve results similar to fully supervised VSOD methods using only 50% pseudo labels.

Weakness

- The overall approach using pseudo labels improves semi and weakly supervised task in general and noise augmentation/suppression works on making network resilient to these pseudo labels. In that regard, the uncertainty aware decoder module as well as the cross frame similarity (global matching) does not have significant contribution on its own. The non-local attention based matching is simple but also already validated approach that is in line with other validated approaches such as optical flow based warping and interpolation. Uncertainty based decoder does have technical merit, so more detailed analysis on that would be a plus for the paper.

- GAN based approach for noise suppression and semi-supervised training is an unconventional take, which is interesting. The issue is with having to train a GAN model with multiple outputs (rgb, optical flow and fused salient features). Is the training process similar for all dataset? While optical flow warping is not considered in the cross frame matching module, the generator is learning to fuse the optical flow information. Have the authors explored utility of all input/output streams in the generator?

- While multi-stage approach is common for semi and weakly supervised approach and the components used here have positive effect on the task, most components are simple use of existing validated approaches with low technical novelty. The uncertainty based dual decoder is more interesting and deserves more exploration.



questions: Please see weaknesses.
limitations: The authors have not explicitly discussed limitations of this approach. One would expect certain obvious limitations based on this approach, such as cross frame matching for certain sequences based on motion/content and the training of GAN.


----
Post-rebuttal
The authors have answered the questions from the initial review.
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: This paper proposes a pseudo label generation and processing algorithms. The proposed UGPLG generates temporally consistent pseudo labels. The proposed adversarial learning strategy can suppress the noise. The experimental results show that the method can achieve better performance with fewer gts.
strengths_and_weaknesses: Strengths
•	Results greatly improve the performance and the algorithm is strongly competitive with current algorithms.
•	From the results of the ablation experiments, the improvements in the article have improved the performance.
•	The semi-supervised method proposed in this article has great application value, and it is very creative to use the discriminator to suppress noise.

Weaknesses
•	The formulas and figures in the paper can be expressed more clearly.

questions: 
•	Please indicate the dimension of Aux(Ffuse) in Equation 5 and the dimensions of L(.) and P(.) in Equation 8. Why the dimension of laux is H×W. Similarly, why use * in Equation 13, the specific meaning should be explained.
•	From Figure2.Stage1, the input of the main encoder is Ffuse, but in Equation 7, the input of the main encoder is a frame, please explain in more detail.
•	How to get the optical flow data mentioned in the paper?

limitations: Two main modules are designed in the paper: UGPLG and NS-GAN. In the ablation experiment, only UGPLG is tested individually. But it is also necessary to test the NS-GAN separately.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The paper presents a method for semi-supervised video salient object detection. It adopts Cross-Frame Global Matching Module (CFGMM) to exploit temporal information and uses Uncertainty-Aware Dual Decoder Module (UADDM) to locate uncertain regions with an auxiliary decoder. Adversarial learning is adopted to generate reliable pseudo-labels.
strengths_and_weaknesses: Pros:

1.	The topic of semi-supervised video salient object detection is interesting and important.

2.	The proposed method outperforms state-of-the-art semi-supervised video salient object detection methods on multiple datasets.

3.     Good ablation studies.



Cons:

1.	Insufficient literature review, especially some very related research areas, e.g. weakly/semi-supervised segmentation, and uncertainty-aware segmentation/salient detection, semi-supervised image salient object detection.

2.	The novelty of this paper is somewhat limited. 1) Adversarial training has been fully exploited in very related areas, e.g. semantic segmentation [18]. 

3.	The training process (including the loss terms) is a little bit complicated. 

4.	The runtime performance and computational complexity is not reported. 


questions: 
1.	There are some letters that are not explained clearly. 

a)	What does ‘k’ mean in Figure 1, RGB_{k+n}?

b)	What is L_t in Eq.(1)?

2.	In Eq.7, why do you use ‘I_{t+n}’ as the input of the main decoder? Why not F_{t+n} or F_{fuse}?

3.	The loss (the training process) seems a little bit too complicated. The design of the loss functions is not well validated. Do we really need so many loss terms? 



limitations: No. Please include a brief section introducing the limitations of this work (more analysis needed).
review_title: null

Meta-review: metareview: In this paper the authors propose an approach for semi-supervised salient object detection using a combination of pseudo-label prediction and adversarial training, showing improved results on a number of benchmarks. Some concerns about more detailed analysis of aspects of the approach were raised, but seemed to be mostly addressed in the authors’ response. Some reviewers also expressed concerns about there being sufficient technical contributions, but seemed satisfied enough with the analysis and strong positive results. 
confidence: Less certain
Decision: Accept