Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: Paper proposes an attractive smoothing algorithm via log-supermodular CRFs, which takes the CRF as a regularizer for the DNNs. . Extensive experiments on a range of applications demonstrate that even conventional models outperform more recent ones with the addition of an attractive CRF.

The paper is written clearly, both the quantitative and qualitative analysis experiments are well done, which can fully verify the effective of the attractive smoothing algorithm. However, I'm not an expert in this field, so I'm not able to appraise it from the aspects of innovation, algorithm design, and whether it has a positive role in promoting the research field. 
strengths_and_weaknesses: N/A
questions: N/A
limitations: N/A
review_title: null

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper proposes the use of super-modular CRF to regularize a DNN model for applications where smoothing is important. The intuition is that while DNNs are easier to scale, they do not take into account the natural constraints in the task e.g. two neighboring image pixels need to have similar colors. In this DNN-CRF hybrid model, there are two DNNs for generating the feature vectors and the CRF parameters respectively, and a CRF-based inference layer. Results are reports on three tasks: stereo matching, image colorization and semantic segmentation. On stereo matching, adding the CRF regularizer improves the performance of three DNN baselines. On semantic segmentation, CRF gives a small boost to the DNN. On image colorization, adding the CRF improves the peak signal to noise ratio.

strengths_and_weaknesses: Strengths:
* Novel use of supermodular CRFs to improve DNN performance across tasks where smoothing is important
* Performance improvements on three tasks

Weaknesses:
* It would be useful to compare performance against alternative neural approaches e.g. RNN or transformer, that can take into account correlations across time steps.
* The paper does not report the impact on training time due to the CRF regularizer. 
* The paper states that "Our approach applies in both discrete and continuous settings" but the approach is evaluated only for scenarios with continuous inputs.
* It would be useful to provide more details on the hybrid model training e.g. are there practical tips to ensure convergence? Does the DNN/CRF training need to be done in some alternating fashion etc? 
* The appendix is not included in the paper.








questions: * It would be useful to provide more details on the DNN-CRF hybrid training including the algorithm, the variational method, update steps, any practical tricks to ensure convergence, etc. 
* What is the impact on training time due to the CRF regularizer?
* Would it be possible to report a metric that indicates whether the proposed method has improved the smoothness objective on each of the tasks?


limitations: There is no explicit discussion of limitations in the paper. Hence, it would be useful to add a paragraph on limitations to the conclusions section. The authors should also add a sentence about the potential negative societal impact in the conclusions.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The authors present log-supermodular CRF as a smoothing layer for a wide class of applications, including optical flow estimation, activity recognition, colorization, and segmentation. They show in experiments that the proposed module can improve the performance of the model over all the mentioned tasks.
strengths_and_weaknesses: Strengths:
- easy to read through and solid experiments.

Weakness:
- the authors only compare the deep models with and without attractive smoothing. I would encourage them to consider some other adaptive, ad-hoc smoothing methods as baselines.
- It's hard to tell the improvement through the figures for tasks like colorization and segmentation. Also, it is questionable whether segmentation needs smoothing.
questions: - Are the potential functions you presented in Equations 1-4 log-supermodular? Why are they would be helpful for smoothing?
- Could you briefly explain what "attractive" means in this context?
- In Figure 1, what is $\mu,\sigma, \lambda$ and  $\Delta\mu,\Delta\sigma, \Delta\lambda$? you may need to give an explicit form of BFE in your methodology.
- Why not compare with some deep network with ad-hoc smoothing techniques?
- In Figure 2, I can only tell the first and second rows have distance differences, while the rest rows do not show much difference.
- In Figure 3, I cannot tell which colorization is better. Are there any other examples that can show the impact of the proposed method?
- In Figure 4, smooth for segmentation may remove all the detailed small blocks. It may be helpful but not necessary.
limitations: The author may mention if their attractive smoothing idea can be (not-)helpful for other CRF applications in text, like part-of-speech tagging and dependency parsing.
review_title: null

Meta-review: metareview: The paper proposes to use   Log-supermodular CRFs  to smooth the DNN models. The paper is well motivated and conduct extensive experiments to  verify the effective of the attractive smoothing algorithm.  
It could be better if the paper conduct more experiments based on different networks such as RNN and Transformer.  Besides, this paper only explores the proposed attractive smoothing. It should make a comparision with other smoothing methods. 

confidence: Less certain
Decision: Accept