Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: The paper addresses the issue of segmenting glass surfaces in an image. The novel contribution is to use semantic context more explicitly in the overall network system design, as often there are no visual features with which to identify glass. The authors also contribute an extensive new data set with carefully labeled glass areas that includes many images from standard data segmentation data sets. The new network design outperforms the state of the art methods.
strengths_and_weaknesses: The network incorporates both a spatial and a semantic (area+category) based backbone, followed by two modules that combine inputs from both backbones using different architectures. The ablation study shows that both modules are important for performance.

The ablation study seems comprehensive, and it includes how best to assign the query/value/key concepts in the context module.

The network design outperforms existing state-of-the-art networks on both prior data sets and the new contributed data set.

The authors are providing a new and useful data set for glass detection.

Figure 3 has space for more annotation of the components. All component blocks should have a label, and the reader should not have to go back to the text to figure out what the components are (for example, if Upernet is used for the decoder, that should be clearly labeled in figure 3.

The paper needs editing for minor details.  For example, in the caption for figure 3

"The CCA Module is positioned at higher level in the encoder component to inference the relationships between contextual meanings and locations of objects. Features from multiple stage are aggregated by decoder to produce output map." -> "The CCA Module is positioned at a higher level in the encoder component to infer the relationships between contextual meanings and locations of objects. Features from multiple stages are aggregated by the decoder to produce the output map."
questions: Did the authors find any situations where the context fooled the network into thinking glass existed when it did not?
limitations: The authors note the limitation created by mirrors or mirror-like surfaces. There are no examples shown where context fools the system into thinking glass is present when it is not (e.g. an open window).
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: 
The paper tackles the issue of glass surface reflection from a single optical image. The authors observe that glasses are difficult to detect as isolated objects (they do not have specific radiometric or geometric properties). Consequently, the authors propose to account for the semantic context around objects, with the aim to ease the task. Concretly, the authors introduce a network architecture that encompass semantic information and low level features. A new dataset of glass is also created. Results are illustrated on a classic benchmark (Glass Detection Dataset, Mei and al. 2020), and the authors' dataset. 

strengths_and_weaknesses: The main idea of the authors, ie, accounting for the semantic context surrounding glass surface so as to ease the detection, is certainly interesting. Thought it has been proposed by others for other applications  (in particular for human detection),  it is particularly interesting in the current setting. To push further the idea, the problem could be formalized as establishing pair-wise dependencies between semantic objects. 

The implementation of the idea makes sense to me, thought the technical description of the approach appears to me sometimes somehow vague. I wonder if this architecture could be re-used for other problems or applications, ie how much it is specific to this sole problem  at hand  or could it be useful beyond glass detection?

Results on the two datasets are encouraging. Especially the detection visualizations (figure 5, 6) show very consistent glass contours. Quantitative results are less assertive (but it is ok). 

The weak point of the paper is probably the sometimes vague technical description of the approach and probably insufficient discussion about technical choices.  For example in eq.(1) ,  specifying the dimensional of the input and output tensors would  be useful. A short description of the UperNet network used for final segmentation would also have been useful. A clearer description of the learning (finetuning) procedure would also be welcome. 


questions: 1- I am little confused about the use of SegFormer and DeepLab, which are according to the authors, "aggregate spatial-wise object location features"  for the former, and give "intrinsic representations of each object category" for the latter. However, these two networks, to my knowledge, are two semantic segmentation networks. I might misunderstand the differences of usage of these two networks, it would be worth to clarify. 

2- The training procedure is important indeed, and a few additional words on this matter would be interesting.  The authors write: "We then further fine-tuned the model using our GSD-S dataset to introduce a more diverse set of object categories for better semantic extraction capacity. Note that the semantic backbone after fine-tuning is fixed and isolated from subsequent training for glass surface detection". I do not understand why a fine tuning then a subsequent training is necessary? What is learnt at each stage and for what purpose? 

3- Equations 2, describe in a very classical way the attention frameawork. What is the dimensionality of f_sp, f_se, Q,K,V. 
limitations: No concern with societal impact.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper proposed a new method for glass surface detection whose key insight is to incorporate semantic context information of the environment. Two new modules are proposed: (1) a Scene Aware Activation (SAA) Module that adaptively filters critical channels for spatial and semantic features and (2) a Context Correlation Attention (CCA) Module that learns the contextual correlations among objects both spatially and semantically. The paper also proposed a large-scale glass surface detection dataset named GSD-S containing detailed semantic annotations. Experiments show that the proposed method achieves state-of-the-art performance on multiple datasets including the proposed GSD-S dataset.
strengths_and_weaknesses: Strengths:
+ The idea of exploring the semantic relationship between glass and surrounding objects to facilitate glass detection is interesting and insightful. 
+ The paper created a larger dataset for glass detection with refined glass masks and semantic annotations, which will be useful for future research on this topic.
+ The paper conducted extensive experiments which are convincing. The method is compared to various kinds of baselines on multiple datasets, and the proposed method achieves very good results consistently. Very detailed ablation studies also justify the design choices well.

Weaknesses:
- I'm not an expert in this area, but I think the technical novelty of the proposed method is a little limited. The method builds upon many existing frameworks, and combines and adapts them to the glass detection problem, which makes the technical novelty incremental, although I think the insights behind the design are good. 
- The method section contains too many details which are overwhelming to read. For example, in Sec 4.2, Eq (1) and Fig 4 (a) give a very detailed description of each individual operation, which kind of goes into the weeds. I think it would be better to cut some unimportant details or move them to the section of implementation details. 

Given the interesting idea, the good performance, and the extensive evaluation, I currently vote for borderline acceptance despite that the technical novelty is a bit limited. I would like to see reviews from other reviewers and rebuttals from the authors to make further decisions.
questions: - I think more analysis to justify the effectiveness of the proposed dataset would be helpful. How can one know if the proposed method is more effective? Do methods trained on the proposed dataset perform better on in-the-wild out-of-distribution data than on existing datasets? 
- Does the proposed dataset contain mostly indoor scenes? How will the method perform on images of glasses outdoor? 
- Although the glass co-occurs a lot with walls, etc, the contents that are seen through the glass can be more random and diverse. How robust is the method to different contents inside the glass area?  
- The first sentence in the caption of Table 1 says "GSD dataset". Should it instead be "GSD-S dataset"? 
- It is unclear which curve belongs to the proposed GSD-S dataset in Figure 2 (b) as "GSD-S" does not exist in the legend (I guess it is "Glass-Seseg"?). It is also not clear what GDD and GSD stand for without citations and captions. Suggest making Figure 2 (b) clearer.  

Minor:
- Last sentence in Fig.3 caption "multiple stage" -> "multiple stages'.
- Seems that Fig 4. (a) "Attended Features" should instead be "Activated Features".




limitations: Yes.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: The authors observe that the glass object usually cooccurs with some specific objects. Based on the observation, the work proposes a method consisting of two main components CCA and SAA to feature selection in spatial and semantic dimension and efficiently model long range dependency.  Extensive experiments are conducted on their newly collected data to demonstrate the effectiveness of the proposed method and its superiority to the state-of-the-art methods.   
strengths_and_weaknesses: Strengths
1. The performance of the proposed method is promising.
2. The collected data is useful to facilitate the research in this direction.

Weaknesses
1. The observation is common and lacks insight, thus making the work less inspiring. In addition, the observation of some object is closely related with some other objects is not only existed in glass object. For example, sea is usually closed with sand.
2. The proposed CCA and SAA modules are not specifically designed for glass object, it is more like a general way to model the correlation among objects. In this case, the methods should be evaluated in more general segmentation task, where the observed relationship is present as well. 
3. Line 215, the authors mention that "We then further fine-tuned the model using our GSD-S dataset". Did all compared models do this fine-tuning? How dose it affect the model performance?
questions: Please address the weaknesses mentioned above. My major concern is that the observation is specifically with glass object, thus the proposed method does not deal with challenges caused by the characteristics of glass object. 
limitations: Yes
review_title: null

Meta-review: metareview: Post rebuttal, three out of four reviewers were in favor of acceptance. The hold-out reviewer pjyT was primarily concerned with the fact that the method isn't specifically tailored to the task (in the sense that other domains have similar features) and therefore the method ought to be tested in more settings. The AC understands pjyT's perspective in the sense that deep problem-specific understanding of a domain often drives methodological contributions. However, the AC does not find this to be a convincing argument for rejecting the paper and thinks the work provides enough of a strong contribution. Accordingly, the AC recommends accepting the paper.
confidence: Certain
Decision: Accept