Venue: NeurIPS 2021 Conference

Reviews:

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a method for image dehazing by polarization-based learning pipeline. It is based on a physical formation model which is considering the spatially variant real-world scattering and taking into account the polarization effects of both transmitted light and airlight. Experiments demonstrate that the proposed method can dehaze robustly with fewer artifacts, and performs better in recovering details.

main_review: It is a very interesting approach to utilize polarization information of both transmitted light and airlight as a feature for image dehazing. By introducing spatially-variant turbulence N(x,c) from polarization information, the effect of light scattering in the air can be well reflected spatially, and the dehazing results seem to be better. Although the input image data is different, the dehazing results from the proposed method are better than the results from other comparison methods and I believe that the proposed method is able to utilize the polarization information.

On the other hand, I am afraid that there may be some assumptions in the scene where the proposed method can be applied as the scenes used in the evaluation experiments are similar. I think polarization information can be very useful, but it can also lead to erroneous results due to noise effects. (e.g. the haze in some pixels is very thick and leaves little information of the transmitted light.) It would be better to conduct evaluation experiments of the proposed method for various scenes (depth, normals, colors). I feel that the efficacy of refinement subnetworks (g2 and g4) as well as some networks for polarization information is significant, so it would be good to do more evaluation of refinement subnetworks.

The paper is well-written and well-structured. There are only a few typos (e.g. ‘dehzaing ’ in Section 6 Conclusion) that can be addressed in another proofreading.

limitations_and_societal_impact: The author fully addresses the physical image formation model of hazy images, and mentions that it does not apply to such as fog or mist. The use of polarization information as a feature will have a significant impact as polarization cameras become more widely used.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper proposes a neural network for image dehazing where the input is
expected to contain polarization information (e.g. multiple polarized images of
the exact same scene). The total irradiance and total degree of polarization
can be obtained directly from the polarized input frames. The paper then
follows the image formation model of [77] (and others) to identify key
quantities to estimate: the infinite airlight color, and the degree of
polarization of the ambient and scene light. Multiple neural networks
are designed to estimate these components; they are coupled using
closed-formed expressions derived from the image formation model to produce the
final dehazed image. The networks are trained independently, then finetuned
jointly on a synthetic dataset.

main_review: This paper is well-written, consistent and produces good result. Despite its
claims, there is not much novelty in the image formation model. Like many
'physically-based' deep learning image restoration approaches, the proposed
technique replaces heuristic measurements of physical quantities (e.g. a patch
of sky to estimate the airlight) by neural network estimators.


### Strengths

+ I found the paper clear and easy to follow.
+ The description of prior art appears complete and relevant.
+ The derivation is sound (although not technically a contribution of this
work).
+ Synthetic test results are convincing and outperform previous single- and
multi- frame baselines.


### Weaknesses

- The complete derivation of section 3.1 is from [77], this should be better acknowledge.
- One the claimed contributions is a generalized hazy image formation model,
but it is not clear how this paper generalizes the ideas of [77]. The spatially
varying noise terms N(x, c) are introduced but never used in the derivation.
- The real-world results (Fig4) are not convincing. The image suffers from severe
grid artifacts, corrupted boundaries, ghosting around buildings. This makes
me question the "robustness" property claimed in the intro (2), it is not evident
at all that synthetic training data + the physical models are sufficient to
make the method practical in the real-world.
- How do the result degrade visually without the "refinement" networks? It
would be helpful to have some extra real/synthetic images in supplemental.


### Misc

- 268: dehzaing -> dehazing
- 28: although [...] however -> one suffices
- 41: for enabling -> to enable
- paragraph l.41 and l.53 of the intro read as a bit redundant. Maybe a contribution list is not  needed,
or the previous paragraph can be shortened?
- 123: divide -> split, decompose
- 134: consider using different notations for the spatial variation terms N(x, c).
I found it confusing to see it appear in theta, beta and A_\inf
- 201: Is Gaussian noise really a good model for the beta and alpha_inf
spatial variations? This choice seems a bit ad hoc
- The results from SPCVE seem very noisy

limitations_and_societal_impact: The paper shows and discuss the limitations of their work.
review_title: null

final_score: rating: 6: Marginally above the acceptance threshold
review_content: summary: This paper presents dehazing algorithm using characteristics of polarized images. It first introduces generalized physical formation model of polarized hazy images, regarding spatially-variant real-world scattering. Taking three differently polarized hazy images as input, the proposed algorithm estimates and refines degrees of polarization (DoP) of the transmitted light T and airlight A, and utilizes these predictions to finally obtain original scene radiance R. By doing so, this paper was able to show outstanding performances, especially on real-world images due to its consideration on spatially-variant DoP and global airlight.
main_review: The physical formulation and the dehazing pipeline that were proposed in this paper are reasonable, clearly described, and properly executed. The paper is well-written and was easy to follow. While I am satisfied overall, I have a few questions.

1. In refining process for both T and R, the authors claimed that the refinement was done by ‘using semantic and contextual information extracted from I(1, 2, 3)’. While removing these refining subnetworks does drop performance severely (table 2), it seems like the subnetworks are simply using I(1, 2, 3) as inputs and no other technical consideration using architecture or loss function was given. Please elaborate on why do the authors think that this kind of simple process can boost so much of a performance.

2. On top of the mention from the paper, ‘comparing with learning-based dehazing methods might be a bit unfair because of the difference in types of input data’, I think it also restricts practicality and applicability. The most commonly used image capture device is an RGB camera, and these ‘learning-based methods’ can be applied to any hazy image while the proposed algorithm cannot. I want to know the authors’ opinion on why the polarization-based dehazing approaches should be researched.

Minor comments: In figure 2, bottom block, initially estimated R is marked as T^.

limitations_and_societal_impact: I appreciate the authors for mentioning the limitation regarding the generality of the proposed hazy image formation model, for other image degradation weather conditions.
review_title: null

final_score: rating: 7: Good paper, accept
review_content: summary: The paper presents a method for polarization-based dehazing of images. Using a camera with a polarization filter array, multiple polarization images can be captured in a single snapshot. These images are used to estimate an unpolarized image $I$ and a degree-of-polarization image $P$. A series of neural networks are trained on synthetic data to estimate the polarization and radiance of the ambient and transmitted scene light conditioned on $I$ and $P$. The results of the dehazed estimated scene radiance are compared with other methods for dehazing using polarization images or a single unpolarized image. Quantitatively, the approach appears to offer roughly 1-2 dB in improvement over previous methods in terms of PSNR. Qualitatively, the method appears to generate improved results on captured data as well.


main_review: Overall, the paper seems to be well written and the method is clearly explained. The results convincingly demonstrate that accounting for polarization in the dehazing task can improve image quality. While the results do not offer an overwhelming improvement over previous techniques for single image dehazing, the improvement is significant. I think the method could inspire future work on polarization-based dehazing and inverse scattering methods.

I have just a few comments which I note below.

I was surprised by the poor performance of the SPCVE method. In their original paper the method appears to offer a good deal of qualitative improvement over the original images whereas the results shown in Figs. 3 and 4 for SPCVE are almost worse than the input images in some cases. What are the main reasons for this difference? Are there situations in which SPCVE is more competitive with the proposed approach?

Part of the motivation for the method is that regions with the same material properties often reflect light with similar polarization properties. I think seeing these properties in captured data would help motivate the method. For example, I would be curious to see the polarization images side by side (rather than in the stacked format of Figure 3, which obscures their differences). I'm also curious to see the intermediate results of the network, such as the predicted degrees of polarization (these could perhaps be included in the supplemental).

How are the simulation parameters described in section 4.1 chosen?

How do the predicted degree of polarization images in the captured data compare to the simulated dataset?

Fig. 2 typo \hat{T}->R after Eq. (7)(b)

limitations_and_societal_impact: Limitations and societal impact appear to be adequately addressed.
review_title: null

Meta-review: The four reviewers thought this paper was above threshold for acceptance. They all found the idea useful and interesting. The author response also helped to clarify some issues raised by the reviewer.
Decision: Accept (Poster)