Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposed a new face rendering technique to synthesize paired NIR-VIS images for improved face recognition in NIR space. Unlike the previous methods, which use image-to-image translation models learned from paired NIR-VIS images, this method use physical-based 3D face rendering. It first reconstructs 3D face meshes and reflectance assets in VIS space, then infers the corresponding reflectance assets in NIR space, and finally synthesizes paired VIS-NIR images at various head poses and illuminations. It also employs a novel ID-MMD loss to close the gap between VIS and NIR features in NIR-VIS face recognition training. The proposed method helped to achieve state-of-the-art face recognition performance on four NIR-VIS benchmarks.
strengths_and_weaknesses: ### Strengths
- The proposed method allows generating infinite pairs of VIS-NIR facial images covering different head poses and illuminations without learning from any real VIS-NIR image pairs.
- The paper proposes a sophisticated algorithm to transform a VIS reflectance asset into its NIR version.
- The paper proposes a novel ID-MMD loss to close the gap between VIS and NIR features in NIR-VIS face recognition training. Ablation studies confirm its effectiveness.
- By using only the synthesized VIS-NIR images, the trained face recognizers can produce competitive NIR face recognition performance compared with the baseline methods trained on real NIR-VIS datasets. After fine-tuning on real NIR-VIS images, these recognizers provide state-of-the-art and near-perfect performance on four NIR-VIS face recognition benchmarks.

### Weaknesses
- The synthesized images, particularly the VIS ones, look pretty unrealistic. The 3D models only cover the facial part, leaving some components missing, including hair, facial accessories, and background.
- The denotations such as \mathcal{R}^{NIR} should be explained earlier, e.g., in the text or the caption of Table 1, rather than in the caption of Fig. 4.
- There is a predefined subset of WebFace260M called WebFace4M. The authors said they randomly selected images to create their WebFace4M dataset. Are they the same dataset? If not, better change the dataset name to avoid confusion.
- It seems the authors did not augment facial expressions; all qualitative figures of synthetic images have neutral expressions. I think the expression is not an important factor in the test benchmarks, but the authors should consider it when developing face recognizers for real-world applications.
questions: - Can we improve the method by using full-head 3D reconstruction, augmented expressions, and augmented accessories?
- There is a predefined subset of WebFace260M called WebFace4M. The authors said they randomly selected images to create their WebFace4M dataset. Are they the same dataset? If not, better change the dataset name to avoid confusion.
limitations: There is no discussion on limitations. The paper limits its application to avoid potential social impacts.
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: The work proposes a NIR-VIS face matching dataset constructed with a physically-based renderer. An ID-MMD loss is employed to facilitate the identity feature learning as well as reduce the modality discrepancy. The work achieves state-of-the-art performance on 4 NIR-VIS face recognition benchmarks. 
strengths_and_weaknesses: Strengths:
(1)	The proposed method is capable to automatically generate multiple NIR-VIS image pairs with identity information reserved, which is of great significance. 
(2)	With the proposed training scheme, the NIR-VIS face matching dataset helps improve the NIR-VIS face recognition performance by a large margin. 

Weaknesses:
(1)	The proposed ID-MMD loss reduces the distance between the NIR-VIS feature centroids of the same identity, which is effective yet not novel [1]. 
[1] Wei, Ziyu, et al. "Syncretic modality collaborative learning for visible infrared person re-identification." ICCV. 2021.
(2)	The training differences from other compared methods should be more detailly stated
(3).  Some related works are missing, e.g., Dual face alignment learning network for NIR-VIS face recognition, Orthogonal modality disentanglement and representation alignment network for NIR-VIS face recognition, Dual-Agent GANs for Photorealistic and Identity Preserving Profile Face Synthesis

questions: (1)	It would be better if the combinations of modality discrepancy reduction losses and id loss, as the combination of losses sometimes can have larger impact than single ones. 
(2)	In DVG-Face, the evaluation metrics of generation quality are Mean Similarity, Mean Instance Similarity and Frechet Inception Distance. Why do the authors take different metrics in this work? Table 2 shows that the proposed method holds the smallest Mean Identity feature Distance. However, it can also be interpreted as lack of diversity. 

limitations: See Weaknesses & Questions
review_title: null

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: In this paper, the authors propose a method to synthesize near-infrared faces by transforming them from visible faces. Based on this, the authors can conduct NIR-VIS face recognition without any existing NIR-VIS face datasets. Besides, they also propose an Identity-based Maximum Mean Discrepancy loss to facilitate identity feature learning. The good performance shows the efficiency of their method.
strengths_and_weaknesses: Strengths:
1. The authors utilize a novel 3d-rendering based method to generate large-scale NIR-VIS paired data.
2. The authors propose an Identity-based Maximum Mean Discrepancy loss to facilitate identity feature learning.
3. The descriptions of implementations of the proposed synthesis method are detailed.

Weakness:
1. The motivation for using a 3d-rendering based generating dataset is not clearly illustrated, and it seems that the authors just utilize id-related loss when training. It is suggested to strengthen more use of the generation.
2. The comparison results are not sufficient. It is suggested to add generated data by different percentages to illustrate the effectiveness of the synthesis method.
3. I suggest more comparative visualizations to verify the effectiveness of the proposed method. In addition, the authors should discuss how generalizable the proposed method is in practical situations.



questions: Please see my comments above.
limitations: The limitations and potential negative societal impact have been adequately addressed.
review_title: null

Meta-review: metareview: The paper received 3 positive reviews. The reviewers all lean towards acceptance after the rebuttal. 

Overall this work can be of large interest to the community working on NIR-VIS Recognition. But I hope the authors will present additional visualized results, as suggested by the reviewers.
confidence: Certain
Decision: Accept