Title: Face2QR: A Unified Framework for Aesthetic, Face-Preserving, and Scannable QR Code Generation
Venue: NeurIPS 2024
Accepted: yes
Keywords: Image Generation

QR Code

Stable Diffusion

Control Network
Abstract: Existing methods to generate aesthetic QR codes, such as image and style transfer techniques, tend to compromise either the visual appeal or the scannability of QR codes when they incorporate human face identity. Addressing these imperfections, we present Face2QR—a novel pipeline specifically designed for generating personalized QR codes that harmoniously blend aesthetics, face identity, and scannability. Our pipeline introduces three innovative components. First, the ID-refined QR integration (IDQR) seamlessly intertwines the background styling with face ID, utilizing a unified SD-based framework with control networks. Second, the ID-aware QR ReShuffle (IDRS) effectively rectifies the conflicts between face IDs and QR patterns, rearranging QR modules to maintain the integrity of facial features without compromising scannability. Lastly, the ID-preserved Scannability Enhancement (IDSE) markedly boosts scanning robustness through latent code optimization, striking a delicate balance between face ID, aesthetic quality and QR functionality. In comprehensive experiments, Face2QR demonstrates remarkable performance, outperforming existing approaches, particularly in preserving facial recognition features within custom QR code designs.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper is pioneering in its integration of face identity with QR codes, proposing a novel pipeline for generating customized QR codes with embedded face identity. The key idea is to leverage diffusion models and control networks to create visually appealing QR codes while preserving face identity. The pipeline introduces an ID-aware QR ReShuffle module to address conflicts between face identity and QR code patterns, and designs an ID-preserved Scannability Enhancement module to improve scannability without compromising the face identity and visual quality. The experiment results showcase a perfect balance between face identity, aesthetic quality and scannability.\n\nSTRENGTHS:\n- As the first paper to combine face identity with QR codes, the proposed pipeline for generating face embedded QR code is innovative and addresses the practical needs for social connection in real-world scenarios.\n\n- The IDRS module presents an interesting solution to conflicts arising from different control signals. By rearranging QR patterns to harmonize varying control conditions, the proposed pipeline leverages information from both face images and QR codes to generate customized QR codes.\n\n- The IDSE module significantly enhances the scannability of QR images using adaptive loss, while concurrently maintaining a faithful representation of face identity.\n\n- Experimental results show that the generated QR codes successfully preserve face identity, yielding impressive visual results. Notably, there is minimal interference from QR patterns in the face region.\n\n- The paper is well-organized, thoroughly discussing motivation and related work. Rigorous experiments enhance the credibility of the results.\n\nWEAKNESSES:\n- The paper mentions that the method is limited by the generative models, but does not present bad cases due to failure of generative models. It is recommended to include such examples to provide readers with a better understanding of the algorithm’s limitation.\n\n- Certain technical details in the paper require further elaboration. For example, definition of error rate is not explictly given. In Figure 3, the image difference visualization $D$ appears to support the claim that “adaptive loss modifies face region more gently”. However, without a comparison of $D$ between adaptive and uniform losses, this claim lacks substantiation.\n\n- The paper lacks a comparison of computational resource requirements with other methods. This omission makes it challenging to assess the practical feasibility of this algorithm.\n\n- Typos exist; for instance, line 174 should read “with a learning rate of 0.002”\n\nQUESTIONS:\n- Can the authors provide anaylsis of bad cases caused by failure of generative models?\n\n- Could the authors provide definition of error rate and comparison of image difference visualization $D$ between adaptive and uniform losses?\n\n- While the method excels in visual quality, what are the computational resource requirements compared to other methods?\n\nLIMITATIONS:\nYes. The paper adequately discusses limitations and broader impact.", "TITLE": "Official Review", "RECOMMENDATION": 7, "REVIEWER_CONFIDENCE": 5, "SOUNDNESS_CORRECTNESS": 4, "DATE": "2024-07-12"}

comments: SUMMARY:
The article introduces a novel pipeline designed to create customized QR codes that integrate aesthetic appeal, facial identification (ID), and scannability. The proposed approach incorporates three key components: (1) ID-refined QR Integration (IDQR) seamlessly incorporates facial ID into the QR code background (2) ID-aware QR ReShuffle (IDRS) addresses and resolves conflicts between facial ID and QR code patterns (3) ID-preserved Scannability Enhancement (IDSE) optimizes the robustness of QR code scanning while preserving both the facial ID and aesthetic quality.

STRENGTHS:
(1) The motivation of Face2QR proposed in the paper is straightforward, and the method proves to be effective based on the quantitative and qualitative results presented.
(2) There is sufficient ablation study to demonstrate the effectiveness of each module in this paper.
(3) This paper is well-written and easy to follow.

WEAKNESSES:
(1) The paper's innovation is relatively weak, with each module and its technology being a combination of previous works.
(2) Although the paper includes numerous quantitative experiments, it only involves up to 20 identities, all of whom are celebrities. This limitation hinders the ability to fully demonstrate the method's effectiveness for a broader range of ordinary users.
(3)The paper does not mention how to set the value of a in line 131.

QUESTIONS:
Quantitative and qualitative experiments on a larger and more diverse group of ordinary users are crucial to demonstrate the scalability and effectiveness of the method.

LIMITATIONS:
The authors adequately addressed the limitations of their work
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-12

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work proposed Face2QR, a pipeline for generating personalized QR codes that balance aesthetics, face identity, and scannability. It mainly introduces three components: ID-refined QR integration (IDQR) for seamless background styling with face ID, ID-aware QR ReShuffle (IDRS) to rectify conflicts between face IDs and QR patterns, and ID-preserved Scannability Enhancement (IDSE) to boost scanning robustness through latent code optimization. Face2QR outperforms existing methods in preserving facial recognition features within custom QR code designs.\n\nSTRENGTHS:\n+ Generating personalized QR codes that balance aesthetics, face identity, and scannability seems an interesting topic and application. \n\n+ The design of the major components in this paper, including IDQR for seamless background styling with face ID, DRS to rectify conflicts between face IDs and QR patterns, and IDSE to boost scanning robustness through latent code optimization, is well-motivated and reasonable.\n\n+ Experiments with user studies show the proposed method has strength compared with existing method like Text2QR, ArtCoder etc.\n\nWEAKNESSES:\n- The proposed method is mainly build upon existing works in areas like Diffusion models, Identity Preserved Generative Models, and latest QR generation method like [41]. It is indeed a good application work but may lack technical contributions. \n\n- The proposed new components are good practice for this specific application about QR code generation. It is a good application paper but I'm not sure whether these insights are general enough to fit a requirement for a NeurIPS paper. For example, it is not clear if the proposed method can generate impact or useful for more general topics such as Identity Preserved Generative Models.\n\nQUESTIONS:\nPlease refer to details in the Weakness section above.\n\nLIMITATIONS:\nSome limitations have been discussed in the submission.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2024-07-13"}

comments: SUMMARY:
The paper presents an interesting framework to generate face-preserving QR code, which is useful in social entertainment applications. To enable this application, the paper first encode the Face ID information into the QR generation process and a refining process is applied to improve the integrity of facial features as well as the scannability of the QR code. Experiments results show the effective of the proposed algorithm. The paper is well presented and would be easy to reproduce.

STRENGTHS:
1. The proposed face ID preserved QR code generation is useful in industry applications. 
2. The presentation of the paper is clear and easy to follow. There are sufficient details to reproduce the paper.

WEAKNESSES:
1. The paper seems to be an engineering report. The novelty of the paper is limited and it seems more likely to be an application paper rather than a neurips submission. 

2. For the experimental evaluations, there are several points which should be well improved.
* The evaluation test set is relatively small. For example, for scanning robustness test, there are only 20 QR codes for the test, which may not be statistically effectively. 
* For the ID-preserving results, it seems the ID results have been compromised compared with the original images. 
* Also, the generated face results seems not be consistent with the QR codes, as shown in Figure 1 and Table 1. It seems to be more like a simple combination of a QR code with a face image.

3. The illustrative results are based on the celebrities or movie stars. How about the results to common face?

QUESTIONS:
The main concern is on the novelty of the paper. Please well justify the novelties of the paper.

LIMITATIONS:
The paper has discussed the potential limitations in the Conclusion section.
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 3
DATE: 2024-07-15