# A Realistic Hand Image Composition Method for Palmprint ROI Embedding Attack
## 1 Introduction  
Palmprint recognition (PPR) has recently been widely studied due to its robustness and accuracy. A typical PPR is composed of two stages. Stage one is the preprocessing stage, in which the palmprint region of interest (ROI) is located and extracted; stage two is the recognition stage, which entails feature extraction, matching, and the final decision. ROI localization is indispensable for PPR, which determines which part of the palmprint will be used for recognition, and it usually requires an input hand satisfying certain conditions. Given the vital role of ROI in PPR systems, ROI attacks aimed at spoofing and evading PPR systems [1; 2; 3] have emerged. While these attacks primarily target the palmprint ROI, their practicality may be limited, as PPR systems typically utilize hand images as input rather than the ROI. Consequently, synthesized hand images are required for impersonation attacks. Generally, two distinct approaches are employed to achieve hand image synthesis: generating a hand image based on the ROI and embedding the ROI into an existing hand image. The merit of the embedding method over the generating method stems from its reduced dependency on training data for a generator to yield realistic outcomes, rendering it a more practical choice. Specifically, the embedding approach employs a hand image as a carrier and integrates the ROI into it, a procedure dubbed ROI embedding. After acquiring composited images through ROI embedding, these images can be employed to launch impersonation attacks. In this paper, we aim to formulate an ROI embedding method tailored to substantiate the impersonation attack through composited hands, namely a hand attack instead of synthesized ROI. As illustrated in Figure 1, the components delineated by the black dashed line represent the palmprint recognition flow, constituting a PPR system that exclusively accepts the entire hand as a valid input. The ROI attack flow attempts to compromise the system by employing an ROI image deemed less practical. Conversely, the hand attack flow poses a more substantial threat, enabling attacks through legitimate inputs. The hand attack flow consists of a series of steps. Initially, the attacking ROI is acquired with the goal of compromising the target ROI. Subsequently, an arbitrary full-hand image is selected as the carrier hand. The third step involves utilizing an ROI embedding method to seamlessly integrate the attacking ROI into the chosen carrier hand. Ultimately, this process creates a composite hand, enabling the execution of the intended attack. The ROI embedding is intuitive yet non-trivial, posing two principal challenges. The first challenge involves embedding localization, necessitating the precise placement of the ROI on the carrier hand at an appropriate scale. However, a singular ROI position is not universally applicable to a hand, given the diverse preprocessing methods in PPR systems. The second challenge pertains to appearance consistency, wherein the attacking ROI's hue, luminance, and skin color must seamlessly integrate with the carrier hand. Moreover, the region around the embedding position in the carrier hand must exhibit continuous texture and natural transitions. Appearance consistency is an intuitive metric for ROI embedding, ensuring that the composited hand image attains desirable visual quality and fidelity, as illustrated in Figure 2. In this study, we focus on the second challenge, assuming the availability of a decent preprocessing method employed by the target PPS system. Figure 1: Hand attack, ROI attack, and palmprint recognition flow. This paper introduces an end-to-end, training-free approach for ROI embedding to achieve a realistic composition of hand images. Our methodology comprises two key components: ROI harmonization and palm blending. In the ROI harmonization phase, we employ a modified style transfer method to closely transform the attacking ROI's appearance to resemble that of the carrier hand.