Fine-Grained Prompt Learning for Face Anti-Spoofing

Venue: acmmm_org_ACMMM_2024_Conference
Published: 2024-03-19 11:27:30

Abstract: There has been an increasing focus from researchers on Domain-Generalized (DG) Face Anti-Spoofing (FAS). However, existing methods aim to project a shared visual space through adversarial training, making it difficult to explore the space without losing semantic information. We investigate the inadequacies of DG that result from classifier overfitting to a significantly different domain distribution. To address this issue, we propose a novel Fine-Grained Prompt Learning (FGPL) based on Vision-Language Models (VLMs), such as CLIP, which can adaptively adjust weights for classifiers with text features to mitigate overfitting. Specifically, FGPL first motivates the prompts to learn content and domain semantic information by capturing Domain-Agnostic and Domain-Specific features. Furthermore, our prompts are designed to be category-generalized by diversifying the Domain-Specific prompts. Additionally, we design an Adaptive Convolutional Adapter (AC-adapter), which is implemented through an adaptive combination of Vanilla Convolution and Central Difference Convolution, to be inserted into the image encoder for quickly bridging the gap between general image recognition and FAS task. Extensive experiments demonstrate that the proposed FGPL is effective and outperforms state-of-the-art methods on several cross-domain datasets.

Main paper: 
 introduction: 
 Face Recognition systems (FRs) are widely used in daily life due to their unique advantages, such as intuitive, real-time, and noncontact. However, these systems also face significant security risks as criminals may use Presentation Attacks (PAs), such as printed photographs, replayed videos, and 3D masks [28] , to compromise the FRs to steal user information. Therefore, Face Anti-Spoofing (FAS) has become a crucial component in enhancing the security of FR systems against such malicious attacks with the growing importance in applications like face unlocking, face payments, and access control systems.The existing Presentation Attack Detection (PAD) algorithms can usually be summarized as appearance-based and temporal-based methods. The goal of appearance-based techniques is to differentiate between genuine and artificial faces by utilizing various appearance cues, such as deep features [63] , color textures [5, 6] , and picture distortion cues [62] . On the contrary, a variety of temporal cues, including facial gestures [48, 49] , rPPG [37, 38] , and optical flow, may be extracted using temporal-based techniques. Shown as Fig. 1 (a), While these techniques have shown promising results in tests conducted within the same dataset, their effectiveness significantly declines when the training and testing data are obtained from different datasets. The chief reason for this deterioration is that these techniques, tailored solely to the training data, fail to bridge the gap between source and target domains, resulting in inadequate generalization. Domain Generalization (DG) [58] is a commonly used technique in cross-domain contexts to address this issue by learning domain invariant representations. Recently, some DG-based algorithms [10, 16, 36] aim to improve the performance of models on unknown datasets. Great generalization performances have been achieved by works that make use of meta-learning [35, 75] , while others make use of adversarial learning [29, 47, 61] . These techniques are dedicated to acquiring a generalized feature space, assuming that the extracted unknown faces can be mapped into the generalization space, and improving the model's generalization by removing Domain-Specific (DS) information. However, attempting to generate a shared feature space for FAS through adversarial training may result in the following issues: (1) Significant disparities in the distribution of different data domains can easily lead to classifiers overfitting DS information. (2) Forcing the removal of DS information through adversarial training can lead to the loss of semantic information or the destruction of semantic structures in visual features.For the first issue, our analysis shows that the leading cause is that the static weights of the classifier cannot dynamically adapt to the changing DS information. Thus, assigning different weights to samples from various domains during training can significantly mitigate the problem. For the second issue, direct editing of visual features via adversarial training or decoupling learning causes loss of semantic information and structural damage. By using an adapter to modulate visual features towards generalization instead of directly editing them, we can avoid this problem. Inspired by Vision-Language Models (VLMs), such as CLIP [43] , which can perform zero-shot inference with a set of weight vectors by embedding the names or descriptions of the target dataset's classes as depicted in Fig. 1 (b), we can adaptively adjust weights for the classifier with text features. Furthermore, we design a lightweight adapter to transfer CLIP knowledge to FAS tasks, improving the model's generalization ability with minimal learnable parameters and avoiding the loss of feature information. Therefore, in this work, we treat DG-based FAS as a high-quality text feature learning procedure with an effective adapter.To avoid time-consuming and performance-unstable prompt engineering for text feature learning, CoOp [73] models a prompt's context words with learnable vectors while putting the [𝐶𝐿𝐴𝑆𝑆] (i.e., the names or descriptions of the target dataset's classes) token in the end or middle position. Furthermore, CoCoOp [72] alleviates overfitting in the base classes by learning a lightweight Meta-Net to generate an input-conditional token (vector) for each image. However, they generally learn a set of prompts as inputs to the text encoder to generate text features and cannot selectively suppress domain-related information. In this work, we propose a novel Fine-Grained Prompt Learning (FGPL), which first motivates prompts to learn semantic information of both content and domain by capturing Domain-Agnostic (DA) and Domain-Specific (DS) features. As is shown in Fig. 1 (c), the model's generalization ability is improved by reducing the correlation between DA and DS prompts. Finally, the joint prompts are further designed to be category-generalized by diversifying the DS prompts. Additionally, we design an Adaptive Convolutional Adapter (AC-adapter), which is implemented through an adaptive combination of Vanilla Convolution and Central Difference Convolution, to be inserted into the image encoder for quickly bridging the gap between general image recognition and FAS task. To sum up, the main contributions of this paper are summarized as follows:• We propose a new strategy called Fine-Grained Prompt Learning (FGPL), which enhances the model's generalization ability by reducing the correlation between Domain-Agnostic and Domain-Specific prompts.  
related work 2.1 face anti-spoofing: 
 During the initial phases, researchers have presented handcrafted feature-based presentation attack detection (PAD) methods [13, 31, 33, 40] . The majority of conventional algorithms are developed using manually crafted features that rely on abundant texture and temporal appearance cues. These cues include LBP [17, 39] , HOG [21] , SURF [31] , SIFT [41] , facial and head movements [3, 17] , such as smiling and nodding, eye-blinking [26, 32, 40] , gaze tracking [2, 4] , and remote physiological signals, for example rPPG [21, 33] . While the aforementioned approaches have yielded noteworthy outcomes, their applicability is restricted to test data originating from the same domain. When the training and testing data are sourced from distinct domains, the performance of these methods significantly deteriorates.In order to cope with the identification of unseen domains, Domain Generalization (DG) becomes a more effective approach to address the unseen domain for FAS. The initial proposition by Shao et al. [47] involves the acquisition of a generalized feature space that is shared among many source domains through the utilization of a multi-adversarial discriminative domain generalization framework. Wang et al. [56] proposed a method for distinguishing between generic Facial Attribute Synthesis features from subject discriminative features, as well as domain-dependent features. Jia et al. [29] proposed a method for obtaining a discriminative and generalized feature space. The study [16] utilizes adversarial domain adaptation as a method to acquire knowledge in a shared embedding space. The paper [36] employs several domain discriminators to acquire knowledge in a comprehensive feature space. The authors in [16, 67] employ disentangled representation learning to separate the features associated with liveness for categorization purposes.To acquire comprehensive knowledge, numerous meta-learningbased approaches [11, 42, 50] have been developed and improved for the purpose of regular optimization. Despite the achievements in pursuing a mutually shared feature space, inherent constraints and drawbacks still need to be acknowledged and addressed, such as the loss of semantic information, which can further weaken category discrimination. Unlike existing strategies [29, 47, 56] for seeking common spaces in the field of FAS, we propose a novel approach using fine-grained prompt learning and taking advantage of the Vision-Language Models (VLMs) model to transform DG-FAS problem into a high-quality text feature learning procedure with an effective adapter.Advances in computer vision [27, 70] have shown that the use of extensive pre-training with paired image-text data can be a viable alternative to achieving superior learning of visual representations without relying exclusively on natural language guidance. Since the Contrastive Language-Image Pretraining (CLIP) [43] was proposed, it has stimulated research into VLMs. Now, CLIP has been successfully applied in many fields, such as [22] , image generation [45] , visual question answering [1] and Domain Adaptation [19] . With further research [8] , the construction of VLMs using independently pre-trained Large-Language Models (LLMs) and visual backbone models allows VLMs to understand both text and images with only a few parameters in the training phase. Inspired by the simultaneous comprehension of image-text in VLMs, we have delved further into the potential use of CLIP to enhance the FAS task.The concept of prompt learning has roots in Natural Language Processing (NLP), a procedure that includes instructions at the beginning of the input sequence. Prompt learning aims to leverage these instructions to execute downstream tasks without necessitating fine-tuning. Many existing studies have used prompt learning [46, 51] . For instance, the CoOp [73] transforms pre-trained VLMs into data-efficient visual learners that outperform the original CLIP's hand-designed prompt learning templates. Subsequent research [14, 23, 60] has further advanced the development of CLIP, addressing a variety of aspects, particularly in terms of generalization capabilities. For example, by ensuring that prompts are relevant to the input image, the CoCoOp [72] demonstrates adaptation to new target domains, while the ProGrad model [76] achieves the same objective through gradient correction techniques. In recent times, there have been proposals such as the CLIP-adaptor [18] and the TIP-adaptor [68] that aim to enhance the transfer-ability of CLIP on downstream tasks by training a more efficient network. The DenseCLIP framework [44] utilizes the CLIP model to address dense prediction tasks through language-guided fine-tuning. Kg-CoOp [64] introduces a new knowledge-guided context optimization to enhance the generalization of learnable prompts to unseen classes. MaPLe [30] proposes multi-modal prompt learning for the Vision and Language branches. FLIP [52] is based on the Vision Transformer (ViT) visual model for fine-tuning, which aligns image representations with textual prompt Learning to achieve recognition of FAS tasks. However, these prompt learning described above are mostly applied in multi-category recognition tasks and require the category center to be learned using all the category names before the text encoding. In contrast, for the task of fine-grained FAS, simplistic and inflexible prompt learning may lead to overfitting of Domain-Specific information. To this end, we explored a Fine-Grained Promote Learning (FGPL) strategy, which dynamically adjusts the classifier weights through prompt learning, improving the generalization ability of FAS model while simultaneously allowing better application of VLMs to FAS tasks. 
method 3.1 overall framework: 
 The architecture of the FGPL is shown in Fig. 2 , which is built on CLIP. Unlike the standard CLIP [43] , we design a fine-grained promote learning instead of fixed templates. Additionally, we add a lightweight AC-adapter to the image branch. In the following section, we explain our proposed approach in detail: Section 3.2 first reviews the CLIP approach and previous methods such as the CoOp. Then, we present our FGPL approach in section 3.3.CLIP [43] comprises two encoders, one for image and one for text, which can jointly train an image encoder and a text encoder to predict the correct pairings of a set of <image, text> training examples. The contrastive learning objective aligns the image and text representation in the same feature space. CLIP encodes the input image 𝐼 ∈ R 𝐻 ×𝑊 ×3 and the corresponding text description 𝑡 into a shared embedding space. A specific description is explained below.Image encoding: The image encoder is responsible for converting an image into a feature vector, which can be implemented using either a ResNet [24] or a ViT [15] . Suppose the training set contains M samples, which denotes 𝑆 = {𝐼 𝑖 ,𝑇 𝑖 } 𝑀 -1 𝑖=0 , where 𝐼 𝑖 ∈ R 𝐻 ×𝑊 ×3 and 𝑇 𝑖 is the textual description corresponding to the image 𝐼 𝑖 . 𝜈 (•) is the visual encoder that encodes 𝐼 𝑖 into a visual feature: 𝑣 𝑖 = 𝜈 (𝐼 𝑖 ), 𝑣 𝑖 ∈ R 𝑑 , Where 𝑑 is the hidden dimension of the CLIP.Text encoding: The text encoder accepts a series of word tokens as its input and generates a vectorized representation, which is implemented using a transformer [54] . The CLIP text encoder provides feature representations for text descriptions by tokenizing words and projecting them to word embeddings. 𝜏 (•) is the textual encoder that encodes 𝑇 𝑖 into textual feature: 𝑡 𝑖 = 𝜏 (𝑇 𝑖 ), 𝑡 𝑖 ∈ R 𝑑 , Where 𝑑 is the hidden dimension of the CLIP.Zero-shot inference: Using the classification problem as an example, CLIP can achieve zero-shot classification by correctly generating the text input. CLIP utilizes a fixed template prompt to form the text input, for example, "a photo of [𝐶𝐿𝐴𝑆𝑆]". The [𝐶𝐿𝐴𝑆𝑆] in the fixed template can be replaced with the actual class name. For fixed template 𝑇 ′ 𝑖 = {A photo of [CLASS i ]}, here 𝑖 is the 𝐾 -𝑡ℎ categories, 𝑖 = {0, 1, 2 . . . , 𝐾 }. Fixed samples are fed into the encoder to get text features {𝑡 ′ 𝑖 |𝑡 ′ 𝑖 = T (𝑇 ′ 𝑖 )} 𝐾 𝑖=0 , therefore, the predicted probability of CLIP is as follows Eq. 1. where sin(•) denotes the similarity calculation and 𝜏 is a temperature parameter.EQUATIONHowever, CLIP fixed template prompts depend on manual settings and require word matching using a reserved validation set, which can be time-consuming. Therefore, CoOp [73] proposes prompt learning, which uses continuous 𝑡 𝑘 representations to more accurately describe semantic features, as in Eq. 2. The v 𝑚 1 is a vector of the same dimension as the word embedding, and 𝑚 1 ∈ {1, 2, . . . , 𝑀 1 }. EQUATIONDue to the variations in Domain-Specific (DS) factors like illumination, background, and camera type, former methods using fixed text templates for domain descriptions often fail to capture complex domain information. This leads to misclassifications driven by spurious correlations among features from different domains. The existing CoOp [73] model, although innovative in employing continuous text sequences through prompt learning, does not integrate domain-specific information and lacks granularity in its prompts. Initialize Vectors. To address these challenges, we introduce the concept of Fine-Grain Prompt Learning (FGPL).FGPL incorporates learnable Domain-Agnostic (DA) and Domain-Specific (DS) contexts into the prompts to further guide the FAS task. The DA context is universal across all domains, while the DS context is tailored to individual domains, embedding pertinent domain information into the prompts. We define the DA and DS vectors as two learnable variables, i.e., V 𝐷𝐴 and V 𝐷𝑆 , with a length of 𝐿, where 𝐿 = 16 in our case. At first, V 𝐷𝐴 is initialized in R 16×512 , while V 𝐷𝑆 is defined in R 𝐷 ×16×512 , containing domain information. Then, to further process and unify the prompts, we expand V 𝐷𝐴 ∈ R 𝐷 ×𝑐𝑙𝑠 ×16×512 to obtain the domain and class information. Also, V 𝐷𝑆 has been expanded to include classification information. The architecture of V 𝐷𝐴 and V 𝐷𝑆 prompts are structured as Eq. 3 and 4:EQUATION)Where 𝐿 is defined as 16 in our case. Construct Prompts. We construct these vectors as Mix-Prompt P 𝑚𝑖𝑥 , Class-Prompt P 𝑐𝑙𝑠 , and Domain-Prompts P 𝑑𝑜𝑚 .First, we construct a mix prompt 𝑝 𝑚𝑖𝑥 containing all information in the sequence { EQUATIONWhere 𝑃𝐹𝑋 refers to the prefix, and 𝑆𝐹𝑋 refers to the suffix of each prompt. 𝑉 𝐷𝐴 and 𝑉 𝐷𝑆 here represent the learnable DA and DS vectors. For the Mix-Prompt P 𝑚𝑖𝑥 , we randomly shuffle the [V 𝐷𝑆 ] related to each domain, which makes the model less concentrated on the domain-specific information. For Class-Prompt P 𝑐𝑙𝑠 and Domain-Prompt P 𝑑𝑜𝑚 , we insert learnable DA and DS vectors, respectively, to focus on different aspects of the image.Finally, the CLIP's text encoder further processes three prompts, with each tokenized prompt in the first stage, to obtain the text features of the images and calculate the cosine similarity of the image and text features. This refined approach to prompt learning improves the adaptability and effectiveness of face anti-spoofing models across varied domains. It sets a new standard for integrating domain-specific information into deep learning frameworks.The pivotal component of the FAS task is the precise identification and analysis of information that discriminates between live and fake representations. Initially, processing visual features through Vanilla Convolution can lead to semantic information loss and structural degradation. Furthermore, utilizing the image encoding capabilities of CLIP may hinder the learning of fine-grained details critical for FAS. This paper introduces a lightweight visual adapter, Adaptive Convolutional Adapter (AC-Adpater), which integrates into CLIP's visual encoding framework to address these issues. This adapter innovatively merges Vanilla Convolution with central difference convolution [66] . By incorporating the latter, the adapter enhances its capability to detect subtle, deceptive patterns by effectively combining luminance and gradient data, thus improving its effectiveness in distinguishing authentic from spoofed images. The adapter module is placed in parallel with the Multi-Head Self-Attention (MHSA) and Multi-Layer Perceptron (MLP) block, given an input 𝐼 ∈ R 𝑁 ×𝑑 :EQUATIONAs shown in the bottom of Fig. 2 , the visual adapter module consists of four convolution layers: SElayer, which implements the Squeeze-and-Excitation (SE) procedure to adjust features across channels dynamically, is employed to enhance the network's capability for feature representation; Central Difference Convolution layer to compute the difference output; 1×1 standard convolution layer and 1×1 linear convolution to perform dimension change operations.At first, the input 𝐼 ∈ R 𝑁 ×𝑑 , where N represents the length of the sequence and d represents the feature level (768), is reconstructed and downed to 𝑑𝑖𝑚 = 8 dimensions and further add non-linear features using GELU. Then, the last procedure's output 𝐼 𝑑𝑖𝑚 is fed to the SElayer, which contains two stages: Squeeze and Excitation. During the squeeze stage, 𝐼 𝑑𝑖𝑚 ∈ R 𝐵×𝑑𝑖𝑚×14×14 , where B and N represent the batch size and length of sequence respectively, are compressed through an adaptive average pooling operation, which outputs the global average for each channel, i.e., 𝐼 𝑑𝑖𝑚 𝑆𝐸 ∈ R 𝐵×𝑑𝑖𝑚 . This procedure aggregates spatial information, compressing each channel's features into a single numerical value. Then, the SElayer, through a fully connected layer, reduces the dimensions of channels, followed by the introduction of non-linearity through a ReLU activation function, which also aids in mitigating the issue of vanishing gradients. Subsequently, the original dimensions are restored through another fully connected layer. Finally, a Sigmoid activation function outputs weights ranging from 0 to 1. These weights are utilized to scale the features in each channel of the original feature map, completing the "Excitation" phase and dynamically adjusting the inter-channel feature responses.Furthermore, the weights 𝑊 𝑑𝑖𝑚 𝑆𝐸 ∈ R 𝐵×𝑑𝑖𝑚×14×14 of the SElayer outputs are passed into the Central Difference Convolution layer along with the production of a Vanilla Convolutional layer. The Central Difference Convolution layer first calculates the sum of the Vanilla Convolution weights across the last two dimensions to form a new kernel difference for computing a new convolution 𝐼 𝑑𝑖𝑚 𝑐𝑑 ∈ R 𝐵×𝑁 ×14×14 . Then, by multiplying the output weights from the SElayer, the central difference convolution of the input is obtained, which is less than that which samples the local receptive field region R and will simultaneously capture the central gradient along with it. After the processing with this layer, the difference in results from the Vanilla Convolutional layer and itself has been calculated to extract differentiated features, shown as Eq. 9:EQUATIONWhere 𝐼 ′𝑑𝑖𝑚 denotes the total output, 𝑐𝑜𝑛𝑣_𝑐𝑑 (𝐼 ) represents the output of the Central Difference Convolution layer, which is the combination of the feature 𝐼 𝑑𝑖𝑚 𝑐𝑑 and the weight 𝑊 𝑑𝑖𝑚 𝑆𝐸 . Besides, the class token is obtained, convolved, and combined with the 𝐼 ′𝑑𝑖𝑚 feature. Finally, the feature dimensions are restored to their original size of R 𝑁 ×𝐵×768 .In the FAS task, capturing semantic information at the intensity level and detailed information at the gradient level is essential for differentiating between live and spoofed faces. Therefore, employing such a hybrid approach that adaptively combines Vanilla and Central Difference Convolution is critical. The AC-adapter's forward propagation process includes dimensionality reduction via linear convolution, integration of SElayer parameters to facilitate both vanilla and center-difference convolution transformations and utilization of center-difference convolution subtracted from Vanilla Convolution to enhance the detection of forgery-specific features. Also, obtained from the adapters' structure, all trainable paths contain adapter modules after freezing the MHSA/MLP so that the whole fine-tuning process can benefit from the visual bias inherent to CNNs. Additionally, the Central Difference Convolution layer is implemented by transforming the receptive field from the original 3x3 neighborhood into sub-neighborhoods corresponding to horizontal-vertical or diagonal directions, making the visual adapter more focused on capturing Anti-Spoofing detail information.The image and text encoders are unfrozen during training, while fine-grained prompt learning and AC-adapter are added. After the encoders, three fine-grained text features and the image feature are then calculated for similarity and the predictions of the input image using Eq. 1. We utilize the standard CrossEntropy loss during training to obtain the fine-grained loss upon three kinds of features and then sum it as the total loss as shown in Eq. 10:EQUATIONSpecifically, the Class-Loss L 𝑐𝑙𝑠 utilizes the Class-Prompt P 𝑐𝑙𝑠 , the Domain-Loss L 𝑑𝑜𝑚 uses the Domain-Prompt P 𝑑𝑜𝑚 and the  
experiments 4.1 experimental setup: 
 Datasets and Protocols: We used two protocols to assess generalizability. For Protocol 1, we used benchmark datasets that are publicly available to the FAS academic community, MUS-MFSD (M) [62] , CASIA-FAD (C) [71] , Idiap Replay-Attack (I) [12] and OULU-NPU (O) [7] . The four datasets differ for various reasons, concerning material, lighting, background, and resolution differences. As a result, there is a significant bias in the domain between these datasets. Following the established testing rules, each dataset is treated as a domain, and the other three source domains are combined. The leave-one-out test protocol is applied to assess the cross-domain generalizability of the method. For example, OCI→M, which uses OULU-NPU, CASIA-FAD, and Idiap Replay attacks as training protocols, was tested on MSU-MFSD. For Protocol 2, we use the large-scale FAS datasets CASIASRF (S) [69] , CASIA CeFA (C) [34] , and WMCA (W) [20] , which are FAS proprietary datasets, as they encompass a broader range of topics, various types of attacks, and diverse sampling environments.Evaluation Metrics: In line with the evaluation principles, we used three metrics, HTER, AUC, and TPR, to assess the model's performance. (1) HTER is a measure of the false rejection and false acceptance error rates, and its value is taken as the average of the false rejection rate (FRR) and the false acceptance rate (FAR). ( 2) AUC measures the algorithm's overall performance, and its value represents the area under the ROC curve. This metric is used to evaluate the performance of the classifier. ( 3) TPR (True Positive Rate) measures the algorithm's accuracy in recognizing spoofed samples. It can select the appropriate threshold based on the specific application requirements.Implementation Details: For the FAS task, we use a pre-trained CLIP [43] model with ViT-B/16 [15] as the image encoder. We keep the parameters in the encoder unchanged, resize the image to 224×224 with a batch size of 50, and train all models for 500 epochs. We use the Adam optimizer and set the learning rate to 1e-6 for training. Additionally, we set the length of the DA token M1 and the DS token M2 to 16 during the initialization of prompt learning.In Protocol 1, we present the results of the state-of-the-art (SOTA) approach and our improved results using the CLIP model. The following conclusions can be drawn from Tab. 1. FAS domain generalization can be effectively improved by using multiple variants of Vision Transformer (ViT). From the HTER metrics, it can be seen that the mean HTER values of CLIP [43] (5.17%), CoOp [73] (4.51%), and CoCoOp (6.21%) are significantly higher than the MADDG [47] (23.09%), D 2 AM [10] (16.09%), IADG [74] (8.39%), SA-FAS [53] (7.82%). The latter searches the commonality space through adversarial training, so the classifier weights do not adapt to dynamically changing domain-specific information. However, former methods conveying natural language semantic information have much more satisfying results. Such methods are well-established [23, 43, 72] that introducing natural language semantics enriches the visual representation and improves its generalization, enabling models to understand and reason about visual content more comprehensively and accurately. The latest research FLIP [52] Compared to the other SOTA methods, our approach significantly outperforms all baselines in terms of the mean value of HTER and most of the DG tasks.In Protocol 2, The results of the different methods are presented in Tab. 2. Compared to ViT [15] and CLIP-V, which only involve image-side processing, and CLIP [43] , which only uses fixed templates, the HTER of our FGPL is significantly improved. Unlike the baseline CLIP, FGPL incorporates fine-grained prompt learning and lightweight AC-adapters. Among them, constructing three-stage fine-grained textual cues achieves the precise localization of DA and DS information in visual features, effectively mitigating the overfitting problem of classifiers to domain-relevant information. On the other hand, the AC-adapter captures detailed deception features using an adaptive combination of Vanilla Convolutional and Central Difference Convolution. Compared to CLIP, CLIP-V, the HTER of FGPL is reduced by 0.66% on average on the MSU-MFSD (M) [62] , Idiap Replay Attack (I) [12] and OULU-NPU (O) [7] datasets.To further validate the innovativeness of our approach, we first removed our FGPL's AC-adapter and replaced it with the original visual encoder in CoOp. Additionally, we replaced our Fine-Grain Prompt Learning with the original text prompts in CoOp. Through these experiments, we can simultaneously and collaboratively verify the effectiveness of our FGPL and AC-adapter. Shown as Tab. 3, the HTER score increased from 3.50% to 4.86%, and the TPR score decreased from 87.14% to 65.00% by removing the AC-adapter. Moreover, by stipping our FGPL, the HTER increased 3.93%. The results show that the two modules we designed complement each other and are indispensable. Effectiveness of FGPL: Shown in Tab. 3, we conducted the OCM→I experiment to confirm the impact of our FGPL. On the one hand, by removing the FGPL, which refines the prompt learning procedure to reduce the notice of DS information, the CoOp-AC adapter's HTER score increases 3.93% and the TPR drops 6.97%. Moreover, its HTER score even increases by 1.36% compared to the original CoOp model. On the other hand, by adding our FGPL into the standard CoOp model, the HTER improves by 1.21%. The results significantly demonstrate the importance of our FGPL. Even if we use an AC-adapter in isolation, allowing us to extract more semantic information from images; however, the absence of textual cues to supervise the training process can lead the model to focus on DS information overly, ultimately resulting in decreased performance. Effectiveness of AC-adapter: In the OCM→I experiments as Tab. 3, from one view, by removing the AC-adapter from our FGPL method, the HTER performance drops 1.36% and the TPR decreases 22.14%. From another view, by solely utilizing the ACadapter based on the CoOp model, the TPR score increases 10.71%. Interestingly, the HTER score performance worsened if we used the AC-adapter without guidance from FGPL. This is because more fine-grained image features are dug, and DS information is also part of them. Without our FGPL DG method, the model can overfit the DS information and reduce the model's generalization performance. This verifies that our AC-adapter indeed captures more specific features and also restates the importance of using both modules as a whole.Mis-classified images: In Fig. 3 , we present examples of misclassified images in Protocol 1. It is easy to notice that none of the fake samples were misclassified in OCI→M and OCM→I. This can be attributed to the fact that the attack types in the training dataset contain the attack types in the test dataset. However, in OMI→C, approximately 6% of the samples are misclassified as fake faces. This issue may be attributed to the high resolution of the samples trained in OULU. Still, the CASIA test dataset samples do not have enough resolution, have low light conditions, or are overly bright. In contrast, the probability of misclassifying a live face as a fake face is extremely low at 2.1% in ICM→O. After all, the four benchmark datasets, OULU-NPU, are the higher resolution database. When the training samples are of low resolution, this can easily lead to recognizing a higher-resolution fake face as a live face in the testing phase. Similarly, the same analytical conclusions apply to Protocol 2.Attention map: To further demonstrate the benefits of FGPL, we used [9] to generate visual attention maps for the FGPL model on the deception samples in Protocol 1 and Protocol 2, respectively. As observed in Fig. 4 , the dataset in Protocol 1 is primarily affected by printing and replay attacks. The baseline CLIP emphasizes untrustworthy spoofing cues like paper texture, edges, and ripples, resulting in the misclassification of samples. In contrast, our FGPL adjusts the model's focus on more subtle fake characteristics and can efficiently locate the fake patterns in each fake domain to make correct classification decisions. Likewise, the same analytical conclusions are applicable to Protocol 2. 
conclusion: 
 In this paper, we consider DG-Based FAS as high-quality textual feature learning and effective adaptor design that improves the model's generalization capability with minimal loss of learnable parameters and feature information. FGPL proposes an effective framework for fine-grained prompt tuning. A refined prompt learner is used to optimize prompts to adjust classifier weights dynamically. A light Adaptive Convolutional Adapter (AC-adapter) quickly bridges the gap between general image recognition and FAS tasks. We have shown that visual language modules learned using pre-trained visual language models (e.g., CLIP) have excellent generalization capabilities in FAS tasks compared to models that only use multiple variants of ViT. Future work could explore more diverse prompt learning in conjunction with FAS image features to further improve the effectiveness of text monitoring in on-the-fly adaptation.  
 
 Appendix:  


Decision: While one reviewer still have serious concerns, most other reviewers lean to acceptance. Strongly urge authors to carefully address reviewers comments, e.g., some important experimental comparison results with the related works STYLIP, CFPL-FAS, and FLIP, are still missing, important comparison results on protocols of limited some domains are still absent, and experimental analyses of TSNE distribution analyses are lacking.

Reviews: review: paper_summary: paper_summary: This paper studies Domain-Generalized (DG) Face Anti-Spoofing (FAS), and introduces Fine-Grained Prompt Learning (FGPL), leveraging Vision-Language Models (VLMs) like CLIP. FGPL adaptively adjusts classifier weights using text features from prompts to mitigate classifier overfitting to different domain distributions. This approach uses prompts to capture both Domain-Agnostic and Domain-Specific features, enhancing category generalization through diversified Domain-Specific prompts. Additionally, we introduce an Adaptive Convolutional Adapter (AC-adapter) to bridge the gap between general image recognition and FAS tasks. Extensive experiments demonstrate the effectiveness of FGPL, outperforming existing methods across various cross-domain datasets.
strength_weakness: strength_weakness: 1. The paper is well organized, the figures are readable and understandable.
2. The proposed method looks logical and technically sound.
3. The experimental results are good. Consistent improvements have been shown in different benchmarks.
limitations: limitations: 1. Concerns on the novelty: Previous work  [0] has also explored fine-grained prompt learning for domain generalization, and STYLIP  [0] also fuses the domain-agnostic features and domain-specific features. Directly applying a similar idea to DG FAS seems does not introduce sufficient novelties. Besides, CFPL-FAS [1] also explored DG FAS via textual prompt learning. Please the authors make extensive discussions on the differences from these works.

[0]. STYLIP: Multi-Scale Style-Conditioned Prompt Learning for CLIP-based Domain Generalization, WACV 2023;

[1]. CFPL-FAS: Class Free Prompt Learning for Generalizable Face Anti-spoofing, CVPR 2024;

2. Lack of comparison results. 1) Since STYLIP [0], CFPL-FAS [1] and FLIP [2] are the most related work, however, there are no comparison results with these methods [0-2] in different benchmarks, and it is questionable the presented method's superiority and advantages. Does the presented method outperform them in Protocol 1 and Protocol 2? 2) There are only comparison results on 3 source domains to the unseen domain, what are the DG results of limited source domains (two domains) or single source domain to the unseen domain?

[2]. FLIP: Cross-domain Face Anti-spoofing with Language Guidance, ICCV 2023;

3. Lack of citations: some important references [3-9] that also study cross-domain FAS are missing.  The reviewer suggests the authors add these missing references and briefly review them in the related work.

[3]. Domain Invariant Vision Transformer Learning for Face Anti-Spoofing, WACV 2023;

[4]. Adaptive Mixture of Experts Learning for Generalizable Face Anti-Spoofing, ACM MM 2022;

[5]. Learning Meta Pattern for Face Anti-Spoofing, IEEE TIFS 2022;

[6]. Domain Generalization for Face Anti-Spoofing via Negative Data Augmentation, IEEE TIFS 2023;

[7]. Selective Domain-Invariant Feature Alignment Network for Face Anti-Spoofing, IEEE TIFS 2023;

[8]. Test-Time Domain Generalization for Face Anti-Spoofing, CVPR 2024;

[9]. Suppress and Rebalance: Towards Generalized Multi-Modal Face Anti-Spoofing, CVPR 2024;

4.  Insufficient experimental analysis: There's a lack of comprehensive experimental analysis, including TSNE feature visualization, visualization of source and target domains, class visualization, and the distribution of individual adapters. These analyses are crucial for understanding the model's behavior and performance, identifying areas for improvement, and assessing how the model performs in different environments.

5. In addition to the results in the manuscript, More realistic FAS protocols in [10-11] including both 2D and 3D attacks should be compared. For example, MCO to CelebA settings could be compared.

[10]. Towards Unsupervised Domain Generalization for Face Anti-Spoofing, ICCV 2023;

[11]. Causal intervention for generalizable face anti-spoofing, ICME 2022;

6. Concerns on the reproducibility： Since CFPL-FAS [1] does not open source their code, it is questionable on the reproducibility of the proposed method.  The implementation details are too short to reimplement by the readers who want to follow this work. Would the authors release their code to the community?
paper_summary: paper_summary: The paper introduces a novel Fine-Grained Prompt Learning (FGPL) approach, leveraging Vision-Language Models (VLMs) like CLIP to address domain generalization challenges in Face Anti-Spoofing (FAS). By extracting both Domain-Agnostic (DA) and Domain-Specific (DS) features and employing an Adaptive Convolutional Adapter (AC-adapter), the method aims to improve the robustness and applicability of FAS systems across diverse datasets. Extensive experimental validation demonstrates the superiority of FGPL over state-of-the-art methods in cross-domain settings.
strength_weakness: strength_weakness: 1. The paper effectively combines VLMs with prompt learning to extract and utilize DA and DS features, a method that stands out for its originality and potential to significantly enhance FAS domain generalization.
2. The experiments are thorough, with tests conducted across multiple challenging datasets, convincingly demonstrating that FGPL outperforms existing techniques. This robust experimental setup not only validates the effectiveness of the proposed method but also establishes a strong benchmark for future research.
3. The introduction of the AC-adapter is particularly noteworthy. This component integrates seamlessly with VLMs to bridge the gap between general image recognition tasks and FAS, providing a practical solution that enhances model adaptability without extensive retraining or manual tuning.
limitations: limitations: 1. The paper brings textuality-based prompting information via CLIP into the process of FAS detection. It is hoped that subsequent experiments will be conducted to verify whether this leads to more information about the domain that cannot be generalised.
2. While the experimental results are strong, the paper could benefit from a deeper discussion on deployment challenges and limitations in real-world scenarios, including how the model performs under more realistic datasets.
paper_summary: paper_summary: This paper proposes a Fine-Grained Prompt Learning (FGPL) based on Vision-Language Models, such as CLIP, to solve the Domain Generalization in Face Anti-Spoofing area. The authors proposed a Fine-Grained Prompts which has two learnable vectors to capture Domain-Agnostic and Domain-Specific features. Furthermore, two learnable vectors are combined with the image feature separately. Concerning the image branch, the authors add an AC-adapter to dig the image features more deeply. The authors also conducted several DG experiments to verify the results.
strength_weakness: strength_weakness: 1.	This paper considers the DG into a combined task of vision-language tasks, using both features to dynamically adjust the classifier’s weight. Although this paper addresses the DG problem in an ordinary motivation, but the utilization of Vision-Language multi-modal approaches is quite unique and novel. The learnable text vectors can learn the more detailed semantic information and guide the classifier more convincingly.
2.	The experiment of the proposed method is promising through the table showed in the paper. Two kinds of wildly used DG datasets have been adopted to verify the efficiency of the method.
3.	The loss function the authors had designed take all of three loss functions concerning the Domain-Agnostic and Domain-Specific features and the overall feature into the model’s training objectives, which concentrate on both global and local discriminative features.
4.	The AC-adapter utilizes the central difference convolution blocks that can help encoder capture semantic information at the intensity level and detailed information at the gradient level, which is essential for differentiating between live and spoofed faces.
limitations: limitations: 1.	The written style and logic of the paper’s content is a little bit fuzzy that may lead readers to misunderstand what the authors want to tell.
2.	In the method part, although authors tried to use several formulas and symbols to describe their methodology more specifically, they should more concentrate on the meanings and the basic mechanisms which each vector can tell, not only what the vector is.
3.	The experiments of the proposed methods though promising compared to the baselines, it can also be compared to other VLMs-based methods, such as FLIP, which can have more convincing results.
paper_summary: paper_summary: This paper focuses on studying the domain-generalized Face Anti-Spoofing (FAS) task and proposes a novel Fine-Grained Prompt Learning (FGPL) method. Specifically, FGPL first motivates the prompt to learn content and domain semantic information by capturing both domain-agnostic and domain-specific features. Furthermore, the prompts are designed to be category-generalized by diversifying the domain-specific prompts. An Adaptive Convolutional Adapter (AC-adapter) is inserted into the image encoder to bridge the gap between general image recognition and the FAS task.
strength_weakness: strength_weakness: 1. This paper presents a novel idea based on recent advanced Vision-Language Models (VLMs) to solve the domain-generalized FAS task. Based on CLIP, this paper mitigates the classifier's overfitting to the domain correlation information with the help of fine-grained text features, which is a solution that can directly hit the pain point. The FGPL proposed in this paper is a technique worthy of reference and further in-depth study by researchers.
2. Distinguishing from previous clip-based studies, the FGPL contains innovative modules such as fine-grained prompt learning and an Adaptive Convolutional Adapter. Among the former, the three-stage fine construction prompt accurately localizes domain-agnostic and domain-specific information in the visual features. The latter utilizes an adaptive combination of two convolutions to successfully transfer CLIP knowledge to the FAS task, enhancing the model's generalization with minimal learnable parameters and preventing the loss of crucial spoofing features. Therefore, this paper presents an innovative and valuable technique to solve the FAS domain generalization challenge by incorporating improved CLIP.
3. The paper is well-written and easy to understand. The insight about domain-generalized problems is interesting and well described, which helps to understand the proposed techniques.
4. Experimental results show the effectiveness of the proposed method.
limitations: limitations: 1. The font size of the headings in the visible sections of the experimental part is inconsistent, and the authors should pay closer attention to minor layout details.
2. The literature could include more articles on Domain-Generalized (DG) Face Anti-Spoofing (FAS) research for 2023 and 2024.
3. The implementation code of the paper's method is not open-source.