PANTHER: Generative Pretraining Beyond Language for Sequential User Behavior Modeling
Authors: Guilin Li
Yun Zhang
Xiuyuan Chen
Chengqi Li
Bo Wang
Linghe Kong
Wenjia Wang
Weiran Huang
Matthias Hwai Yong Tan
Venue: NeurIPS 2025 poster ()
Keywords: Sequential User Modeling
Fraud Detection
Generative Pretraining
Transformer Models
Representation Learning
Primary area: deep_learning
TL;DR: PANTHER is a hybrid framework that combines self-supervised generative pretraining with lightweight discriminative modeling to enable real-time fraud detection in large-scale payment platforms, delivering a 38% improvement in online performance.

Abstract: Large language models (LLMs) have shown that generative pretraining can distill vast world knowledge into compact token representations. While LLMs encapsulate extensive world knowledge, they remain limited in modeling the behavioral knowledge contained within user interaction histories. User behavior forms a distinct modality, where each action—defined by multi-dimensional attributes such as time, context, and transaction type—constitutes a behavioral token. Modeling these high-cardinality, sparse, and irregular sequences is challenging, and discriminative models often falter under limited supervision. To bridge this gap, we extend generative pretraining to user behavior, learning transferable representations from unlabeled behavioral data analogous to how LLMs learn from text. We present PANTHER, a hybrid generative–discriminative framework that unifies user behavior pretraining and downstream adaptation, enabling large-scale sequential user representation learning and real-time inference. PANTHER introduces: (1) Structured Tokenization to compress multi-dimensional transaction attributes into an interpretable vocabulary; (2) Sequence Pattern Recognition Module (SPRM) for modeling periodic transaction motifs; (3) a Unified User-Profile Embedding that fuses static demographics with dynamic transaction histories, enabling both personalized predictions and population-level knowledge transfer; and (4) Real-time scalability enabled by offline caching of pre-trained embeddings for millisecond-level inference.Fully deployed and operational online at WeChat Pay, PANTHER delivers a 25.6\% boost in next-transaction prediction HitRate@1 and a 38.6\% relative improvement in fraud detection recall over baselines. Cross-domain evaluations on public benchmarks (CCT, MBD, MovieLens-1M, Yelp) show strong generalization, achieving up to 21\% HitRate@1 gains over transformer baselines, establishing PANTHER as a scalable, high-performance framework for industrial user sequential behavior modeling.

PANTHER: Generative Pretraining Beyond Language for Sequential User Behavior Modeling
Guilin Li
Yun Zh
Xiuyuan Chen 1.+
Chengqi Li
Bo Wang"
Linghe Kong'
Wenjia Wang²
Weiran Huang 1.3.1
Matthias Hwai Yong Tan
Shanghai Jiao Tong University WeChat Pay, Tencent
"Shanghai Innovation Institute *City University Hong Kong
'Hong Kong University of Science and Technology (Guangzhou)
Abstract
Large language models (LLMss) have shown that generative pretraining can distill vast world knowledge into compact token representations. While L.LMs encapsu- late extensive world knowledge, they remain limited in modeling the behavioral knowledge contained within user interaction histories. User behavior forms a distinct modality, where each action-defined by multi-dimensional attributes such as time, context, and transaction consticoss a behavioral token. Modeling these high-cardinality, sparse, and irregular sequences is challenging, and discrimi- native models often falter under limited supervision. To bridge this gap, we extend generative pretraining to user behavior, learning transferable representations from unlabeled behavioral data analogous to how LLMs learn from text. We present diseriminative PANTHER, hybrid generatie-diiermminaave framework that unifies be- havior pretraining and downstream adaptation, enabling large-scale sequential representation learning and real-time inference. PANTHER introduces: (1) Structured Tokenization to compress multi-dimensional transaction attributes into interpretable vocabulary; (2) Sequence Pattern Recognition Module (SPRM) for modeling periodic transaction motifs; (3) a Unified User-Profile Embedding that fuses static demographics with dynamic transaction histories, enabling both per- sonalized predictions and population-leye level knowledge transfer; and (4) Real-time scalability enabled by offline caching of pre-trained embeddings for millisccond- level inference. Fully deployed and operational online at WeChat Pay, PANTHER delivers a 25.6% boost in next-transaction prediction HitRate @ and 38.6% relative improvement in fraud detection recall over baselines. Cross-doman eval- uations on public benchmarks (CCT, MBD, MovieLens-IMM Yelp) show strong generalization, achieving up 21 % HitRate I gains over transformer baselines, establishing PANTHER as a scalable, high-performance framework for industrial user sequential behavior modeling.
Introduction
Online payment platforms, such as WeChat Pay, Alipay, and PayPal, process billions of transactions daily, supported by critical applications like fraud detection, credit risk assessment, and personalized marketing [1]. Modeling payment behavior at this scale is challenging: data volumes are extreme; categorical features (e.g.. merchant category. payment channel) are high-cardinality: and real same risk decisions must be delivered under 100 ms.
"qqual contribution. Contact: (guilinli @étencent.com. Corresponniig author. Contact: weiran. huang @ outlook.com.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Daily Commate
Scenic Arca Vasi
Daily Commote
Mall Shopping
Sahway
Cantazn
Taxi
Cinama
Pupcsen
Purking
Subway
Cantian
Tlaci
Chocks
Cantazn
Cuffez
Necmmóliee
Hittorke) Paymeet
8:00
12:00
18.01
19:00
1ou15
21:00
8:09
12,00
Iavte
10:00
12:00
14:00
16:00
Bebavter Sequence
Figure
Illustration of the periodic pattern of user behaviors
Recent advances in self-supervised pretraining have transformed representation learning in language. vision, and recommendation [2-6]. Large language models (LLMs) demonstrate that generative pretraining on unlabeled text can compress extensive world knowledge into token representations. Payment platforms, however, require models that capture behavioral knowledge the individualized regularities, intents, and deviations embedded in users' interaction histories. In this modality, each action is structured event rather than word; we therefore view a transaction behavioral token defined by multi-dimensional attributes (time, context, device, counterparty, amount). Extracting user-relevant information from these high-cardinality. sparse, and irregular sequences is essential for understanding users and for operational decision- decision-making at scale.
Mainstream industrial approaches, adapted from recommendation systems (e.g., DeepFM [7], DCN [8], AutoFIS [9]. DIEN [10]), rely primarily on supervised discriminative models for transaction- classification. At billion-user scale they face four persistent limitations: (1) label scarcity positive examples are too few to span the combinatorial feature space; (2) overfitting to high-dimensional categories-models memorize spurious co-occurrences rather than meaningful risk patterns; (3) latency-driven truncation long and (4) static embeddings- biased toward frequent labels and brittle under cold-start and long-tail distributions [1]].
the ability to leverage long-range behavior;
Although sequential recommenders increasingly adopt generative objectives. they typically use generation as an end task (next-item prediction) (for example HSTU [6]) rather than as a pretraining mechanism to compress user knowledge into transferable representations. In contrast, we adopt pretrain-vdday perspective: learn generalizable user embeddings from unlabeled behavioral logs offline, then online adapt them with lightweight discriminative heads to satisfy production latency constraints. This hybrid generative- santtiv--iieriminatii design targets user understanding rather than only item generation, and supports multiple downstream decisions (fraud detection, transaction prediction, recommendation).
Orthogonal to this objective/framework difference, user behavior sequence data inherently differs from other sequential modalities. Unlike natural language, which is governed by grammatical structures, user behavior sequences exhibit rich recurring patterns daaterns-diily daily routines, weekly cycles, and seasonal reflectnnn habitual user behaviors (illustrated in Figure 1). Standard sequential architectures, including Transformers, process events individually through self-attention mechanisms. Although powerful, these methods inadequately capture local periodicities and global relational patterns inherent to payment data, often diluting signals from lengthy sequences and missing subtle yet crucial anomalies indicative of fraud.
We propose PANTHER (Pattern AttentioN Transformer with Hybrid User ProfilER), a hybrid mnnerative-disciminee
framework that unifies user behavior pretraining with downstrean adapta- tion for real time inference. Offtine, a PANTHER transformer is pretrained on billions of transactions to predict subsequent events, producing compact user-profile embeddings and event likelihoods that encode long-term intent and temporal dynamics. Online, a lightweight classifier fuses these cached representations with current transaction context to compute risk within milliseconds. This design leverages generative pretraining for representational power while keeping inference efficient for tightthroughput systems.
To make pretraining effective on user behavior sequences and inference feasible online, PANTHER introduces three modeling components and one systems mechanism:
1. Token Space Compression. A structured tokenization scheme integrates contextual and counter- party attributes into unified tokens and applies frequency-awan compression reduce dimension- ality. filtering noise and stabilizing generative learning over heterogeneous inputs.
Pattern-Aware Convolutional Cross-Attention. A Sequence Pattern Recognition Module (SPRM) blends multi-scale (depthwise) convolutions with cross-attention to capture local pe- riodicities (e.g., daily/weekly cycles) together with broader contextual dependencies-ppreet preservving cyclical signals while maintaining global relations.

---

3. User Profile Embedding with Contrastive Personalization. A dedicated user-profile token provides persistent access user context across the sequence; a contrastive objective arranges users with similar payment behaviors nearby in latent space, improving personalization under sparsity and cold-start.
Real-Time Hybrid Inference. Pretrained user/profile embeddings are cached offline and fused online with context, recent patterns, and deviation features to produce millisecond-level posterior meeting production latency constraints.
We empirically validate PANTHER real-world WeChat Pay data, demonstrating strong generaliza- tion across fraud detection, transaction prediction, personalized user modeling, and recommendation. PANTHER yields 25.6% improvement over Transformer baselines internal WeChat Pay bench- marks and a 21 % HR @1 gain on MovieLens- 1M; on Yelp. it improves NDCG@5 by 29.6% over DCN. A production
WANTHER-based fraud system at WeChat Pay improves Top-0. 1% recall by 38.6% in online A/B tests, enhancing security across billions of daily transactions.
In summary, PANTHER provides a scalable, efficient approach to modeling complex sequential user behaviors by extending generative pretraining beyond language to the behavioral modality, compressing user knowledge into transferable representations, and bridging pretraining with real-time inference for industrial sequential decision decision-making.
2 Related Work
Sequential Deep Learning and Generative Recommendation. Deep learning methods for se- quential modeling have significantly advanced recommendation and personalization, evolving from early RNN- and CNN-based models (e.g., GRU4Rec [12], Caser [13]) to recent transformer-he based approaches (e.g.. SASRec [14], BERT4Rec [15]). Generative sequential models, such as HSTU [6],
TIGER [16], DiffuRecSys [17], and HLLM [18] have further advanced the field by modeling com- plex temporal dependencies and uncertainties. Despite these advancements, several aspects remain under-explored: explicit modeling of periodic behaviors (e-g., daily or weekly patterns), dedicated long-term personalized user embeddings, and computational strategies for low-latency inference. PANTHER uniquely addresses these challenges by incorporating convolutional cross-attentie for periodic behavior modeling, contrastive user personalization embeddings, and cached representations enabling efficient real-time inference.
Fraud Detection in Financial Systems. Fraud detection in financial systems involves severe class imbalance, label scarcity, and real-time inference constraints. Early supervised approaches. including logistic regression, decision trees, and gradient boosting, perform effectively on structured data but face challenges with sparse, noisy, high-dimensional transaction data and extreme class imbalance [19]. Graph-based methods like R-GCNs [20] and heterogeneous GNNs [21] effectively capture relational patterns among entities, yet scalability and real-time latency billion-user scenarios remain open research areas. PANTHER complements these methods by leveraging transformer-bast based generative pretraining, efficiently modeling temporal user behaviors and maintaining millisecond- level inference.
3 PANTHER
3.1 Model Overview and Problem Formulation
PANTHER employs a two-stage architecture for payment fraud detection, combining offline gen- erative pretraining with real-time inference. Let 2/ denote our user base where each user E U generates a sequence of payment events X,, [m1, .2...., =z,] with 2, represeenting compressed transaction tokens (see $3.2). Our system aims to estimate the fraud probability:
Pryy= = Foow , Cnew, Xx),
(1)
for each new transaction Vnew with contextual features Crews given the user's historical sequence X,,-
Stage 1: Pretraining for Next Transaction Prediction. We first learn user behavior patterns
through next payment behavior prediction. Each user behavior sequence augmented with learnable profile token proofll encoding static attributes (see $3.4). Our cransformerbbaee model with SPRM
3

---

Backward propegation--
L= (1 Aym XLeu
Next Rehavior Prediction S
Generation Grm
Contrasivee Leaming Lose Lea
Updatted User Profile Emb e.
SPRM
Y Multi-Hend
Attention
Self-Arteution
4
Dissimilar
Hidden states R
User Profiiee
User Proffle
Dilated
Leamable prototypes
*N layers
Input Embeikding
User Profile
Emhedding
Behavise Embedding User Profile Emneedling
Structured Tokenization
(beverape.
Jelric
jstrving
User ID, Gender, Age, City, Payment
Bood,
(sport,
(grocory. 50 1001
<100
statistics....
Payment Rehavice Sequence
User Profile Pyrctl
Figure 2: Key components PANTHER: Structured Tokenization, SPRM and user profile embedking
modules $3.3) optimizes:
ggen Buwall
log Pro Fett Eprofile)
where the loss is the negative log-likelihood of the ground-truth next behaviors over the sequence. This produces two key pretrained outputs: (1) user profile embeddings e, E IRd, and (2) behavior predictors X,, that predict next behavior Pro(#1+1|) Xx,) through linear and softmax layer.
Stage 2: Hybrid Inference for Real-Time Fraud Detection.
For live transactions, we compute
risk scores through feature fusion:
Pr(y 1|-) = 9o (+((Coow) Sene (X:!! (X.I.-1001
A Fnew
context
recent patterns
long-tere term profile
behavior deviation
where the function represents a general downstream discriminative model, which can take various flexible forms depending specifie tasks cases, rb(-) embeds transaction context features, fene encodes the last 100 transactions via TexICNN [22], and measures the distance between predicted and observed behaviors.
This design strikes balance between accurate long-term behavior modeling and responsive real-time decision-making making it particularly well-suited for high-throughput fraud detection systems.
3.2 Structured Tokenization for Payment Behaviors
Unlike natural language, payment behaviors lack predefined semantic units, as each transaction is defined by a combination of contextual features (e.g., payment channel, discretized amount) and counterparty attributes (e.g., merchant category, risk level). To efficiently capture this structured, multi-dimmenional information, we introduce structured tokenization framework that integrates these attributes into a unified representation.
Each transaction token formed as the Cartesian product of contextual and counterparty features:
T (c,, aj, my, r1) E x A R.
where, for instance, might represent payment channel such as CreditCard or RedPocket, and aj could capture the transaction amount, discretized into ranges like $10-50 and $50-100. On the counterparty side, my can capture the merchant category, while r indicate the associated risk level, reflecting the merchant's historical reliability (e.g.. HighRisk, LowRisk). This tokenization approach captures the key transactional semantics by transforming raw payment behaviors into compact, domain-spectnn specific tokens (e.g., (e.g., CreditCard 1 1000110_Fueel111 Lowe_sLowiss)) effectively embedding transactions within structured, context-rich feature space. This design allows the

---

pretraining model to learn meaningful representations of transaction behavior, preserving critical financial signals while reducing sparsity.
However, the raw combinatorial space |V] = |C| x [A] x |M] x |R] R 2M, which prohibitively large, leading severe sparsity and overfitting risks. To address this, adopt frequency-based compression, leveraging real-world transaction distributions to retain only the top K = 60, 000 most frequent tokens, covering over 96% of historical transactions. Less frequent, long-tail combinations are mapped to a unified [UNK] token, effectively reducing the vocabulary size by 97% (from 2M to 60k), while preserving high-impact and interpretable patterns. It maintains critical transaction semantics while significantly reducing the model's 's computational footprint.
3.3 Sequence Pattern Recognition Module (SPRM)
Payment sequences, unlike natural language, lack formal grammar, yet exhibit structured recurring patterns, such as sequential routines and periodic behaviors (as illustrated in Figure 1). Accurately modeling these patterns is critical for applications like next-payment prediction and fraud detection, as they encapsulate context-aware user habits. However, standard self-attention in Transformers processes each event independently, incurring quadratic complexity and lacking inductive biases for local and periodic structures. This inefficiency can dilute signals in long sequences and obscure subtle deviations from the routine. To address these limitations, we introduce the Sequence Pattern Recogni- tion Module (SPRM), which incorporates two lightweight inductive biases to explicitly capture local and periodic patterns payment sequences. Operating in parallel with the multi- head self-attention, the SPRM enhances the final representation by adding its output to the Transformer's 's output, forming a composite result that leverages both global context and structured pattern recognition.
(i) Local Pattern Aggregation. To capture the multi-scale nature transactional routines, we apply depthwise dilated convolutions to the token embeddings H R using range of kernel sizes w and dilation rates r:
(*()) Convall=r, w=w (H), Hp Concaty (H, (*)
Here, kernels with smaller dilation rates (e.g. w 3, 1) capture immediate temporal clusters, while larger dilated kernels (c.g., 3, 2) capture periodic or recurring patterns, even in the presence of occasional noise or sporadic behaviors. This convolutional aggregation efficiently captures multiscale transactional patterns in linear time, providing a compact representation that encodes both short-term and long-term transactional routines while remaining robust to random fluctuations within those patterns.
(ii) Proiotype-Driven Pattern Matching. To further enrich these multiscale embeddings, we introduce m. learnable prototypes P E IRd each representing canonical spending motif (e.g.. weekend leisure, weekday commute). These prototypes act as structured anchors in the embedding space, providing scaffold for the model to map observed behaviors to known patterns.
QK
Q = HH K == PWW,, V VWP
X, = Softmax
V.
vdx.
This cross-attention mechanism compresses each local segment onto the nearest prototypes, effectively aligning real-world sequences to interpretable, high-level motifs, while preserving local context. Unlike vanilla self-attention, which treats every token context-free, this approach enforces structured alignment, encouraging the model to form sparse, interpretable codes that highlight subtle departures from routine behavior.
Complexity Analysis. For a sequence of length T, standard self-attention incurs a quadratic O(T²) complexity. In contrast, SPRM introduces only linear cost for dilated convolutions, O(T), and a modest (Tm) for prototype cross-attention. where m & T (typically m = 64 practice). This results in overall complexity of approximately O(T), making it feasible for long payment sequences without sacrificing long-range dependency capture.
3.4 User Profile Embedding for Personalized Payment Predictions
In PANTHER, we propose a novel user profile embedding that enables personalized transaction predictions by learning shared behavioral patterns among demographically similar users. This embedding combines static user attributes with dynamic transaction histories to form compact latent
5

---

Table Next-transaction prediction results on WeChat Pay. Relative improvements reflect PANTHER'S gains over the Transformer baseline.
Method
HR@1
HR@5
HR@1 10
Transforner
0.1952
0.4121
0.5308
SASRec
0.2041
0,4280
0.5347
0.4271
0.5320
PANTHER (SPRM only)
0.2243(+14.99)
0.4493(+9.0326)
0.5421(+2.133)
Profile as First Tnken
(.2.01(+17.955)
(14654((+12455%)
0,5468(+3.015 %%
Profile Positional Encoding
0.2351(+200449)
0.4774(+11.855%
0.5568(+4.900%)
Profile CL
0.2452 0.0005 (+25.6
0.4837 0,0009 (+17.37%)
0.5647 0.0012 (+6.39%)
representation, which simultaneously serves two purposes: (1) as personalized positional encoding that contextualizes user-specific transaction sequences, and (2) as learnable similarity anchor that adaptively refines historical behavior patterns through contrastive learning.
Our contrastive objective follows an information theooretic formulation:
exp(- e//l2/T)
LLL= =
Σ log
(5)
E exp(-1les - =x12//T)
(K)))Pos
where Pos denotes positive pairs of users sharing demographic attributes (age= same geographic region, etc.), Ul represents the user population, and controls the similarity concentration temper- ature. This objective maximizes mutual information between demographically similar users while maintaining separation from dissimilar counterparts through the denominator's hard negative mining over all non-positive pairs.
The complete optimization objective combines both components:
(1 Agon +
XCLL
Individual fidelity
Population structure
This dual-objective formulation yields three key advantages: (1) geometrically meaningful embed- dings where user similarity correlates with both demographic alignment and behavioral consistency, (2) improved sample efficiency through knowledge transfer between similar users, and (3) inherent regularization that prevents overfitting to individual transaction outliers.
4
Experiments
4.1 Real- World Deployment & Performance Validation
We validate the core contributions of PANTHER through its large-scale deployment at WeChat Pay, focusing on two key aspects: pretraining efficacy (next-transaction prediction) and downstream fraud detection capabilities. This deployment addresses the challenges identified in Section 3.1, showcasing how PANTHER can enhance fraud detection and personalized user services in real-world setting.
4.1.1 Next TxttTTransacton Prediction Benchmark
Task & Dataset:
models are pretrained on 5.3B anonymized transactions (38M users over 6 months) to model user-specific behavior. The raw transactions data, consisting six attributes (amount, merchant category, etc.), are tokenized by 60k interpretable tokens with the structured tokenization scheme.
Experimental Setup: In the transactnnnn prediction task, we evaluate PANTHER's 's ability to predict the next transaction based on user's 's historical data. The model leverages the SPRM and unified user-profile embeddings for this purpose. We employ two widely-adopted evaluation metries: HR @ K (Hit Ratio at K) and NDCG@ @ K (Normalized Discounted Cumulative Gain at K). Specifically. HR K measures
fraction of test instances which the ground-truth item appears among the top-K predicted items. NDCG@t K assesses the ranking quality by assigning higher weights to relevant items placed at top positions, normalized by the ideal discounted gain. The code is available at https:/ / //g/ thub. com com/WeChatPay- Pretraining/ ANTHER.

---

Cruss Attention Scores Heutmap
Pattorn Bosis
0.136
0.169
0,152
0.056
0.036
0.067
0.001
0,001
0.004
0.127
0.115
0,017
0.255
0.302
0.233
0,010
0,010
0.007
0.2
0.004
0.017
0.008
0.111
0,007
0.003
0.323
0.319
0.314
8.1
Spending
Shopping
Spending
1 TTarc/aabrrm
3h48: Tomepe
Ga Stalen Dakke Vadte
Boal bakatile
2 Rabed Goads C.
10omm & Trib CCees
13am FGaen
Figure 3: Demonstration of the cross attention score between recurring three-gram user payment hehaviors and the learnable pattern prototypes.
We benchmark PANTHER against several strong baseline models, including Transformer [23], SASRec [14], and HSTU [6].
Key Findings: As shown Table 1, PANTHER outperforms baseline models significantly. The SPRM module alone improves HR@1 by 14.9%, highlighting the value of sequential behavior patterns.
Incorporating user profiles via learnable positional encoding boosts HR@ @ by an additional 5.5%. Adding contrastive learning objectives results in total HR @ improvement of 25.6%, demonstrating the effectiveness of context-aware embeddings and knowledge transfer.
Visualization of Learned Patterns: Figure 3 shows how PANTHER maps consecutive user behaviors to learned behavior prototypes (P1, P2, P3). Each column represents a sequence of three consecutive behaviors identified as recurring patterns in the user's history. The heatmap values indicate the strength of alignment between behaviors and prototypes, with higher values signifying stronger matches. This visualization highlights how the model captures and recognizes repeating transaction motifs over time.
4.1.2 Hybrid Real-Time Fraud Detection
We evaluate PANTHER's s fraud detection performance through 10-day full-traffic A/B test on WeChat Pay's production system, using the DeepFM model [7] the baseline. DeepFM relies on handcrafted features and TexICNN encoder for sequence processing. PANTHER operates in hybrid configuration, combining offline pretrrirreeee user-profile embeddings (e.) and behavioral anomaly scores (A) with real-time transaction features (Equation 3), achieving substantial recall improvements, shown in Figure 4.
The hybrid PANTHER model improves fraud recall by 109.5% at the 0.01% 10% threshold, with smaller gains at higher thresholds (0.1% +38.6%, 1% +12.1%). The larger gains extreme thresholds high- light the model's effectiveness in detecting low- freeuuencee, high-risk fraud cases using personalized embeddings (e,) and behavioral anomaly scores (A).
Deployment Strategy: This hybrid approach introduces minimal overhead (5ms higher than the baseline), while offering three key advantages: (1) personalized fraud detection via user-specific embeddings, (2) explainable anomaly detection through interpretable deviation scores, and (3) scalable production deployment by separating compute-intensive pretraining from real-time inference.
4.1.3 Merchant Risk Assessment via Behavior Sequence Pretraining
In addition to fraud detection, we developed framework for merchant risk assessment based on behavior sequence pretraining. This approach identifies merchants potentially involved in fraud by analyzing deviations in user behavior at the merchant level, incorporating components for behavior prediction, deviation measurement, risk aggregation, and merchant classification.
Next Behavior Deviation Scoring. Deviation measured as the standardized difference between the predicted likelihood of a behavior and the user's 's typical behavior distribution. It reduces false positives for users with naturally varied transaction patterns: Au,m (m) where fl, and o.
d.
the mean and Std Dev of predicted probabilities across all potential behaviors for user

---

Model Improvement by Configuration Top Percentage
Improvement (Re aelvv to Baseline)
109.5%
Top 0.01%
Top 0.1%
Top 1%
61.6%
64.2%
38.6%
16.8%
9.3%
21.1%
12.1%
0.0% 0.0% 0.0%
4.1%
DeepFM (Baseline)
User Profile Embedding Behavioral Anomaly Score Model Configuration
Beth Components
Figure 4: Fraud recall improvement operational thresholds (Recall (occallTTopKK).
Merchant Risk Scoring and Classification. acting with merchant m and compute the quantile-ban baase risk score:
We aggregate deviation scores for all users trans-
Rm = Quantile,, ({Au,
Lmm)),
where Um the set of interacting with merchant m. Robust statistical features extracted and used as inputs for classifier (e.g., XGBoost) to label merchants as high or low risk.
Practical Impact. The framework showed strong performance in practice, achieving 76% accuracy for flagged high-risk merchants a significant improvement over the baseline method's ' 50% accuracy.
4.2 Benchmark Performance
After validating PANTHER's s real-world viability, evaluate its performance on public benchmark datasets to quantify its improvements over existing methods. This section compares PANTHER against strong baselines on public transaction datasets and recommendation benchmark, demonstrat- ing consistent performance gains across domains beyond the WeChat Pay setting.
4.2.1 Datasets & Tasks
We evaluate PANTHER's pretraining performance four datasets, including two financial datasets and two recommendation datasets, to assess its effectiveness and generalizability. For downstream tasks, compare fraud detection performance CCT and recommendation tasks Yelp (since only these datasets include user profile features).
1. Credit Card Transactions (CCT) [24]: A synthetic dataset with 20 million transactions from 2,000 users, used for fraud detection with embedded anomalies.
2. MBD-mini [25]: An anonymized banking dataset tracking monthly product purchases, focusing repetitive consumer behaviors.
3. MovieL.ens-1M. [26]: A widely-used recommendation dataset with million user-item interactions, ideal for evaluating sequential models con-financial domains.
4. Yelp [27]: A public recommendation dataset containing millions of business interac- tions (c.g., restaurant reviews), commonly used for sequential recommendation tasks.
Each dataset split chronologically into training, validation, and test subsets.
Implementation details:
We apply the same model hyperparameters to the open benchmark datasets as those used for the WeChat Pay dataset. Full implementation details are provided in Sec- tion B and our supplementary code.
4.2.2 Next Payment Behavior Prediction Benchmarking
We assess PANTHER on next behavior prediction. As summarized in Table for MBD-mini, CCT, MovieLens- IM and Yelp, it consistently outperform strong baselines like SASRec and HSTU.

---

Table Experimental results of various methods on transaction and recommendation datasets.
Dataset
Method
IR@1
IRR5
IIR @ 10
Transformer
.0441
.1735
.2864
SASRec (442(+0.23%)
(.299--.35%)
2287((+0.24))
MBD-mini
HSTU (432(-2.04%%
.711(-1.27%)
2851(-0.45%)
Payment
PANTHER
(454(+2.955%)
.1766(+1.79%)
.2923 .0005(+2.06%)
Transformer .0500
.1381
.1779
SASRec 0447(-10.60%)
.1887(+0443%)
1994(+0.76%%
CCT
HSTU 0537(47.40%%)
.419(+2.75%)
.025(+2.322%)
PANTHER 0076((15520%%
1554((12.53%)
.276 .0004(+9.55%
Transformer 00799
.1266
.2733
Recommendation
Moviel.ens-IM
SASRec 0627(+8.29%)
1906(+4.88)
2255(+2.88%)
HSTU 0699(+20.73%)
.1772(77.999)
3043(+9.748)
PANTHER .7755(21..66%
2103(+15.17%)
3078 .000((+11.01%) %)
Transformer .0851
.1769
.2300
SASRec 0875(+2.74%)
1820(+2.911)
2385(+3.71%)
Yelp
HSTU 0879(+3.21
.1878(+6.199) 19%)
2496(+8.55%)
PANTHER
.0929(+9.17%) %)
2200((24.63 %)
.2924 .0006(+27.14%%
The higher HR @ 10 reported SASRec [14] is due to sampled negative evaluation (~100 items). Our protocol follows HSTU with full-ranking over 33,700 items.
Table 4: Fraud detection performance on CCT
Table 3:
Recommendation Performance on Yelp
Model
Recall
Accuracy
Score
Model
HR@1 1
HR@5
NDCG@5
TabBERT-MLP TabBERT-LSTM
0.760 (baseline)
DCN
0.612
0.963
0.534(baseline)
0.860 (+13.2%)
PANTHER
0.773
0.982
0.692(+29.6%)
DCN
0.931
0.871
0.888 (+ 16.8%
PANTHER
0.978
0.896
0.911 (+ 19.9%)
WeChat Pay, SANTHER-large achieves 25.56% HR@1 improvement over Transformer, reflecting its capabiliiyy to model sporadic, large-scale financial transactions. On MBD-mini and CCT, PANTHER improves HR by 2.95% and 15.2%, respectively, demonstrat- ing its broad applicability other payment transaction data. On the MovieLens- 1M and Yelp datasets, PANTHER improves HR@ by 21.8% and 9.17% over Transformer, surpassing HSTU and showing strong generalization beyond payment data. Overall, PANTHER demonstrates robustness across domains and highlights the advantages of larger model configurations for complex user-item interactions.
Key Observations:
4.2.3 Hybrid Fraud Detection Recommendation
We evaluate the PANTHER on downstream fraud detection and recommendation tasks, by introducing its pretrained embeddings (e,.) and next-behavior predictions (X,) to the baseline models.
Hybrid Recommendation on Yelp. We demonstrate that PANTHER not only excels in fraud detection but also performs effectively in recommendation tasks. For example, on the Yelp dataset, the NDCG@s 5 metric improves by 29.6% over the DCN baseline (Table 3), highlighting the value of pretrained embeddings in enhancing recommendation performance.
Hybrid Fraud Detection CCT. On the CCT dataset, incorporating PANTHER's pretrained embeddings and predictions improves fraud detection recall by 19.9% over the TabBERT-MLP ([28) baseline (Table 4), enhancing the model's 's effectiveness in detecting fraudulent activity.
These results confirm that PANTHER's s hybrid method significantly boosts performance across tasks by leveraging learned user profiles and behavior predictions.

---

4.3 Transferability
We examine PANTHER's ability to transfer learned representations across new users, datasets, and domains, demonstrating the power of generative pretraining in low-label and cross-doman scenarios. PANTHER shows exceptional transferability, with a 301.4% improvement over cold-start baselines when transferring across user demographics. Additionally, fine-tuning on external datasets after pretraining on WeChat Pay leads to an average HR improvement of 16.66% on MBD-mini and CCT, showcasing the model's s adaptability across diverse transaction contexts. More detailed results and experiments are provided Section C. These findings highlight PANTHER'S ability to generalize effectively with minimal retraining, making it highly suitable for real-world applications with sparse labeled data.
4.4 Summary of Experimental Findings
Our experiments show that PANTHER consistently outperforms strong baselines across tasks. achieves robust next-transaction prediction with noisy, sparse data by leveraging the Sequence Pattern Recognition Module and adaptive user embeddings. PANTHER also demonstrates strong transferability, excelling across diverse domains, including transaction data (CCT, MBD-mini) and recommendation tasks (MovieLens- IM, Yelp). Real-world deployment at WeChat Pay shows a 38% improvement fraud recall at top 0.1 1... Ablation studies confirm that key components, like SPRM and contrastive learning, significantly enhance performance. Overall, it proves to be versatile, scalable solution for sequential behavior modeling, with strong generalization across domains.
5 Conclusions
We introduced PANTHER, a generative pretraining framework that addresses real-world payment data complexities by combining noise suppression, pattern recognition, and user personalization. Through token space compression and innovative attention mechanisms, PANTHER uncovers subtle, cyclic behaviors and incorporates long-term user context, producing high-fidelity user embeddings. This design supports critical financial applications such as fraud detection, credit scoring, next- payment prediction, and user segmentation. Beyond industry use, PANTHER highlights an important direction for the machine learning community: leveraging massive unlabeled transaction logs to enhance efficiency and adaptability. As financial and e-commerce data volumes rise, PANTHER's integration of generative modeling and personalization enables more secure, accurate, and user-centric services. However, its lack of interpretability is a limitation, as complex representations may hinder transparency in high-stake applications. Future work will focus on improving explainability for broader applicability in regulated domains.
Acknowledgments
This work is supported by the National Natural Science Foundation of China (No. 62406192), Opening Project of the State Key Laboratory of General Artificial Intelligence (No. SKLAGI2024OP12,, and Tencent WeChat Rhino-Birn Focused Research Program.
References
[1] Enterprise Apps Today. Global digital payment platforms user statisties (2022). https:
/ //www.entettt enterppiseappstooeeeeo com/ ttnm/stats/oni /nw//nnnnneeeennn http//onnnnnnnnnnntt.oeeeeeeoeee payment-statisticc
statistics html, 2022.
Accessed: 2025-01-23.
[2] Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume I (Long and Short Papers), pages 4171-4186 2019. doi: 10.18653/1/N19-1444 URL https://aclaathhllggg wwwpaaooogyoooooooooooooooo org/N19 11te/N99-10000 1423/.
[3] Yu Sun, Shuohuan Wang. Yukun Li, Shikun Feng, Hao Tian, Hua Wu, and Haifeng Wang. Ernie 2.0: A continual pre-training framework for language understanding. Proceedings of the AAAI conference artificial intelligence, volume 34, pages
2020.
10

---

[4] Mark Chen, Alec Radford, Rewon Child, Jeffrey Wu, Heewoo Jun, David Luan, and Ilya Sutskever. Generative pretraining from pixels. In International conference on machine learning. pages 1691-1703. PMLR, 2020.
[5] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 16000-16009, 2022.
[6] Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhaojie Gong, Fangda Gu, Michael He, Yinghai Lu, and Yu Shi. Actions speak louder than words: priraeen-prrmmeter sequential transducers for generative recommendations. In Proceedings of the st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 58484-58509, 2024. URL https: / proceeeiccceeeesee mlr. presa/ v235/ zhazzhai22 html.
[7] Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. Deepfm: mnctorization-machin based neural network for ctr prediction. Proceedings of the 26th Interna- tional Joint Conference on Artificial Intelligence (IJCAI), pages 1725-1731, 2017.
[8] Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang. Deep & cross network for ad click predictions. In Proceedings of the ADKDD' 17, pages 1-7. 2017.
[9] Bin Liu, Chenxu Zhu, Guilin Li, Weinan Zhang, Jincai Lai, Ruiming Tang, Xiuqiang He, Zhenguo Li, and Yong Yu. Autofis: Automatic feature interaction selection in factorization models for click-through rate prediction. In Proceedingy of the 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2636-2645, 2020.
[10] Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. Deep interest evolution network for click-through rate prediction. Proceedings of the AAAI Conference on Artificial Intelligence (1541)::911100 2019. doi: 10. 10.1609/aai..v3/0 33015941. URL httpp::///de
org/10.10 1609/6009/00 aaai. v33101. 33015941.
[11] Paul Covington, Jay Adams, and Emre Sargin. Deep neural networks for youtube recommen- dations. Proceedings of the 10th ACM Conference on Recommender Systems (RecSys), pages 191-198, 2016.
[12] Balazs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. Session-based recommendations with recurrent neural networks. In Proceedings of the 4th International Conference on Learning Representations (ICLR), 2016. URL https: //ta:://aae //arxi///iiv/////a .rrg/ aba/ 1511.06939.
[13] Jiaxi Tang and Ke Wang. Personalized top-n sequential recommendation via convolutional sequence embedding. In Proceedings of the 1th ACM International Conference on Web Search and Data Mining ( (WSDM), pages 567-575, 2018. doi: 10.1445/11995523155555
[14] Wang-Chir Wang-Cheng Kang and Julian McAuley, Self-attentive sequential recommendation. In 2018 IEEE International Conference on Data Mining (ICDM), pages 197-206, 2018. doi: 10.1 RDDM.208.000335
[15] Fei Sun, Jun Liu, Jian WU, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. Bert4rec: Se- quential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pages 1441-1450, 2019. doi: 10.1145//357384.337788 URL https: / //doi. 01444//3353884.333 3357895.
[16] Shashank Rajput, Nikhil Mehta, Anima Singh, Raghunandan Hulikal Keshavan, Trung Vu, Lukasz Heldt, Lichan Hong, Yi Tay, Vinh Q. Tran, Jonah Samost, Maciej Kula, Recommender systems with genera-
Ed H. Chi, and Maheswaran Sathiamoorthy. tive retrieval. In Advances in Neural Information Processing Systems, volume 36. 2023. URL https: https://prooeeddaii http://ooooceiinnn.eeeeeeeeeeepe wttppa/reeeeeeeeeeeeeear....... files/ wtlle//papee//22///aaa 00lca00110066accc00llaaaaa111111 Abstract- Conference hhtren.eee.ml.......

---

[17] Zihao Li, Aixin Sun, and Chenliang Li. Diffuree: A diffusion model for sequential recommen- dation. ACM Transactions on Information Systems, 42(3), 2024.
[18] Junyi Chen, Lu Chi, Bingyue Peng, and Zehuan Yuan. Hilm: Enhancing sequential recom- mendations via hierarchical large language models for item and user modeling. arXiv preprint 12X1::2409.12740, 2024.
[19] Richard J Bolton and David J Hand. Statistical fraud detection: A review. Statistical Science, 17(3):235-255 2-22, 2002. doi: 10.1214/ss/104272793
[20] Jaime D. Srevedo-Viloriaa Luisa Roa, Soji Adeshina, Cesar Charalla Olazo, Andrés Rodríguez- Rey, Jose Alberto Ramos, and Alejandro Correa-Bahnsen, Relational graph neural networks for fraud detection in super-app environment. arXiv preprint arXiv:2107. 13673, 2021. doi: 10. 00448500/aXx..2107.33777
URL httph::t///rrt
org/ awwaaas///a 2107 13673.
[21] Ziqi Liu, Chaochao Chen, Xinxing Yang, Jun Zhou, Xiaolong Li, and Le Song. Heterogeneous graph neural networks for malicious account detection. Proceedings of the 27th ACM International Conference Information and Knowledge Management, pages 2077-2085, 2018. doi: 10.114453266220632222011
[22] Yoon Kim. Convolutional neural networks for sentence classification. Proceedings of the 2014 Conference Empirical Methods in Natural Language Processing (EMNLP), pages 1746-1751, 2014.
[23] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Informa- tion Processing Systems, volume 30, 2017.
[24] Eric Altman et al. Credit card transactions (cct) dataset, 2019. URL hwwph:tsss.s...eeeeeeee wwwgs/////w.kkall...ee com/datasets/eat tman20111 taatat/tteeeteeeeaaaaaaaa-----ee cardaatt...d Accessed: 2025-01-31.
[25] AI Lab. Mbd-mini dataset. http:p//hhnniiiiggeee//////ttttt 2024.
[26] GroupLens Research. Movielens Im dataset, 2003. URL https://grouplenn.com datasets/movveeeens///
[27] Yelp. Yelp dataset, 2014. URL https://businest http://bbusness.ell...p.p www..ellppp.....eeeeeeee www.///att/eeoorrrecoooooo
open- nenn-datsee/..
[28] Inkit Padhi, Yair Schiff, Igor Melnyk, Mattia Rigotti, Youssef Mroueh, Pierre Dognin, Jerret Ross, Ravi Nair, and Erik Altman. Tabular transformers for modeling multivariate time series. In ICASSP 2021-2021 IEFE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 3369. IEEE, 2021.
12

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract accurately claims the the background, introduced method and its performance.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We briefly discussed the limitations of PANTHER in the conclusion chapter.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
13

---

The paper does not include theoretical results. Focus of this work is on the problem formulation and design of pre-training framework.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Our experiments on public benchmark datasets are reproducible. Experiment configurations are fully disclosed.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
14

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification:
The code reproduce the experiments public datasets is submitted.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: Cooeeeueeeeeeeeeeeeee
/ cttppssccppbbccccc/pepcppppccclc / ssi onPolicy) for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: Settings of experiments on public datasets are specified in the paper and appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
The standard error of the repeated experiments is reported for our proposed method, providing clear indication of the statistical significance and reliability of the results.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by emor bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
15

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CL, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The required compute resources are reported in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: //nurrip/////..././/// cc/ httpp//urrpp.cccppliii.c///////p csGuidelines!
Answer:
[Yes]
Justification:
The research conforms with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No. they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Potential societal impacts are discussed in the conclusion section.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
16

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
The paper poses such risks.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: Existing assets used in the paper cited and credited. Licenses included in the appendix.
Guidelines:
The answer NA means that the paper does use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
17

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. cww datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
The released code of the paper are well documented.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crodwsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
18

---

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The core method development in this research does involve LLMs as any important, original, or non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy pettpa//nnerips: for what should or should not be described.
19

---

Appendix
A
Notation Table
We summarize the frequently used notations in Table 5.
Table 5: Notations used in this paper
Notation
Descrpition
X, [21, ,2,,,
The payment behavior sequence of length L for user u E U.
21 V
The transaction token at time step t, V is the set of possible transaction tokens.
P(y Tnew, Cnew, X
Fraud probability for new transaction Vnew with contextual feature Cuew, given X., The is an indicator variable for fraud.
Egen
The generative loss function for next behavior prediction
Po(ær | (E (profle)
The probability of the t-th transaction 2; given the previous transactions I<L, user profile features profil and model pa- rameters 0.
eu
The learned user profile embedding for user u.
(-)
Real-time fraud detection model that integrates features in- cluding recent patterns, predicted deviations, and user pro- files for risk prediction, typically DCN network.
(-)-
The embedding function for transaction context features, typically in the form of linear layer.
f-nc(.)
Sequence encoder for user's recent short-term payment be- haviors, typically TexICNN model.
((Enew kL+1)
The deviation between the observed new transaction and the predicted next behavior L
(c,,ag, E C A X MX x R
Example definition of transaction token formed by the Carte-
sian product of the contextual and counterparty features.
PSPMM E Rmxd
The set of m learnable prototypes of embedding dimension d in the Sequence Pattern Recognition Module (SPRM).
H RTxd
Token embeddings of dimension for input sequence of length T
LCL
The contrastive loss function for user profile embedding learning
(i,j) E Ppos
positive pairs of users sharing similar demographic attributes
Cj,
User profile embeddings for users i and j. respectively.
A
The hyper-parameter balancing the two losses: Lgen- LCL
Rm
The risk score for merchant m,, computed from the user deviation scores interacting with the merchant.
20

---

B Implementation Details
In this section, we provide the full implementation details for MBD-mini, CCT, Movielens, and Yelp datasets, including model configurations, training procedures, and tokenization methods.
B.1 CCT, Yelp, and MBD-mini Experiments
For the CCT, Yelp, and MBD-mini dataset, we configured PANTHER with 4 layers and attention heads, Training was conducted with batch size of 128 at learning rate 1 x 10 10 The training utilized single GPU over span of 2 hours CCT, 6 hours on Yelp, and 12 hours hours on MBD-mini.
The Transformer, SASRec, and HSTU models were configured with the same learning rate, number of layers, and batch size PANTHER.
B.2 MovieLens-IMM 1M Experiments
For the MovieLens- 1M experiments, PANTHER was trained with batch size of 128 and learning rate of 1 x 10-3, Specifically, PANTHER was built with 2-layer, 1-head configuration. All baseline models were configured identically to PANTHER.
These experiments followed the same training pipeline as WeChatPay, with hierarchical tokenization and discretization applied transaction attributes. Given the smaller dataset sizes, training was completed within shorter time frame while preserving model scalability.
B.3 Tokenization of benchmark datasets
For the CCT dataset, user behavior tokens are constructed from payment amounts, payment methods, and merchant categories, resulting in vocabulary of 16,847 tokens. User profiles include available card and card-holder information such as card brand, card type, and user age. For the MBD-mini dataset, user behavior tokens are derived from transaction attributes including amount, currency, event type, and the source and destination types, yielding vocabulary of 40,791 distinct tokens.
In the Yelp dataset, user behavior tokens formed by combining business's city, category, star rating, and review count, where continuous features are bucketized. The original vocabulary of 40K tokens be compressed to 17K tokens, covering 95% of all user- buer-business interactions. User profiles consist of attributes such as the number of friends, number of reviews, and average star ratings. For the Movielens dataset, movie IDs directly used behavior tokens.
B.4 Efficiency Evaluation
We compare GPU memory usage and inference time of the SPRM against the Transformer baseline across increasing sequence lengths, in Table 6. These experiments illustrate how SPRM scales more efficiently both memory consumption and latency, particularly when handling longer sequences where the Transformer model fails due to memory overflow.
Table 6: Efficiency comparison between Transformer and SPRM across different sequence lengths
Sequence Length
Transformer Memory
Transformer Inference Time(s)
SPRM
SPRM Inference Time(s)
Memory(GB)
1024
8.4
72.2
1.9
65,4
2048
29.7
113.9
3.2
70.3
4096
OOM
5.7
74.1
8192
00M
10.8
93.8
21

---

C Transferability of PANTHER
We examine PANTHER 's capacity to transfer learned representations new users, new datasets, and new domains. The experiments showcase the advantage of generative pretraining in label scarcity and cross-domain scenarios.
Table 7: Comparison of PANTHER with cold-start and user transfer settings on WeChatPay.
WeChatPay
HR@1 1
HR@5
HR@ 10
HR@ 50
Cold-Start
.0581
.0834
.0963
.1308
User- Transfer
.2332(+301 %)
.449((439%%)
.5417(+463%)
.7280(+457%
C.1 User-Level Transferability
We pre-train PANTHER on one set of WeChatPay users and evaluate on another disjoint set for cold-start recommendation setting. As shown in Table 7, the pre- pre-trained PANTHER significantly outperforms the cold-start baseline, demonstrating its ability to preserve learned behavioral patterns and adapt new users with minimal retraining. This finding is crucial for financial applications where new users frequently arrive and labeled data are sparse.
C.2 Data-Level Transferability
Table 8: Experimental results PANTHER on CCT and MBD-mini datasets after pretraining on WeChatPay dataset. The values in parentheses indicate the relative improvement compared to directly finetuning on these datasets
Dataset
HR@1
HR@5
HR@
HR@50
MBD-mini .0539(+16.66%)
.1980(+10.43%)
.3143(6..72%)
.7123(+3.82%)
CCT
.0248(+13.76%)
.0729(+10.99%) .1050(66.59%
.2706(+4.08%)
To assess cross-dataset adaptability, we pre-train PANTHER on WeChatPay and fine-tune it on MBD-mini and CCT. Table 8 shows an average HR @ improvement of 16.66%, demonstrating effective transfer of our generative pretraining across diverse transaction contexts. The model retains valuable cross-data signals, such as user spending patterns, even when the merchant or product space changes.
D Ablation Experiments
To examine the sensitivity of PANTHER to the balance between Leen and LCL, we vary the loss coefficient À and report the corresponding performance. The results are summarized in Table 9.
Table 9: Performance on WeChat Pay under different values of the loss coefficient
A
HR @
HR @5
HR @ 10
0.1
0.2452
0.4837
0.5647
0.2
0.2441
0.4862
0.5644
0.4
0.2435
0.4869
0.5653
0.8
0.2430
0.4832
0.5629
22


Reviews:
[{"review_id": "O4Yvs8ABko", "reviewer": "Reviewer_DVzH", "summary": "The paper introduces PANTHER, a hybrid framework combining self-supervised generative pretraining and lightweight discriminative modeling for fraud detection in large-scale payment systems. Key innovations include structured tokenization, a sequence pattern recognition module (SPRM), and a contrastively-learned user profile embedding. The model demonstrates strong real-world performance (e.g., deployed at WeChat Pay) and generalizes well to public benchmarks.", "questions": "1. The two baseline models used in the experiments, DeepFM and DCN, are relatively outdated. Why did the authors choose these models instead of more recent baselines ? While I understand that lightweight discriminative models like DeepFM and DCN are favorable for fast inference, including comparisons with more recent and stronger baselines would help more comprehensively demonstrate the essential role and contribution of PANTHER.\n2. On page 6, the authors formulate the complete optimization objective as a balance between individual fidelity and population structure, controlled by the hyperparameter \\lambda. However, the paper does not include the experimental analysis on how different values of \\lambda affect model performance. Such an ablation study would be valuable to understand the sensitivity of the model to this trade-off and to justify the chosen value.\n3. Tokenization for public datasets (Appendix A) lacks specifics: How are continuous features (e.g., Yelp’s \"review count\") bucketized ? Why does MovieLens dataset directly use raw movie IDs (not structured tokens) ?", "limitations": "See the Weaknesses and Questions parts.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "1. The method presents several targeted architectural innovations. Structured tokenization converts multi-field transaction data into meaningful and compact semantic units, addressing the challenge of high-cardinality categorical features. The SPRM module enhances behavioral pattern modeling by combining dilated convolutions with prototype-based attention. Furthermore, the use of contrastive user embeddings integrates both static profiles and dynamic histories, significantly improving performance in cold-start scenarios.\n\n2. The proposed approach is supported by extensive experimental evidence. It shows significant improvements in real-world A/B testing, generalizes well to other domains in benchmark comparisons, and includes detailed ablation studies that clarify the contribution of each component. \n\n3. The framework has been deployed in a large-scale payment platform, where its combination of offline generative pretraining and efficient online inference achieves a good trade-off between detection accuracy and latency.\n\n4. The paper introduces the function $g_\\phi(\\cdot)$ in Equation (3) as part of the hybrid inference model, but its exact form or implementation is not explained. \n\n5.  On page 4, the authors state that token compression reduces the vocabulary size from approximately 2 million to 60,000, claiming a 99.7% reduction. However, the correct reduction is 97% (i.e., (1 - 60,000 / 2,000,000) × 100%). This appears to be a calculation error and should be corrected.", "was_revised": true, "final_justification": "I would like to keep my rating as is.", "rebuttal": "**W1: The paper introduces the function  in Equation (3) as part of the hybrid inference model, but its exact form or implementation is not explained.**\n \n**A1:** Thank you for pointing out the need for clarity regarding the function $g_\\phi$ in Equation (3) (Eq. 134). The function $g_\\phi$ represents a general downstream discriminative model, which can take various flexible forms depending on specific tasks or use cases. Its primary role is integrating embedded context features, recent transaction encodings, long-term user profile embeddings $\\mathbf{e}_u$, and behavior deviations to compute a unified risk score. \n\nIn our practical implementation, we use a DeepFM structure for capturing feature interactions, combined with a TextCNN for recent transaction sequence modeling, followed by a feed-forward network. The model is trained end-to-end using binary cross-entropy loss on labeled data. We will clearly specify both the general and practical forms in Section 3.4.\n\n----\n\n**W2: Correct the token compression reduction claim from 99.7% to the accurate 97% based on the given values.\"** \n\n**A2:** Thank you for pointing out the correction regarding the token compression claim. The original reduction from 2 million to 60,000 tokens corresponds to a 97% reduction—not 99.7% as previously stated. We will revise the text to reflect the accurate figure to ensure clarity and correctness. \n\n----\n\n **Q1:Justify the use of outdated baselines (DeepFM, DCN) and consider adding stronger recent models to better highlight PANTHER’s contribution.**\n \n**A3:** Thank you for your insightful question. PANTHER’s hybrid approach is designed to integrate seamlessly with any discriminative model, which is why we chose DeepFM/DCN for this study. While the public datasets (Yelp, MovieLens) used in our experiments are standard for sequence model evaluation, they have limited feature complexity, which restricts the training of more advanced base models. To address this, we will include comparisons with more recent models, such as xDeepFM and AutoInt, in the appendix to showcase PANTHER’s consistent improvements.\n\nIn our real-world fraud detection system, we use DeepFM for feature interactions, TextCNN for sequence modeling, and a sequence retargeting model for behavior selection. This setup serves as a baseline, with PANTHER’s generative pretraining outputs yielding a 38.4% recall improvement. Since the core contribution of our work is the generative pretraining, we chose not to focus on this specific baseline setup in detail.\n\n----\n\n**Q2: Include an ablation study on the hyperparameter $\\lambda$ to show its impact and justify the chosen trade-off in the optimization objective.**\n\n**A4:** Thank you for the suggestion. We have conducted an ablation study on the hyperparameter $\\lambda$, and the results are shown in the table below. As observed, performance remains relatively stable across a range of $\\lambda$ values, with minimal fluctuation in HR@1, HR@5, and HR@10. This stability can be attributed to the fact that the contrastive loss converges quickly in the early stages of training, causing its value to diminish rapidly and have little impact on the overall loss thereafter.\n\n|$\\lambda$|HR@1|HR@5|HR@10|\n|:-:|:-:|:-:|:-:|\n|0.1 (paper)|0.2452|0.4837|0.5647|\n|0.2|0.2441|0.4862|0.5644|\n|0.3|0.2450|0.4881|0.5666|\n|0.4|0.2435|0.4869|0.5653|\n|0.5|0.2445|0.4873|0.5666|\n|0.8|0.2430|0.4832|0.5629|\n|0.9|0.2415|0.4791|0.5577|\n\nWe hope this ablation study clarifies the impact of $\\lambda$. \n\n----\n\n**Q3. Tokenization for public datasets lacks specifics: How are continuous features (e.g., Yelp’s \"review count\") bucketized ? Why does MovieLens dataset directly use raw movie IDs ?**\n \n**A5:**  For continuous features like Yelp's \"review count,\" we bucketize them using quantiles to ensure approximately balanced bucket sizes across the data distribution. \n\nFor MovieLens, we directly use raw movie IDs to maintain consistency with HSTU's experimental setup, enabling fair comparison with their baselines. We'll expand Appendix A with these details for clarity."}, {"review_id": "FoxtTU21VY", "reviewer": "Reviewer_13bX", "summary": "The paper proposes a large sequential behavioral modeling model called PANTHER. The modeling approach proceeds in two steps. In the first step, the model is trained to predict the next transaction with a self-supervised learning objective. The model is trained to predict the next transaction based on structured tokens which take into account merchant and transaction information and risk profiles. The model also predicts user embeddings which are learned with contrastive learning with hard negative sampling. The user embedding and the predicted next transaction are used during inference, along with embeddings of the last 100 transactions, to predict whether the next transaction is fraudulent, using a light weight model. The resulting model is evaluated on real world data from WeChat pay against state-of-the-art baselines including Transformer. The model is also evaluated on some public datasets, and demonstrates promising performance.", "questions": "1. Could you explain how your model will adapt to a new user that it is not trained on? How are the user embeddings generated?\n2. Could you please provide the summary statistics of this dataset? What are the fields? How many users is it from? What is the time range, # of transactions, average length of transaction, etc. What are the different features in this dataset? Will a subset of this dataset be made public? How is the fraudulent data labeled? Can there me false negatives (i.e. fraudulent transactions labeled as normal). How is the train and test split is chosen?\n3. What is the context length of the model? How many tokens can it take at a time?", "limitations": "The authors have briefly discussed the limitations, but I would encourage the authors to discuss have a dedicated section on limitations.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "## Strengths\nThe paper is well written, the problem is well motivated, and the method is novel to the best of knowledge. The resulting model achieves promising performance. The model is evaluated on a large real world dataset, and against state-of-the-art baselines. The code is also publicly available (in the supplementary material). \n\n## Weaknesses\n[Major] I am not sure if the paper is **double blind** given that it is evaluated on datasets collected by WeChat Pay. It is reasonable for me to assume that such a dataset can only be provisioned by Tenecent. That said, I do not think this has clouded by judgement of the paper. \n\n1. Please expand and define HR (Hit Rate) and NDCG on first use, and define them in the paper. \n2. There are a bunch of moving parts / components in the paper. I would encourage the authors to run / present ablation experiments for their design choices. For example, what is the impact of personalized user embeddings? What is the impact of using a transformer instead of the CNN architecture? What is the impact of the prototypes? \n3. Scaling laws: I wonder what is the impact of pre-training PANTHER on multiple transactional datasets? What is the impact of scaling the data (# of transactions and users) and the model size? \n\n## Minor Issues\n1. Typo in line 68", "was_revised": true, "final_justification": "This is a good paper with solid contributions. I do not have any issues with the paper, and would vote for acceptance.", "rebuttal": "**W0: Concern regarding the mention of a real-world dataset in the context of the evaluation.**\n\n**A1:** Thank you for raising this concern—we truly appreciate your thoughtful feedback. In preparing the submission, we carefully adhered to NeurIPS guidelines. This practice aligns with numerous accepted NeurIPS papers from industry settings (e.g., those associated with Google, Meta, and Alibaba), where evaluations on proprietary or internal datasets are discussed to highlight practical relevance while preserving the core principles of anonymity. For example, Google's TpuGraphs paper introduces a performance prediction dataset on tensor programs running on Tensor Processing Units (TPUs), their proprietary hardware. We believe this approach balances the need for meaningful context with the integrity of the review process.\n\n----\n\n**W1:Please expand and define HR (Hit Rate) and NDCG on first use, and define them in the paper.**\n\n**A2:** Thank you for this helpful suggestion. We will define HR (Hit Rate) and NDCG (Normalized Discounted Cumulative Gain) on first use in the camera-ready version: \"HR@K measures the fraction of instances where the ground-truth item appears in the top-K predictions. NDCG@K evaluates ranking quality by rewarding relevant items appearing earlier, normalized against the ideal ranking.\" This ensures accessibility for unfamiliar readers.\n\n----\n\n**W2: Encourage ablation studies to assess the impact of key design choices like user embeddings, transformer vs. CNN, and prototype usage.**\n\n**A3:** Thank you for your insightful suggestion regarding ablation studies. We agree that they enhance the robustness of the paper and have already incorporated several in the submission:\n- **Impact of Personalized User Embeddings**: In our paper, Table 1 presents the results of integrating the Unified User-Profile Embedding in the next-transaction prediction task on WeChat Pay data. The variant \"PANTHER (SPRM only)\" (0.2243 HR@1, +14.9% over the Transformer baseline) represents the model without user profile integration. In contrast, \"+ Profile + CL\" (0.2452 HR@1, +25.6% overall) demonstrates the added value of personalized embeddings with contrastive learning. For the downstream fraud detection task, Figure 4 shows a similar ablation, where \"+ User Profile Embedding\" achieves a 16.8% relative recall improvement at Top-0.1% over the DeepFM baseline.\n\n- **Impact of Prototypes**: Since prototypes are a core component of the SPRM module, the \"PANTHER (SPRM only)\" variant in Table 1 includes them. In comparison, baselines like Transformer and SASRec do not incorporate this structured approach, highlighting the prototypes' contribution to the +14.9% HR@1 gain.\n\nTo address your other points (e.g., Transformer vs. CNN in SPRM and the isolated impact of prototypes on downstream fraud detection), we conducted additional ablation experiments. These assess: (1) replacing the Transformer with a CNN in the model architecture, and (2) evaluating fraud detection with a configuration excluding SPRM (and thus prototypes). We will include these results in the camera-ready version.\n|Variant|WeChatPay-FraudRecall(Top0.1%)|WeChatPay-HR@1(%)|\n|:-:|:-:|:-:|\n|Full PANTHER|+38.6%|0.2452|\n|W/o User-Profile|+16.8%|0.2243|\n|W/o SPRM|+23.8%|0.2179|\n|Replace Transformer with CNN|+19.5%|0.1879|\n|Fraud Detection Baseline|+0%|-|\n \n----\n\n**W3:Explore the impact of pre-training on multiple datasets and how scaling data and model size affects PANTHER’s performance.**\n\n**A4:** Thank you for this insightful question on scaling laws. The supplementary material (Section B.2) already explores pre-training on multiple transactional datasets via transfer learning. Specifically, pre-training on WeChat Pay and fine-tuning on CCT/MBD-mini results in an average +16.66% HR@1 improvement over direct fine-tuning (Table 6), demonstrating effective cross-context transfer.\nFor data scaling (# transactions/users) and model size —we ran new experiments on WeChat Pay for next-transaction prediction. \n \n Scaling from 0.45M to 44M parameters yields a +8.1% HR@1 and +8.5% HR@10 improvement. The smaller model converges at ~0.1B users, while the medium model converges at ~0.15B users. These results will be added to the supplementary material in the camera-ready version.\n\n|ScalingType|Config|HR@1|HR@10|GPUDays| \n|:-:|:-:|:-:|:-:|:-:| \n|Model Size (Small)|0.45M params|0.2014|0.4900|3| \n|Model Size (Medium)|44M params|0.2177 (+8.1%)|0.5320 (+8.5%)|12| \n\n----\n\n**Q1: \"Could you explain how your model will adapt to a new user that it is not trained on? How are the user embeddings generated?\"** \n\n**A5:**  PANTHER generates user embeddings by combining static demographic features (e.g., age, location) with dynamic transaction histories through the Unified User-Profile Embedding module, which is trained on a predict-next-behavior task using a subset of users.\n\nFor new, unseen users, the model leverages the patterns learned from the trained subset and generalizes them to make predictions, enabling zero-shot adaptation. Supplementary B.1 demonstrates this: untrained users achieve HR@1 = 0.2332, compared to 0.2452 for trained users, resulting in a +301% improvement over cold-start baselines (Table 5). This shows that PANTHER can predict the next behavior for new users based on learned patterns from others.\n\nTo further address adaptation for new users with limited data (e.g., very short transaction histories), we conducted experiments where behavior sequences for unseen users were subsampled during inference. These results show robust performance even with minimal data. For instance, with a sequence length of 10, HR@1 reaches 0.3083, surpassing the full-sequence baseline (HR@1=0.2452 for seq=256). This counterintuitive improvement may arise from some users engaging in highly predictable activities—such as frequent game top-ups—that are easier to model with sparse data. As sequences lengthen, performance gradually stabilizes, demonstrating PANTHER’s effectiveness in cold-start scenarios. We will include this analysis and the full table in the camera-ready supplementary material.\n|SeqLength|HR@1|HR@5|HR@10|\n|:-:|:-:|:-:|:-:|\n| 10          | 0.3083 | 0.5501 | 0.6004 |\n| 20          | 0.2831 | 0.5369 | 0.5983 |\n| 40          | 0.2534 | 0.5077 | 0.5763 |\n| 80          | 0.2262 | 0.4515 | 0.5244 |\n| 160         | 0.2246 | 0.4431 | 0.5136 |\n| 200         | 0.2310 | 0.4599 | 0.5238 |\n| 240         | 0.2383 | 0.4700 | 0.5374 |\n| 256         | 0.2452 | 0.4837 | 0.5647 |\n\n----\n\n**Q2: Could you please provide the summary statistics of this dataset? What are the fields? How many users is it from? What is the time range, # of transactions, average length of transaction, etc. What are the different features in this dataset? Will a subset of this dataset be made public? How is the fraudulent data labeled? Can there me false negatives (i.e. fraudulent transactions labeled as normal). How is the train and test split is chosen?**\n\n**A6:** Thank you for these detailed questions about the WeChat Pay dataset, which help clarify its characteristics and our experimental setup. Below, we address each point:\n\n- **Summary statistics**: The dataset encompasses behaviors from approximately 1 billion users over the past year. The average transaction sequence length per user is around 300, covering both social payments (e.g., peer-to-peer transfers) and commercial payments (e.g., merchant purchases).  \n\n- **Fields/Features**: The dataset includes multi-dimensional transaction attributes such as payment amount, pay channel (e.g., QR code, app), pay source (e.g., bank-linked, wallet), merchant category (e.g., retail, gaming), merchant risk level, timestamps, user demographics (static: age, location). These are compressed via Structured Tokenization for modeling.\n\n- **Public subset availability**: Due to privacy and proprietary concerns, we cannot release any subset of the WeChat Pay dataset. However, to promote reproducibility, we evaluate PANTHER on fully public open-source datasets (CCT, MBD-mini, MovieLens-1M) in the paper, where it shows strong generalization.\n\n- **Fraud labeling**: Fraudulent transactions are labeled based on user reports to customer service, followed by thorough investigation to confirm. \n\n- **False negatives**: Yes, false negatives are possible, as some fraud may go unreported by users. This is a common challenge in real-world fraud datasets, mitigated in our setup by focusing on recall improvements.\n\n- **Train/test split**: For the fraud detection task, we use a temporal split: the past 60 days of data for training (to learn recent patterns) and the subsequent 10 days for evaluation (to simulate real-time deployment and avoid data leakage).\n\nWe will add a dedicated subsection in the camera-ready version summarizing these dataset details for clarity.\n\n----\n\n**Q3:What is the context length of the model? How many tokens can it take at a time?**\n\n**A7:**  PANTHER is designed to handle a maximum sequence length of 512 behaviors (tokens) per user during both training and inference. This choice balances computational efficiency—longer sequences quadratically increase complexity in attention-based components—with empirical performance: experiments show that extending beyond 512 yields diminishing returns in metrics like HR@1."}, {"review_id": "RKtx03JRk9", "reviewer": "Reviewer_MhX6", "summary": "This paper introduces PAINTER, a hybrid framework that enables real-time, large-scale fraud detection in payment systems. PAINTER leverages generative pretraining on unlabeled transaction data and user profile embedding to capture personalized user behavior. Experiments show significant improvements in fraud detection, next-transaction prediction, and other recommendation tasks, outperforming existing methods.", "questions": "See weaknesses. I list some other questions here.\n\n1. Could the authors provide some statistical summaries of the WeChat Pay dataset to illustrate its scale and characteristics?\n2. How are these textual payment behaviors transformed into token embeddings? Does this process utilize a pretrained BERT or large language model, and is this module updated during model training?\n3. Given that the frequent-token set will challenge as transaction distribution evolve over time, is it necessary to retrain the model in such scenarios?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Strengths:**\n1. The paper tackles an important and timely issue: efficient and effective fraud detection in real-time, billion-scale scenarios, which is crucial for large-scale payment systems that require both speed and accuracy.\n2. The pretraining module for next transaction prediction, which fuses high-dimensional data alongside local and global relational pattern recognition, presents an interesting approach for characterizing user behavior.\n3. The comprehensive evaluation across multiple tasks and datasets demonstrates the robustness and generalization of the proposed framework. The model performs well on the real-world datasets and dataset from WeChat Pay, achieving notable improvements in different prediction tasks.\n\n**Weaknesses:**\n1. While the authors emphasize the efficiency of their approach, there is no specific experimental validation in this paper. Incorporating an experiment that measures how training time or computational resources for both the proposed model and baseline methods scale with increasing data volume would strengthen this claim.\n2. Several existing approaches (e.g., Perceiver) adopt prototype-inspired techniques to reduce transformer complexity to $\\mathcal{O}(mT)$. It would be better if the authors clarified how their Pattern Matching Transformer outperforms these models in the context of user behavior modeling.\n3. The structured tokenization scheme is relatively simple and relies on a frequency-based vocabulary. This approach risks mapping low‐frequency but critical fraud indicators to an UNK token, thereby weakening the model’s ability to detect novel or rare fraud patterns. Moreover, once deployed, the fixed vocabulary cannot be updated dynamically; in environments where transaction distributions shift over time, the tokenization strategy needs timely updating, and the model requires retraining.\n\n[1] Perceiver: General Perception with Iterative Attention", "was_revised": true, "final_justification": "The authors have addressed part of my concerns. I would like to raise my score.", "rebuttal": "**W1: Add experiments to validate efficiency claims by measuring training time and resource usage as data volume scales.**\n\n**A1:**  Thank you for highlighting  the importance of quantitatively validating our efficiency claims. We have conducted additional experiments to quantitatively illustrate how our SPRM module scales efficiently compared to a Transformer baseline.\n\nSpecifically, we measured GPU memory usage and inference time as input sequence lengths increased, serving as a proxy for larger per-user data volumes (batch size=128, single A100 GPU). The results clearly demonstrate SPRM's efficiency advantage:\n \n|Sequence  Length|Transformer Memory(GB)|Transformer InferenceTime(s)|SPRM Memory(GB)|SPRM InferenceTime(s)|\n|:-:|:-:|:-:|:-:|:-:|\n|1024|8.4|72.2|1.9|65.4|\n|2048|29.7|113.9|3.2|70.3|\n|4096|OOM|-|5.7|74.1|\n|8192|OOM|-|10.8|93.8|\n\nSPRM effectively handles sequences up to 8192 tokens without memory overflow, whereas the Transformer baseline fails beyond 4096 tokens. We will include these experiments in the final camera-ready version to substantiate our efficiency claims comprehensively.\n\n----\n\n**W2:. Clarify how the proposed Pattern Matching Transformer outperforms prototype-based models like Perceiver for user behavior modeling.**\n\n**A2:** Thank you for your suggestion regarding prototype-based models like Perceiver. The Perceiver uses cross-attention to map high-dimensional input data into a compact latent space, with queries (Q) derived from the latent array and keys (K) and values (V) coming from the raw input. In contrast, PANTHER uses convolutional layers to aggregate local patterns and match them with learnable prototypes. \n\nWe conducted experiments comparing PANTHER and Perceiver, both with 1 million parameters, on the next transaction prediction task. The results show that PANTHER can outperform Perceiver across all metrics (HR@1, HR@5, HR@10). \n\n| Model        | HR@1   | HR@5   | HR@10  |\n|:-:|:-:|:-:|:-:|\n| PANTHER (ours)    | 0.2452 | 0.4837 | 0.5647 |\n| Transformer | 0.1952 | 0.4121 | 0.5308 |\n| Perceiver   | 0.1881 | 0.3593 | 0.4236 |\n\nFurthermore, we find that PANTHER is significantly more efficient to train. With the same number of training steps, PANTHER's training time is only 17.3% of Perceiver's.  \n\n----\n\n**W3: The current simple, frequency-based tokenization risks missing rare but critical fraud patterns due to UNK mappings, and its fixed vocabulary requires frequent updates and retraining to handle evolving transaction behaviors effectively.**\n\n**Q3: Given that the frequent-token set will challenge as transaction distribution evolve over time, is it necessary to retrain the model in such scenarios?**\n\n**A3:** Thank you for raising this important concern regarding our structured tokenization approach. While it is true that frequency-based vocabularies might map rare events to UNK tokens, our PANTHER pretraining strategy deliberately focuses only on modeling normal user behaviors. Specifically, it compresses historical patterns into personalized embeddings that establish a robust baseline of typical actions per user, rather than explicitly modeling rare or anomalous events. For example, some users regularly engage in large transactions with new contacts for business purposes, whereas others rarely exhibit such behavior—our pretraining method precisely captures these user-specific norms.\n\nIn the downstream fraud detection task, the model effectively identifies rare yet critical fraud signals by comparing recent user behaviors, leveraging contextual features and sequential information as detailed in Line 134, against the pre-trained personalized baseline. Significant deviations, such as unusual large payments or unexpected transaction patterns, are flagged as potential fraud.\n\nFor distribution shifts, the model's predictive logic stays relatively static and is retrained periodically (e.g., monthly) with vocabulary updates, while we perform regular inference to incorporate new behaviors into embeddings—no full retraining required. We'll clarify this separation in Section 3.2 and expand the limitations section on UNK impacts and adaptive strategies.\n\n----\n\n**Q1: Could the authors provide some statistical summaries of the WeChat Pay dataset to illustrate its scale and characteristics?**\n\n**A4:** Thank you for these insightful questions, which allow us to clarify essential aspects of our dataset and embedding approach:\n\nThe WeChat Pay dataset consists of transaction behaviors from approximately 1 billion users collected over the past year. On average, each user has around 300 transactions, encompassing diverse transaction types such as social payments (e.g., peer-to-peer transfers) and commercial transactions (e.g., merchant purchases). Key dataset features include multi-dimensional transaction attributes such as payment amount, payment channel, payment source, merchant category (e.g., retail, gaming), merchant risk scores, and timestamps.\n\n----\n\n**Q2: How are these textual payment behaviors transformed into token embeddings? Does this process utilize a pretrained BERT or large language model, and is this module updated during model training?**\n \n**A5:** In our PANTHER framework, textual payment behaviors are transformed into token embeddings through a hybrid approach. Specifically, merchant names are embedded by prompting a pretrained large language model (LLM) (e.g., \"Please introduce the business scope and potential customer base of this merchant\"), extracting the final-layer embedding output via average pooling, and discarding the generated textual description. These embeddings are precomputed and stored in a lookup table, remaining fixed during the training process to ensure computational efficiency and embedding consistency."}, {"review_id": "OBTZD6hh1i", "reviewer": "Reviewer_JVxW", "summary": "This paper presents a framework for fraud detection in payment platforms, seemingly inspired by self-attentive sequential recommendation systems [17]. The proposed approach consists of an unsupervised pre-training step to derive long term user profiles in the form of user embeddings, followed by a supervised step that can be trained using a limited amount of labelled examples.", "questions": "Q1: Why do the authors use only SASRec [17] and HSTU [19] in all their comparisons? \n\nQ2: Could the authors provide a link to HSTU [19]?\n\nQ3: Why aren't the SASRec results provided by the authors consistent with the results given in [17] for the Movielens 1M dataset?", "limitations": "Yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\nS1: Evaluation of several different tasks and datasets including \n- WeChat data for next transaction prediction and merchant risk assessment \n- a publicly available Credit Card Transaction dataset with labels of fraudulent transactions \n- Movielens and Yelp datasets for recommendations on sequential data.\n\nWeaknesses: \n\nW1: The paper is not clearly written, the notation and the terms used are not properly introduced: \n- HR and NDCG are not defined\n- T is not introduced\n- \\psi is introduced, but how it is trained/computed is not described.\n\nW2: P is used in many different meanings, which makes the paper very difficult to parse: \n- In Equations (1), (2) and (3), P means probability\n- In Section 3.3.2, P appears to represent a learnable prototype of dimension d\n- In Section 3.4, P denotes positive pairs of users sharing demographic attributes\n\nW3: The proposed solution is compared with the same two baselines (SASRec [17] and HSTU [19]) independent of the task & dataset:   \n- SASRec [17] is from 2018.  This reviewer was not able to locate HSTU [19] in the proceedings of ICML 2024.\n- The Hit Rate @ 10 reported in [17] for Movielens is 0.8245, significantly higher than the value reported in this paper, which is 0.2853.\n- It is very unlikely that these two baselines (one of which cannot be located) could produce state-of-the-art results for all the tasks and all the datasets evaluated. The authors should have compared their solution with the methods that produce state-of-the-art results.", "was_revised": true, "final_justification": "Even though the quality of the writing and the evaluation in the original submission did not meet my expectations, the authors have convinced me that they were willing to address some of these issues in the final version.", "rebuttal": "**W1: The paper is not clearly written, the notation and the terms used are not properly introduced:**\n- HR and NDCG are not defined\n- T is not introduced\n- \\psi is introduced, but how it is trained/computed is not described.\n\n**A1:** We appreciate the reviewer's feedback. We would like provide a detailed illustration to the notations you mentioned as follows.\n- **HR and NDCG:** Our use of HR (**Hit Rate**) and NDCG (**Normalized Discounted Cumulative Gain**) follows the convention of influential works like SASRec [17] and HSTU [12], where these are often treated as standard metrics in sequential recommendation and user behavior modeling tasks. We agree that explicit definitions improve clarity for all readers and will add them to the revised manuscript.\n- **T:** Our notation T consistently refers to the **sequence length** throughout the paper, and it's explicitly defined in line 187 (\"For a sequence of length T\"). We acknowledge that its earlier appearance in line 161 might cause confusion. Our intention there was simply to illustrate that the time complexity exhibits a quadratic growth trend. To prevent any ambiguity, we will remove this early mention and rephrase the sentence to \"incurring a quadratic time complexity ~~of O(T^2)~~.\"\n- **$\\psi$**: Our notation $\\psi$ in Eq. (3) denotes a **general embedding function** mapping  context attributes   to dense vectors. In our experiment, it is implemented by an one-layer MLP and is trained end-to-end on labeled data during downstream fine-tuning.\n- Notation table: to further enhance the paper's clarity, we will include a comprehensive notation table in the appendix, listing all key terms and symbols used throughout the manuscript.\n| **Symbol**  | **Description** |\n| --- | --- |\n| $\\mathbf{X}_u=[x_1,x_2,\\ldots,x_L]$   | The historical payment behavior sequence  for a user $u\\in \\mathcal{U}$.  |\n| $x_t \\in \\mathcal{V}$ | The transaction token at time step $t$, $\\mathcal{V}$ is the set of possible transaction tokens.|\n| $\\mathrm{Pr}(y = 1 \\mid x_{\\mathrm{new}}, \\mathbf{c}_{\\mathrm{new}}, \\mathbf{X}_u)$   | Fraud probability of a new transaction $x_{\\mathrm{new}}$ with context $\\mathbf{c}_{\\mathrm{new}}$, given history $\\mathbf{X}_u$. $y$ is an indicator for fraud. | \n| $\\mathcal{L}_{\\text{gen}}$  | Generative loss function for next behavior prediction.| \n| $\\mathrm{Pr}\\_\\theta(x\\_t \\mid x\\_{<t}, x\\_{\\text{profile}})$   | Probability of transaction $x_t$ given history $x_{<t}$, user profile $x_{\\text{profile}}$, and model parameters $\\theta$. |\n| $e_{u}$ | Learned user profile embedding for user $u$.|\n| $g_\\phi(\\cdot)$   | Real-time fraud detection model that integrates recent patterns, predicted deviations, and user profiles. |\n| $\\psi(\\cdot)$ | Embedding function for transaction context features (e.g., a linear layer). |\n| $f_\\mathrm{enc}(\\cdot)$ | Sequence encoder (e.g., TextCNN) for short-term user behavior. |\n| $\\Delta(x_{\\text{new}}, \\hat{x}_{L+1})$ | Deviation between observed new transaction $x_{\\text{new}}$ and predicted next behavior $\\hat{x}_{L+1}$. |\n| $\\tau = (c_i, a_j, m_k, r_l)\\in \\mathcal{C} \\times \\mathcal{A} \\times \\mathcal{M} \\times \\mathcal{R}$ | A transaction token composed from contextual and counterparty features.  |\n| $\\mathcal{P} \\in \\mathbb{R}^{m\\times d}$  | Set of $m$ learnable prototypes of dimension $d$ in the Sequence Pattern Recognition Module (SPRM). |\n| $H\\in\\mathbb{R}^{T\\times d}$| Token embeddings of dimension $d$ for an input sequence of length $T$.|\n| $\\mathcal{L}_{\\text{CL}}$   | Contrastive loss function for user profile embedding learning |\n| $(i,j)\\in Pos$| Positive user pairs with similar demographic attributes. |\n| $e_i, e_j$  | User profile embeddings for users $i$ and $j$.|\n| $\\lambda$   | Hyperparameter balancing $\\mathcal{L}{\\text{gen}}$ and $\\mathcal{L}{\\text{CL}}$. |\n| $R_m$ | Risk score for merchant $m$, derived from user deviation scores associated with the merchant.|\n----\n\n**W2: P is used in many different meanings, which makes the paper very difficult to parse.**\n\n**A2:** We regret the overloaded use of P, which indeed risks confusion. In the revision, we'll disambiguate by changing notations: e.g., $\\mathrm{Pr}$ for probability in Eqs. (1)-(3), $\\mathcal{P} $ for prototypes in Section 3.3.2, and Pos for positive pairs in Section 3.4. This should make the paper much easier to parse without altering content.\n\nWe believe these targeted revisions will resolve the weaknesses and enhance readability—your input has been invaluable, and we hope this demonstrates our commitment to strengthening the submission.\n\n----\n\n**W3:**\n- Why only SASRec and HSTU were used for comparison\n- Link to HSTU [19]  \n- Discrepancy in SASRec results\n\n**A3:**  Thank you for highlighting these important points, providing us an opportunity to clarify and strengthen our submission. We believe the confusion arises primarily from a citation error and differences in evaluation protocols, as detailed below. We emphasize that all our results are fully reproducible using the provided supplemental code, including scripts for data processing, model training, and evaluation across internal and public datasets.\n1. **Selection of Baselines (SASRec and HSTU):** We chose SASRec as a widely recognized sequential baseline and HSTU as the current state-of-the-art (SOTA) generative pretraining model, which significantly outperforms other notable methods (e.g., BERT4Rec, GRU4Rec by up to 65.8% in NDCG). Due to page limitations, we prioritized these two highly relevant baselines, focusing on demonstrating clear improvements. Nevertheless, we will incorporate additional results, such as BERT4Rec from the HSTU paper, in the appendix for comprehensive coverage.\n2. **Correct Reference for HSTU:** The correct citation for HSTU [19] is \"Actions Speak Louder than Words: Trillion-Parameter Sequential Transducers for Generative Recommendations\" by Zhai et al. (ICML 2024, arXiv:2402.17152). We will ensure this reference is properly cited in the final version.\n3. **Discrepancy in SASRec Results:** The noted discrepancy in HR@10 for SASRec on MovieLens-1M (0.2853 reported by our paper/HSTU vs. 0.8245 in [17]) is due to differences in evaluation protocols. Specifically, [17] employs a sampled negative setup (**~100 negatives per test item**), which significantly simplifies ranking, whereas our evaluation (following HSTU) considers the entire item set (**~3,700 items**), offering a more realistic and challenging assessment. Notably, HSTU independently confirms our reported performance (0.2853). To avoid confusion, we will explicitly clarify this evaluation distinction with a footnote in Section 4.1.\n\nFinally, we emphasize our methodological contributions tailored specifically for challenging scenarios such as high cardinality, sparsity, and real-time processing needs:\n- **Token Space Compression** reduces dimensionality and noise through structured contextual tokenization.\n- **Pattern-Aware Convolutional Cross-Attention (SPRM)** efficiently captures local and global patterns with linear complexity.\n- **Contrastive Personalized User Embedding** integrates static and dynamic user information to enhance cold-start performance.\n\nThese innovations have yielded substantial empirical improvements, achieving gains of 25.6%+ on WeChat Pay benchmarks, 21-29.6% on open datasets, and a 38.6% recall improvement in real-world fraud detection. We believe these clarifications reaffirm the robustness and novelty of our work and look forward to further improving the manuscript based on your valuable feedback."}]

Author rebuttal:

Meta-review: This paper presents PANTHER, a hybrid framework that combines self-supervised generative pre-training with lightweight discriminative modeling for sequential user behavior modeling, targeting billion-scale fraud detection with millisecond latency. The model introduces structured tokenization, the Sequence Pattern Recognition Module (SPRM), and unified user-profile embeddings, and it has been deployed at WeChat Pay, achieving a 38.6% recall improvement in fraud detection while also showing strong cross-domain generalization.

The strengths are the clear motivation and industrial relevance of large-scale fraud detection (Reviewer MhX6), the principled combination of generative pre-training with efficient discriminative inference (Reviewer 13bX, Reviewer DVzH), and strong empirical results both in production and on public benchmarks (Reviewer JVxW after rebuttal acknowledged the robustness of the improvements). Reviewers highlighted the novelty of structured tokenization, the effectiveness of SPRM for handling long sequences, and the impact of personalized user embeddings. The authors also provided ablations, scaling analyses, and efficiency validations in rebuttal, which strengthened confidence in their claims.

The weaknesses include some clarity and notation issues (Reviewer JVxW), limited initial baseline coverage with outdated or insufficient comparators (Reviewer DVzH, Reviewer JVxW), and questions about reproducibility since the main WeChat Pay dataset is proprietary (Reviewer 13bX, Reviewer JVxW). Concerns were also raised about the simplicity of structured tokenization and its handling of rare fraud patterns (Reviewer MhX6). The authors addressed most points in rebuttal by clarifying notation, correcting errors, expanding baselines (e.g., TabBERT), and adding efficiency and ablation studies, which reviewers generally accepted, with some raising their scores.

Overall, the paper makes a significant contribution by demonstrating how large-scale self-supervised pre-training can be effectively combined with lightweight inference for real-time fraud detection, with both academic and industrial impact. While limitations remain in clarity and reproducibility, the technical contributions, empirical strength, and deployment evidence support acceptance.

Final Recommendation: Accept
Decision: Accept (poster)