## 3 Materials and Method
### Proposed Method
#### 3.3.1 Policy-Learning Module  
The policy-learning module represents the core component of the proposed DRL-based pricing-strategy framework, responsible for mapping high-dimensional economic state inputs to distributions over pricing actions. The entire process can be abstractly formulated by the following mathematical expression:  
\[\pi^{*}=\arg\max_{\pi}\mathbb{E}_{\pi}\left[\sum_{t=0}^{\infty}\gamma^{t}r_{t}\right] \tag{17}\]  
Here, \(\pi^{*}\) denotes the optimal pricing policy, and \(\pi(a_{t}|s_{t})\) represents the policy function, which outputs a probability distribution over actions \(a_{t}\) given the state \(s_{t}\). The state \(s_{t}\) comprises the current price \(p_{t}\), competitor pricing, user click sequences, inventory levels \(I_{t}\), seasonal factors, and external events. The action \(a_{t}\) refers to the pricing decision made by the platform at time \(t\). The immediate reward \(r_{t}\) is defined as follows:  
\[r_{t}=\lambda_{1}\cdot\text{Profit}_{t}+\lambda_{2}\cdot\text{SalesGrowth}_{t }+\lambda_{3}\cdot\text{StabilityPenalty}_{t} \tag{18}\]  
In this formulation, \(\text{Profit}_{t}\) denotes the actual profit at time \(t\), \(\text{SalesGrowth}_{t}\) measures the relative increase in sales volume compared to a historical average, and \(\text{StabilityPenalty}_{t}\) penalizes abrupt pricing fluctuations to encourage smooth strategy evolution. The weighting coefficients are set as \(\lambda_{1}=0.6\), \(\lambda_{2}=0.3\), and \(\lambda_{3}=0.1\), emphasizing a profit-oriented optimization objective while incorporating considerations for sales growth and pricing stability. The learned mapping function from state to action distribution \(\pi\theta(a_{t}|s_{t})\) guides the platform in selecting optimal pricing strategies at each time step. The input features of this module include current product prices, competitor prices, inventory levels, user click-behavior sequences, seasonality factors, and external event variables. These features are vectorized via an embedding layer and fused with positional embeddings to retain both sequential and structural information within the transformer backbone network. The embedded state tensor \(X\in\mathbb{R}^{T\times d}\), where \(T\) denotes the sequence length and \(d\) is the embedding dimension, is processed through multiple transformer encoder layers for state representation modeling, as shown in Figure 2.  
The transformer trunk is composed of three stacked encoder layers, each consisting of a multi-head self-attention mechanism and a feed-forward network, both integrated with residual connections and layer normalization (Add and Norm). In each self-attention layer, the input \(X\) is projected into queries (\(Q\)), keys (\(K\)), and values (\(V\)), and the attention weight matrix is computed as \(A=\text{softmax}(QK^{\top}/\sqrt{d_{k}})\). The contextual representation is then obtained via \(AV\), followed by multi-head concatenation and a linear projection to yield the fused feature representation. This design enables the policy network to capture intricate dependencies between heterogeneous features such as price trends, user engagement, and competitive actions, which is particularly suitable for economic behavior modeling scenarios characterized by complex causal structures and market dynamics. For \begin{table}
\begin{tabular}{l l l l} \hline
**Layer** & **Input Dim** & **Output Dim** & **Param** \\ \hline Feed-Forward Network (FFN) & 128 & 128 & 65,536 \\ LayerNorm \(\times\) 2 & 128 & 128 & 256 \\ Dropout & – & – & 0 \\ GRU-based Value Head & 128 & 1 & 98,432 \\ Policy Head (MLP) & 128 & 3 & 384 \\ \hline
**Total** & – & – & **214,272** \\ \hline \end{tabular}
\end{table} 
Table 2: _Cont._instance, the mechanism can emphasize recent user behavior spikes during promotional periods or prioritize inventory fluctuations when supply constraints emerge. Following the transformer backbone, the output features are passed to the strategy head to generate action probabilities.