## 3 Materials and Methods
### Proposed Method
#### 3.3.2 User-Centered LLM  
The user-centered LLM is the core component of the proposed framework, which is designed to protect users' sensitive information by incorporating differential privacy techniques while ensuring efficient and accurate data processing. The user-centered LLM is based on the Transformer architecture, which is enhanced with advanced technologies and strategies to achieve dual goals of privacy protection and efficient computation, as shown in Figure 3.  
The implementation of the user-centered LLM involves multiple neural network layers each with specific functions and parameter settings. This model adopts a deep neural network structure, comprising 12 layers of Transformer encoders. Each layer includes a multi-head self-attention mechanism and a Feed-Forward Neural Network (FFN). In the multi-head self-attention mechanism, 8 attention heads are set, with each head having a dimension of 64, resulting in a total attention head dimension of 512 per layer. Specifically, the input word vectors \(X\) are transformed into query \(Q\), key \(K\), and value \(V\) matrices through linear transformations:  
\[Q=XW_{Q},\quad K=XW_{K},\quad V=XW_{V} \tag{12}\]  
where \(W_{Q}\), \(W_{K}\), and \(W_{V}\) are the learnable weight matrices. The multi-head self-attention mechanism is calculated as follows:  
\[\text{Attention}(Q,K,V)=\text{softmax}\bigg{(}\frac{QK^{T}}{\sqrt{d_{k}}} \bigg{)}V \tag{13}\]  
Here, \(d_{k}\) represents the dimension of the keys. In the multi-head self-attention mechanism, multiple attention heads are computed in parallel to capture different features of the data. The mathematical expression is  
\[\text{MultiHead}(Q,K,V)=\text{Concat}(\text{head}_{1},\text{head}_{2},\dots, \text{head}_{h})W_{O} \tag{14}\]  
where \(\text{head}_{i}=\text{Attention}(QW_{Q_{i}},KW_{K_{i}},VW_{V_{i}})\), and \(W_{O}\) is the output weight matrix. Each Transformer layer's FFN includes two linear transformations and a ReLU activation function, which is given by  
\[\text{FFN}(x)=\max(0,xW_{1}+b_{1})W_{2}+b_{2} \tag{15}\]  
Figure 3: User-centered LLM architecture. Here, \(W_{1}\) and \(W_{2}\) are weight matrices, and \(b_{1}\) and \(b_{2}\) are bias vectors. To stabilize the training process and accelerate convergence, each sublayer (including the multi-head self-attention mechanism and the FFN) is followed by layer normalization and a residual connection. Specifically, the input \(X\) after layer normalization and residual connection is given by  
\[\text{output}=\text{LayerNorm}(X+\text{SubLayer}(X)) \tag{16}\]  
where \(\text{SubLayer}(X)\) represents the sublayer operations. In the model input layer, text data are converted into word vectors with a size of \(512\times 512\), where \(512\) represents the sequence length and another \(512\) represents the dimension of the word vectors. Image data are processed into a pixel matrix of \(224\times 224\times 3\), representing the image's width, height, and number of channels. After processing through convolutional layers, the image features are transformed into appropriate dimensions to be fused with text features. In each layer, Gaussian noise is added to the model parameters to achieve differential privacy protection. Assuming the model parameters at a certain layer are \(\theta\), the parameters with added noise \(\theta^{\prime}\) are calculated as  
\[\theta^{\prime}=\theta+\mathcal{N}(0,\sigma^{2}) \tag{17}\]  
where \(\mathcal{N}(0,\sigma^{2})\) denotes Gaussian noise with mean 0 and variance \(\sigma^{2}\). This method effectively prevents user data from being leaked during training and inference. The user-centered LLM designed in this paper ensures that the users' sensitive information is fully protected during data processing through a multi-layered privacy protection mechanism. The combination of the multi-head self-attention mechanism and the FFN enables the model to efficiently capture different features of the data, enhancing the model's generalization capability and accuracy.