# Proxy-Based Semi-Supervised Cross-Modal Hashing
## 3 Methodology
### Overview of PSSCH Framework  
As illustrated in Figure 1, the proposed PSSCH framework, PSSCH, consists of three networks: ImgNet, TxtNet, and CPNet.  
ImgNet and TxtNet. Building on the ideas from [31; 40; 41], we introduce feature extraction based on the Transformer encoder in cross-modal retrieval, comprising 12 stacked  
\begin{table}
\begin{tabular}{l l} \hline \hline
**Notation** & **Definition** \\ \hline \(\mathcal{D}\) & Training Dataset \\ \(N\) & Number of Samples \\ \(\mathcal{H}\) & Hash Functions \\ \(\mathcal{G}\) & Feature Extractor \\ \(I\) & Indicator Function \\ \(\mathbf{x_{i}}\) & The i-th Image Data \\ \(\mathbf{y_{i}}\) & The i-th Text Data \\ \(C\) & the number of categories \\ \(K\) & The number of bits in the hash code \\ \(\mathbf{F}\) & Features of Data \\ \(\mathbf{P^{f}}\) & Feature Proxies \\ \(\mathbf{P^{h}}\) & Hash Proxies \\ \(\mathbf{B}\) & Hash Code \\ \(\mathbf{S}\) & Similarity matrix \\
**epoch num** & The total number of iterations \\ \hline \hline \end{tabular}
\end{table}
Table 1: A summary of common symbols and notations.  
Figure 1: The proposed PSSCH framework is shown in the figure. PSSCH consists of three networks: ImgNet, TxtNet, and CPNet, where the CPNet takes the one-hot encoding of categories as input and, through forward propagation, generates feature proxies and hash proxies. The feature proxies are updated using loss \(\mathcal{L}_{f}^{l}\) and loss \(\mathcal{L}_{f}^{T}\). The cosine similarity between the unlabeled data features and feature proxies is calculated through the pseudo-label generation module to generate continuous pseudo-labels. Finally, the hash codes of the data, hash proxies, and continuous pseudo-labels are used to compute the loss through Adaptive Dual-Label Loss, and the network parameters of the TxtNet, ImgNet, and CPNet are updated via backpropagation.  
encoder blocks. Each block includes Layer Normalization, Multi-Head Self-Attention, and MLP components, with a total of 12 attention heads. The process of extracting image semantic features is represented as \(\mathbf{F}_{x}^{l}=\mathcal{G}_{x}(\mathbf{X}^{l},\mathbf{\theta}_{x})\) and \(\mathbf{F}_{y}^{u}=\mathcal{G}_{x}(\mathbf{X}^{u},\mathbf{\theta}_{x})\), where \(\mathbf{F}_{x}^{l}\) and \(\mathbf{F}_{x}^{u}\) represent the semantic features of labeled and unlabeled images, respectively. \(\mathcal{G}_{x}\) denotes the image semantic encoder, and \(\mathbf{\theta}_{x}\) refers to its parameters.  
Similarly, the text Transformer encoder comprises 12 encoder blocks, each with 8 attention heads for Multi-Head Self-Attention [29; 31]. The extraction of text semantic features is expressed as \(\mathbf{F}_{y}^{l}=\mathcal{G}_{y}(\mathbf{Y}^{l},\mathbf{\theta}_{y})\) and \(\mathbf{F}_{y}^{u}=\mathcal{G}_{y}(\mathbf{Y}^{u},\mathbf{\theta}_{y})\), where \(\mathbf{F}_{y}^{l}\) and \(\mathbf{F}_{y}^{u}\) represent the semantic features of labeled and unlabeled text, respectively. \(\mathcal{G}_{y}\) represents the text semantic encoder, and \(\mathbf{\theta}_{y}\) corresponds to its parameters.  
CPNet. The CPNet consists of four simple fully connected layers, with the dimensions of the intermediate layers matching the dimensions of image and text semantic features, and the output layer having a dimension equal to the hash code length \(K\). One-hot encoding represents categorical information directly in a sparse discrete format, providing a clear distinction between different categories; we use the one-hot encoding of categories as input, and, through forward propagation, it produces the feature proxies \(\mathbf{P}^{f}=\left\{\mathbf{p}_{i}^{f}\right\}_{i=1}^{C}\) and hash proxies \(\mathbf{P}^{h}=\left\{\mathbf{p}_{i}^{h}\right\}_{i=1}^{C}\).