# Multi-Label Weighted Contrastive Cross-Modal Hashing
## 3 The Proposed Methodology
### Optimization  
The learning process of the proposed MLWCH consists of two stages: (i) the hash representation learning stage and (ii) the hashing function learning stage. We adopt the Adam adaptive algorithm [63] for optimization. In the hash representation learning stage, we minimize Equation (26) to optimize the hash representation learning model \(\mathcal{H}_{1}^{v}(\cdot,\mathbf{\theta}_{1}^{v})\) and \(\mathcal{H}_{1}^{t}(\cdot,\mathbf{\theta}_{1}^{t})\) as follows:  
\[(\mathbf{\hat{\theta}}_{1}^{v},\mathbf{\hat{\theta}}_{1}^{t})=\underset{\mathbf{\theta}_{1 }^{v},\mathbf{\theta}_{1}^{t}}{arg\min}\ \mathcal{L}_{rep}. \tag{31}\]In the hash function learning stage, we minimize Equation (30) to optimize the hash function learning model \(\mathcal{H}_{2}^{c}(\cdot,\mathbf{\theta}_{2}^{c})\) and \(\mathcal{H}_{2}^{t}(\cdot,\mathbf{\theta}_{2}^{t})\) as follows:  
\[(\mathbf{\theta}_{2}^{c},\mathbf{\theta}_{2}^{t})=\underset{\mathbf{\theta}_{2}^{c},\mathbf{ \theta}_{2}^{c}}{arg\min}\mathcal{L}_{func}. \tag{32}\]  
Overall, the entire optimization procedure of MLWCH is presented in Algorithm 1.  
```
0: Number of training image-text pairs \(N\); number of epochs \(e\); hash code length \(k\); batch size \(b\); learning rate of the network \(\eta_{1}\) and \(\eta_{2}\); hyperparameter \(\alpha\), \(\lambda\), \(\gamma\), \(\tau\);
0: Optimized parameters \(\mathbf{\hat{\theta}}_{1}^{v}\), \(\mathbf{\hat{\theta}}_{1}^{t}\), \(\mathbf{\hat{\theta}}_{2}^{v}\), \(\mathbf{\hat{\theta}}_{2}^{t}\).
1: Construct a semantic similarity matrix \(\mathcal{S}\) from multi-label set \(\{l_{i}\}_{i=1}^{N}\);
2:repeat
3: Iterative training \(e\) times;
4: // Hash representation learning stage: optimizing objective function Equation (31)
5:for each \(i\in[1,\left\lceil\frac{N}{b}\right\rceil]\)do
6: Randomly select \(b\) training image-text pairs;
7: Generate continuous hash representation \(\mathbf{x}_{v},\mathbf{x}_{t}\) through \(\mathcal{H}_{1}^{v}(\cdot,\mathbf{\theta}_{1}^{v})\), \(\mathcal{H}_{1}^{v}(\cdot,\mathbf{\theta}_{1}^{t})\);
8: Calculate the loss \(\mathcal{L}_{rep}\) by Equation (26) and update the parameters \(\mathbf{\theta}_{1}^{v}\), \(\mathbf{\theta}_{1}^{t}\) through back propagation as follows: \(\mathbf{\theta}_{1}^{v}\longleftarrow\mathbf{\theta}_{1}^{v}-\eta_{1}\times\frac{ \partial}{\partial\mathbf{\theta}_{1}^{v}}\left(\mathcal{L}_{con}^{weight}+\alpha \mathcal{L}_{total}^{sim}\right)\) \(\mathbf{\theta}_{1}^{t}\longleftarrow\mathbf{\theta}_{1}^{t}-\eta_{1}\times\frac{ \partial}{\partial\mathbf{\theta}_{1}^{t}}\left(\mathcal{L}_{con}^{weight}+\alpha \mathcal{L}_{total}^{sim}\right)\)
9:endfor
10: // Hash function learning stage: optimizing objective function Equation (32)
11:for each \(i\in[1,\left\lceil\frac{N}{b}\right\rceil]\)do
12: select \(b\) training image-text pairs and hash representation \(\mathbf{x}_{v},\mathbf{x}_{t}\);
13: Map the original feature \(\mathbf{v}\) and \(\mathbf{t}\) into a hash code through \(\mathcal{H}_{2}^{v}(\cdot,\mathbf{\theta}_{2}^{v})\), \(\mathcal{H}_{2}^{t}(\cdot,\mathbf{\theta}_{2}^{t})\);
14: Calculate the loss \(\mathcal{L}_{func}\) by Equation (30), and update the parameter \(\mathbf{\theta}_{2}^{v}\), \(\mathbf{\theta}_{2}^{t}\) through back propagation as follows: \(\mathbf{\theta}_{2}^{v}\longleftarrow\mathbf{\theta}_{2}^{v}-\eta_{2}\times\frac{ \partial}{\partial\mathbf{\theta}_{2}^{v}}\left(\mathcal{L}_{sup}+\mathcal{L}_{ quan}\right)\) \(\mathbf{\theta}_{2}^{t}\longleftarrow\mathbf{\theta}_{2}^{t}-\eta_{2}\times\frac{ \partial}{\partial\mathbf{\theta}_{2}^{t}}\left(\mathcal{L}_{sup}+\mathcal{L}_{ quan}\right)\)
15:endfor
16:until epoch ==
```  
**Algorithm 1** Optimization procedure for MLWCH.