# Attention-Based Personalized Compatibility Learning for Fashion Matching
## 3 Methodology
### PCE-Net
#### 3.2.1 Attention-Based Compatibility Embedding Modeling  
We propose a more effective way of measuring the compatibility between the top \(t_{i}\) and bottom \(b_{j}\). To accomplish this, we suggest that the model learns its compatibility embeddings in latent compatibility space. In this space, complementary top-bottom pairs should be closer than incompatible pairs. Additionally, we argue that there should be a gap between the interactive features of matching top-bottom pairs and mismatched top-bottom pairs, i.e., \(c_{ij}\), \(c_{ik}\), thus turning the task of predicting compatibility into a classification problem. To learn the preliminary features of items in visual and text modalities, we employ convolutional neural networks (CNN) which have demonstrated excellent performance in learning representations [30; 31; 32]. It is imperative to note that all fashion items have visual and textual modalities. For example, the information on colors, patterns, and shapes of a fashion item can be extracted from its image, while its textual description can provide information on the brand, material, and category. These two modalities provide complementary information crucial for understanding the fashion items at the feature level. Therefore, we integrate both modalities' information to learn the compatibility embeddings between fashion items. In Section 3.1, we introduced \(v_{i}^{t}(v_{j}^{b})\in\mathbb{R}^{D_{v}}\) and \(t_{i}^{t}(t_{j}^{b})\in\mathbb{R}^{D_{t}}\) to represent the global visual and textual features of the top \(t_{i}\) and bottom \(b_{j}\), respectively, from various ConvNet modules. Inspired by previous works [6; 9; 10], we incorporated two attention branches to capture features that aid in compatibility embeddings modeling. These branches allow us to obtain attentive visual and textual features, which we denote as \(\overset{\sim}{v_{i}^{t}}(\overset{\sim}{v_{j}^{b}})\in\mathbb{R}^{D_{v}}\) and \(\overset{\sim}{t_{i}^{t}}(\overset{\sim}{t_{j}^{b}})\in\mathbb{R}^{D_{t}}\), respectively. Here, \(D_{t}\) represents the dimensionality of the latent compatibility space. **Visual Attention. ** To enable the compatibility embeddings module to automatically capture pertinent fine-grained visual characteristics such as color, pattern, and shape, we incorporate visual dot product attention to generate attention weights based on global visual features, \(v_{i}^{t}(v_{j}^{b})\in\mathbb{R}^{D_{v}}\). Specifically, we employ the visual attentive representation learning of the top portions as an illustration. The visual attention weight, \(\omega_{v}^{t_{i}}\), can be calculated according to Equation (5) by applying the following formula:  
\[e_{t_{i}}=V_{a}^{T}tanh(U_{a}(v_{i}^{t})^{T}) \tag{5}\]  
where \(U_{a}\in\mathbb{R}^{D_{v}\times D_{v}}\) and \(V_{a}\in\mathbb{R}^{D_{v}}\). The visual attention weight \(\omega_{v}^{t_{i}}=e_{t_{i}}\). Then we calculate the attentive visual features \(\overset{\sim}{v_{i}^{t}}\in\mathbb{R}^{D_{v}}\) of top \(t_{i}\),  
\[\overset{\sim}{v_{i}^{t}}=\omega_{v}^{t_{i}}v_{i}^{t} \tag{6}\]  
Likewise, we can calculate the attentive visual features \(v_{j}^{\widetilde{b}}\in\mathbb{R}^{D_{v}}\) of bottom \(b_{j}\),  
\[e_{b_{j}}=V_{a}^{T}tanh(U_{a}(v_{j}^{b})^{T}) \tag{7}\]  
\[\omega_{v}^{b_{j}}=e_{b_{j}} \tag{8}\]\[\overset{\sim}{v_{j}^{b}}=\omega_{v}^{b_{j}}v_{j}^{b} \tag{9}\]  
Next, we utilize inner products to quantify the visual compatibility interaction between the attentive visual features of the top portion \(t_{i}\) and the bottom portion \(b_{j}\). \[\overset{\sim}{v_{ij}}=\overset{\sim}{v_{i}^{t}}\;(\overset{\sim}{v_{j}})^{T} \tag{10}\]  
where the inner product encodes the visual interaction scores between fashion items. **Text Attention. ** We propose the integration of a text attention branch into the compatibility embedding network. This branch aims to capture the text features of each individual top and bottom, as well as the interactive text features of top-bottom pairs.