# An NLP-Based Perfume Note Estimation Based on Descriptive Sentences
## 5 Experiments
### Results and Discussion  
#### 5.2.1 Experiment 1: Quantitative Analysis  
Table 4 shows both Hit@\(k\) at \(k=\{5,10\}\) and MRR performances of the investigated sentence embedding models with and without the proposed fine-tuning scheme. From the middle panel of Table 4, denoted by \"Without the proposed fine-tuning\", all the four embedding models achieve better (i.e., higher) hit values at \(k=10\) than those at \(k=5\). This is because we have more candidates at \(k=10\) and thus it is considered an easier problem. Among the four models, MiniLM (12 Layers) and MPNet-base achieve the best and worst Hit@5 values, respectively. Different from the case of \(k=5\), however, MPNet-base yields the highest Hit@10 value. In terms of the MRR measure, similar to the case of Hit@5, MiniLM (12 Layers) produces the highest MRR value. In a similar manner, perfume note estimation performances (Hit@5, Hit@10 and MRR) obtained by applying the proposed fine-tuning strategy are tabulated in the bottom-most panel of Table 4. By directly comparing the values with those shown in the middle panel, it is observed that the estimation capability is largely improved. Particularly, we achieve 0.355%, 0.282% and 0.345% higher Hit@5, Hit@10 and MRR performances on average than those values shown in the middle panel. Moreover, the sample standard deviations are \begin{table}
\begin{tabular}{c c c} \hline \hline
**Experiments** & **Mode** & **Brief Description** \\ \hline Experiment 1 & \begin{tabular}{c} Quantitative \\ analysis \\ \end{tabular} & \begin{tabular}{c} Performance comparison with \\ and without the fine-tuning using the \\ four sentence embedding models \\ \end{tabular} \\ \hline Experiment 2 & \
\begin{tabular}{c} Qualitative \\ analysis \\ \end{tabular} &
\begin{tabular}{c} Evaluation of the retrieved base \\ notes with and without fine-tuning \\ \end{tabular} \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Evaluation scenario. reduced about 0.016% \(\sim\) 0.059%, meaning that the proposed idea is helpful in stabilizing the model performance. As an extension to the results shown in Table 4, the Hit@\(k\) performance of the investigated embedding models are evaluated over \(k=1,2,\cdots,50\) and represented in a form of cumulative match characteristic (CMC) curves as shown in Figure 4. It is observed from Figure 4 that the proposed fine-tuning strategy helps in enhancing the Hit@\(k\) performances with a big gap, particularly at lower \(k\) values. At \(k\geq 28\), MiniLM (six Layers) with the proposed fine-tuning achieves 100% of Hit@\(k\) values, while 0.855% \(\sim\) 0.935% of Hit@\(k\) values are resulted at \(k\geq 28\) without the fine-tuning. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Mode**} & **Investigated Sentence** & \multicolumn{3}{c}{**Perfume Note Estimation**} \\  & **Embedding Models** & \multicolumn{3}{c}{**Performance, \%**} \\ \cline{3-5}  & **Hit@5** & **Hit@10** & **MRR** \\ \hline \multirow{5}{*}{Without the proposed fine-tuning fine-tuning} & MPNet-base & 0.371 & 0.694 & 0.237 \\ \cline{2-5}  & DistilRoBERTa & 0.387 & 0.500 & 0.221 \\ \cline{2-5}  & MiniLM (6 Layers) & 0.403 & 0.597 & 0.263 \\ \cline{2-5}  & MiniLM (12 Layers) & 0.484 & 0.548 & 0.319 \\ \cline{2-5}  & Average (\(\pm\)std) over the four models & 0.411 & 0.585 & 0.260 \\ \cline{2-5}  & MPNet-base & 0.433 & (\(\pm\)0.072) & (\(\pm\)0.037) \\ \hline \multirow{5}{*}{With the proposed fine-tuning} & MPNet-base & 0.758 & **0.887** & 0.573 \\ \cline{2-5}  & DistilRoBERTa & **0.790** & 0.855 & 0.601 \\ \cline{2-5}  & MiniLM (6 Layers) & **0.790** & 0.871 & **0.632** \\ \cline{2-5}  & MiniLM (12 Layers) & 0.726 & 0.855 & 0.612 \\ \cline{2-5}  & Average (\(\pm\)std) over the four models & 0.766 & 0.867 & 0.605 \\ \cline{2-5}  & Average (\(\pm\)0.027) & (\(\pm\)0.013) & (\(\pm\)0.021) \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Comparative analysis of the perfume note estimation performance (Hit@\(k\) at \(k=\{5,10\}\) and MRR values) with and without the proposed fine-tuning strategy.