## 3 Results
### Keyword Extraction Precision  
We extracted the top 1, 5, 10, and 20 keywords from each of these unsupervised keyword extraction models for each of our 50 initial seed words. As the complete set of smoking-related keywords are not available in the dataset, we used precision to measure the performance of the keyword extraction. To compare the precision of the models, two human annotators independently assessed the extracted keywords. Each extracted keyword was deemed correct only when both annotators agreed that it described the same smoking status as its input seed word. As shown in Table 3, our SPPMI-based keyword expansion method showed superior performance in extracting smoking status-related keywords from bilingual EHRs. It is interesting to note that the basic SPPMI model does not significantly improve keyword extraction precision. Instead, it was the lower-dimensional word vectors created from the SVD that exhibited superior and robust performance. As the number of generated keywords increased, the precision inevitably decreased because the number of words related to smoking status in our dataset is limited. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Methods** & **\# of Keywords** & **Top 1** & **Top 5** & **Top 10** & **Top 20** \\ \hline Word co-occurrence & 38.00\% & 35.60\% & 30.40\% & 29.50\% \\ PMI vector & 42.00\% & 36.40\% & 31.20\% & 27.40\% \\ NPMI vector & 40.00\% & 35.60\% & 32.00\% & 27.70\% \\ PMI score & 38.00\% & 36.80\% & 37.00\% & **34.50\%** \\ NPMI score & 36.00\% & 38.40\% & 37.40\% & 34.20\% \\ SPPMI (k = 1) & 42.00\% & 36.40\% & 31.20\% & 27.40\% \\ SPPMI (k = 5) & 38.00\% & 34.80\% & 31.80\% & 27.40\% \\ SPPMI (k = 15) & 36.00\% & 30.40\% & 27.00\% & 27.20\% \\ SPPMI-SVD (k = 1, d = 100) & 46.00\% & 40.80\% & **38.40\%** & 31.50\% \\ SPPMI-SVD (k = 1, d = 500) & 46.00\% & **41.20\%** & 35.40\% & 29.80\% \\ SPPMI SVD (k = 1, d = 1000) & 50.00\% & 38.80\% & 31.20\% & 29.50\% \\ SPPMI-SVD (k = 5, d = 100) & **56.00\%** & 37.60\% & 32.00\% & 29.30\% \\ SPPMI-SVD (k = 5, d = 500) & 42.00\% & 36.00\% & 32.80\% & 30.00\% \\ SPPMI-SVD (k = 5, d = 1000) & 42.00\% & 34.40\% & 32.80\% & 30.10\% \\ SPPMI SVD (k = 15, d = 100) & **56.00\%** & 40.40\% & 33.40\% & 30.60\% \\ SPPMI-SVD (k = 15, d = 500) & 44.00\% & 32.40\% & 29.80\% & 29.30\% \\ SPPMI-SVD (k = 15, d = 1000) & 44.00\% & 33.20\% & 31.00\% & 28.50\% \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison of precision between SPPMI-based keyword expansion and five baseline models (word co-occurrence, PMI vector, NPMI vector, PMI score, and NPMI score). The values of d represent the number of singular values used in the SVD or the dimension of word vectors in word2 vec. The values of c indicate the context size used in training word2 vec. It is also interesting to note that word2 vec shows poor precision. There are several reasons why word embedding methods generally do not work effectively in our data. One of the most critical issues is the huge number of unique words relative to the size of the data. When doctors are writing clinical notes, they are often simultaneously interacting with their patients. Due to this real-time nature, the expressions in the notes are often short and abbreviated. Consequently, they do not strictly adhere to standardized expressions, and their expression styles often differ between doctors. Due to this issue, semantically similar yet structurally different terms are prevalent in our dataset. Without normalizing these terms, word embedding methods fail to generalize. However, bilingual term normalization itself is another critical future research topic that is beyond the scope of this paper. The detailed precision for each of the three smoking classes (never smoker, past smoker, and current smoker) is included in Tables S1-S3 in Supplementary Materials. Additionally, Table 4 includes a few examples of extracted keywords from each smoking status.