## 2 Materials and Methods
### SPPMI-Based Keyword Extraction  
In this work, we introduce SPPMI (Shifted Positive Point Mutual Information) [15]-based keyword expansion to extract smoking status-related keywords from bilingual EHRs. It is an end-to-end unsupervised method, thus not requiring any annotated data or model training. Therefore, it is easily applicable in biomedical practice as manual annotation is both time-consuming and especially costly in the medical field. SPPMI-based keyword extraction uses three main steps: text preprocessing, seed word preparation, and keyword extraction (Figure 1). During the text preprocessing step, we identified 170 commonly used medical acronyms and replaced them with their full expressions. Unlike in English, not all words in Korean are delimited by spaces. As an agglutinating language, these words are often delimited by a set of special words or particles [16]. To tokenize Korean phrases (eo-jeol) into the most semantically relevant words, we applied the Python package soynlp ([[https://github.com/lovit/soynlp](https://github.com/lovit/soynlp)]([https://github.com/lovit/soynlp](https://github.com/lovit/soynlp)), accessed on 8 September 2021) to the texts written in Korean. Without relying on any predefined dictionaries, soynlp finds the boundaries between Korean words by estimating the probability of those boundaries at the character level. After text preprocessing, we prepared a list of known smoking status-related seed keywords. They served as a basis for finding other keywords that describe a patient's smoking status. Two medical professionals analyzed frequently occurring words in our data and generated 50 smoking status-related keywords: 11 never-smoker-related keywords, 16 past-smoker-related keywords, and 13 current-smoker-related keywords. We limited \begin{table}
\begin{tabular}{c c c c} \hline \hline  & **Family Medicine** &
\begin{tabular}{c} **Pulmonary and Critical** \\ **Care Medicine** \\ \end{tabular} & **Total** \\ \hline Current smokers & 1046 & 84 & 1130 \\ Past smokers & 547 & 431 & 978 \\ Never smokers & 399 & 144 & 543 \\ Unknown & 1520 & 540 & 2060 \\ Total & 3512 & 1199 & 4711 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Overall summary of SNUH clinical notes data. Figure 1: Overall summary of our SPPMI-based keyword extraction method. our seed words to unigrams or bigrams for clarity and computational efficiency. For Korean keywords, their unigrams and bigrams are defined in terms of words identified by soynlp. A few examples of seed keywords from each smoking status are provided in Table 2.  
During the keyword extraction step, we identified semantically similar words to each of our seed words. To calculate the semantic similarity, we applied SPPMI to represent both our seed words and all the words in our EHR data as numerical vectors. We then calculated and ranked the pairwise cosine similarity between each seed word vector and other word vectors. Words with the highest cosine similarity to the seed words are identified as the extracted keywords. In SPPMI, each word is initially represented as a vector of its pointwise mutual information (PMI) scores [17] with every other word in the dataset. As described in equation 1, PMI provides a probabilistic measure of association between two words by comparing their jointly occurring probability with their individual probabilities. Because they can effectively capture word similarity, PMI and its variants, such as normalized PMI (NPMI) [18], are frequently used in NLP [19; 20; 21]. \[\text{PMI}(\text{word}_{1},\text{ word}_{2})=\ln\biggl{(}\frac{p(\text{ word}_{1},\text{ word}_{2})}{p(\text{word}_{1})\times p(\text{word}_{2})}\biggr{)} \tag{1}\]  
Equation (1). Pointwise mutual information (PMI). As shown in Equation (2), SPPMI shifts the PMI values of each word vector by a global constant (\(\log k\)). If lower-dimensional word vectors are desired, matrix factorization, such as singular value decomposition (SVD), is additionally applied.