  Nowadays, mainstream natural language pro-cessing (NLP) is empowered by
pre-trained language models. In the biomedical domain, only models pre-trained
with anonymized data have been published. This policy is acceptable, but there
are two questions: Can the privacy policy of language models be different from
that of data? What happens if private language models are accidentally made
public? We empirically evaluated the privacy risk of language models, using
several BERT models pre-trained with MIMIC-III corpus in different data
anonymity and corpus sizes. We simulated model inversion attacks to obtain the
clinical information of target individuals, whose full names are already known
to attackers. The BERT models were probably low-risk because the Top-100
accuracy of each attack was far below expected by chance. Moreover, most
privacy leakage situations have several common primary factors; therefore, we
formalized various privacy leakage scenarios under a universal novel framework
named Knowledge, Anonymization, Resource, and Target (KART) framework. The KART
framework helps parameterize complex privacy leakage scenarios and simplifies
the comprehensive evaluation. Since the concept of the KART framework is domain
agnostic, it can contribute to the establishment of privacy guidelines of
language models beyond the biomedical domain.
