Taxonomy-Based Automation of Prior Approval Using Clinical Guidelines (2023)

Abstract: Performing prior authorization on patients in a medical facility is a time-consuming and challenging task for insurance companies. Automating the clinical decisions that lead to authorization can reduce the time that staff spend executing such procedures. To better facilitate such critical decision making, we present an automated approach to predict one of the challenging tasks in the process called primary clinical indicator prediction, which is the outcome of this procedure. The proposed solution is to create a taxonomy to capture the main categories in primary clinical indicators. Our approach involves an important step of selecting what is known as the “primary indicator” – one of the several heuristics based on clinical guidelines that are published and publicly available. A taxonomy based PI classification system was created to help in the recognition of PIs from free text in electronic health records (EHRs). This taxonomy includes comprehensive explanations of each PI, as well as examples of free text that could be used to detect each PI. The major contribution of this work is to introduce a taxonomy created by three professional nurses with many years of experience. We experiment with several state-of-the-art supervised and unsupervised techniques with a focus on prior approval for spinal imaging. The results indicate that the proposed taxonomy is capable of increasing the performance of unsupervised approaches by up to 10 F1 points. Further, in the supervised setting, we achieve an F1 score of 0.61 using a conventional technique based on term frequency–inverse document frequency that outperforms other deep-learning approaches.

## introduction real-world applications in the natural language processing (nlp) domain are known to perform better when the language models that support them are trained and fine-tuned on the domain in ques-tion @xcite @xcite . one domain where this idea is applicable at a high level is the healthcare domain. applications herein must adhere to the domain-specific vocabulary and guidelines. prediction tasks require large amounts of sensitive data that contain information about patients and other details about the facilities that provide treatment. while the data sensitivity and protection challenges alone can be considered overwhelming due to the caveats of anonymization and privacy efforts, other atypical challenges based on knowledge and representation add to the complexity of nlp solutions in healthcare. knowledge from overworked staff, such as nurses and physicians, is critical to obtaining high-quality corpora to train nlp models. due to the lack of time, medical personnel are often unwilling to participate in annotation tasks to transfer knowledge @xcite @xcite . furthermore, when staff can participate in annotation tasks, facilities are usually unwilling to release annotations for public consumption, making their use by other healthcare systems extremely difficult. in this work, we present several experiments (unsupervised and supervised) using state-of-the-art (sota) deep-learning techniques and compare them to more conventional techniques like term frequency-inverse document frequency (tf-idf). the experiments predict what is known as the "primary indicator" from a set of clinical guidelines for spinal imaging that are readily available on the web 1 . the primary indicator is the first step of several for determining whether or not a patient should be approved for a spinal imaging procedure. typically, indicators consist of findings such as the presence of pain, trauma, and fracture when approval is required by a facility to perform a procedure. primary indicators of spine injuries are generally chosen by clinical personnel in a facility without automation using carefully prepared guidelines written by highly skilled physicians in the field. as a way of narrowing down the guidelines for language model prediction and facilitating future iterations of machine learning experiments, our work introduces a taxonomy available for public use annotated by three clinical professionals skilled in the area of nursing. our experiments show that the use of taxonomy from skilled professionals can be used to increase performance for the real-world task at hand, especially in an unsupervised manner. the annotations created in this work are for use by the medical nlp community for investigative purposes and can be considered the main contribution therein. although we can achieve high f1 performance using transformer models @xcite on common corpora known as pubmed @xcite and or mimic-iii @xcite . due to this training procedure, these models are capable of performing well in biomedical corpora such as bc5cdr, i2b2, and others. however through this work we demonstrate on the contrary, that traditional models built on tf-idf typically outperform deep learning models in terms of performance on unstructured corpus of insurance claims. we also contrast various fasttext @xcite based models for unsupervised approaches with and without the added taxonomy. the rest of the paper is organized as follows. first, we provide an overview of the limited existing work in this domain in section 2. next, in section 3, we outline the problem that we aim to address. the construction of the taxonomy and annotation approaches is explained in section 4.2. subsequently, in section 5, we discuss the approaches employed, along with the experimental details. finally, we present a comprehensive analysis of the results in section 6, and discuss future work in section 7. ## related work work that deals with real-world clinical data is sufficiently limited due to the prohibitive nature and sensitivity of facilities and patients. most models and published work use some form of fine-tuning on models trained with corpora like the pubmed @xcite and mimic-iii @xcite . however, the approaches presented in this section, while not comprehensive, cover sota approaches in the supervised and unsupervised clinical domain. supervised -various techniques such as selfsupervised and contrastive learning are used by different studies. @xcite , a self-supervised model, uses a transformer-based language model and a knowledge graph known as umls @xcite to classify entities of names. in their approach, they do not use clinical-based guidelines. in other work, they used masked-language modeling (mlm) called neigh-bert @xcite that is capable to classify entities and link them using the umls as a guide. our work does not use the umls -our intent is to provide a nurse-based taxonomy and several baseline approaches and to show the impact of the taxonomy without the complexities of finding entities. our supervised approaches include two other commonly-used approaches known as biobert @xcite and bluebert @xcite . the biobert @xcite model uses weights trained on a general domain from wikipedia and the google books corpus @xcite and then pre-trains it using pubmed @xcite abstracts. bluebert is similar to biobert with the additional inclusion of the mimic-iii @xcite corpus in the fine-tuning procedure. in this work, we use both models and fine-tune them on our datasets. our data comprise an unstructured corpus of insurance claims in the form of free text, which includes patient health records vital to making a decision of whether or not a claim should be approved. unsupervised work generally relies on methods of clustering along with the usage of external sources of knowledge like taxonomies or structured data such as umls. target classes and input data are encoded using the same embeddings, and a distance measurement like cosine similarity is used to calculate the similarity between class representation and the input data. embeddings can be created at the word, sentence, or document level. biosentvec @xcite and @xcite can both be used to generate embeddings. some sota work uses biowordvec @xcite @xcite for both supervised and unsupervised tasks approaches. we use biowordvec in our work to compare and contrast with other techniques such as biosentvec @xcite . the results of other works that used biowordvec suggest that this embedding performs well in unsupervised setting @xcite @xcite . we found little work for the clinical domain that uses taxonomies together with embeddings. however, work from @xcite is quite similar to ours because it uses biosentvec @xcite to create embeddings and has a classifier, albeit supervised, for named-entity recognition (ner). in their work, the task was based on entity finding, similar to @xcite and others; here, we forego supervision outside of the annotations that are created. other work @xcite in a similar way with a supervised model. the majority of other related work that uses a taxonomy is based on a clustering technique such as latent dirichlet allocation (lda) @xcite . this form of clustering first clusters words from groups of documents and topics as a form of weak supervision, in which topics can be mapped to a taxonomy. since we already have a taxonomy, lda does not add any value to the data or to our approach, hence, we do not use lda in this work. ## problem statement the overall objective is to mimic the behavior of clinicians in the prior authorization process. as an initial step, this research aims to address the aforementioned challenges and develop a multiclass classification approach that can accurately predict one of the 34 primary indicators from electronic health records. ultimately, our goal is to improve the efficiency and effectiveness of information retrieval and knowledge management in the healthcare domain. ## taxonomy, data acquisition, and annotation methods these sections elaborate on the annotation process and the taxonomy of the corpora provided. ## modeling "primary indicator" (pi) experiments are broken down into several tasks related to sota in the field covered in section 2. specifically, we separate the settings into two types: supervised and unsupervised to show the benefit of the taxonomy while also applying the latest techniques to solve the real-world problems at hand. we first set baselines of how far supervised techniques can reach before moving on to showing the advantage of using our method on unsupervised techniques. the following two sections explain the supervised and unsupervised experiment settings. to evaluate our models, we use weighted f1 score in order to account for the high-class imbalance present in corpus (see appendix 3 for details). all the hyperparameters for the aforementioned approaches are detailed in the appendix ## results in this section, we present our experimental results for both the supervised and unsupervised approaches in under supervision, the tf-idf model outperforms other deep-learning models based on bert @xcite and fasttext @xcite . this is due to the fact that other approaches like biowordvec @xcite and biosentvec @xcite on average have a 48 percent outof-vocabulary (oov) word detriment. this forces pre-trained word-embedding models to perform poorly when the words are not available. while models based on the bert @xcite architecture are typically known to outperform conventional models such as tf-idf, the limitation of 512-word tokens for these experiments degrades the resulting performance. in our corpus, the documents are generally about three times larger than the 512-word-token limit (an average of 1700 tokens). in this real-world setting, the adaptation of the baseline nlp models was necessary along with the experimentation of taxonomy to better understand the value of knowledge representation for the task. as shown in figure 1 , the unsupervised approaches including both sent2vec (moghadasi and zhuang, 2020) and fasttext @xcite show that the use of the introduced taxonomy increases the performance considerably when compared to a simple "cut and paste" approach directly from the clinical guidelines. we also note that when the fasttext @xcite model is trained on our training data, it outperforms other off-the-shelf approaches like biosentvec @xcite when using the introduced taxonomy. we believe that the underperformance is due to both the domain and the lack of vocabulary (covering only nearly 50% of the vocabulary in the test set). the nursing annotations are somewhat more descriptive for annotators 2 and 3. we believe that this is due to the domain knowledge. however, in some cases, annotator 1 described more specific cases. another note that we should present -annotators did not annotate for 3 classes [hd-16.1, sp-2.2, sp-2.8]. this was due to the fact that those primary indicators were irrelevant and are not currently used in the clinical guidelines. in our experiments, we excluded those primary indicators from all sets. annotators also indicated that the inflammatory spondylitis primary indicator is nearly the same as ankylosing spondylitis class. in that case, we updated both class labels as one only. ## conclusion and future work we introduce a novel corpus-based taxonomy from a real-world clinical setting. this taxonomy is created from publicly available guidelines and used as a corpus of instrumentation in an unsupervised setting. the corpus itself, created by three nursing annotators with several years of experience, illustrates how domain knowledge can increase the performance of the spinal imaging primary indicator in a set of clinical guidelines (also public). experiments in the supervised setting show that we are able to achieve decent f1 results with state-ofthe-art techniques based on deep learning. our next steps are to include the taxonomy in the supervised setting in hopes of achieving f1 scores of at least 80% which will make this approach viable to use in a real-world setting. additionally, we intend to create a classifier that is capable of processing further indications from the clinical guidelines. yifan peng, shankai yan, and zhiyong lu. 2019b. transfer learning in biomedical natural language processing: an evaluation of bert and elmo on ten benchmarking datasets. corr, abs/1906.05474. faisal rahutomo, teruaki kitasuka, and masayoshi aritsugi. 2012. semantic cosine similarity. in the 7th international student conference on advanced science and technology icast, volume 4, page 1. matías rojas, jocelyn dunstan, and fabián villena. 2022. clinical flair: a pre-trained language model for spanish clinical natural language processing. in proceedings of the 4th clinical natural language processing workshop, pages 87-92. ayush singh, saranya krishnamoorthy, and john ortega. 2022. neighbert: medical entity linking using relation induced dense retrieval. y zhang, q chen, z yang, hf lin, and zy lu. 2019. biowordvec, improving biomedical word embeddings with subword information and mesh. sci data 6: 52. sicheng zhou, nan wang, liwei wang, hongfang liu, and rui zhang. 2022. cancerbert: a cancer domainspecific language model for extracting breast cancer phenotypes from electronic health records. journal of the american medical informatics association, 29(7):1208-1216.