kswanjitsu/RABAC

This model disambiguates clinical sense acronyms and abbreviations within clinical notes.

Be sure to use the prompt "dejargon: " before any input and include the end of sentence token </s> at the end of your text.

To use this model:

>>> from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

>>> tokenizer = AutoTokenizer.from_pretrained("")
>>> model = AutoModelForSeq2SeqLM.from_pretrained("")

>>> text = "dejargon: the pt is a 32 yo f w/ a pmhx of afib, htn, dm2 who presents for SOB."

>>> input_ids = tokenizer.encode(str(text), return_tensors='pt')
>>> output = model.generate(input_ids, max_length=500)

>>> print(tokenizer.decode(output[0], skip_special_tokens=True))

the patient is a 32 year old female with a past medical history of atrial fibrillation, hypertension, type 2 diabetes mellitus who presents for shortness of breath.</s>

It uses the SciFive NLI large as a base and it was fine tuned on MTsamples to detect abbreviations such as "HTN" and resolve them to their long forms i.e. "hypertension."

We used a list of ~1000 acronyms and abbreviations commonly found in clinical notes and performed reverse substitution into mtsamples as these notes are all dictated and do not contain acronyms and abbreviations. This created a gold standard dataset of ~5000 notes containing acronyms and abbreviations and their targets with long forms to train and evaluate the model.

We held out 20% of the ~5000 notes to test the model.

Macro metrics: (total correctly disambiguated acronyms and abbreviations per note) accuracy = 98.9%, recall = 98.7%, precision = 93.8%, f1 = 96.1%

Micro metrics: (individual correctly disambiguated acronyms and abbreviations per note) Still to be calculated, of note, less frequently occuring acronyms such as 'IS' for incentive spirometry perform less well than commonly occurring acronyms and abbreviations such as pt (patient), or afib (atrial fibrillation).

Benchmarks:

• Rouge1_precision: 0.994687
• Rouge1_recall: 0.976285
• Rouge1_fmeasure: 0.985218
• Rouge2_precision: 0.991907
• Rouge2_recall: 0.972631
• Rouge2_fmeasure: 0.981505
• RougeL_precision: 0.995465
• RougeL_recall: 0.976081
• RougeL_fmeasure: 0.985006