Package: zensols.mimicsid
MIMIC-III corpus parsing and section prediction with MedSecIdThis repository contains the a Python package to automatically segment and
identify sections of clinical notes, such as electronic health record (EHR)
medical documents.  It also provides access to the MedSecId section annotations
with MIMIC-III corpus parsing from the paperA New Public Corpus for Clinical
Section Identification: MedSecId.  See themedsecid repositoryto reproduce
the results from the paper.This package provides the following:The same access to MIMIC-III data as provided in themimic package.Access to the annotated MedSecId notes as an easy to use Python object graph.The pretrained model inferencing, which produces a similar Python object
graph to the annotations (provides the classPredictedNoteinstead of anAnnotatedNoteclass.Table of ContentsObtainingDocumentationInstallationUsagePrediction UsageAnnotation AccessDifferences from the Paper RepositoryTrainingPreprocessing StepTraining and TestingTraining Production ModelsModelsMedCAT ModelsPerformance MetricsVersion 0.0.2Version 0.0.3CitationDockerChangelogCommunityLicenseObtainingThe easiest way to install the command line program is via thepipinstaller:pip3installzensols.mimicsidBinaries are also available onpypi.Adockerimage is now available as well.DocumentationSee thefull documentation.
TheAPI referenceis also
available.InstallationIf you only want to predict sections using the pretrained model, you need only
toinstallthe package.  However, if you want to access the
annotated notes, you must install a Postgres MIMIC-III database asmimic
package install section.UsageThis package provides models to predict sections of a medical note and access
to the MIMIC-III section annotations available onZenodo.  The first time it
is run it will take a while to download the annotation set and the pretrained
models.See theexamplesfor the complete code and additional documentation.Prediction UsageTheSectionPredictorclass creates section annotation span IDs/types and
header token spans.  See the example below:fromzensols.nlpimportFeatureTokenfromzensols.mimicimportSectionfromzensols.mimicsidimportPredictedNote,ApplicationFactoryfromzensols.mimicsid.predimportSectionPredictorif(__name__=='__main__'):# get the section predictor from the application context in the appsection_predictor:SectionPredictor=ApplicationFactory.section_predictor()# read in a test note to predictwithopen('../../test-resources/note.txt')asf:content:str=f.read().strip()# predict the sections of read in note and print itnote:PredictedNote=section_predictor.predict([content])[0]note.write()# iterate through the note object graphsec:Sectionforsecinnote.sections.values():print(sec.id,sec.name)# concepts or special MIMIC tokens from the addendum sectionsec=note.sections_by_name['addendum'][0]tok:FeatureTokenfortokinsec.body_doc.token_iter():print(tok,tok.mimic_,tok.cui_)Annotation AccessAnnotated notes are provided as a PythonNote class, which contains most of
the MIMIC-III data from theNOTEEVENTStable.  This includes not only the
text, but parsedFeatureDocumentinstances.  However, you must build a
Postgres database and provide a login to it in the application as detailed
below:fromzensols.configimportIniConfigfromzensols.mimicimportSectionfromzensols.mimicsidimportApplicationFactoryfromzensols.mimicimportNotefromzensols.mimicsidimportAnnotatedNote,NoteStashif(__name__=='__main__'):# create a configuration with the Postgres database loginconfig=IniConfig('db.conf')# get the `dict` like data structure that has notes by `row_id`note_stash:NoteStash=ApplicationFactory.note_stash(**config.get_options(section='mimic_postgres_conn_manager'))# get a note by `row_id`note:Note=note_stash[14793]# iterate through the note object graphsec:Sectionforsecinnote.sections.values():print(sec.id,sec.name)Differences from the Paper RepositoryThe papermedsecid repositoryhas quite a few differences, mostly around
reproducibility.  However, this repository is designed to be a package used for
research that applies the model.  To reproduce the results of the paper, please
refer to the [medsicid repository].  To use the best performing model
(BiLSTM-CRF token model) from that paper, then use this repository.Perhaps the largest difference is that this repository has a pretrained model
and code for header tokens.  This is a separate model whose header token
predictions are "merged" with the section ID/type predictions.The differences in performance between the section ID/type models and metrics
reported involve several factors.  The primary difference being that released
models were trained on the test data with only validation performance metrics
reported to increase the pretrained model performance.  Other changes include:Uses themednlp package, which usesMedCATto parse clinical medical
text.  This includes changes such as fixing misspellings and expanding
acronyms.Uses themimic package, which builds on themednlp packageand parses
[MIMIC-III] text by configuring thespaCytokenizer to deal with pseudo
tokens (i.e.[**First Name**]).  This is a significant change given how
these tokens are treated between the models and term mapping (Pt.becomespatient).  This was changed so the model will work well on non-MIMIC data.Feature sets differences such as provided by theZensols Deep NLP package.Model changes include LSTM hidden layer parameter size and activation
function.White space tokens are removed inmedsecid repositoryand added back in
this package to give additional cues to the model on when to break a
section.  However, this might have had the opposite effect.There are also changes in the libraries used:PyTorch was upgraded from 1.9.1 to 1.12.1spaCywas upgraded from 3.0.7 to 3.2.4Python version 3.9 to 3.10.TrainingThis document explains how to create and package models for distribution.Preprocessing StepTo train the model, first install the MIMIC-III Postgres database per themimic
packageinstructions in theInstallationsection.Add the MIMIC-III Postgres credentials and database configuration toetc/batch.conf.Comment out the lineresource(zensols.mimicsid): resources/model/adm.confinresources/app.conf.Vectorize the batches using the preprocessing script:$ ./src/bin/preprocess.sh.  This also creates cached hospital admission and
spaCy data parse files.Training and TestingTo get performance metrics on the test set by training on the training, use the
command:./mimicsid traintest -c models/glove300.conffor the section ID
model.  The configuration file can be any of those in themodelsdirectory.
For the header model use:./mimicsidtraintest-cmodels/glove300.conf--overridemimicsid_default.model_type=headerTraining Production ModelsTo train models used in your projects, train the model on both the training and
test sets.  This still leaves the validation set to inform when to save for
epochs where the loss decreases:Update thedeeplearn_model_packer:versioninresources/app.conf.Preprocess (see thepreprocessing) section.Run the script that trains the models and packages them:src/bin/package.sh.Check for errors and verify models:$ ./src/bin/verify-model.py.Don't forget to revert filesetc/batch.confandresources/app.conf.ModelsYou can mix and match models across section vs. header models (seePerformance
Metrics).  By default the package uses the best
performing models but you can select the model you want by adding a
configuration file and specifying it on the command line with-c:[mimicsid_default]section_prediction_model=bilstm-crf-tok-fasttextheader_prediction_model=bilstm-crf-tok-glove-300dThe resources live onZenodoand are automatically downloaded on the first
time the program is used in the~/.cachedirectory (or similar home directory
on Windows).MedCAT ModelsThe dependencymednlp packagepackage uses thedefault MedCAT
model.Performance MetricsThe distributed models add in the test set to the training set to improve the
performance for inferencing, which is why only the validation metrics are
given.  The validation set performance of the pretrained models are given
below, where:wF1is the weighted F1mF1is the micro F1Mf1is the macro F1accis the accuracyFundamental API changes have necessitated subsequent versions of the model.
Each version of this package is tied to a model version.  While some minor
changes of each version might present language parsing differences such as
sentence chunking, metrics are most likely statistically insignificant.Version 0.0.2NameTypeIdwF1mF1MF1accBiLSTM-CRF_tok (fastText)Sectionbilstm-crf-tok-fasttext-section-type0.9180.9250.7970.925BiLSTM-CRF_tok (GloVE 300D)Sectionbilstm-crf-tok-glove-300d-section-type0.9170.9220.8090.922BiLSTM-CRF_tok (fastText)Headerbilstm-crf-tok-fasttext-header0.9960.9960.9590.996BiLSTM-CRF_tok (GloVE 300D)Headerbilstm-crf-tok-glove-300d-header0.9960.9960.9620.996Version 0.0.3NameTypeIdwF1mF1MF1accBiLSTM-CRF_tok (fastText)Sectionbilstm-crf-tok-fasttext-section-type0.9110.9170.7920.917BiLSTM-CRF_tok (GloVE 300D)Sectionbilstm-crf-tok-glove-300d-section-type0.9290.9330.8100.933BiLSTM-CRF_tok (fastText)Headerbilstm-crf-tok-fasttext-header0.9960.9960.9650.996BiLSTM-CRF_tok (GloVE 300D)Headerbilstm-crf-tok-glove-300d-header0.9960.9960.9620.996CitationIf you use this project in your research please use the following BibTeX entry:@inproceedings{landes-etal-2022-new,title="A New Public Corpus for Clinical Section Identification: {M}ed{S}ec{I}d",author="Landes, Paul  andPatel, Kunal  andHuang, Sean S.  andWebb, Adam  andDi Eugenio, Barbara  andCaragea, Cornelia",booktitle="Proceedings of the 29th International Conference on Computational Linguistics",month=oct,year="2022",address="Gyeongju, Republic of Korea",publisher="International Committee on Computational Linguistics",url="https://aclanthology.org/2022.coling-1.326",pages="3709--3721"}Also please cite theZensols Framework:@inproceedings{landes-etal-2023-deepzensols,title="{D}eep{Z}ensols: A Deep Learning Natural Language Processing Framework for Experimentation and Reproducibility",author="Landes, Paul  andDi Eugenio, Barbara  andCaragea, Cornelia",editor="Tan, Liling  andMilajevs, Dmitrijs  andChauhan, Geeticka  andGwinnup, Jeremy  andRippeth, Elijah",booktitle="Proceedings of the 3rd Workshop for Natural Language Processing Open Source Software (NLP-OSS 2023)",month=dec,year="2023",address="Singapore, Singapore",publisher="Empirical Methods in Natural Language Processing",url="https://aclanthology.org/2023.nlposs-1.16",pages="141--146"}DockerAdockerimage is now available as well.To use the docker image, do the following:Create (or obtain) thePostgres docker imageClone this repositorygit clone --recurse-submodules https://github.com/plandes/mimicsidSet the working directory to the repo:cd mimicsidCopy the configuration from the installedmimicdbimage configuration:make -C docker/mimicdb SRC_DIR=<cloned mimicdb directory> cpconfigStart the container:make -C docker/app upTest sectioning a document:make -C docker/app testdumpsecLog in to the container:make -C docker/app devloginOutput a note to a temporary file:mimic note 1118471 > note.txtPredict the sections on the note:mimicsid predict note.txtLook at the section predictions:cat preds/note-pred.txtChangelogAn extensive changelog is availablehere.CommunityPlease star this repository and let me know how and where you use this API.
Contributions as pull requests, feedback and any input is welcome.LicenseMIT LicenseCopyright (c) 2022 - 2024 Paul Landes