Package: zensols.mimic
MIMIC III Corpus ParsingA utility library for parsing theMIMIC-IIIcorpus.  This usesspaCyand
extends thezensols.mednlpto parse theMIMIC-IIImedical note dataset.
Features include:Creates both natural language and medical features from medical notes.  The
latter is generated using linked entity concepts parsed withMedCATviazensols.mednlp.Modifies thespaCytokenizer to chunk masked tokens.  For example,[,**,First,Name**]becomes[**First Name**].Provides a clean Pythonic object oriented representation of MIMIC-III
admissions and medical notes.Interfaces MIMIC-III data as a relational database (either PostgreSQL or
SQLite).DocumentationSee thefull documentation.
TheAPI referenceis also
available.ObtainingThe easiest way to install the command line program is via thepipinstaller:pip3installzensols.mimicBinaries are also available onpypi.InstallationInstall the package:pip3 install zensols.mimicInstall the database (either PostgreSQL or SQLite).MedCAT ModelsThe dependencyzensols.mednlppackage uses thedefault MedCAT
model.SQLite ConfigurationSQLite is the default database used for MIMIC-III access, but, it is slower and
not as well tested compared to thePostgreSQLdriver.  See theSQLite database fileusing theSQLite instructionsto create the SQLite
file from MIMIC-III if you need database access.Once you create the file, configure it with the API using the following
additional configuration in the--configfile is also necessary (or in~/.mimicrc):[mimic_sqlite_conn_manager]db_file=path: <some directory>/mimic3.sqlite3PostgreSQLPostgreSQL is the preferred way to access MIMIC-II for this API.  The MIMIC-III
database can be loaded by following thePostgreSQL instructions, or consider
thePostgreSQL Docker image.  Then configure the database by adding the
following to~/.mimicrc:[mimic_default]resources_dir=resource(zensols.mimic): resourcessql_resources=${resources_dir}/postgresconn_manager=mimic_postgres_conn_manager[mimic_db]database=<needs a value>host=<needs a value>port=<needs a value>user=<needs a value>password=<needs a value>The Python PostgreSQL client package is also needed, which can be installed with:pip3installzensols.dbpgUsageTheCorpusclass is the data access object used to read and parse the corpus:# get the MIMIC-III corpus data acceess object>>>fromzensols.mimicimportApplicationFactory>>>corpus=ApplicationFactory.get_corpus()# get an admission by hadm_id>>>adm=corpus.hospital_adm_stash['165315']# get the first discharge note (some have admissions have addendums)>>>fromzensols.mimic.regexnoteimportDischargeSummaryNote>>>ds=adm.notes_by_category[DischargeSummaryNote.CATEGORY][0]# dump the note as a human readable section-by-section>>>ds.write()row_id:12144category:Dischargesummarydescription:Reportannotator:regular_expression----------------------0:chief-complaint(CHIEFCOMPLAINT)-----------------------Unresponsiveness-----------1:history-of-present-illness(HISTORYOFPRESENTILLNESS)------------Thepatientisa...# get features of the note useful in ML models as a Pandas dataframe>>>df=ds.feature_dataframe# get only medical features (CUI, entity, NER and POS tag) for the HPI section>>>df[(df['section']=='history-of-present-illness')&(df['cui_']!='-<N>-')]['norm cui_ detected_name_ ent_ tag_'.split()]normcui_detected_name_ent_tag_15historyC0455527history~of~hypertensionconceptNNSee theapplication example, which gives a fine grain way of configuring the
API.Medical Note SegmentationThis package uses regular expressions to segment notes.  However, thezensols.mimicsiduses annotations and a model trained by clinical informatics
physicians.  Using this package gives this enhanced segmentation without any
API changes.CitationIf you use this project in your research please use the following BibTeX entry:@inproceedings{landes-etal-2023-deepzensols,title="{D}eep{Z}ensols: A Deep Learning Natural Language Processing Framework for Experimentation and Reproducibility",author="Landes, Paul  andDi Eugenio, Barbara  andCaragea, Cornelia",editor="Tan, Liling  andMilajevs, Dmitrijs  andChauhan, Geeticka  andGwinnup, Jeremy  andRippeth, Elijah",booktitle="Proceedings of the 3rd Workshop for Natural Language Processing Open Source Software (NLP-OSS 2023)",month=dec,year="2023",address="Singapore, Singapore",publisher="Empirical Methods in Natural Language Processing",url="https://aclanthology.org/2023.nlposs-1.16",pages="141--146"}ChangelogAn extensive changelog is availablehere.CommunityPlease star this repository and let me know how and where you use this API.
Contributions as pull requests, feedback and any input is welcome.LicenseMIT LicenseCopyright (c) 2022 - 2024 Paul Landes