Exploration into Open Electronic Health Records (EHRs)
By: Eric North
Tags: Electronic Health Record, Postgresql, Openehr, Data Science, Healthcare

Photo by Luis Melendez on Unsplash

Disclaimer

The content of this article is intended for an academic audience, I make no gaurantee nor do I accept any liability whatsoever. You as the reader are completely at your own risk in regard to exploring or otherwise making use of the material contained herein. It is your responsibility to abide by all applicable privacy laws for your District, State, Province, Country and/or jurisdiction, specifically in regard to any and all material mentioned or referred to in this article. Consider yourself warned.

Introduction

In this article I will discuss a publically-available Electronic Health Record (EHR) dataset. Ever the optimist, my hope is that one day there will be a widespread availability of de-identified EHR. Researchers who are equipped with this information should be in a better position to properly test their healthcare applications. Potential applications include training a non-linear programming model to help predict patient outcomes given certain treatements and procedures. Others involve improvements to interoperability and information exchange both within and among healthcare facilities. A third application is the benchmarking of various backends. Last but not least, the use of Big Data should help inform operational and managerial improvments including better care and service delivery models. The result: more and more people receiving optimum healthcare!

The landscape of publically-available EHRs at the time of this writing is, to put it bluntly, sparse. I mean, *really* sparse. Many websites appear helpful at first glance, to the point of offering a plethora of reports and various other research tools. Unfortunately most of the underlying data used to generate these products is highly obfuscated. It is comparable to the tips of so many icebergs! Getting to the source data is next to impossible unless you have privileged access as a member of a partnering institution or government agency.

Despite some initial set-backs I did not give up in my journey to find a public dataset of EHRs. My criteria for this gathering exercise consisted of the following:

Privacy. The data must be properly de-identified to protect the privacy of the patients and their families. Apart from the obvious choice of doing nothing, simulated data is a decent alternative to help alleviate privacy concerns. See this link for simulated EHRs. Unfortunately, simulated data may not provide enough real-world fidelity, relevance, noisiness or scores of other criteria depending on the researcher’s intended application. A third option includes de-identified patient records whereby a healthcare facility or group has agreed to release records after they have passed the privacy ‘sniff test’. In these instances, it is prudent on the part of the researcher to have such a decision in-hand to accompany their work. This can include an approved record-of-discussion, reference number or other form of auditable evidence as to the data’s de-identification and releasability from a privacy standpoint. See the above disclaimer. You will be wise to tread very cautiously as there can be severe penalties in your country in regard to breaches of privacy of patient health records.

Ease of access, as in acces without any strings attached. Any websites offering ‘partner programs’ and ‘free trials’ were excluded from my search. Others on my list of exclusions included websites requiring people to provide their request for informatoin whereupon the underlying organization determined if the request was valid. Often these websites advertised latencies ranging from weeks to months before the data was provided, if at all.

My third criteria for this research effort involved interoperability and ease of incorporation into a data science application via one or more known and well-supported formats. In more colloquial terms: if I can ingest it into the Python ecosystem quickly and easily, it is good-to-go. This part is what I like to refer to as ‘mungieness’.

Lastly, I was interested in a data-set containing representative samples. It should be based on a large population spanning over several years if possible. It should also be of comparable size to one produced by an enterprise-level application on the order of, say, a medium to large-scale Medical Records System (MRS).

Let me reiterate: what is the goal of this article? To obtain a representative dataset containing de-identified EHRs. Equipped with such a dataset, I intended to geek out by formulating and testing several hypothoses underpinned by research themes in healthcare. Some themes that I found interesting included:

How well can a facility deliver timely and effective services to its patients?

Are there any metrics that can be used by administrative staff and other employees to tease out efficiencies and thus help optimize the delivery of care?

Spoiler alert: the dataset that I stumbled across unfortuantely did not help me answer these questions. According to the authors of a paper on ORBDA: An openEHR benchmark dataset for performance assessment of electronic health record servers, the data in their study was intended for benchmarking tests of an OpenEHR system: ‘we discourage the use of ORBDA in other contexts than benchmarking assessment.’[1] Nonetheless, the sheer volume of information (measured in GB) meant that I could at least test some of my analytics toolsets using what is, in my opinion, a very realistic by-product of healthcare information technology. Please see this link to get access to the data-set.

Procedure

The data is available for download in several files and there are accompanying SQL scripts to assist researchers with loading it into a PostgreSQL database. My tests were performed on a laptop running Ubuntu 20.04.

First things first, you will want to install PostgreSQL:

sudo apt-get install postgresql

I recommend having two terminal windows open, one to run shell commands and a second to run commands from the postgresql prompt once you are logged in to your database.

Download the ORBDA datafiles and scripts from this website and place them in a suitable folder.

Next, unzip both the scripts as well as datafiles.

In order to get PostgreSQL on your local machine up-and-running, a helpful tutorial can be found here. Log in using the default username. The administrator password for your Ubuntu account will be required for this step:

sudo su postgres

Perform the above step in a second terminal window. Once you are logged in, run:

psql

The prompt should change to something along the lines of:

postgres=#

Using this new prompt you may now enter PostgreSQL commands to interact with your databases.

Before creating the database, open the script called ‘orbdaCreateDB.sql’ and change the collation as per your region otherwise you will get an error. For my region, I had to change it from:

LC_COLLATE = ‘en_US.UTF-8’

LC_CTYPE = ‘en_US.UTF-8’

to:

LC_COLLATE = ‘en_CA.UTF-8’

LC_CTYPE = ‘en_CA.UTF-8’

Once changed, save the script. Switching back to the first terminal window, run this script.

psql -U postgres -f orbdaCreateDB.sql

Provided it completed successfully you should now have a database called ‘orbda’. In the second terminal window in the PostgreSQL prompt, type:

\c orbda

You should get a message stating:

You are now connected to database “orbda” as user “postgres”.

orbda=#

Subsequent commands you run in this prompt should execute on the orbda database.

Switch back to the first terminal and run the table creation script:

psql -U postgres -d orbda -f orbdaCreateTables.sql

If all goes well, you can switch to the second terminal to see the list of newly-created tables. In the postgresql prompt type ‘\dt’ to see a list of the tables:

orbda=# \dt

List of relations

Schema | Name | Type | Owner

— — — — + — — — — — — — — -+ — — — -+ — — — — —

public | bariatrics | table | postgres

public | chemotherapy | table | postgres

public | hospitalisation | table | postgres

public | medication | table | postgres

public | miscellaneous | table | postgres

public | nephrology | table | postgres

public | radiotherapy | table | postgres

(7 rows)

If you have reached this point in the article, you are just about ready to play something fun called the waiting game. Before doing this, please open the ‘importingFiles.sql’ script to change the directory to the location where you placed the files in the above step. Open the ‘importingFiles.sql’ and modify the path to an absolute path to the files as appropriate. If you do not do this you will get an error upon attempting to import the data.

Once the script is updated and saved, navigate to the first terminal and run:

psql -U postgres -d orbda -f importingFiles.sql

Once the import script is running you should see the number of entries being written to the database in succession. Mouse over to the second terminal to inspect the table details as the data is imported. Their size *should* increase. Execute this command in your PostgreSQL prompt in-between brief pauses:

\dt+

Once the import begins you will likely have to wait several hours for it to finish. Your mileage may vary.

A happy little consequence occured when I discovered one of the folder locations was mis-named, resulting in having to re-run the ‘importing files’ script once I corrected the mistake. I noticed that not only did the table I was attempting to append get bigger, a number of other tables did as well. Apparently some of the records for these tables were missed during the previous import. Running the script a third time resulted in no new entries being made to the database.

If all goes well, you should have a PostgreSQL database with gigabytes full of de-identified EHRs. Inspect one of the tables using the following command in your postgresql prompt:

SELECT * FROM nephrology LIMIT 1;

Conclusion

This was a quick tutorial in getting started with a publically-available set of electronic health records available at this link. I’ll reiterate my above friendly reminder that the data contained in these EHRs should not be used for studies or other purposes as the researchers’ intended scope was in regard to the benchmarking of OpenEHR systems. Further details are contained at their website in regard to transforming the information from the relational database format of PostgreSQL to OpenEHR. You are encouraged to follow the additional steps should you wish to continue with the conversion.

That’s it for this article, all the best with your data science applications in healthcare information technology!

Reference

[1] ORBDA: An openEHR benchmark dataset for performance assessment of electronic health record servers, Douglas Teodoro, Erik Sundvall, Mario João Junior, Patrick Ruch, Sergio Miranda Freire, 2018.