Package: google-fhir-views
FHIR ViewsIntroductionFHIR Views is a way to define simple, tabular views over complex FHIR data and
turn them into queries that useSQL on FHIR conventions, or other data
sources in the future. It is installed as part of a simplepip install google-fhir-views[r4,bigquery]command.FHIR Views has two main concepts:A viewdefinition, which defines the fields and criteria created by a
view. It provides a Python API for convenience, but ultimately a
view definition is a set of FHIRPath expressions that we'll explore below.A viewrunner, which creates that view over some data source.For example, let's create a simple view of patient resources for patients born
before 1960:importdatetimefromgoogle.fhir.viewsimportbigquery_runner,r4# Load views based on the base FHIR R4 profile definitions.views=r4.base_r4()# Creates a view using the base patient profile.pats=views.view_of('Patient')# In this case we interpret the 'current' address as one where period is empty.# This can be adjusted to meet the needs of a specific dataset.current=pats.address.where(pats.address.period.empty()).first()simple_pats=pats.select([pats.id.alias('id'),pats.gender.alias('gender'),pats.birthDate.alias('birthdate'),current.line.first().alias('street'),current.city.alias('city'),current.state.alias('state'),current.postalCode.alias('zip')]).where(pats.birthDate<datetime.date(1960,1,1))With support forSQL on FHIR v2, the above
view can also be defined as:simple_pats_config={'resource':'Patient','select':[{'alias':'id','path':'id',},{'alias':'gender','path':'gender',},{'alias':'birthDate','path':'birthDate',},{'alias':'street','path':'address.where(address.period.empty()).first().line.first()',},{'alias':'city','path':'address.where(address.period.empty()).first().city',},{'alias':'state','path':'address.where(address.period.empty()).first().state',},{'alias':'zip','path':'address.where(address.period.empty()).first().postalCode',},],'where':[{'path':'birthDate < @1960-01-01'},],}views=r4.base_r4()simple_pats=views.from_view_defination(simple_pats_config)If you run the above in a Jupyter notebook or similar tool, you'll notice that
the view builder supports tab suggestions that matches the fields in the FHIR
resource of the given profile. In fact, this is just a Pythonic way to build
FHIRPath expressions to be used by the runner, with suggestions available
by just pressing tab:That builder is convenient for Python users, but you can also see the FHIRPath
expression themselves by just getting the string representation of the view,
such as by runningprint(simple_pats). Notice every column and the 'where'
criteria are defined by FHIRPath expressions, while every column must define
its name in the alias() function appended to the FHIRPath:View<http://hl7.org/fhir/StructureDefinition/Patient.select(
  id.alias(id),
  gender.alias(gender),
  birthDate.alias(birthdate),
  address.where(period.empty()).first().line.first().alias(street),
  address.where(period.empty()).first().city.alias(city),
  address.where(period.empty()).first().state.alias(state),
  address.where(period.empty()).first().postalCode.alias(zip)
).where(
  birthDate < @1960-01-01
)>In other words, anyrunnerimplementation would basically use the FHIRPath
expressions to select and filter the underlying data. The example below
will use a BigQuery runner, which translates FHIRPath expressions into SQL,
but runners in Apache Spark and directly on JSON will follow. This could
also be exported as a simple JSON structure and passed to remote services
to evaluate the FHIRPath expressions and produce a view for the user.Now that we've defined a view, let's run it against a real dataset. We'll run
this over BigQuery:# Get a BigQuery client. This may require additional authentication to access# BigQuery, depending on your notebook environment. Typically the client# and runner are created only once at the start of a notebook.fromgoogle.cloudimportbigqueryasbqclient=bq.Client()runner=bigquery_runner.BigQueryRunner(client,fhir_dataset='bigquery-public-data.fhir_synthea',snake_case_resource_tables=True)runner.to_dataframe(simple_pats,limit=5)Which produces this table:idgenderbirthdatestreetcitystatezip06759d2b7-38b4-4798-97c0-d171a53e013amale1916-03-21659 Bayer Wall Apt 61BostonMassachusetts02108141dbee4d-d355-413f-a040-93ca037fe646male1951-12-05226 Sipes Ranch Unit 37LynnfieldMassachusetts019402e194d708-8989-4e0c-a8e1-eda7351672cemale1947-09-24638 Pouros Wall Suite 52LynnfieldMassachusetts0194034bccdc85-c040-45dd-ada3-a55064439a01male1943-06-20825 Jakubowski ExtensionTewksburyMassachusetts0187648dca4c3c-d2d5-460f-9168-5f18e5d29b2bmale1945-12-13319 Cronin LightHubbardstonMassachusetts01452That's it! Now the returned dataframe contains a table of the example patients
described in the query, pulled from the FHIR data stored in BigQuery. Examples
below will show more sophisticated use cases such as turning a FHIR view into
a BigQuery virtual view or incorporating clinical content from code value sets.At this time we support a BigQuery runner to consume FHIR data in BigQuery as
our data source, but future runners may support other data stores, FHIR servers,
or FHIR bulk extracts on disk.Working with code valuesMost meaningful analysis of healthcare data involves navigating clinical
terminologies. In some cases these value sets come from an established authority
like theValue Set Authority Center, and other
times they are defined and maintained locally for custom use cases.FHIR Views offers a convenient mechanism to create and use such value sets in
your queries. Here is an example that defines a collection of LOINC codes
indicating LDL results:LDL_TEST=r4.value_set('urn:example:value_set:ldl').with_codes('http://loinc.org',['18262-6','18261-8','12773-8']).build()Now we can easily query observations with a view that uses the FHIRPathmemberOffunction:# Creates the base observation view for convenience, typically done once per# base type in a notebook.obs=views.view_of('Observation')ldl_obs=obs.select([obs.subject.idFor('Patient').alias('patient'),# Below is a Pythonic shorthand -- users could type# `obs.value.ofType('Quantity').value` instead for the FHIRPath ofType# expression, but the shorthand helps autocompletionobs.valueQuantity.value.alias('value'),obs.valueQuantity.unit.alias('unit'),obs.code.coding.display.first().alias('test'),obs.effectiveDateTime.alias('effectiveTime')]).where(obs.code.memberOf(LDL_TEST))runner.to_dataframe(ldl_obs,limit=5)patientvalueunittesteffectiveTime0903156da-ca5d-4ec3-ad36-073a9437afe4153.058mg/dLLow Density Lipoprotein Cholesterol2014-06-20 11:30:15+00:0013d268dce-fed4-4bc7-b156-c78e810c5183149.379mg/dLLow Density Lipoprotein Cholesterol2013-06-10 16:20:36+00:002fdf7c87b-1c8f-4d09-8d51-e622f747a7c888.047mg/dLLow Density Lipoprotein Cholesterol2013-10-07 00:08:45+00:0039007c0ff-a0ad-48dc-adc2-c0908c06fba8108.145mg/dLLow Density Lipoprotein Cholesterol2016-03-18 10:31:54+00:004cc5a2dd6-37b6-4f15-9da7-53f3b85e337064.5849mg/dLLow Density Lipoprotein Cholesterol2012-05-26 13:27:46+00:00Working with external value sets and terminology servicesYou can also work with value sets defined by external terminology services. To
do so, you must first create a terminology service client.This example uses the UMLS terminology service from the NIH. In order access
this terminology service, you need tosign up here.You should then enter
the API key foundon your profile pagein
the place of 'your-umls-api-key' below.fromgoogle.fhir.r4.terminologyimportterminology_service_clienttx_client=terminology_service_client.TerminologyServiceClient({'http://cts.nlm.nih.gov/fhir/':('apikey','your-umls-api-key'),})Before making queries against an externally-defined value set, you must first
get the codes defined by the value set and write them to a BigQuery table. You
only need to perform this step once. After doing so, you'll be able to reference
the value set definitions you've written in future queries.injury_value_set_url='http://cts.nlm.nih.gov/fhir/ValueSet/2.16.840.1.113762.1.4.1029.5'wound_disorder_value_set_url='http://cts.nlm.nih.gov/fhir/ValueSet/2.16.840.1.113762.1.4.1219.178'runner.materialize_value_set_expansion((injury_value_set_url,wound_disorder_value_set_url),tx_client)To make queries against an externally-defined value set which you've saved to
BigQuery, you can simply refer to its URL.injury_conds=cond.select([cond.id.alias('id'),cond.subject.idFor('Patient').alias('patientId'),cond.code.alias('codes')]).where(cond.code.memberOf(injury_value_set_url))runner.create_database_view(injury_conds,'injury_conditions')Saving FHIR Views as BigQuery ViewsWhilerunner.to_dataframeis convenient to retrieve data for local analysis,
it's often useful to create such flattened views in BigQuery itself. They can be
easily queried with much simpler SQL, or used by a variety of business
intelligence or other data analysis tools.For this reason, the BigQueryRunner offers acreate_database_viewmethod that
will convert the view definition into aBigQuery View, which can then
just be consumed as if it was a first-class table that is updated when the
underlying data is updated. Here's an example:runner.create_database_view(ldl_obs,'ldl_observations')By default the view is created in the fhir_dataset used by the runner, but this
isn't always desirable (for example, a user may want to do their analysis in
their own, isolated dataset). Therefore it's common to specify aview_datasetwhen creating the runner as the target for any views created. Here's an example:runner=bigquery_runner.BigQueryRunner(client,fhir_dataset='bigquery-public-data.fhir_synthea',view_dataset='example_project.diabetic_care_example')