# Presentation: 275613

## ICRi WG Feedback on  caBIG Enterprise Use Cases

**ICRi WG Feedback on  caBIG Enterprise Use Cases**

***July 23, 2008***

## Agenda 

**ICRi Participants**
**Clinical Use Case**
- Proposed revisions to Clinical Informatics Use Case
- ICRi commentary on Clinical Informatics Use Case
- Proposed revisions to Clinical Informatics Use Case Levels
**Translational Research Use Case**
- ICRi commentary on Translational Research Use Case
- Proposed revisions to Translational Research Use Case
- Proposed revisions to Translational Research Use Case Levels
**Appendix**
- Original Use Cases
- Additional Notes

**Notes:**
I will give a some background, go through the Clinical UC and then the Translational Research UC. At the end of the deck there is an Appendix which has the original use cases and some supplementary notes.

## ICRi Participants

**Participants**
- David Steffen – Working Group Lead, (Baylor)
- Buddy Dennis (UCLA)
- Kiran Keshav (Columbia)
- Jomol Mathew (Dana Farber)
- Leonard McMillan (UNC)
- Rakesh Nagarajan (Wash U)
- Daniel Rubin (Stanford)
- Grace Stafford (Jackson Lab)
- Baris Suzek  (Georgetown)
- Bruce Trock  (Johns Hopkins)
- Himanso Sahni (SAIC)

**Program**
- Elaine Freund
- Juli Klemm
- Li Kramer
- Subha Madhavan

**Notes:**
ICR Interoperability working group participants are listed, many names are familiar across the program, and there is a mix of incredibly talented clinicians, scientists and engineers all of who were involved in the review of this project.

## ICRi UC Review Process

- ICRi WG Lead met with ICR WS leadership.
- Written presentation and initial comments went out to ICRi participants
- Convened an ICRi teleconference. UC were reviewed in detail considering process and language
- Shared written commentary
- Convened an ICRi teleconference. UC reviewed again for process and language followed by a discussion of levels
- Written commentary

**Notes:**
The ICRi Working Group is chartered to identify, understand, and facilitate responses to informatics needs to support integrative and translational cancer research. One of their deliverables is to define use cases for interoperability and identify gaps limiting interoperability, so we brought them the enterprise UC.
As you can see there were many meetings and written exchanges. They looked at the step by step scenario/process in detail and then the computability levels - with the process/content being their higher priority.

## Revised Clinical UC

**Assumptions:**
- Look at process of clinical information from a clinical trial where, based on available information, a doctor assigns a patient to particular arm of a clinical trial.
- Assume tooling exists or works with tools available in the community.
**Revised Scenario:**
- A clinician consents a patient to participate in a breast cancer clinical trial.
- Clinician sends patient for baseline imaging to radiologist.
- Patient goes to an imaging center, and gets an MR image for her breast cancer.
- Radiologist report is sent back to the clinician.
- Patient provides bio sample to the pathologist.
- Pathologist sends sample to molecular lab service.
- Lab service runs labs and sends the results back to the pathologist (FSH. IHC, GxP, SNP, et al.).
- Pathologist collates result from labs and his/her analysis of the tissue and send the pathology report back to the clinician.
- Research coordinator uses pathology report and radiology report to determine eligibility.

**Notes:**
Here is the revised clinical UC. An assumption is that the tools exists. 
One of the important decisions to come out of the lengthy discussions was shifting the UC point of view. ICRi decided to (Read first bullet).  
Portions of the scenario that are revised are highlighted in yellow. The first assumption is captured as an additional line to the scenario at the very beginning. 
Genomic service was changed to molecular lab service to convey the breadth of molecular analyses available.
Clinician and research coordinator – they are different people and want to distinguish roles.

## Summary of Challenges

- Some of ICRi not comfortable considering caBIG tools supporting clinical care, but were for clinical research
- Others saw the distinction and wanted a clinical care component to be included with clinical research
- It was very important to ICRi to distinguish roles for each actor and use agreed upon terms to reflect accurately what each does
- ICRi is going to add a multi-center trial scenario**.**
- Levels
- Not able to be sufficiently expressive. More description and/or more levels are needed.
- Currently an incredible range of how different centers do the same tasks.

**Notes:**
Some of the folks in ICRi were not comfortable considering caBIG tools supporting clinical care, but they were for clinical research. Factors include widely implemented vendor solutions, the direction and maturity level of caBIG tools, and philosophical approaches of some centers. Others actually wanted a mix of clinical care and research since it reflects the direction they are headed. This resulted in the assumption shifting the point of view of the use case. 
This group paid was very thoughtful about phrasing, language and roles. Clinicians provide care and are different from research coordinators.
Multi center trial is worthy of it’s own scenario which ICRi is developing.

Levels not really sufficient and they had trouble agreeing on Level assignments in the UC. There is such a wide range in how the same tasks are carried out across centers – it was hard to reach consensus on what is typical at several key steps in the scenario.

## Levels

| Level | Description | Examples |
| --- | --- | --- |
| 0 | <NEEDED> | <NEEDED> |
| 0a | Verbal Communication | Non-permanent, e.g. verbal communication |
| 1 | Non-electronic data | Mail, phone |
| 1a | Non-electronic data | Paper documents |
| 2 | Machine-transportable data | PC-based and manual fax, secure e-mail of scanned documents |
| 3 | Machine-organizable data | Secure e-mail of free text or incompatible/proprietary file formats, HL-7 message  |
| 4 | Machine-interpretable data | Automated entry of LOINC results from an external lab into a primary care provider’s electronic health record  |

**Notes:**
ICRi would have liked more detailed descriptions and/or more levels to more accurately convey levels of communication and computability. Levels with an “a” next to a number is where they inserted, but then they stopped because of time constraints and the complex considerations. 

## Level Commentary- Clinical Scenario

- A clinician consents a patient to participate in a breast cancer clinical trial. (Adds at Level 0 & 1a, and several at Level 4)
- Clinician sends patient for baseline imaging to radiologist. (add at Level 0)
- Patient goes to an imaging center, and gets an MR image for her breast cancer. (Level 3) (one decrease to Level 2)
- Radiologist report is sent back to the clinician. (Level 2) (moves to Level 3, 4)
- Patient provides bio sample to the pathologist. (Level 0)
- Pathologist sends sample to molecular lab service. (Adds Level 0)
- Lab service runs labs and sends the results back to the pathologist (FSH. IHC, GxP, SNP, et al.). (Level 3) (one increase to Level 4)
- Pathologist collates result from labs and his/her analysis of the tissue and send the pathology report back to the clinician. (Level 2) (moves to Level 3 and 4)
- Research coordinator uses pathology report and radiology report to determine eligibility. (Adds at Level 0 and several at Level 4)

**Notes:**
They prioritized which items they would want to have higher level of computability by increasing those that were most important –The assumption is the goal is to have as many steps at Level 4 as possible. The collation is shown in red. Some steps have a wide range, and those that resonated the most are highlighted in yellow. 

## ICRi Commentary on Tln’l UC

**Use Case 2: “Translational Research Use Case”**
- Drew much more consensus.
- Address hypothesis generation and querying of electronic databases to assess feasibility. When IRBs are engaged varies.
- Distinguish different actors

**Notes:**
The translational research use case drew much more consensus. Mainly discussion centered around queries that would be done at the start of the process to assess feasibility of a study to address a hypothesis. In some cases researches might have access to de-identified data and bring a full proposal to an IRB. In other cases, a very high level proposal is made to an IRB, access to patient information is granted and the actual proposal is then fleshed out.
Roles again considered.

## Revised Translational UC

- Clinical Researcher D creates a hypothesis based on bench research, clinical research, and population research (this is from survey of the literature, searches of electronic databases, and interactions with colleagues.)
- A retrospective query is performed on a research data repository (with IRB approval if necessary) to determine feasibility of the study.
- D writes a clinical protocol for validation of a biomarker as predictive of tumor shrinkage in the context of treatment and submits it to the sponsor.
- The protocol is approved and the trial is initiated.
- Patients either apply for a study or are recruited by a clinician.
- Participants are screened on the basis of eligibility.  Screening data and sample are supplied to clinic trial coordinator (coordinator) by patient for enrollment evaluation.
- Coordinator sends sample to the basic scientist for biomarker screening.
- Basic scientist sends the result of biomarker screening to the coordinator.
- Based on eligibly criteria and biomarker results, the coordinator sends the enrollment notification to patient.
- Clinical coordinator refers patient to radiologist for imaging.

**Notes:**
The changes from the original use case are Highlighted in yellow. 
The second bullet reflects the feasibility study point. 
They added patients can apply to be part of a study. 
The rest of the changes are to separate out the clinician and clinical trial coordinator, who have different roles.

## Revised Translational UC – cont’d

- Radiologist acquires MR images.
- Radiologist analyzes images and creates image annotations and passes along to the coordinator.
- Clinical coordinator collects tissue samples and sends to the pathologist (for diagnosis and for distribution to the basic researcher (biomarker assessment).
- Basic researcher runs the biomarker assays and sends analysis results to coordinator.
- The pathologist sends staging and grading to the coordinator. (This is repeated throughout the study.)
- The coordinator starts the treatment, and the cycle of image collection, tissue collection and molecular data collection continue throughout the trial, and all information is available to the coordinator throughout the duration of the trial.
- The trial closes and all the data are provided to the statistician.
- Statistician communicates the clinical significance of and evidence for biomarker response prediction.
- The statistician, approved by the P.I., sends the report to the sponsor
- Clinical Researcher, basic scientist and statistician write a scientific paper reporting the results.
- Supporting data is made available via an appropriate repository as per the data sharing plan approved by the sponsor.

**Notes:**
The coordinator, PI and clinical research scientist are separated out, as they are usually different people. 
Finally the transfer of information at the end of trial is carefully worded since a sponsor can be pharmaceutical company.

## Level Commentary - Translational UC

- Clinical Researcher D creates a hypothesis based on bench research, clinical research, and population research (this is from survey of the literature, searches of electronic databases, and interactions with colleagues.) (Level 1) (increases to levels 1a, 2 and 4)
- A retrospective query is performed on a research data repository (with IRB approval if necessary) to determine feasibility of the study. (Adds to Levels 3 and 4)
- D writes a clinical protocol for validation of a biomarker as predictive of tumor shrinkage in the context of treatment and submits it to the sponsor. (Level 3)
- The protocol is approved and the trial is initiated.  (Add at Level 2)
- Patients either apply for a study or are recruited by a clinician. (Add at Level 1a)
- Participants are screened on the basis of eligibility.  Screening data and sample are supplied to clinic trial coordinator (coordinator) by patient for enrollment evaluation. (Level 3) (one increase to Level 4)
- Coordinator sends sample to the basic scientist for biomarker screening. (Level 0)
- Basic scientist sends the result of biomarker screening to the coordinator.   (Level 3) (increases to Level 4)
- Based on eligibly criteria and biomarker results, the coordinator sends the enrollment notification to patient. (Level 2) (one decrease to Level 1)
- Clinical coordinator refers patient to radiologist for imaging. (Level 1) (increase to Level 1a, 3, and a decrease to Level 0)

**Notes:**
On to level assessment.
One of the challenges in setting levels was the broad range of existing levels of computability at centers, so the base line is not always the same. Each person in the group was asked to address computability and reflect those items of higher priority by elevating the Level (noting that a Level 4 is often the desired end goal). The collated results are shown in red. Those items receiving the most changes are highlighted in yellow.
Read the two. 

## Level Commentary – Translational UC – cont’d

- Radiologist acquires MR images. (Level 3)  (one decrease to Level 2)
- Radiologist analyzes images and creates image annotations (Level 3) and passes along to the coordinator.  (unanimous increase to Level 4)
- Clinical coordinator collects tissue samples (Level 0) and sends to the pathologist (for diagnosis and for distribution to the basic researcher (biomarker assessment). (Level 0) (adds of Level 3 and 4 for associated documentation)
- Basic researcher runs the biomarker assays and sends analysis results to coordinator. (Level 3) (increases to Level 4)
- The pathologist sends staging and grading to the coordinator. (This is repeated throughout the study.)  (Level 2) (increases to Level 3 and 4)
- The coordinator starts the treatment, and the cycle of image collection, tissue collection and molecular data collection continue throughout the trial, and all information is available to the coordinator throughout the duration of the trial. (Level 2) (increases to Level 3 and mostly 4)
- The trial closes and all the data are provided to the statistician. (Level 3) (increases to Level 4)
- Statistician communicates the clinical significance of and evidence for biomarker response prediction. (Level 3) (increases to Level 4)
- The statistician, approved by the P.I., sends the report to the sponsor (Level 2) (increases to Level 4)
- Clinical Researcher, basic scientist and statistician write a scientific paper reporting the results. (Level 1)
- Supporting data is made available via an appropriate repository as per the data sharing plan approved by the sponsor. (Adds at Level 3 and 4)

**Notes:**
Again the priority items are highlighted in yellow. I think so many of these items are highlighted because of the mix of people commenting. Different groups had different priorities, but the ones highlighted cumulatively received the most comments. 

## Appendix: Original Use Cases

## 	
	Clinician B sends patient for baseline imaging to radiologist. Patient A goes to an imaging center, and gets an MR image for her breast cancer (Level 3, image file.)  Radiologist report is sent back to the clinician (Level 2) Patient gives sample to the pathologist (Level 0).  Pathologist sends sample to genomic lab service C (Level 0.) Lab service runs labs and sends the results back to the pathologist (FSH. IHC, GxP, SNP, et al.) (Level 3.)  Pathologists collates result from labs and his/her analysis of the tissue and send the pathology report back to the clinician (Level 2.)  Physician uses pathology report and radiology report to make diagnosis, and discusses treatment options with patient (Level 1.)  (These activities generally In practice occur as an iterative loop, and often non-concurrently.)

**	****Clinician B sends patient for baseline imaging to radiologist. Patient A goes to an imaging center, and gets an MR image for her breast cancer (Level 3, image file.)  Radiologist report is sent back to the clinician (Level 2) Patient gives sample to the pathologist (Level 0).  Pathologist sends sample to genomic lab service C (Level 0.) Lab service runs labs and sends the results back to the pathologist (FSH. IHC, GxP, SNP, et al.) (Level 3.)  Pathologists collates result from labs and his/her analysis of the tissue and send the pathology report back to the clinician (Level 2.)  Physician uses pathology report and radiology report to make diagnosis, and discusses treatment options with patient (Level 1.)  (These activities generally In practice occur as an iterative loop, and often non-concurrently.)**

**Clinical Informatics Use Case (Levels from Eric Pan’s Interoperability Taxonomy, extended by Mead et al.)**

## 	Clinical Researcher D creates a hypothesis based on bench research, clinical research,  and population research (this is from survey of the literature and interactions with colleagues. ) (Level 1.)  D writes a clinical protocol for validation of a biomarker as predictive of tumor shrinkage in the context of treatment and submits it to the sponsor (Level 3. PDF)  The trial is approved and initiates.  Patients are recruited directly (patients themselves) or indirectly (via clinician) for screening. Participants are screened on the basis of eligibility.  Screening data and sample are supplied to trialist by patient for enrolment evaluation (Level 3 (ex. CRF)).  Trialist sends sample to the basic scientist for biomarker screening (Level 0). Basic scientist sends the result of biomarker screening to the trialist (Level 3).  Based on eligibly criteria and biomarker results, the trialist sends the enrolment notification to patient (Level 2.)  Clinical trialist refers patient to radiologist for imaging (Level 1.) Radiologist acquires MR images (Level 3.)  Radiologist analyzes images and creates image annotations (Level 3) and passes along the trialist.  Clinical trialist collects tissue samples (Level 0) and sends to the pathologist (for diagnosis and for distribution to the basic researcher (biomarker assessment) (Level 0). Basic researcher runs the biomarker assays and sends analysis results (Level 3) to trialist. The pathologist sends staging and grading to the trialist (Level 2.)  (This is repeated throughout the study.)  The trialist starts the treatment, and the cycle of image collection, tissue collection and molecular data collection continue throughout the trial, and all information is sent back to the trialist throughout the duration of the trial (Level 3.) The closes and ends, and all the data is provided to the statistician, (Level 3.) Statistician communicates the clinical significance of  and evidence for biomarker response prediction (Level 3.)  The statistician, approved by the clinical trialist, sends the report to the sponsor (Level 2.)  Trialist, basic scientist and statistician write a scientific paper reporting the results. (Level 1)  Data goes back to the sponsor (Level 3.)

**Translational Research Use Case **

## ICRi Commentary on Clinical UC

**Intent of scenario is not clear.**
- Not hypothesis or question driven.
- Focus description to highlight where the issues of interoperability (data formats, communication standards, heterogeneous storage systems, etc) are key. Shifts the focus from how data moves to inventorying the full spectrum of data clinicians and researchers consume, where it comes from, and what barriers exist to making sense of it.
- Reads as purely clinical, standard of care treatment with no research component. Current caBIG reference standard implementations, while satisfactory in a research context for the most part, would fall far short of what is required for clinical standard of care software.
**Clinical care and clinical research data integration vary.**
- Clinical informatics illustrates the difference between clinical care and clinical research data, and standard of care data is not translational. Usually there are separate systems for clinical and research data and research does not have access to patient health information.
- Incorporation of research data in an EMR is the future.
**Conclusions**
- ICRi to comment on the process to support clinical decisions where a research finding ties back into current clinical support systems as rules, alerts, triggers**. **

## Levels Commentary

- COMMENTS:
- Not clear what the levels are communicating.  _We believe that different centers will have very different levels for each step in this use case._  For example, at some centers (e.g. BCM) consenting is entirely paper-based and thus is at level 1.  At other institutions as possible at Level 4.
- (e.g. UCLA) this is an electronic process and thus would be at level 3 or level 4.  Should these levels represent what can be done today with caBIG tools or what people want?  What we think is “typical”?  What does typical mean given the high level of variability.
- Utility is not clear since many institutions are targeting paperless systems and ultimately the goal is to have as many things
- INSTRUCTIONS:
- Update the levels associated with the specific (bulleted) items in the use case according to the table and considering Levels are for more highly specifying the use case.
- Specify at what level you want to execute that level in the use case, e.g. what is desirable.  Not what can be done today, but what we ideally would like to do.  Not what an institution can do today, but a typical use case grounded in reality. We should identify those things that will never be at Level 4 (e.g. not data), things that are significant pain points that we want to have at level four.  Other things we might compromise on in the short run at least.  Images maybe can never be at Level 4, for example.  Some things might never be intended to be only at level 4.

## ICRi Commentary on Scenario 2

**When an IRB is engaged varies and influences the hypothesis generation and feasibility process **(i.e. to see if there are enough patients to ask a particular question).
- In some cases a experiment is proposed to the IRB and on approval, access to information is granted to then do queries. A full protocol would not be developed without being sure it is a reasonable research question.
- In other cases where there is de-identified data, IRB approval for initial queries is not required.