# PRODUCT REVIEW / ÉVALUATION DE PRODUIT

**Product:** Isabel Pro – the DDX Generator

**URL:**
https://www.isabelhealthcare.com/products/isabel-pro-ddx-generator

**Cost:** Free 30-day trial, Standard individual subscription USD $149/yr Premium individual subscription USD $219.99/yr Institutional subscription also available

## Purpose and Intended Audience

Isabel Healthcare began in 1999 as a non-profit organization after its founder Dr. Mark Graber's daughter suffered a near fatal misdiagnosis. In 2004, the organization converted to a for-profit business to support individual healthcare providers, and institutions including hospitals and medical schools. Isabel Pro or differential diagnosis (DDX) generator (henceforth, Isabel Pro) is a web-based diagnosis decision-making tool as part of the Isabel product suite of diagnosis decision support systems. Instead of rules-based algorithms, Isabel Pro uses statistical natural language processing (SNLP) to provide diagnosis retrieval for physicians and healthcare providers to avoid "System 1 diagnosis errors" in clinical settings. Sample cases can also be built to support the teaching of medical students and residents. Patients can use the free Isabel Symptom Checker to make sense of and research their symptoms. It allows patients to describe their symptoms in plain language and helps patients figure out their next steps (e.g. visit a clinic, visit the emergency, etc.). This product review is intended for the former product, Isabel Pro.

## Product Description

Isabel Pro is a software tool that uses SNLP engine to search in a database of disease presentations or illness scripts for physicians. Isabel Pro determines likely diagnosis based on the patient's demographics

and clinical features, including time-sensitive "Don't Miss Diagnoses."

Symptoms can be searched using natural language in a database of over 10 000 diagnoses of which 6 000 are diseases and 4 000 are drugs. The database is manually built and populated with knowledge about each disease from different sources. It does not give a sufficient precision but, instead, presents different potential diagnoses and match rates.

## Isabel Pro Features and Usability

The interface is intuitive to use. It takes information entered and matches it across electronic sources to create a checklist of potential diagnoses. For teaching purposes, cases can also be generated and uploaded to Isabel Pro to support students in the diagnosis decision-making process. Additional training and support are also available from Isabel representatives.

### Search

The search feature has the ability to retrieve results using natural language. Physicians can begin the differential diagnosis process by entering patient data (e.g. age, gender, pregnancy status, travel history) into the "Clinical Features" box. Next, physicians can enter "Abnormal Clinical Features" including chief complaints, common medical abbreviations such as SOB (i.e. shortness of breath), labs, vitals, and comorbidities extracted from the patient's medical record. The search box can also directly link to a library's subscription of evidence-based resources.

### Ranked Diagnoses

By clicking on the "Get Checklist" button, the system generates a ranked list of diagnoses arranged by a color bar of "Likelihood Indicator." Outputs are filtered for the relevant age, gender, and region of the patient. The list can also be sorted by specialty or red flags (i.e. conditions requiring immediate attention). In a separate tab, drug side effects that may have caused the symptoms are also listed.
<figure><img src="image_1.png" /><figcaption>**Fig.1.** Isabel Pro user interface</figcaption></figure>

## Results

In the options listed on the right of the ranked diagnoses, every generated differential diagnosis can be emailed, printed, saved, or copied. The interface also allows users to provide direct feedback to Isabel representatives through a “feedback” icon on the page.

## Other Features

Medical publishers have also partnered with Isabel Healthcare to provide links to resources a library may have, including Minute Consult from Wolters Kluwer,

DynaMed Plus from EBSCO, and Best Practice from the British Medical Journal (BMJ).

## Sample Case<fnref n="1" />

### Case description:

A 34-year-old woman who was 14 weeks pregnant presented to the emergency department with 5 days of

<footnote n="1">¹ Case modified from Isabel case template retrieved from http://www.isabelhealthcare.com</footnote>
nonspecific abdominal pain, nausea, vomiting. On examination, she appeared well with normal vital signs and had some mild diffused abdominal tenderness.

Test results:

- White blood cell count of 19 000 cells/uL

- Urinalysis shows positive for nitrates and leukocytes esterase

As test results are added, Isabel Pro produces a modified ranked list of diagnoses. Acute appendicitis and peritonitis move up on the ranked list and highlighted in dark orange as "most likely" diagnoses.

Knowledge Resources

Users can click on each diagnosis to access knowledge resources integrated into Isabel Pro.

<figure><img src="image_2.png" /><figcaption>Fig.2. Isabel Pro results based on initial clinical features</figcaption></figure>
<figure><img src="image_3.png" /><figcaption>**Fig.4** Pop-up window of knowledge resources for a selected diagnosis</figcaption></figure>

<figure><img src="image_4.png" /></figure>
## Product Integration

### Knowledge and Library Resources

The tool has the capability to integrate with many popular databases, clinical tools, and library resources such as PubMed, Embase, MEDLINE, UpToDate, DynaMed, Lexicomp, Micromedex, CDC, and TRIP. When integrated with DynaMed or UpToDate, results will link directly to relevant DynaMed or UpToDate topics. If neither is available, the links will resort to PubMed.

### Integration into EMR

Isabel Pro can be integrated in an electronic medical records (EMR) system. Data from pre-assigned EMR fields can be submitted to Isabel Pro using the Isabel User Interface. Current EMR vendors that offer an Isabel interface are Epic, Cerner, NextGen, Allscripts, SystmOne, T-Systems, Better Day, and VersaSuite.

## Strengths and Weaknesses

Isabel Pro is selected by the American Medical Association (AMA) as the diagnosis tool for its portal and endorsed by the British Medical Journal (BMJ). The tool undergoes continuous validation processes, including controlled trials and peer-reviewed studies, covering various aspects of the system. For users, the Isabel Healthcare support team provides on demand support, customizable interfaces using the Isabel API, and seamless integration with a library's databases, clinical tools and resources, or EMR system. Users can also capture and print what they learned in Isabel Pro for a credit to be issued towards their continuing medical education (CME).

While Isabel Pro is intuitive and user-friendly, knowledge of medical terms may be a drawback for some users. It can also be costly for individuals and institutions on top of their existing subscriptions. For the full potential use of the tool, it is best integrated with other resources, most of which require a library subscription.

## Peer Review and Validation Studies<fnref n="2" />

A number of Isabel Healthcare and independent studies have studied the usefulness of Isabel Pro for attending physicians, residents, nurses, and medical students in the diagnosis decision-making process. A summary of findings is listed below.

* Ramnarayan et al. (2007) found that Isabel performs with an acceptable degree of clinical accuracy. In particular, Isabel was helpful for reminding junior doctors of key diagnoses in the emergency department. Isabel displayed the final discharge diagnosis in 95% of inpatients and 90% of “must-not-miss” diagnoses suggested by the expert panel.

* Bond et al. (2012) tested four diagnosis programs and found that the performance testing supports the use of Isabel and DxPlain.

* El-Kareh et al. (2013) reported accuracy rates for DDX generators in the range of 70-95%. However, they suggest the development of alternative metrics to measure diagnostic performance.

* In a pilot study, Henderson and Rubin (2013) found that Isabel was of limited utility for secondary medical care in the general practice setting in the UK

* In an audit study, Semigran et al. (2015) examined the clinical performance across 23 symptom checkers and a wide range of conditions in the UK, US, the Netherlands, and Poland. Their study found that symptom checkers had a deficit in diagnoses and triage. Isabel achieved 44% accuracy for correct symptom ranked first and 69% ranked in the top three.

* Riches et al. (2016) highlight in their systematic review and meta-analysis that DDX generators have the potential to improve diagnostic practice and reduce diagnostic error.

The literature on DDX generators is complex, with a variety of study designs. The lack of standardized appraisal and criteria for DDX generators often result in poor quality and insufficient evidence from the existing literature to recommend the routine use by

<footnote n="2">² Visit https://www.isabelhealthcare.com/validation/peer-reviews for other peer reviews and clinical studies.</footnote>
physicians. DDX generators are not intended to replace physicians but rather augment the diagnosis decision-making process. Further studies that examine the efficacy, ranking of diagnoses, cost-effectiveness, and time should be addressed before further recommendations can be made.

## Product Comparisons

DDX generators have moved away from rules-based systems that were developed in the 1970s and 1980s. Tools, such as DxPlain, QMR, Meditel, and Diagnosis Pro, tend to associate each symptom with a particular disease and assigned probability. On the other hand, Isabel Pro is part of a new generation of tools that uses natural language pattern recognition through a database of diagnosis presentations to provide a list of ranked diagnoses. A competing tool similar to Isabel Pro is VisualDx, which is a system primarily based on digital images and allows physicians and clinicians to build visual differential diagnosis based on the patient's test findings.

## Compatibility

Isabel Pro is web-based and compatible with all browsers and mobile platforms. While no mobile app is available, users can save a shortcut of a web link to their device's home screen for easy access.

## Currency

Database contains 10 000 diagnoses including 6 000 diseases and 4 000 drugs. Records are updated on a weekly and monthly basis with submissions from 30 different sources, including Isabel Healthcare, users, and physicians.

## Cost/Value

Isabel Pro is a subscription-based product. Annual individual subscriptions are available for two versions: standard USD $149/yr and premium USD $219.99/yr. The premium package includes access to 5 Minute Consult. Institution subscriptions are negotiated for each institution. Discounts are also available for students.

## Contact Information

To discuss your library's needs, please contact an Isabel Healthcare representative for an inquiry or visit https://www.isabelhealthcare.com/contact-isabel-healthcare.

## Acknowledgement

Thanks to Patrick Garrett, Isabel Sales Associate, for providing a product demo and addressing follow-up questions.

## Statement of Competing Interests

No competing interests declared.

## References

1. Amy, L. R., Borowitz, S. M., Brown, P. A., Mendelsohn, M. J., & Lyman, J. A. (2006). Impact of a Web-based diagnosis reminder system on errors of diagnosis. *AMIA ... Annual Symposium proceedings. AMIA Symposium*, 2006, 843.

2. Bond, W. F., Schwartz, L. M., Weaver, K. R., Levick, D., Giuliano, M., & Graber, M. L. (2012). Differential diagnosis generators: an evaluation of currently available computer programs. *Journal of General Internal Medicine*, 27(2), 213-219.

3. El-Kareh, R., Hasan, O., & Schiff, G. D. (2013). Use of health information technology to reduce diagnostic errors. *BMJ Quality & Safety*, 22(Suppl 2), ii40-ii51.

4. Henderson, E. J., & Rubin, G. P. (2013). The utility of an online diagnostic decision support system (Isabel) in general practice: a process evaluation. *JRSM Short Reports*, 4(5), 1-11.

5. Ramnarayan, P., Cronje, N., Brown, R., Negus, R., Coode, B., Moss, P. Britto, J. (2007). Validation of a diagnostic reminder system in emergency medicine: a multi-centre study. *Emergency Medicine Journal: EMJ*, 24(9), 619-624.

6. Riches, N., Panagioti, M., Alam, R., Cheraghi-Sohi, S., Campbell, S., Esmail, A., & Bower, P. (2016). The effectiveness of electronic differential
diagnoses (DDX) generators: a systematic review and meta-analysis. *PloS one*, 11(3), e0148991.

7. Semigran, H. L., Linder, J. A., Gidengil, C., & Mehrotra, A. (2015). Evaluation of symptom checkers for self diagnosis and triage: audit study. *British Medical Journal*, **351**, h3480.

Lili Yuxi Ren

*Reference Librarian*
The Hospital for Sick Children
Toronto ON
Email: lily.ren@sickkids.ca