Evaluation of clinical decision support provided by medication review software
Creators: name: Curtin, CM
Publisher: name: University Of Tasmania
Year: 2023

description: Aim The purpose of this investigation was to evaluate the clinical decision support capacity of commercial computer software designed to assist pharmacists performing medication reviews. The primary hypothesis was: If medication review software is related to pharmacist knowledge, then the detection of therapeutic problems will result in a similar frequency and scope of identified problems as those identified by pharmacists. Method Home medication review data collected during 2008 for a previous study were used for this investigation. The data contained original pharmacist findings of drug-related problems (DRPs), patient demographics, medications, laboratory results and diagnoses. Two commercial software applications advertising decision support were assessed, Monitor-Rx (MRX) utilising simple rules triggered by the presence of medication and Medication‚Äövë¬¢ Review Mentor (MRM) utilising an advanced artificial intelligence rules-based approach. The previously collected data were entered into each of the applications and the DRPs identified by each tool were recorded. Additionally, published prescribing criteria, Beers (2003 and 2012 versions), Screening Tool of Older Person's Prescriptions and Screening Tool to Alert doctors to Right Treatment (STOPP/START) and Prescribing Indicators in Elderly Australians (PIEA) were also adapted so as to be applied computationally over the same set of patient data. DRPs were assigned broad DOCUMENT classifications and examined by frequency and type. A common vocabulary of descriptive classifications capturing essential DRP concepts was developed to allow detailed comparison between the various DRP sources. The ability of software to identify the same classifications in the same patients as pharmacists was assessed as a crude measure of clinical relevance. A panel of pharmacology experts assessed the DRPs identified by pharmacists, MRM, MRX and the STOPP/START prescribing criteria for their opinions concerning clinical relevance, excessive DRP findings, missed DRPs and the appropriateness of recommendations. A qualitative survey of pharmacists who used MRM was also undertaken to obtain their opinions of the decision support capability of MRM. Results In total, across 570 patients, pharmacists identified 2020 DRPs, MRM 3209, PIEA 1492, STOPP 1032, Beers03 404 and Beers12 399. Ten percent of the volume of DRPs identified by MRM were found to be duplicated DRPs, where the same essential problem was identified more than once for a patients, typically via different rules. Using a smaller sub-sample of 100 patients, MRX identified 1265 DRPs. Pharmacist DRPs encompassed the widest range of DOCUMENT classifications, followed by MRM, then the sets of prescribing criteria and finally MRX. A list of 141 descriptive classifications was developed which described the various DRP concepts in depth. Pharmacist-only descriptive classifications involving compliance and not-classifiable DRPs were excluded from assessment, since it was impossible to detect these DRPs without access to additional patient data that was not included in pharmacists written reports. Pharmacist DRPs were associated with 113 different descriptive classifications, MRM 100 and MRX 17. MRM was able to identify 90 differing classification types that were also identifiable by pharmacists. MRM was able to identify the same problems in the same patients as the reviewing pharmacists identified in 389 instances, whereas MRX identified the same problems in the same patients in only 11 instances. Assessment of expert opinions found that experts generally agreed that MRM presented clinically relevant DRPs (80%) and appropriate recommendations for DRP resolution. This finding contrasted strongly for MRX, with experts of the opinion that MRX presented few clinically relevant DRPs (13%). Similarly, relatively few experts agreed that MRM presented too many DRP findings (19%) whereas the vast majority of experts agreed MRX presented an excessive number of findings (93% of opinions). Pharmacists who used MRM agreed with the expert panel regarding MRM. These pharmacists also found MRM to be easy to use (mean 76 on scale of 0 to 100) and useful (mean 5.6 on a scale of 1 to 7). The pharmacists agreed that MRM identified clinically relevant DRPs (73%) and also agreed that MRM identified clinically relevant DRPs that would otherwise have been overlooked by pharmacists (73%). Discussion Both MRM and MRX identified a greater number of DRPs than pharmacists. However, MRM was considered, by both the expert panel members and pharmacist subscribers, to identify an acceptable number of mostly clinically relevant DRPs and to present appropriate recommendations to resolve DRPs. In contrast, the expert panel thought MRX identified an excessive number of mostly irrelevant DRPs. The contrast between the relevance of MRM and lack of relevance of MRX highlighted the different approaches used by each product. MRX utilised a simple approach through the identification only of medications of interest, whereas MRX incorporated a range of variables such as examining all medications, medication doses, medical history, laboratory results. The integration of a range of variables allowed MRM to provide far greater context to the DRPs found for each patient. Application of the DOCUMENT classifications and the descriptive classifications found that MRX was capable of identifying a very limited range of DRP types, whereas MRM was capable of identifying a wide range of DRP types, approaching the range of problem types identifiable by pharmacists. The descriptive classification comparison of MRM and MRX findings with the sets of prescribing criteria found that MRX was more closely aligned with the Beers12 criteria, whereas MRM was more closely aligned with the STOPP/START criteria. Interestingly the STOPP/START criteria were also considered, by the expert panel, to provide clinically relevant findings. This may be due to patient contextualisation via the incorporation of medication-medication interactions and medication-diagnosis interactions within many of the STOPP/START criteria. However, compared to MRX, MRM and pharmacists' original findings, STOPP/START found the smallest number of problems. These findings were naturally limited to the set of 74 criteria which implemented in this investigation. An additional finding was the STOPP/START criteria were found to be the closest of all the sets of prescribing criteria to the pharmacists' findings, both in terms of scope of problem types as well as by frequency. Greater contextualisation of DRPs certainly provided greater clinical relevance, as shown with the STOPP/START prescribing criteria and exemplified by MRM. However, STOPP/START was limited to a set of specific consensus-based rules, limiting opportunities to expand on the identification of clinically relevant DRPs. MRM did not have this limitation, allowing an expert in the the knowledge domain of medication reviews to add and refine numerous rules incorporating patient-specific data to maximise the detection of clinically relevant DRPs. A strong rationale for the use of MRM, or similarly implemented technologies, was not only MRM's clinical relevance but also MRM's practical usefulness in the detection of missed opportunities ‚Äö- pharmacist subscribers confirmed that MRM did identify clinically relevant DRPs that the pharmacists themselves had missed. Conclusion The implementation of overly simple rules, such as the mere presence of a medication as was used by MRX, was insufficient to provide good decision support as it resulted in an excessive abundance of a narrow range of mostly clinically irrelevant DRPs. Automated versions of the prescribing criteria STOPP/START, PIEA, and Beers proved to be a slightly better tool for identifying DRPs as took into account a broader spectrum of information about the patient's condition (typically diagnoses), allowing them to identify more targeted and relevant problems. However, the use of artificial intelligence technology by MRM allowed for both greater contextualisation and variety of clinically relevant DRPs. MRM identified a higher frequency of DRPs than did pharmacists. In part, the greater frequency of DRPs identified by MRM may represent clinically relevant DRPs that were missed by reviewing pharmacists. This capacity to supplement clinically relevant DRPs complements the consistency and thoroughness of the pharmacists medication reviews. In light of the performance of MRM seen in this research, it is reasonable to expect that future clinical decision support system (CDSS) applications using the multiple-classification ripple-down rules (MCRDR) approach could provide significant benefits over the much simpler technologies that are typically utilised, with very limited success, worldwide. These benefits include the identification of clinically relevant problems both more frequently, and more consistently, yet with very few clinically irrelevant problems identified. Furthermore, this consistently high performance level leads to better uptake and acceptance rates by users, ensuring that the problems are not only identified, but are actually acted upon when appropriate to do so. Such CDSS implementations might be successfully incorporated into a wide variety of healthcare settings, such as hospital, general and specialist practice, and community pharmacy. Given the results of this thesis, that the technology now exists, and that quality patient electronic health record (EHR) data is gradually becoming more available, it seems that the time has come for this technology to be applied more widely.