Impact of Redesign of a Clinical Decision Allergy and Drug Interactions Alerts in an Electronic Prescribing System on Patient Safety – A Quantitative Descriptive Study

Background: Electronic medication management systems (EMS) generate medication alerts such as Drug-Drug interaction (DDI) and allergy at the drug order entry point for clinicians to improve patients’ safety. EMS that provide non-clinically significant alerts contribute to alert fatigue and pose a risk for patients’ harm. The primary aim is to assess the impact of redesign of allergy and DDI alerts on alerts’ trigger and overrides rates. The secondary aim is to assess the impact of the redesign of the alerts on reported patients’ harm. Methodology: A retrospective cross sectional 2 stage study was conducted. Stage 1 involved analysis of inpatients’ electronic drug orders in the hospital’s EMS that triggered an allergy, or a DDI alert from October to December 2019 in a 650 bed Australian hospital. A report on the 50 commonly overridden allergy and DDI alerts was reviewed by a multidisciplinary team to assess the clinical significance of the alerts using a risk matrix tool, frequency of overrides as well as published literature on adverse effects. Subsequently, non-clinically significant allergies and DDI alerts were deactivated in EMS system in March 2020. Stage 2 of the study involved the same analysis conducted in stage 1 (March to May 2021). The number of alerts overrides, alert trigger rates and number of related reported incidents involving patients’ harm were analysed. Results: A total of 288,267 and 288,133 prescriptions orders were reviewed in the 2 stages respectively. A total of 12 DDI and 37 allergy alerts were deactivated in stage 2. Redesign of the alerts reduced the trigger rate of allergy alerts (4.96% to 3.77%, P < 0.0001) and DDI alerts (5.30% to 4.73%, P < 0.0001). A statistically significant reduction in the number of incidents with reported patients’ harm related to overrides of alerts was observed in the post intervention phase. The allergy alert trigger rate was reduced from 4.96% to 3.77%, P = 0.0172. Conclusion: The study demonstrated that using an evidence-based approach and a risk assessment matrix to deactivate non-clinically significant alerts


INTRODUCTION
Electronic medication management systems (EMS) are used worldwide across multiple health settings.EMS with clinical decision support (CDS) aids have been shown to improve clinical practice, reduce medication errors and improve patients 'safety [1][2][3].They improve drug order legibility, standardize clinicians' documentations, and provide CDS aids at the point of drug order entry [4,5].
CDS generates alerts that act as a safeguard by providing an additional step where clinical judgement is required to complete the process.Common types of medication-related alerts include drug-drug interaction alerts, drug-allergy alerts, drug-laboratory alerts, duplicate drug alerts and drug dosing alerts [2,3].
Alerts may be displayed when a patient has been ordered treatment that may be related to a known documented allergy/adverse drug reaction (ADR) or possible interactions between prescribed medications [2,3].The prescriber may decide to change the treatment plan or proceed by overriding the alert [5].
A review article conducted by van der Sijs et al identified that up to 96% of medication alerts were overridden by the users [6].Moreover, studies suggest that on average more than half of alert overrides were classified as appropriate [7,8].When a large proportion of alerts are overridden or deemed to be irrelevant by the users, the prescribing process is hindered; and more importantly, 'alert fatigue' can occur [9].Alert fatigue is a state when users become less responsive to alerts and thereby reduces the efficacy of this decision support tool in promoting patient safety [9].
Drug-Drug Interaction (DDI) and allergy alerts are common types of alerts that can be triggered unnecessarily and contribute to alert fatigue [9,10].DDI and allergy alert rules in EMS are often based on general and sometimes outdated databases which have been reported to have low specificity and over-ratings of DDI and allergy severity due to legality considerations [11][12][13].Internationally, a few studies have attempted to implement local adaptation of third-party DDI and allergy databases to improve the specificity and clinical effectiveness of these alerts [14,17].In these studies, the absolute reduction in the percentage of alert overrides achieved vary from 0-50% [14][15][16]18].To date, no similar study has been conducted in Australia.
A recent study by Slight et al published on the economic cost of reported ADR from inappropriate medication related alerts overrides in the United States in the inpatient settings [19].The study was conducted on data retrieved from inpatients allergy alerts prescribing settings over a one-year period.It estimated that about 5.5 million medication-related alerts might have been inappropriately overridden, resulting in approximately 196 600 ADR [19].This was projected to cost between $871 million and $1.8 billion for treating preventable ADRs [19].The study also estimated that clinicians and pharmacists would have jointly spent 175 000 hours responding to 78.8 million alerts with an opportunity cost of $16.9 million.Additionally, a study published in 2021 by Khalil identified that well designed alerts in EMS resulted in patients and clinicians benefits as well as reduction in hospital acquired complications and cost of hospital admissions [20].It is in this context as well as the scarce data on allergy and drug interaction alerts data in the Australian settings that we conducted this study [19].
The primary aim of the study is to assess if a redesign of allergy and DDI alerts in the EMS reduce trigger alert and overrides rates by clinicians.The secondary aim of this study is to assess the impact of the redesign of the alerts on reported patients' harm.

METHODOLOGY
This retrospective cross-sectional study was conducted over 2 stages.Stage 1 involved retrospective analysis of medication orders that triggered an allergy, or a drug interaction alert over a 3-month period (October to December 2019) in a teaching hospital in Victoria, Australia.The hospital has 650 beds across multiple sites and service a population of over 168,000 in the Mornington Peninsula, Victoria.The hospital has the EMS (Cerner Millennium™) in place for 7 years and is a leader in the implementation of electronic medication management systems in Australia.
Allergy documentation was recorded upon admission in the patients' electronic medication health records.Alerts were generated electronically if a patient is prescribed a same or similar class of medication to the documented allergies in EMS.Additionally, a drug interaction alert was generated for new medication order that interacted with another charted medication.When an alert was generated in any of these two circumstances, the prescriber would review the alert and either cancel the order or continue with prescribing after considering risk and benefit to the patients.Refer to Figure 1.This quality Improvement study complied with the Standards for Quality Improvement Reporting Excellence (SQUIRE).

STAGE 1 (PRE INTERVENTION PHASE)
Stage 1 of the study included a retrospective data extraction from the EMS for the total number of medication orders and the number of orders triggering an allergy or a drug interaction alert from October to December 2019.Medication orders with allergy or drug-drug interaction alerts that were overridden by prescribers were also analyzed to establish the percentage of the overridden alerts.
Data on reported medication incidents related to overriding of alerts were retrieved from the hospital Victorian Hospital Incident Management System (VHIMS).VHIMS is an electronic hospital medication management system that records all patients' harm and near misses and used across all Australian hospitals in various iterations.Medication incidents that have caused a near miss harm incidents are encouraged to be submitted at this health service.However, all medication incidents with harm are mandated to be entered and verified in the hospital database.According to the Institute for Safe Medication Practices and the Australian Council on Healthcare Standards, near misses' medication incidents is a medication error that happened but did not reach the patient [21,22,23].Conversely, medication error with harm is defined as a medication error that reached a patient and required a clinical intervention beyond routine observation or monitoring [21,22,23].

THE INTERVENTION PHASE
The data from stage 1 informed the intervention phase.The usefulness of the 50 most overridden and non-clinically significant drug allergy alerts and DDI alerts were reviewed by a multidisciplinary review group of experienced health professionals in Feb 2020.Consensus was achieved by the review group through assessment of the risk associated with each drug combination using a risk matrix tool, frequency of overrides as well as published literature on likelihood of harm and adverse effects [25].All discrepancies in clinicians' opinions were discussed until a consensus is reached.The final list of recommended deactivated and/or modified alerts were presented to the hospital Medication Management Committee for review and endorsement before implementation in the EMS.After approval from the hospital digital team and the department of Health, the proposed change in alerts deactivation and/or modifications were upgraded in Cerner in March 2020 (Tables 1a, 1b and 2a, 2b).

Table 1a
Drug interaction alerts proposed for deactivation.

Reasons for deactivations
1. Drug-drug interaction not reported in most established databases.
2. Inconsistent report of potential risks from the drug-drug interaction in literature.
3. Lack of report on potential risks from the drug-drug interaction in literature.
4. Lack of clinical significance as concurrent use is well accepted in practice.

Figure 1
Prescribing process in EMS.

Table 1b
Drug interaction alerts proposed to remain unchanged .
1. Deactivation is deemed to be associated with high level of risk.
2. Documented evidence in the literature for contraindication.
3. Alert provides clinically significant advice for users.

STAGE 2 (POST INTERVENTION PHASE)
Stage 2 of this study included a retrospective data extraction and analysis on the total number of medication orders, the rate of alert triggers and overrides from March to May 2021.The 3 months duration of data collection of the post intervention phase was in line with similar published studies [24].The number of related reported incidents involving patients' harm or near misses were analysed as undertaken in Stage 1.
In both stages of the study, data collection was retrospective without direct contact with the clinicians to avoid the potential of Hawthorn effect.

STATISTICAL ANALYSIS
Descriptive statistics were used to analyze the audit data.Statistical differences in the outcomes from pre-and post-intervention stages were assessed using inferential statistical tests.Chi-square test was used to analyze categorical data (e.g., trigger rate and overrides rates).A p-value <0.05 was considered statistically significant.Statistical analyses were carried out with IBM SPSS Statistics (IBM Corp., Armonk, NY, USA).

RESULTS
A total of 288267 and 288133 prescriptions orders were recorded in the pre and post interventions phases respectively.A total of 12 DDI combinations were deactivated in the post intervention phase, while the other 38 alerts remained unchanged due to their clinical usefulness and evidence of potential harm if the alerts were deactivated (Table 1a and 1b).Additionally, a total of 37 allergy alerts were deactivated based on the multidisciplinary review group advice while the other 13 alerts remained unchanged due to insufficient or conflicting published evidence (Tables 2a and 2b).

ALLERGY ALERTS
The total number of triggered allergy alerts decreased in the post intervention phase from 14311 alerts to 10856 alerts.This corresponded to a reduction in the percentage of allergy alerts overridden by clinicians from 81.98% to 77.24%, p < 0.0001.Moreover, the allergy alert trigger rate was reduced from 4.96% to 3.77%, P < 0.0001 in the post intervention phase (Table 3).

DRUG-DRUG INTERACTIONS ALERTS
The total number of DDI alerts has slightly reduced in the post intervention phase from 15282 alerts to 13621 alerts.However, the rate of DDI alerts that were overridden remained the same (81.84%and 81.48 %, P = 0.41 pre and post intervention respectively).The DDI alert trigger rate reduced post the intervention phase from 5.30% to 4.73%, P < 0.0001 (Table 4).

REPORTED PATIENTS' HARM
The number of reported incidents related to clinicians overriding DDI and allergy alerts for the 3 months reported pre intervention are 5 and 7 respectively.Post the intervention phase, the number of DDI and allergy alerts overrides were reduced to 0 and 1 respectively (Table 5a and 5b).This corresponds to a significant reduction in the total number of incidents related to these alerts in the post intervention phase, RR = 0.0837(0.0109-0.6440),P = 0.0172 (Table 6).

DISCUSSION
This study identified that using an evidence-based risk assessment approach to deactivate non clinically significant allergy and DDI alerts reduced the alerts trigger rates in electronic medication management system at this hospital.This systemized method of deactivating alerts in EMS has subsequently reduced the rate of reported patients' harm in this health service.This study has shown that deactivating non clinically significant allergy alerts in EMS significantly reduced the number of triggered and overridden allergy alerts.The trigger rate of drug allergy alerts per 100 medication orders dropped from 4.96 alerts/100 orders to 3.77 alerts/100 order, P < 0.0001.This corresponded to a reduction of unnecessary allergy alerts by 1152 alerts per month.As a result, a higher percentage of drug allergy alerts were being accepted by clinicians from (82% to 77%).The new alert rate identified observed in anotherthis study is like other similar studies published in a systemic review by Poly et al. [24] Additionally, similar results were also observed by Brodowy et al, which reported an improvement of allergy alerts override rate from 94% to 90% after removal of non-clinically significant allergy alerts [26].
It is likely that deactivating the non-clinically significant allergy alerts has contributed to clinicians clinically assessing the alerts before accepting or overriding them.Conversely, deactivating the non-clinically significant DDI interactions has not influenced the percentage of DDI alerts overridden, despite a reduction corresponded to of 553 less DDI alerts that are triggered per month reduction corresponded to 553 less DDI alerts that are triggered per month.This could be due to the large number of inbuilt DDI into the EMS and the need to further refine and deactivate a larger number of the DDI alerts to observe a significant effect on the overall rate of DDI alerts overridden.Additionally, the lack of published literature on clinical outcomes for deactivating various DDI alerts and higher level of assessed clinical risk as deemed by the review working group resulted in a smaller number of DDI alerts being deactivated.
Results from this study align with other published evidence that demonstrated a reduction in alerts trigger rate but not alerts overridden rate, which is likely explained by the magnitude of alerts deactivation [14,15].Contrarily, a study by Cornu et al identified an improvement in DDI alerts override rate from 98% to 48% due to adoption of context-specific DDI alerts, interruptive alert interface in addition to deactivation of low severity DDI alert rules [16].Lower level of alert fatigue is a main contributing factor to the results observed in Cornu's study, where the DDI alert trigger rate was 10.4 alerts/1000 orders post intervention, compared to 4.73 alerts/100 orders in our study.Correlating to the findings of other studies, it is suggested that lowering alerts trigger rate to a clinically significant level is essential to reduce the override rate.
This study has also assessed the relationship between overriding allergy and DDI alerts and reported patients' harm.Deactivating non-clinically significant allergy and DDI alerts has statistically reduced the number of reported incidents related to overriding alerts in our study, P = 0.0172.This finding aligns with well published studies, which have shown that well designed drug safety alerts in EMS can reduce incidence of adverse drug events and improve patient safety [20,27].Furthermore, analysis of expected impact of drug alerts by Weingart et al suggested a reduction of 402 adverse drugs events, 39 hospitalizations and 3 deaths annually [28].
There is a scarcity of literature investigating the impact of redesign alerts in EMS on clinical patient outcomes.Simpao et al in their study deactivated clinically irrelevant DDI alerts in the EMS using a visual analytics dashboard [15].This resulted in a reduction of medication serious safety event rate from 0.18 events per 10,000 adjusted patient days to 0.08 after the study period.We observed a larger reduction in the reported medication incidents rate in our study.It is difficult to compare incident rates between studies due to several reasons.Firstly, the redesign of alerts system in our study involved both allergy and DDI alerts.Secondly identification of medication incidents relies on the voluntary reporting by clinicians, which could be variable across different  settings and health services.Lastly near misses are included in our medication incident rate but excluded in the study by Simpao et al [15].Further research is required to explore users' experience with upgraded alerts in the EMS system to assess clinicians'and perception with theiron the usefulness of the upgraded alerts.Additionally, concepts used to deactivate alerts in this study can potentially be used to deactivate the entire library of built-in DDI and allergy databases in the EMS or other types built-in CDS alerts (such as Venous thromboembolism and renal and drug prescribing alerts) rendering the electronic system much user friendly.
There are several limitations to this study.Firstly, alert fatigue' cannot be measured directly.Therefore, markers such as the alert override rate and self-measured alert adherence were used instead.There is no optimal level of alert overridden rate reported in the literature; though a lower alert override rate is likely to associate with less non-clinically relevant alerts and hence reduced 'alert fatigue'.A proportion of allergy alert overrides are inevitable for alerts that arise from documented intolerance or mild adverse effects of a similar medication in the patients 'notes and is dependent on the clinician accuracy of documentation.However, it is important to acknowledge that not all overridden alerts are useless.In some instances, the prescriber may override an alert judiciously and appropriately as alternative management strategies have been employed (eg.Additional monitoring, patient education or dose change).Measuring these instances is challenging if it is not documented well in the patients' notes.Secondly, although measuring patients' harm is useful, recording near misses with the potential for harm is also important.Reporting of near misses in this health service is voluntary and may have contributed to observed underreporting of near misses' cases in both phases of the study.Thirdly, other changes/updates implemented within the EMS during the study period may potentially confound the results.Finally, it is important to note that accuracy of ADR and medication history taking is highly variable amongst clinicians.Detailed history taking and investigation processes are vital to ensure accurate documentation of all ADR and prescribed medications.Therefore, the accuracy of the allergy and DDI alerts and overrides are dependent on user accuracy for entering the initial information.

CONCLUSION
This study has demonstrated that using an evidencebased approach and a risk assessment matrix to deactivate non clinically significant clinical decision support aids in electronic prescribing systems reduce alerts trigger rates and potentially contribute to a decrease in resulting patients' harm.

Table 2a
list for drug allergy alerts proposed for deactivation Reasons for deactivations 1. Different chemical structure.2.Alternative option to those allergic to Drug A. 3. Documented evidence in the literature.

Table 2b
Drug allergy alerts proposed to remain unchanged.1. Insufficient evidence in the literature.2. Documented evidence in the literature for contraindication.Khalil and Hua International Journal of Digital Health DOI: 10.29337/ijdh.40

Table 3
outline of number and percentage of allergy alerts overridden and allergy alert trigger rate pre and post intervention stages.

Table 4
outline of number and percentage of DDI alerts overridden and DDI alert trigger rate pre and post intervention stages.

Table 5a
Medication incidents due to overriding DDI alerts.

Table 5b
Medication incidents due to overriding drug allergy alerts.

Table 6
Number of reported incidents related to overriding of allergy and DDI alerts.