Abstract
Background: Variability in antimicrobial prescribing may indicate an opportunity for improvement in antimicrobial use. We sought to measure physician-level antimicrobial prescribing in adult general medical wards, assess the contribution of patient-level factors to antimicrobial prescribing and evaluate the association between antimicrobial prescribing and clinical outcomes.
Methods: Using the General Medicine Inpatient Initiative (GEMINI) database, we conducted a retrospective cohort study of physician-level volume and spectrum of antimicrobial prescribing in adult general medical wards in 4 academic teaching hospitals in Toronto, Ontario, between April 2010 and December 2019. We stratified physicians into quartiles by hospital site based on volume of antimicrobial prescribing (days of therapy per 100 patient-days and antimicrobial-free days) and antibacterial spectrum (modified spectrum score). The modified spectrum score assigns a value to each antibacterial agent based on the breadth of coverage. We assessed patient-level differences among physician quartiles using age, sex, Laboratory-based Acute Physiology Score, discharge diagnosis and Charlson Comorbidity Index. We evaluated the association of clinical outcomes (in-hospital 30-day mortality, length of stay, intensive care unit [ICU] transfer and hospital readmission) with antimicrobial volume and spectrum using multilevel modelling.
Results: The cohort consisted of 124 physicians responsible for 124 158 hospital admissions. The median physician-level volume of antimicrobial prescribing was 56.1 (interquartile range 51.7–67.5) days of therapy per 100 patient-days. We did not find any differences in baseline patient characteristics by physician prescribing quartile. The difference in mean prescribing between quartile 4 and quartile 1 was 15.8 days of therapy per 100 patient-days (95% confidence interval [CI] 9.6–22.0), representing 30% higher antimicrobial prescribing in the fourth quartile than the first quartile. Patient in-hospital deaths, length of stay, ICU transfer and hospital readmission did not differ by physician quartile. In-hospital mortality was higher among patients cared for by prescribers with higher modified spectrum scores (odds ratio 1.13, 95% CI 1.04–1.24).
Interpretation: We found that physician-level variability in antimicrobial prescribing was not associated with differences in patient characteristics or outcomes in academic general medicine wards. These findings provide support for considering the lowest quartile of physician antimicrobial prescribing within each hospital as a target for antimicrobial stewardship.
Antimicrobial resistance is an important global health threat that is associated with increased infection-related morbidity and mortality, incremental health care costs and diversion of health care resources away from other priority areas.1–3 The key modifiable driver of antimicrobial resistance in humans is antimicrobial prescribing and overuse.4 Variability in antimicrobial prescribing may signal overuse and an opportunity for improvement.5–10 Defining the magnitude, consequences and possible contributors to variability in antimicrobial prescribing is the requisite initial step in identifying ways to improve practice.
Despite substantial study of variability in antimicrobial prescribing in the community and long-term care settings,9,11–14 few studies have evaluated antimicrobial prescribing variation among patients admitted to hospital, specifically to general medicine wards. Patients cared for in general medicine wards represent a large percentage of all hospital admissions in Canada. In the pre-COVID-19 era, patients admitted for 6 of the top 10 reasons for hospital admission were commonly treated in general medicine wards.15 Further, antimicrobial use is common on general medicine wards, with about 30% of patients receiving antimicrobial therapy.16,17
We sought to evaluate physician-level variability in antimicrobial prescribing for patients admitted to general medicine wards, the contribution of patient-level factors toward this variability and the association between physician-level antimicrobial prescribing practices and patient outcomes.
Methods
Study design and setting
This retrospective cohort study included adult patients admitted to general medicine wards at 4 academic teaching hospital sites (Mount Sinai Hospital, St. Michael’s Hospital, Toronto Western Hospital and Toronto General Hospital) participating in the General Medicine Inpatient Initiative (GEMINI) in Toronto, Ontario, Canada.18 The GEMINI database compiles clinical and administrative data across participating hospitals in Ontario. We chose these 4 hospitals as they were among the initial 7 hospitals to provide data to GEMINI and had validated pharmacy drug data for analysis at the time of study.
Hospitals in Ontario are publicly funded and provide general and specialized inpatient care to patients from all socioeconomic backgrounds. In GEMINI hospitals, general medicine admissions account for about 40% of all patients admitted to hospital and 25% of all bed-days.18
The general medicine service model for the hospitals in this study has been previously described18 and involves care provided by clinical teaching teams (medical students, residents, staff physicians and allied health professionals) under the supervision of the most responsible physician (MRP). Service blocks for the MRP are typically 2 weeks in length but may vary from 1 to 4 weeks.19 Patients admitted to the general medicine service are assigned to MRPs from the emergency department based on the rotational admission schedule, without preference for a specific MRP.19,20 This supports assignment of patients to any MRP regardless of diagnosis or likelihood of need for antimicrobials.
Other clinicians may also affect antimicrobial prescribing in the participating hospitals, including pharmacists, infection prevention and control providers and infectious disease consultants.
We reported this study in accordance with the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement.21
Eligibility criteria
We assessed eligibility of all patients admitted to general medicine wards between Apr. 1, 2010, and Dec. 31, 2019. This time period represents the earliest data available in the GEMINI database and excludes the effect of the SARS-CoV-2 pandemic on antimicrobial prescribing. We included adult patients (aged ≥ 18 yr) admitted to general medicine wards when the MRP was either an internist (including internal medicine subspecialists) or a family medicine–trained hospitalist attending on the inpatient general medicine service. We attributed patient-days to physicians using the Canadian Institute for Health Information (CIHI) definition for an MRP (i.e., “responsible for the care and treatment of the patient for the majority of the visit to the health care facility”).22 For example, for an admission of 15 days in which the patient was cared for by 2 MRPs during the stay, the assigned MRP would be the physician with 8 or more days caring for the patient.
We excluded patients not admitted directly from the emergency department, patients with a total hospital length of stay greater than 30 days (to increase attribution accuracy, as this was the longest length of MRP service blocks and allowed fewer transitions of MRPs) and any patient whose MRP had fewer than 100 admissions to general medicine over the period of study (to ensure statistical stability and exclude clinicians with little service time and, thus, nonrepresentative antimicrobial prescribing).19 For patients transferred to the intensive care unit (ICU) during their admission, we censored ICU-related antimicrobial prescribing data and patient-days and limited analyses to prescribing data and patient-days to the period during which they were managed on the medical ward. Patients could be included in the study more than once if they had more than 1 admission to a participating hospital.
Data sources
The GEMINI database includes patient-specific demographic, administrative and clinical data that are collected from electronic health records of participating hospitals and are typically used for reporting to CIHI. These data are highly reliable and have accuracy ranging from 98% to 100% in a manual validation of more than 20 000 data points.23 At the time of this analysis, GEMINI data included nearly 250 000 patient admissions to 7 Toronto hospitals.23 Current holdings represent more than 30 different hospitals, 1.2 million admissions and 2.2 billion data points.24
Patient characteristics
Data collected included sex, age, Charlson Comorbidity Index score,25 Laboratory-based Acute Physiology Score,26 admission date and day of the week, admission time to the general medicine service, discharge diagnosis (using codes from the Canadian version of the International Classification of Diseases, 10th Revision [ICD-10-CA]) and general medicine admissions within 30 days before the index admission. We selected these patient factors as they have been used in previous studies and reflect the relative patient complexity, acuity and likelihood of receiving antimicrobial therapy during hospital stays.18,27
To calculate antimicrobial prescribing for each patient, we extracted data from GEMINI on antimicrobial agents prescribed during their admission, including the antimicrobial name and start and stop dates.
No data were missing for any components that we extracted from GEMINI.
Measures of antimicrobial prescribing
We analyzed aggregate data on antimicrobial prescribing at the physician level. We described volume of antimicrobial prescribing as antimicrobial days of therapy per 100 patient-days and the percentage of antimicrobial-free days. We described the intensity of antimicrobial prescribing using the modified spectrum score.28,29
The primary measure used for analysis was days of therapy per 100 patient-days (Appendix 1, Section S1, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.221732/tab-related-content). This measure offers a standardized method to evaluate antimicrobial prescribing.30,31 If a patient received more than 1 antimicrobial on a specific day, each antimicrobial was considered separately (e.g., a patient who received 2 antimicrobials on each day of a 5-day admission would be considered to have had 10 antimicrobial days of therapy). To address limitations of days of therapy (e.g., overestimating when combination therapy is prescribed, even if appropriate), we calculated the percentage of antimicrobial-free days, defined as the proportion of total patient-days per physician without antimicrobial exposure.31
No validated tool exists to assess differences in aggregate intensity of antimicrobial prescribing. To obtain an exploratory understanding of variability in the spectrum of antibacterial prescribing, we used a modified tool that was initially developed to help quantify the relative spectrum of antibacterial agents for patients in the ICU.28,32 Spectrum of activity refers to the relative coverage of different bacterial organisms for each antimicrobial, taking into account resistance rates in the local setting.33 For example, piperacillin–tazobactam, a broad-spectrum combination of ureidopenicillin and a β-lactamase inhibitor, effectively treats methicillin-susceptible Staphylococcus aureus, Streptococcus spp. and many enteric gram-negative organisms such as Escherichia coli and Klebsiella spp., along with Pseudomonas aeruginosa. Penicillin G, a narrow-spectrum agent, effectively kills β-hemolytic streptococci and Treponema spp. The modified spectrum score assigns a numerical value to each antibacterial agent (e.g., 42.25 for piperacillin–tazobactam, 4.0 for penicillin G), corresponding to the clinically relevant organisms covered by each drug, with a maximum theoretical value of 60 (Appendix 1, Section S2). We modified the original spectrum score to include antimicrobial agents prescribed outside of the critical care environment (e.g., penicillin and nitrofurantoin).29
Clinical outcomes
We evaluated the association between physician-level antimicrobial prescribing and patient outcomes, namely inpatient death, hospital length of stay, 30-day hospital readmission at any participating GEMINI hospital and ICU transfer. These are widely used outcomes in health systems research and are particularly useful for assessment of antimicrobial prescribing, as infection and treatment may lead to changes in these outcomes.19,34,35
Statistical analysis
To evaluate physician-level variability in prescribing, we calculated within-hospital differences between the highest and lowest values of each measure of antimicrobial prescribing. We visualized variability in prescribing between physicians by site, with 95% confidence intervals (CIs) estimated by bootstrapping for each physician (Appendix 1, Section S3).
We categorized physicians into quartiles of prescribing measures (days of therapy, antimicrobial-free days and modified spectrum score) within each hospital. We then evaluated the balance of patient-level characteristics across quartiles. We calculated standardized mean differences (SMDs) as means divided by standard deviations (SDs), using the 2 quartiles with the largest difference for each patient-level measure (referred to as the maximum standardized difference); values of greater than 0.1 suggested imbalance.36
We assessed clinical outcomes by physician-level prescribing quartile. This was done using trend tests, adjusted for clustering of admissions within physicians, and with multilevel models, with admissions nested within physicians. We used logistic multilevel models for binary outcomes and negative binomial multilevel models for hospital length of stay to estimate associations between physician-level prescribing measures and patient-level clinical outcomes. We reported results for binary outcomes as odds ratios (ORs) with 95% CIs. We reported rate ratios (days/admission) for length of stay.
Sensitivity analyses
We performed several sensitivity analyses to evaluate whether patient-level factors influenced physician-level antimicrobial prescribing.
To assess whether observed differences were indeed related to differences in underlying latent physician prescribing practices, we split the sample into 2 time periods (Apr. 1, 2010, to Feb. 14, 2015, and Feb. 15, 2015, to Dec. 31, 2019). We filtered the data set to physicians who had at least 100 encounters in each half of the study period and estimated the correlation between prescribing in the 2 halves.
To evaluate for potential bias resulting from nonrandom patient assignment to MRPs, we constructed a matched population that would be more balanced in baseline patient characteristics between physician prescribing quartiles 1 and 4. We assigned patients of quartile 1 physicians (controls) to those of quartile 4 physicians, using 1:1 nearest neighbour propensity score matching, with a caliper width of 0.2. We derived the propensity score using the Charlson Comorbidity Index, day of admission, time of admission, previous 30-day admission and primary ICD-10-CA diagnosis as these variables are available in the GEMINI database and are potentially predictive of antimicrobial administration.37–42 We then recalculated the prescribing measures within this matched population to evaluate whether prescribing variability was still present.
To minimize the potential for misattribution of MRP status and antimicrobial prescribing, we conducted an analysis restricting inclusion to patients who were admitted to and discharged from the general medicine team with the same MRP.
To minimize the potential for effects to be driven by patients being cared for with a palliative care approach that involved no active medical management, where antimicrobial prescribing may be appreciably different, we conducted an analysis excluding patients with a discharge diagnosis of palliative care (ICD-10-CA code Z51.5).
Finally, to evaluate whether adjustment for patient characteristics significantly affected physician-level variability in antimicrobial prescribing, unadjusted (physician-level fixed effects) and adjusted (physician-level fixed effects plus patient-level variables), we estimated and compared prescribing patterns. A high degree of concordance between the physician fixed-effect coefficients estimated in the unadjusted and adjusted models would suggest a relatively small influence of patient-level factors in explaining physician-level practice variation. We conducted this analysis for the full cohort, as well as the restricted cohorts of same MRP at admission and discharge and with exclusion of patients with a discharge diagnosis of palliative care.
We performed all statistical analyses using R, version 3.5.0 (R Foundation for Statistical Computing).
Ethics approval
Research ethics board approval was obtained from all participating hospitals (Unity Health Toronto Research Ethics Board no. 15–087 for St. Michael’s Hospital, Toronto General Hospital and Toronto Western Hospital and the Sinai Health Ethics Board no. 15–0075-C for the Mount Sinai Hospital).
Results
The cohort consisted of 124 physicians with 124 158 patient admissions (Figure 1). The median volume of physician antimicrobial prescribing across the cohort was 56.1 (interquartile range [IQR] 51.7–67.5) days of therapy per 100 patient-days (Table 1). The median physician percentage of antimicrobial-free days was 62.3% (IQR 56.9%–64.3%) and the median physician modified spectrum score was 24.9 (IQR 24.3–25.5). (Appendix 2, Tables S1 and S2, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.221732/tab-related-content). The variability of each measure across sites is shown in Figure 2, with each dot representing 1 physician within each hospital site.
Patient characteristics were balanced across the 4 quartiles of physician-level days of therapy per 100 patient-days, with the standardized maximum difference not exceeding 0.1 (Table 1). Characteristics were similar when examined by quartiles of antimicrobial-free days or modified spectrum score (Appendix 2, Tables S1 and S2).
The difference in mean prescribing between physician quartile 4 and quartile 1 of the primary measure was 15.8 (95% CI 9.6–22.0) days of therapy per 100 patient-days. The median physician in quartile 4 had a volume of antimicrobial prescribing that was 30% higher than that of the median physician in quartile 1.
The difference in mean physician prescribing between quartile 4 and quartile 1 for percentage of antimicrobial-free days was 8.1% (95% CI 5.6%–10.5%); for the modified spectrum score, the difference was 1.9 (95% CI 1.5–2.3) (Appendix 2, Tables S1 and S2).
Clinical outcomes
In-hospital death occurred in 5571 (4.5%) patients; 6972 (5.6%) patients were admitted to the ICU and 16 765 (13.5%) of the 124 158 hospital admissions were readmitted within 30 days of discharge. The mean hospital length of stay was 6.2 days. No significant differences were noted for any clinical outcome when assessed by quartile of days of therapy, antimicrobial-free days or modified spectrum score (Table 2).
Multilevel modelling showed no significant association between clinical outcomes and volume of physician-level antimicrobial prescribing (Table 3). In-hospital mortality was positively associated with modified spectrum score (OR 1.132, 95% CI 1.035–1.239); however, no difference was noted in other outcome measures related to modified spectrum score.
Sensitivity analyses
Comparing prescribing between the 2 halves of the study period, we estimated a Pearson correlation coefficient of 0.79 (p < 0.01), which indicated fairly stable prescribing over time.
After propensity score matching to further balance patient characteristics between quartiles 1 and 4, the difference in mean prescribing between these 2 quartiles was 8.1 (95% CI 1.3–17.5) days of therapy per 100 patient-days (Appendix 2, Table S3).
When the cohort was restricted to patients admitted and discharged from the same MRP, and in the analysis restricted to those without a discharge diagnosis code of palliative care, we did not observe any significant differences in the case mix between quartile 4 and quartile 1, as in the main analysis (Appendix 2, Tables S4 and S5).
Variability in physician-level days of therapy, modified spectrum score and antimicrobial-free days remained robust to adjustment by patient-level characteristics (Appendix 2, Tables S6, S7 and S8).
For outcomes analyses, we did not observe significant differences by quartile when the cohort was restricted to patients admitted to and discharged from the same MRP and when restricted to those without a discharge diagnosis code of palliative care (Appendix 2, Tables S9 and S10). Multilevel modelling, whether restricted to patients with the same MRP or to those without a discharge diagnosis of palliative care, did not show an association between modified spectrum score and mortality. We did not observe other associations between prescribing measures and clinical outcomes (Appendix 2, Tables S11 and S12).
Interpretation
This multicentre study of antimicrobial prescribing in patients admitted to general medicine wards in academic hospitals showed variability in physician-level volume of antimicrobial prescribing, which was not related to patient-level factors. Physician-level antimicrobial prescribing was not associated with patient clinical outcomes. The positive association between the intensity of antimicrobials prescribed (measured by the modified spectrum score) and odds of in-hospital mortality is unexpected and may be caused by an actual association, unmeasured confounding, multiple measurements or other unexplained factors.
When evaluating the change in prescribing patterns over time by physicians, the strong positive correlation between the 2 halves of the study period suggests that physicians had differences in prescribing practices, and that findings were not just a product of random fluctuations.
Variations in antimicrobial prescribing have been shown in ambulatory settings and long-term care facilities. Jung and colleagues7 and Stenehjem and colleagues43 showed significant physician-level variation in antimicrobial prescribing for acute respiratory infections in ambulatory care and urgent care settings, respectively. Daneman and colleagues44 found significant prescribing variability in long-term care facilities, where higher antimicrobial prescribing was associated with patient harms.12
Antimicrobial prescribing in hospitals is a complex clinical, logistical, social and emotional task, and requires additional study.39,45–52 Because these factors are often difficult to measure at scale, using data from electronic health records is a practical method to help identify potential drivers of local variation in prescribing within a particular institution. Other research methodologies, such as qualitative research, may be helpful in teasing out the relative contributions of the various components that affect prescribing.
Antimicrobial stewardship interventions tailored to addressing drivers of variability and specific groups of higher-tier prescribers may be effective. A recent randomized controlled trial by Schwartz and colleagues53 found that provision of peer-comparison feedback letters reduced the duration of antimicrobial therapy in the community setting. A similar study in the United Kingdom found an approximate 5% reduction in antimicrobial prescribing when letters were sent to primary care physicians in the top 20% of antimicrobial prescribers.54 Although the literature is mixed,55 peer comparison remains a relatively low-intensity intervention to reduce antimicrobial exposure at scale. Developing peer-comparison interventions that target prescribers in the top 3 quartiles, rather than only the highest quartile or outliers, may help further reduce antimicrobial prescribing in the hospital setting.
Optimal antimicrobial prescribing behaviour is difficult to define and will naturally have acceptable variation between clinicians. For example, community-acquired pneumonia can be treated in most patients with 5–7 days of antimicrobial therapy.56 Although this range of therapy duration is likely appropriate, clinicians who are most comfortable with a duration of 7 days may find themselves in a higher quartile of prescribing. Feedback on prescribing variation may support self-reflection for clinicians and prompt local practice discussions to reduce variation.
Ultimately, without discernable benefit in outcomes of patients of physicians who prescribe more frequently, less antimicrobial exposure may be possible, leading to lower risk of antimicrobial resistance.
Limitations
As this was a retrospective observational study using electronic patient data, unmeasured confounding cannot be excluded. Other factors affecting antimicrobial prescribing may have driven individual decision-making. For example, confounding by indication may have influenced our findings by differential distribution of infections that require use of more antimicrobial agents (e.g., infective endocarditis). Based on the MRP assignment process from the emergency department, our assumption was that each MRP would have an approximately equal chance of encountering common infectious diseases.
Accurate attribution of patients to physicians is a prerequisite to our study; any errors in attribution to MRPs have the potential to affect the results. However, it is encouraging that we did not observe any significant differences when the cohort was restricted to patients for whom the admitting and discharging physician were the same. In this study, we chose attribution of antimicrobial prescribing to the physician designated as the MRP (i.e., 100% attribution to MRP) rather than a blended model of initiation versus continuation of antimicrobials. This attribution model may have disproportionately affected those with shorter average service block time.
In academic hospitals, the MRP is often not the clinician who ordered the antimicrobial. Most antimicrobials are ordered by trainee physicians either at the behest of or with agreement from the MRP. Therapy is often started overnight or when other clinicians are responsible for care. The differences in prescribing measures may represent a combination of both team management and personal prescribing practices that may differ from individual prescribing. In addition, we were unable to capture the impact of other health care providers. Pharmacists, infection prevention and control providers and infectious disease consultants, among others, were active in all hospital sites but were not evaluated in this analysis, which may have affected variability of antimicrobial prescribing.
The lack of a validated, aggregate spectrum score was a limitation of our study. Although we wished to assess the intensity of antibacterial therapy, we found no appropriate measure in the literature. We did assess the derivation of the spectrum score and found it applicable to our local pattern of antimicrobial resistance, so decided to use this as an exploratory metric.29 Further study is needed to validate this approach.
Data were collected from urban academic teaching hospitals. Generalizability to community hospitals and nonurban teaching centres may be limited. Further research is needed to evaluate the variability of antimicrobial prescribing and association with outcomes in other institutions.
Finally, none of the chosen metrics could be used to assess the appropriateness of antimicrobial therapy.
Conclusion
This large, multicentre retrospective cohort study of 4 urban academic hospitals showed variability in the volume of antimicrobial prescribing at the physician level that was not associated with patient-level characteristics or clinical outcomes. These findings provide support for considering the lowest quartile of physician antimicrobial prescribing within each hospital as a target for antimicrobial stewardship.
Footnotes
Competing interests: Michael Fralick is a consultant for ProofDx, a startup company that has created a point-of-care test for COVID-19 using CRISPR. Amol Verma is a provincial clinical lead for quality improvement at Ontario Health (part-time employee) and has the potential to acquire minority interests in a health artificial intelligence start-up company, Signal1, which acquired a patient deterioration early warning system that he coinvented. Fahad Razak is a principal investigator of the GEMINI platform. No other competing interests were declared.
This article has been peer reviewed.
Contributors: Mark McIntyre, Andrew Morris, Amol Verma and Fahad Razak contributed to the conception and design of the study. All authors contributed to the acquisition and interpretation of the data. Sudipta Saha analyzed the data. Mark McIntyre, Sudipta Saha and Fahad Razak drafted and revised the manuscript. All authors reviewed the manuscript critically for important intellectual content, gave final approval of the version to be published and agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. Amol Verma and Fahad Razak share co–senior authorship for this manuscript.
Funding: No funding was received for this study.
Data sharing: Data from this manuscript can be accessed upon request to Fahad Razak (fahad.razak{at}mail.utoronto.ca), to the extent that is possible in compliance with local research ethics board requirements and data-sharing agreements.
- Accepted July 7, 2023.
This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) licence, which permits use, distribution and reproduction in any medium, provided that the original publication is properly cited, the use is noncommercial (i.e., research or educational use), and no modifications or adaptations are made. See: https://creativecommons.org/licenses/by-nc-nd/4.0/