Abstract
BACKGROUND: The Ottawa chronic obstructive pulmonary disease (COPD) Risk Scale (OCRS), which consists of 10 criteria, was previously derived to identify patients in the emergency department with COPD who were at high risk for short-term serious outcomes. We sought to validate, prospectively and explicitly, the OCRS when applied by physicians in the emergency department.
METHODS: We conducted this prospective cohort study involving patients in the emergency departments at 6 tertiary care hospitals and enrolled adults with acute exacerbation of COPD from May 2011 to December 2013. Physicians evaluated patients for the OCRS criteria, which were recorded on a data form along with the total risk score. We followed patients for 30 days and the primary outcome, short-term serious outcomes, was defined as any of death, admission to monitored unit, intubation, noninvasive ventilation, myocardial infarction (MI) or relapse with hospital admission.
RESULTS: We enrolled 1415 patients with a mean age of 70.6 (SD 10.6) years and 50.2% were female. Short-term serious outcomes occurred in 135 (9.5%) cases. Incidence of short-term serious outcomes ranged from 4.6% for a total score of 0 to 100% for a score of 10. Compared with current practice, an OCRS score threshold of greater than 1 would increase sensitivity for short-term serious outcomes from 51.9% to 79.3% and increase admissions from 45.0% to 56.6%. A threshold of greater than 2 would improve sensitivity to 71.9% with 47.9% of patients being admitted.
INTERPRETATION: In this clinical validation of a risk-stratification tool for COPD in the emergency department, we found that OCRS showed better sensitivity for short-term serious outcomes compared with current practice. This risk scale can now be used to help emergency department disposition decisions for patients with COPD, which should lead to a decrease in unnecessary admissions and in unsafe discharges.
One of the most common conditions causing respiratory difficulty in older patients is chronic obstructive pulmonary disease (COPD). It is a respiratory disorder largely caused by smoking, which is characterized by progressive, partially reversible airflow obstruction, systemic manifestations and exacerbations of respiratory distress that increase in frequency and severity over time.1,2 Exacerbations of COPD, defined as the presence of an increase in at least 2 of breathlessness, sputum volume or sputum purulence, are a frequent cause of emergency department visits.3 Chronic obstructive pulmonary disease is a major cause of morbidity and mortality in North America and Europe, and the rates of hospital admissions and visits to the emergency department have been rising over the past decade.4 Many patients with COPD can be treated aggressively for exacerbation of COPD in the emergency department and improve sufficiently to be discharged safely within several hours. Discharging patients with adequate medications can prevent relapse or return to the emergency department because of worsening respiratory symptoms.3 Many COPD exacerbations, however, are so severe that the patient must be admitted to hospital to ensure adequate management and a safe outcome. We previously showed a relatively low admission rate (38%) from Canadian emergency departments, but 50% of adverse events occurred in the patients who were not admitted to hospital.5
Unfortunately, there is little evidence to inform disposition decisions by physicians. Existing guidelines for disposition decisions are consensus-based and have not been validated.6,7 Several authors have developed risk-stratification scores for COPD exacerbations; however, most are limited by predicting death only among admitted patients or have not be been validated clinically in real time in the emergency department.8–10 There have been no robust evaluations of risk factors to assist with the admission decision for patients with COPD in the emergency department. Prospective studies conducted in the emergency department are limited by small sample size, no reassessment of response to therapy and no prospective validation.11–15
Our group previously developed the Ottawa COPD Risk Scale (OCRS), based upon a multicentre, prospective data collection from 945 patients in the emergency department with exacerbation of COPD.5 With multivariate analyses, we created a scale comprising 10 items from history, physical examination and bedside tests (Figure 1). The total score estimates the risk of short-term serious outcomes within 14 days, ranging from 2.2% to 91.4%. Our primary goal with the current study was to validate prospectively the OCRS for its accuracy in predicting short-term serious outcomes. Secondary outcomes were its acceptability with clinicians and its potential effect on patient safety and hospital admissions. Our study allowed physicians to apply the OCRS explicitly in real-time for patients with exacerbations of COPD.
The Ottawa COPD (chronic obstructive pulmonary disease) Risk Scale (OCRS) is used in the emergency department (ED) to identify patients with acute COPD who are at high risk for short-term serious outcomes. To date, no patients have had a score greater than 10. Note: CABG = coronary artery bypass graft, ECG = electrocardiogram, HR = heart rate, PVD = peripheral vascular disease, Sao2 = oxygen saturation.
Methods
Study design and setting
We conducted this multicentre prospective cohort study in 6 large tertiary care hospitals (The Ottawa Hospital General Campus, Ottawa; Kingston General Hospital, Kingston, Ontario; The Ottawa Hospital Civic Campus, Ottawa; University of Alberta Hospital, Edmonton; Foothills Medical Centre, Calgary; and Mount Sinai Hospital, Toronto) in Canada from May 2011 to December 2013. These 6 emergency departments had a combined annual census of 400 000 patient visits.
Study population
We included all adults aged 50 years or more who presented to the emergency department with acute shortness of breath or respiratory distress caused by exacerbation of COPD and who might be considered well enough to be discharged by the attending physician. Consecutive eligible patients were enrolled in the emergency departments for as many as 16 hours a day, depending on staff availability.
We defined exacerbation of COPD as an increase in at least 2 of the following 3 criteria: breathlessness, sputum volume or sputum purulence. Patients either had a previous diagnosis of COPD or the diagnosis was made in the emergency department based on a 1-year history of chronic dyspnea or cough with sputum production. Patients must have had a history of 15 pack-years or more of cigarette smoking and evidence of at least moderate airflow obstruction (as defined in the guideline from the Global Initiative for Chronic Obstructive Lung Disease)7 in the emergency department. We included all patients regardless of their disposition, i.e., both discharged and admitted cases. However, we excluded those patients who were still extremely ill after 2 to 12 hours of management in the emergency department: resting oxygen saturation less than 85%; heart rate of 130 beats/min or more; systolic blood pressure less than 85 mm Hg; presence of confusion, disorientation or dementia; primary presentation for ischemic chest pain requiring treatment; acute ischemic ST–T interval changes on initial electrocardiography; death expected within weeks from chronic illness; attended the emergency department from a long-term– or chronic-care facility; on long-term hemodialysis; or were enrolled in the study in the previous 2 months.
Data collection
Attending physicians and residents in emergency medicine were trained locally and then assessed patients for the 10 components of the OCRS; they recorded their findings on an emergency department data form. Furthermore, site research assistants were trained to ensure proper collection of clinical and laboratory data for the study. The OCRS score was not available to the physicians who made the final decision about admission. We defined “fails reassessment after treatment” as when the patient had a resting oxygen saturation (Sao2) less than 90% on room air or usual oxygen, or a heart rate greater than 120 beats/min. The nurses and respiratory therapists recorded their findings on the emergency department data form (Supplementary Figure 1 in Appendix 1, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.180232/-/DC1). The central study nurse coordinator regularly evaluated the quality of patient assessments.
Outcome measures
The primary outcome measure was short-term serious outcome, defined as death from any cause within 30 days of the visit to the emergency department; or any of the following within 14 days of the index visit to the emergency department, regardless of whether the patient was admitted originally: (1) The patient was admitted to a monitored unit (i.e., intensive care, coronary care, acute monitoring or stepdown units but not a telemetry unit). Use of the monitored unit as a criterion is an important outcome that almost always reflects severity of illness; such patients would likely have substantial morbidity if they had been discharged, because they were confined to their beds and required constant monitoring of vital signs. (2) The patient required endotracheal intubation or noninvasive ventilation after hospital admission, unless the patient was using noninvasive ventilation at home. (3) The patient had a diagnosis of myocardial infarction (MI) as defined by the Joint ESC/ACCF/AHA/WHF Task Force for the Third Universal Definition of Myocardial Infarction.16 (4) The patient underwent a major procedure defined as coronary artery bypass graft, percutaneous coronary intervention, other cardiac surgery or new hemodialysis. (5) The patient was discharged after the initial visit to the emergency department and subsequently returned to the emergency department for any related medical problem within 14 days and followed by admission to hospital. Relapse without admission was not deemed a short-term serious outcome. Fourteen days was chosen as a meaningful timeline for emergency department outcomes by a consensus of the investigators.5
We reviewed hospital and provincial death records to determine if a short-term serious outcome occurred. We also had physicians answer this question, “How comfortable would you be using this scale to assist making a disposition decision for this patient?”, using a 5-point scale on the data form (from very comfortable to very uncomfortable).
Statistical analysis
Our primary analyses used the original OCRS score calculated from the criterion interpretation, as determined by the study steering committee. We conducted secondary analyses to evaluate the scale on the data form as interpreted by the physicians. The scale is intended to provide clinicians with an estimate of risk rather than a rigid “yes or no” cut point to guide admissions. Regardless, we estimated classification accuracy using example cut points of 1 or more and 2 or more points to evaluate the potential effect of implementing OCRS into practice, using the criterion interpretation. We calculated sensitivity and specificity with 95% confidence intervals (CIs), as well as potential admission proportions. We assessed the possible effect by comparing actual admission proportions to those predicted at different threshold levels of total point scores.
We assessed clinical sensibility in 2 modes: overall accuracy was calculated as percentages with 95% CIs for interpretation of the original OCRS by the treating physicians versus the criterion interpretation and data about the physicians’ responses to the theoretical question about use of the scale were tabulated in a simple descriptive format.
Ethics approval
The study protocol was approved by the research ethics boards at each hospital. The research ethics boards of 2 of the hospitals (Kingston General Hospital and University of Alberta Hospital) determined that written informed consent was required, whereas those at the other 4 sites waived the need for written consent for this observational study. The study was approved first by the Ottawa Health Science Network Research Ethics Board.
Results
From May 2011 to December 2013, 5449 patients were screened for eligibility at the 6 hospitals, and 1415 were enrolled in the study (Figure 2). There were 2719 patients who were eligible for the study; however, 1185 were not enrolled primarily because they presented to the emergency department when research staff were not available. Among the enrolled participants, mean age was 70.6 (SD 10.6) years, 804 (56.8%) arrived via ambulance, 1380 (97.5%) had a previous diagnosis of COPD, 349 (24.7%) were on oxygen at home, 1015 (71.7%) were taking inhaled steroids and 202 (14.3%) were taking oral steroids (Table 1). The patients who were excluded had similar characteristics to those who were enrolled (Appendix 1 Supplemental Table 1).
Flow diagram for participants in the study. Note: ECG = electrocardiography, SSO = short-term serious outcome.
Characteristics of participants with acute chronic obstructive pulmonary disease in the emergency department
Among the 1415 participants who were enrolled (Table 2), there were 135 (9.5%) short-term serious outcomes, with higher rates in those admitted compared with those discharged from the emergency department (11.0% v. 8.3%, p < 0.01). Of the 636 participants who were admitted to the hospital on their index visit, 45 (7.1%) were admitted to a monitored unit, 54 (8.5%) required noninvasive ventilation or intubation, 5 (0.8%) had an MI, and 65 (8.3%) had a relapse and were readmitted to hospital. Of concern, 65 (8.3%) of these short-term serious outcomes occurred in the 779 participants who were not admitted to hospital at the initial visit to the emergency department.
Outcomes for participants with acute chronic obstructive pulmonary disease at their visit to the emergency department
We compared the classification performance and expected admission proportions for the OCRS with current practice at the 6 study hospitals (Table 3). The incidence of a short-term serious outcome ranged from 4.6% for a total score of 0 to 100% for a score of 10. Use of the scale could improve upon the sensitivity of current practice, in which only 51.9% (70/135) of short-term serious outcome cases were admitted at the first visit to the emergency department. Choosing total point scores of 1 or 2 as the threshold for admission would be associated with sensitivities for a short-term serious outcome of 79.3% or 71.9%, respectively. These theoretical admission thresholds would lead to absolute admission rates of 56.6% or 47.9%, respectively, compared with the observed admission rate of 45.0% at the study hospitals.
Classification performance and expected admission proportions based on different admission cut points for the Ottawa COPD (chronic obstructive pulmonary disease) Risk Scale compared with current practice at the 6 study hospital sites
We compared the classification of risk category (i.e., low, medium, high, very high) by the physicians to the criterion interpretation and found overall agreement of 64.9% for the exact category and 94.8% for the exact category ± 1.
On the 5-point scale of comfort in using OCRS, the physicians indicated that they would be uncomfortable or very uncomfortable in only 13.4% of cases.
Interpretation
In this clinical validation of a risk-stratification tool for COPD in the emergency department, we found that 9.5% of the 1415 participants with COPD had short-term serious outcomes, with a concerning proportion occurring in those discharged home from the emergency department. Compared with current practice, an OCRS score threshold of 1 or more would increase sensitivity by 50% but would require 25% more admissions. Alternately, a threshold of 2 or more would improve sensitivity by 38% while leading to only a slight increase in admissions. There was an excellent spread of the incidence of short-term serious outcome by score in a very similar pattern to the derivation study.5 Physicians displayed reasonable accuracy in interpretation, as well as good acceptance of OCRS.
A larger study involving 113 centres in France was limited because there was no evaluation of response to therapy or there was a lack of follow-up data on mortality.17,18 The Dyspnoea, Eosinopenia, Consolidation Acidaemia and atrial Fibrillation (DECAF) Score was derived and validated in the United Kingdom to predict mortality among inpatients, but it has never been evaluated clinically.8,9 A large prospective cohort study involving 16 hospitals within the Spanish National Health Service collected data on patients with COPD to develop a risk score, but this has not been applied clinically.10,19 Other prospective studies that were only conducted after admission to hospital were limited by similar issues, and most attempted only to predict mortality.20–23 Others were conducted retrospectively on existing databases, focused on inpatients or mortality and were not directly applicable to the emergency department.24–30
This study has multiple strengths including the prospective and rigorous collection of real-time clinical data that includes response to treatment. The large number of cases with short-term serious outcomes from this multicentre study allowed a robust analysis with stable estimates of performance. The use of other important outcomes besides mortality increases the clinical relevance of OCRS for physicians determining whether to admit or discharge their patients with acute COPD.
It is neither necessary nor efficient to admit all patients with COPD to the hospital because many will respond to treatment in the emergency department and would not benefit from admission to hospital. We know that admission to hospital is associated with risks to patients (e.g., hospital-acquired infections, sleep deprivation, deconditioning). Regardless, many patients with COPD who are admitted to hospital have short-term serious outcomes, i.e., they die, or require noninvasive ventilation or intubation. In addition, many patients are discharged from the emergency department after treatment only to die or relapse and return to the emergency department and require admission to hospital.
We intend that OCRS be used in a pragmatic fashion to ensure that patients are managed in a safe and efficient manner, and we do not advocate for a specific risk score total that would require admission. Depending on local practice, OCRS could lead to either an increase or a decrease in hospital admissions. More important than altering the admission rate is ensuring admission of the right patients, i.e., those at highest risk of a poor outcome. Many nonmedical factors are important in the decision to admit a patient, such as support at home or availability of early follow-up.31 We see OCRS as an important tool to help physicians gauge the medical risk for their patients with COPD, while they determine the need for admission or early follow-up.
Limitations
We believe the correct approach was to include both admitted and discharged patients. Admission status may confound the likelihood of a short-term serious outcome occurring, because some patients who are admitted to hospital will not have short-term serious outcomes because they receive more intensive therapy in hospital; the same patients might have had short-term serious outcomes if they had been discharged from the emergency department. Our goal is to develop a risk scale that aids the appropriate admission of patients at high risk (i.e., sensitivity) and minimizes the admission of patients at low risk (i.e., specificity). We could only do this by including patients who were admitted to hospital.
We chose not to evaluate spirometry in the emergency department because it is often not available and was felt to be of limited usefulness for predicting short-term outcomes. We were unable to enroll a large number of eligible patients because they presented outside of normal business hours.
We could detect neither selection bias nor threat to the validity of our findings. It is possible, although unlikely, that some return visits to the emergency department were not identified in patients who returned to a different hospital. Five of the 6 hospitals could monitor such visits through shared electronic medical records with other hospitals in their city. If such events did occur, we do not think this would have had a substantial effect on our results. Although 5 of the 6 study hospitals also participated in the derivation study, we see this as only a minor limitation in that the current study enrolled new patients prospectively and introduced the risk scale that had not yet been derived in the first study. Although physicians displayed reasonable accuracy in interpreting the OCRS scores, this could likely be improved by specific training in any future implementation initiatives.
Conclusion
The OCRS showed better sensitivity for short-term serious outcomes compared with current practice, excellent stratification of risk and good acceptance by physicians. This risk scale has been clinically validated and can now be used to estimate medical risk and help with decisions about patient disposition. This should lead to a decrease both in unnecessary admissions and in unsafe discharge decisions for patients with COPD in the emergency department.
Acknowledgements
The authors thank the following individuals for their invaluable assistance: Jane Reid, RN (Kingston General Hospital, Kingston, Ont.); Tami Clavet, RN (The Ottawa Hospital, Civic and General Campuses, Ottawa); Danielle de Vuyst, RN (University of Alberta Hospital, Edmonton); Renée Vilneff, RN (Foothills Medical Centre, Calgary). The authors also thank the following individuals from the coordinating centre in Ottawa: Connor Sheehan, Angela Marcantonio, Sheryl Domingo, My-Linh Tran (Ottawa Hospital Research Institute); and Li Chen (University of Ottawa Heart Institute). Finally, the authors are grateful to the many physicians, nurses and respiratory therapists at the study hospitals for their assistance with patient identification and data collection.
Footnotes
Competing interests: Andrew McRae received a research grant from the Canadian Institutes of Health Research for the submitted work. Ian Stiell received a research grant from CIHR that was paid to his institution. No other competing interests were declared.
This article has been peer reviewed.
Contributors: Ian Stiell conceived the idea, co-wrote the manuscript and secured research funding. Catherine Clement administrated the study, co-wrote the manuscript, and collected data. Jennifer Brinkhurst coordinated the study, collected data and contributed to the writing of the manuscript. George Wells provided statistical assistance. Robert Brison, Bjug Borgundvaag, Shawn Aaron, Andrew McRae, Lisa Calder, Jeffrey Perry, Brian Rowe and Alan Forster assisted with study design, supervised the recruitment of patients and management of data, and revised the manuscript. All of the authors supervised the conduct of the trial and data collection, revised the manuscript critically for important intellectual content, gave final approval of the version to be published and agreed to be accountable for all aspects of the work.
Funding: Peer-reviewed funding from the Canadian Institutes of Health Research (CIHR) was received for this study. Ian Stiell holds a Distinguished Professorship from the University of Ottawa. Ian Stiell, Jeffrey Perry and Shawn Aaron hold Clinical Research Chairs from the University of Ottawa. Brian Rowe was supported by a Tier I Canada Research Chair in Evidence-based Emergency Medicine from CIHR through the Government of Canada. Dr. Rowe is currently supported by CIHR as the Scientific Director of the Institute of Circulatory and Respiratory Health. Bjug Borgundvaag is a Clinician Scientist in the Department of Family and Community Medicine at the University of Toronto and is supported by the Schwartz/Reisman Emergency Medicine Institute at Sinai Health System.
Data sharing: Study data are not available.
Disclaimer: George Wells is a biostatistical consultant for CMAJ and was not involved in the editorial decision-making process for this article.
- Accepted October 2, 2018.
References
In this issue
Article tools
Jump to section
Related Articles
Cited By...
- Comparison of two scores for short-term outcomes in patients with COPD exacerbation in the emergency department: the Ottawa COPD Risk Scale and the DECAF score
- Predicting Adverse Events Among Patients With COPD Exacerbations in the Emergency Department
- Most important risk factor missing from COPD score
- An author responds to "Most important risk factor missing from COPD score"
- To keep patients with COPD out of hospital, look beyond the lungs
More in this TOC Section
Similar Articles
Collections
