Emergency Medicine

Assessing resident and attending error and adverse events in the emergency department

a b s t r a c t

Background: There is a paucity of data looking at resident error or contrasting errors and adverse events among residents and attendings. This type of data could be vital in developing and enhancing educational curricula Objectives: Using an integrated, readily accessible electronic error reporting system the objective of this study is to compare the frequency and types of error and adverse events attributed to emergency medicine residents with those attributed to emergency medicine attendings.

Methods: Individual events were classified into errors and/or adverse events, and were attributed to one of three groups–residents only, attendings only, or both (if the event had both resident and attending involvement). Error and adverse events were also classified into five different categories of events–systems, documentation, di- agnostic, procedural and treatment. The proportion of error events were compared between the residents only and the attendings only group using a one-sample test of proportions. Categorical variables were compared using Fisher’s exact test.

Results: Of a total of 115 observed events over the 11-month data collection period, 96 (83.4%) were errors. A ma- jority of these errors, 40 (41.7%), were attributed to both residents and attendings, 20 (20.8%) were attributed to residents only, and 36 (37.5%) were attributed to attendings only. Of the 19 adverse events, 14 (73.7%) were at- tributed to both residents and attendings, and 5 (26.3%) adverse events were attributed to attendings only. No adverse events were attributed solely to residents (Table 1). Excluding events attributed to both residents and attendings, there was a significant difference between the proportion of errors attributed to attendings only (64.3%, CI: 50.6, 76.0), and residents only (35.7%, CI: 24.0, 49.0), p = 0.03. (Table 2). There was no significant dif- ference between the residents only and the attendings only group in the distribution of errors and adverse events (Fisher’s exact, p = 0.162). (Table 2). There was no statistically significant difference between the two groups in errors that did not result in adverse events and the rate of errors proceeding to adverse events (Fisher’s exact, p = 0.15). (Table 3). There was no statistically significant difference between the two groups in the distribution of the types of errors and adverse events (Fisher’s exact, p = 0.09). Treatment related errors were the most common error types, for both the attending and the resident groups.

Conclusions: Resident error, somewhat expectedly, is most commonly related to Treatment interventions, and rarely is due to an individual resident mistake. Resident error instead seems to reflect concomitant error on the part of the attending. Error, in general as well as adverse events, are more likely to be attributed to an attend- ing alone rather than to a resident.

(C) 2022

  1. Introduction

Medical error is a well-known and significant cause of patient mor- bidity and mortality. According to the Harvard Medical Practice Study released in 1991, 3.7% of admitted patients experience treatment

* Corresponding author at: Department of Emergency Medicine, Beth Israel Deaconess Medical Center, West, Clinical Center 2, One Deaconess Road, Boston, MA 02215, United States of America.

E-mail address: [email protected] (J.L. Adler).

complications, of which approximately 65% may be attributed to errors in medical care [1]. The landmark publication by the Institute of Medi- cine in 2000, “To Err Is Human,” reported that nearly 100,000 patients die per year secondary to Medical errors occurring in the hospital envi- ronment [2]. This data focused national attention toward issues of pa- tient safety and medical error [3,4]. More recent data indicate that the incidence of adverse events attributable to medical error among hospi- talized patients may be increasing despite renewed efforts to limit error [5]. A follow-up report from the IOM, published in 2015, suggests that diagnostic error remains rampant, and postulates that nearly every

https://doi.org/10.1016/j.ajem.2022.01.015

0735-6757/(C) 2022

person will experience a diagnostic medical error at some point in their life [6]. Most recently, medical error and its association with malpractice risk has further concentrated attention toward patient safety and med- ical error prevention [7].

Many hospitals have a quality assurance process in place; however, the specific protocols vary widely across institutions, and there is no sin- gle mechanism for the identification of cases, assessment of potential error, and implementation of responses to error. Thus, the efficacy of having a quality assurance process in place varies across institutions based on the individual elements of the process, the rigor of evaluation, and the methods of utilizing results to inform ongoing care.

In response to this heightened attention to medical error, the Ac- creditation Council for Graduate Medical Education (ACGME) imple- mented requirements for incorporation of quality improvement and patient safety milestones into accredited residency training programs [8]. In such institutions, with the emphasis on training of junior physi- cians, there may be higher rates of error inherent to the process of train- ing physicians. With this mandate, academic institutions have instituted more rigorous quality assurance processes. Evaluation of data obtained from these Quality assurance processes in academic settings may help improve the practice of emergency medicine. The incidence and categorization of errors and adverse events within the field of emer- gency medicine continues to be an area of developing research; there remains a paucity of data looking at resident error or contrasting errors and adverse events among residents and attendings. Such investigation may be useful in creating educational curricula.

Using an integrated, readily accessible electronic error reporting sys- tem, the objective of this study is to compare the frequency and types of error and adverse events attributed to emergency medicine residents with those attributed to emergency medicine attendings.

  1. Methods
    1. Study design and setting

We conducted a prospective, observational study of consecutive pa- tients presenting to an urban, tertiary care academic medical center Emergency Department (ED) with an annual volume of 57,000 patients between October 13, 2015 and September 14, 2016. Implementation of our unique data acquisition system occurred on October 13, 2015 and served as the starting date for included data. The end date of September 14, 2016 is based on the time at which initial review of the collected data to date began. Our ED is associated with a 3-year ACGME Emer- gency Medicine residency program. This ED maintains an electronic QA database linking all standardized quality reviews to patient records utilizing a web-based dashboard, wherein QA issues are identified using both automatED triggers and manual flags by staff. Automated triggers for QA review include 72-h returns with admission, death within 24 h of presentation to ED, floor transfers to ICU <24 h from ad- mission, and morbidity and mortality conference cases. Additionally, any nurse, physician, or other health care provider involved in patient care can flag a perceived QA issue through the hospital wide EHR or via direct flag links built in the ED dashboard. All automated QA triggers are automatically transferred into the QA database. Patient or physician complaints received by ED administration are forwarded into the QA database for review. This database maintains the data relating to each QA case, reports from the individual cases reviewers and findings of a final arbitrating QA review committee. This system was then utilized to collate and review findings of the QA committee on medical errors and adverse events assigned to resident and attending physicians based on the QA review committee determination. Errors were catego- rized as follows: 1) Treatment errors, including administration of med- ications or other therapeutic interventions, as well as erroneous absence of the implementation of indicatED treatments. 2) Diagnostic errors, which includes failure to identify the correct diagnosis, determination of an incorrect diagnosis, or other flaws in cognitive diagnostics.

3) Systems errors, which includes any errors that arise within the con- text of the practice environment as a function of the operations and flow of that environment. For example, development of a pressure ulcer related to prolonged immobilization on a gurney and prolonged ED length of stay would be considered a systems error. 4) Documenta- tion errors, which encompasses errors in written communication be- tween providers and disruption in continuity of care. 5) Procedural errors, including technical mistakes or unintentional outcomes that arose in the performance of a medical procedure.

    1. Selection of participants

All patients presenting to the ED during the specified period were el- igible for inclusion in this study. The study population ultimately consisted of any patient flagged in the electronic quality assurance (QA) system as described in greater detail above, and in a previous publication [9]. The QA database stores all submitted cases for further two-tiered evaluation. Initially, each case is randomly assigned to an ED attending not involved with the care to determine if the case met error inclusion criteria based on an 8-point Likert scale. This Likert scale was described in a previous publication [9]. In brief, there are seven standardized categories for the assigned reviewer to address per each case review. These are as follows: 1) Was error made by the ED team; 2) Was there an adverse event resulting from the care of the ED team; 3) Documentation; 4) Resource over-utilization; 5) Performance of procedures; 6) Medical judgement of the ED team; and 7) Coordina- tion of care. Those cases meeting inclusion criteria, which was a score of 4 or more in any of the above seven categories were then referred to a 20-person interdisciplinary committee for formal QA committee review. The QA committee is comprised of faculty attending physicians, members of nursing leadership, and a chief resident. Additionally, all residents are expected to participate in at least two meetings as part of their academic requirements. Cases that involve other specialties are referred to their respective departments after review by the ED QA committee. Final determination and attribution of an error is based on consensus of the committee [8]. Once the review is completed and entered into the QA database, notification of error and adverse events are sent automatically via email to the providers, including relevant residents and attendings, with a plan for remediation. This information is stored in the QA database for future reference as needed.

    1. Data analysis

All cases reviewed by the QA committee are assigned a case number. Cases falling within the study dates were compiled into a file to identify each case with an error or adverse event. Individual events were classi- fied into error events or adverse events, and were attributed to one of three groups–residents only, attendings only, or both (if the event had both resident and attending involvement). The errors were also classified into those errors leading to adverse events and those errors not leading to adverse events (“near misses”). This was accomplished by sorting the cases by their case numbers, identifying repeated occur- rences of the case number within the original dataset, and noting if the repeat entry was classified as an adverse event. The earliest occur- rence of the case number was then tagged as an error leading to an ad- verse event, and the repeat was then removed. If there were no repeats of a case number, then the single occurrence was tagged as an error not leading to an adverse event (“near miss”). Error and adverse events were classified into five different categories of events–systems, docu- mentation, procedural, treatment and diagnostic. One-sample tests of proportions were used to compare the proportions of error events be- tween the residents and the attending, the proportions of standalone error events between the residents and attendings, and the proportions of attendings/residents contributing to each of the five categories of er- rors/adverse events. Statistical analyses were conducted with the use of STATA, version 15.1 (College Station, TX, StataCorp LP, USA).

  1. Results

Five thousand seven hundred and seventy-three cases were screened for review by individual attending physicians; these screening reviews identified 381 (6.6%) cases that met previously described criteria for further review by the arbitrating QA committee. Based on consensus of the QA committee, 115 error and adverse events were identified. Of the total of 115 observed events, 96 (83.4%) were error events. The majority of errors 40 (41.7%) were attributed to both resi- dents and attendings; 20 (20.8%) were attributed to residents only, and 36 (37.5%) were attributed to attendings only. Of the 19 adverse events, 14 (73.7%) were attributed to both residents and attendings, and 5 (26.3%) adverse events were attributed to attendings only. No adverse events were attributed solely to residents (Table 1).

For clarification, when we excluded errors and adverse events attrib- uted to both residents and attendings (n = 54), there was a significant difference between the proportion of error events attributed to attend- ings (64.3%, 95% CI: 50.6, 76.0), and residents (35.7%, 95% CI: 24.0, 49.0),

p = 0.03, n = 61 (Table 2). We were unable to examine the difference in proportion of adverse events attributed to attendings versus residents because of zero counts for residents (Table 2).

The error events attributed to either resident-only or attending only (n = 56) were then stratified into two groups–errors that did not lead to adverse events (n = 51), hereafter referred to as “near misses,” and errors that did lead to adverse events (n = 5). Among the 51 near mis- ses, 31 (60.8%, 95% CI: 46.4, 73.5) were attributed to attendings only

(about two-thirds of all near misses), and 20 (39.2%, 95% CI:26.5, 53.6) were attributed to residents only. There was no significant difference in the proportion of near misses between residents and attendings (p = 0.003).

Five errors were found to have led to an adverse event; all of these adverse events were attributed to the attending alone. We were unable to examine the difference in proportion of errors leading to adverse events attributed to attendings versus residents because of zero counts for residents.

Errors were then further categorized according to common themes. As depicted in Table 3, there were significant differences (p < 0.01) between the proportions of systems error/adverse events attributed to residents (13.3%, 95% CI: 2.7, 45.3) versus attendings (86.7%, 95% CI 54.6, 97.2), and in the proportions of diagnostic error/adverse events (p = 0.03) attributed to residents (18.2%, 95% CI: 3.4, 57.9) versus

attendings (81.8%, 95% CI 42.0, 96.5). There were no differences between attendings and residents when examining documentation, procedural and treatment errors, and adverse events.

When examining the types of errors/adverse events attributed to both residents and attendings, there were a total of 54 cases identified, falling into the following categories: Diagnosis errors (n = 23, 42.6%, 95% CI: 29.8, 56.4), treatment errors (n = 19, 35.2%, 95% CI: 23.4,

49.2), systems errors (n = 6, 11.1%, 95% CI: 2.9, 23.1), procedural errors (n = 5, 9.3%, 95% CI: 3.8, 20.9), and documentation errors (n = 1, 1.9%, 95% CI: 0.2, 12.7). Treatment related and diagnostic errors were the most common error types, for both the attending and the resident groups.

  1. Discussion

In a controlled training setting, we found that both errors and ad- verse events were more frequently attributed to attending physicians.

Table 1

Attribution of errors and adverse events, entire cohort (n = 115).

Attribution

Errors (n = 96)

Adverse Events (n = 19)

Resident only

20 (21%)

0 (0%)

Attending only

36 (37%)

5 (26%)

Both

40 (42%)

14 (74%)

Table 2

Attribution of error events and adverse events, resident-only and attending-only (n = 61).

Errors (n = 56) Adverse events (n = 5)

Attribution

n (%)

p-value

n (%)

p-value

Resident only

20 (35.7)

0.03

0 (0.0)

NA

Attending only

36 (64.3)

5 (100.0)

When analyzing the errors individually we were able to identify five main categories of error: diagnostic, treatment, procedural, documenta- tion and system errors.

Highest rates of error were in treatment, where both residents and attendings had similar identified number of errors, although the propor- tion of total events comprised by treatment errors was greater for resi- dents. Attendings also had a higher number and proportion of diagnostic error as compared to residents. This is likely a function of the trainee role, wherein attendings take ultimate responsibility for di- agnostic decisions in the ED. Diagnostic errors are an established area of patient safety risk, with ED physicians at particularly high risk of such error given a reliance on cognitive shortcuts and pattern recognition in order to manage a high volume and high acuity of patient care typi- cally found in the ED [7].

Treatment errors might be expected to be prevalent in a group of training physicians, with their more limited Knowledge base and their primary role in ordering treatments. However, we would expect much crossover between attending and physician treatment error given the supervisory role of the attending physicians, as erroneous treatment plans should be identified prior to these plans reaching the patient; therefore, attending physicians would be expected to be accountable for nearly all of the identified errors. Prior to data acquisition, we antic- ipated that residents would have a higher rate and incidence of errors that occur with procedural Skill acquisition; however, a greater number of errors were ultimately attributed exclusively to attendings, albeit with similar overall percentage of attributed errors. This may again re- flect the training environment at our institution in which attendings are expected to actively oversee procedures, and therefore a resident is rarely assigned sole responsibility for procedural errors. In contrast, when attendings are primarily seeing patients without any resident in- volvement, any procedural errors are theirs alone.

Clearly delineating the attribution of error between the residents and attendings involved in patient care is quite challenging, despite our robust QA system, given the team-based approach to patient care practiced in our facility. The root cause of an error and/or adverse event cannot always be traced to a single provider within the team. Res- idents often initiate workups and treatments prior to full discussion with attendings, as progressive autonomy is a core component of resi- dency training.

Further complicating evaluation of errors and adverse events is the inherent complexity and multi-factorial aspects of the current medical care environment. It is not always possible to distinguish between events that occur due to systemic factors as opposed to those that can be attributed solely to an individual provider. Systemic issues such as boarding, suboptimal staffing, and resource limitations frequently con- tribute to an environment fraught with opportunity for errors and

Table 3

Types of error events and adverse events, by attribution (n = 61).

Classification of

Resident only

Attending only

p-value

event?,??

(n = 20)

(n = 41)

Systems (n = 15)

2 (13.3)

13 (86.7)

<0.01

Documentation (n = 4)

2 (50.0)

2 (50.0)

>0.99

Procedural (n = 10)

3 (30.0)

7 (70.0)

0.23

Treatment (n = 21)

11 (52.4)

10 (47.6)

0.83

Diagnostic (n = 11)

2 (18.2)

9 (81.8)

0.03

* Includes error events and adverse events.

?? Percentages are across rows.

adverse events. Attributing error or adverse event to an individual or in- dividuals within this context must take into consideration the systemic issues that may have contributed to the occurrence of an error or ad- verse event. The overarching goal of our QA is identification of opportu- nities for quality improvement, rather than serving as a punitive mechanism for our residents and attendings. We find that this goal is best served by establishing an overtly non-punitive system in which providers are encouraged to actively participate in identification of issues in order to reduce potential for future error, whether by high- lighting systems issues for further review, or by providing educational support to individuals or the entire group to reduce likelihood of recurrence.

    1. Limitations

Limitations associated with this study include the small study size; despite a comprehensive review system we identified very few errors and adverse events. Single center design of this study may lead to iden- tification of errors more reflective of the specific quality protocols in place at this institution rather than being generalizable to the entire field of emergency medicine. There is also a lack of standardized guide- lines defining a resident versus attending error. One might argue that each action performed by a resident and every system issue could be at- tributed to the attending on duty, however we measure error based on consensus of a 20-person multidisciplinary QA review committee. Our committee assumed that an attending physician, for example, might recommend a certain medication or dose, but would not be expected to routinely review each order to ensure that it is written correctly and therefore should not be held responsible for a resident error in this case. In contrast, diagnostic errors may originate with trainee mis- diagnosis, but there is a general expectation in our training program that one of the fundamental roles of the attending physician is to ensure misdiagnosis secondary to inexperienced trainees does not translate into an error or adverse event that reaches the patient. Therefore, diag- nostic errors are far less commonly attributed solely to the residents.

  1. Conclusions

Resident error, somewhat expectedly, is most commonly related to treatment interventions, and rarely is due to an individual resident mis- take. Resident error instead seems to reflect concomitant error on the part of the attending. Error, in general, as well as adverse events, are more likely to be attributed to an attending alone rather than to a resi- dent. Resident error instead seems to reflect concomitant error on the part of the attending either through lack of supervision or an individual attending’s own errors. Further work is needed to identify and imple- ment strategies to target these areas. Future data collection could be fur- ther categorized by level of training to better understand the rates and types of errors that occur with the progression of Emergency Medicine physicians from the beginning of their training to a supervisory role as senior residents, through junior to senior attendings. Future expansions of this research will include a greater range of dates to increase the sam- ple size. Given the unique nature of our QA data collection tool, expan- sion to other medical centers will require identifying sites with similar methods of data acquisition or creating data collection criteria that can

be applied to any EHR. Greater understanding of the rates and types of errors that occur relative to stage in training will help facilitate educa- tional models that can strive to improve patient safety in the academic Emergency Department setting.

CrediT authorship contribution statement

Jamie L. Adler: Writing – original draft, Writing – review & editing, Data curation. Kiersten Gurley: Writing – review & editing. Carlo L. Rosen: Writing – review & editing. Richard E. Wolfe: Writing – review & editing. Shamai A. Grossman: Conceptualization, Writing – review & editing, Data curation.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influ- ence the work reported in this paper.

References

  1. Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard medical practice study I. N Engl J Med. 1991;324(6):370-6.
  2. Leape LL, Brennan TA, Laird N, et al. The nature of adverse events in hospitalized pa- tients. Results of the Harvard Medical Practice Study II. N Engl J Med. 1991;324(6): 377-84.
  3. Kohn LT, Corrigan JM, Donaldson MS, editors. To err is human: Building a safer health system [Internet]. The National Academies Press; 2000. http://www.nap.edu/ openbook.php?record_id=9728.
  4. Crossing the quality chasm: a new health system for the 21st century [internet]. The National Academies Press; 2001. http://www.nap.edu/openbook.php?record_ id=10027.
  5. Public Health Council Presentation on Serious Reportable Events in Calendar Year 2015 – August 2016. Serious Reportable Event (SREs) [Internet]. [cited 2017, Mar 3]. http://www.mass.gov/eohhs/gov/departments/dph/programs/hcq/serious- reportable-event-sres.html.
  6. National Academies of Sciences, Engineering, and Medicine. Improving diagnosis in health care. Washington, DC: The National Academies Press; 2015..
  7. Gurley KL, Grossman SA, Janes M, Yu-Moe CW, Song E, Tibbles CD, et al. Rosen CL comparison of emergency medicine malpractice cases involving residents to non- resident cases. Acad Emerg Med. 2018 Sep;25(9):980-6. https://doi.org/10.1111/ acem.13430.
  8. Accreditation Council for Graduate Medical Education. Common Program Require- ments. http://www.acgme.org/What-We-Do/Accreditation/Common-Program- Requirements; 2016.
  9. Gurley KL, Burstein JL, Wolfe RE, Grossman SA. Using a rule-based system to define error in the emergency department. J Am Coll Emerg Physicians Open. 2020;1(5): 887-97.

Further Reading

  1. AHRQ Common Formats. [Cited 2017]. https://www.psoppc.org/psoppc_web/ publicpages/commonFormatsOverview.
  2. Handel DA, Wears RL, Nathanson LA, Pines JM. Using information technology to im- prove the quality and safety of emergency care. Acad Emerg Med. 2011;18(6): e45-51.
  3. Handler JA, Gillam M, Sanders AB, Klasco R. Defining, identifying, and measuring error in emergency medicine. Acad Emerg Med. 2000;7(11):1183-8.
  4. Stang AS, Wingert AS, Hartling L, Plint AC. Adverse events related to emergency department care: a systematic review. PLoS One. 2013.;8(9):e74214.
  5. Cosby K. A framework for classifying factors that contribute to error in the emer- gency room. Ann Emerg Med. 2003;42(6):815-23.

Leave a Reply

Your email address will not be published. Required fields are marked *