The inaccuracy of determining overcrowding status by using the National ED Overcrowding Study Tool

Published:August 02, 2014DOI:



      Emergency department (ED) crowding has become more common, and perceptions of crowding vary among different health care providers. The National Emergency Department Overcrowding Study (NEDOCS) tool is the most commonly used tool to estimate ED crowding but still uncertain of its reliability in different ED settings.


      The objectives of this study are to determine the accuracy of using the NEDOCS tool to evaluate overcrowding in an extremely high-volume ED and assess the reliability and consistency of different providers’ perceptions of ED crowding.

      Material and methods

      This was a 2-phase study. In phase 1, ED crowding was determined by the NEDOCS tool. The ED length of stay and number of patients who left without being seen were analyzed. In phase 2, a survey of simulated ED census scenarios was completed by different providers. The interrater and intrarater agreements of ED crowding were tested.


      In phase 1, the subject ED was determined to be overcrowded more than 75% of the time in which nearly 50% was rated as severely overcrowded by the NEDOCS tool. No statistically significant difference was found in terms of the average length of stay and the number of left without being seen patients under different crowding categories. In phase 2, 88 surveys were completed. A moderate level of agreement between health care providers was reached (κ = 0.5402, P < .0001). Test-retest reliability among providers was high (r = 0.8833, P = .0007). The strength of agreement between study groups and the NEDOCS was weak (κ = 0.3695, P < .001).


      Using the NEDOCS tool to determine ED crowding might be inaccurate in an extremely high-volume ED setting.
      To read this article in full you will need to make a payment
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to The American Journal of Emergency Medicine
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Bernstein S.L.
        • Asplin B.R.
        Emergency department crowding: old problem, new solutions.
        Emerg Med Clin North Am. 2006; 24: 821-837
        • Olshaker J.S.
        Managing emergency department overcrowding.
        Emerg Med Clin North Am. 2009; 27: 593-603
        • Hoot N.R.
        • Aronsky D.
        Systematic review of emergency department crowding: causes, effects, and solutions.
        Ann Emerg Med. 2008; 52: 126-136
        • Weiss S.J.
        • Derlet R.
        • Arndahl J.
        • Ernst A.A.
        • Richards J.
        • Fernandez-Frackelton M.
        • et al.
        Estimating the degree of emergency department overcrowding in academic medical centers: results of the National ED Overcrowding Study (NEDOCS).
        Acad Emerg Med. 2004; 11: 38-50
        • Jones S.S.
        • Allen T.L.
        • Flottemesch T.J.
        • Welch S.J.
        An independent evaluation of four quantitative Emergency Department Crowding Scales.
        Acad Emerg Med. 2006; 13: 1204-1211
        • Weiss S.J.
        • Ernst A.A.
        • Nick T.G.
        Comparison of the national emergency department overcrowding scale and the emergency department work index for quantifying emergency department crowding.
        Acad Emerg Med. 2006; 13: 513-518
        • Anneveld M.
        • van der Linden C.
        • Grootendorst D.
        • Galli-Leslie M.
        Measuring emergency department crowding in an inner city hospital in The Netherlands.
        Int J Emerg Med. 2013; 6: 21
        • McCarthy M.L.
        • Aronsky D.
        • Jones I.D.
        • Miner J.R.
        • Band R.A.
        • Baren J.M.
        • et al.
        The emergency department occupancy rate: a simple measure of emergency department crowding?.
        Ann Emerg Med. 2008; 51: 15-24
        • Richardson L.D.
        • Asplin B.R.
        • Lowe R.A.
        Emergency department crowding as a health policy issue: past development, future directions.
        Ann Emerg Med. 2002; 40: 388-393
        • Hwang U.
        • McCarthy M.L.
        • Aronsky D.
        • Asplin B.
        • Crane P.W.
        • Craven C.K.
        • et al.
        Measures of crowding in the emergency department: a systematic review.
        Acad Emerg Med. 2011; 18: 527-538
        • Weinstock M.A.
        • Colditz G.A.
        • Willett W.C.
        • Stampfer M.J.
        • Rosner B.
        • Speizer F.E.
        • et al.
        Recall (report) bias and reliability in the retrospective assessment of melanoma risk.
        Am J Epidemiol. 1991; 133: 240-245
        • Friesema I.H.
        • Veenstra M.Y.
        • Zwietering P.J.
        • Knottnerus J.A.
        • Garretsen H.F.
        • Lemmens P.H.
        • et al.
        Measurement of lifetime alcohol intake: utility of a self-administered questionnaire.
        Am J Epidemiol. 2004; 159: 809-817
        • Trzeciak S.
        • Rivers E.
        Emergency department overcrowding in the United States: an emerging threat to patient safety and public health.
        Emerg Med J. 2003; 20: 402
        • McCarthy M.L.
        • Zeger S.L.
        • Ding R.
        • Levin S.R.
        • Desmond J.S.
        • Lee J.
        • et al.
        Crowding delays treatment and lengthens emergency department length of stay, even among high-acuity patients.
        Ann Emerg Med. 2009; 54: 492-503
        • Derlet R.W.
        • Richards J.R.
        • Kravitz R.L.
        Frequent overcrowding in US emergency departments.
        Acad Emerg Med. 2001; 8: 151-155
        • Weiss S.J.
        • Ernst A.A.
        • Derlet R.
        • King R.
        • Bair A.
        • Nick T.G.
        • et al.
        Relationship between the national ED overcrowding scale and the number of patients who leave without being seen in an academic ED.
        Am J Emerg Med. 2005; 23: 288-294
        • Bernstein S.L.
        • Aronsky D.
        • Duseja R.
        • Epstein S.
        • Handel D.
        • Hwang U.
        • et al.
        The effect of emergency department crowding on clinically oriented outcomes.
        Acad Emerg Med. 2008; 16: 1-10
        • Guttmann A.
        • Schull M.J.
        • Vermeulen M.J.
        • Stukel T.A.
        Association between waiting times and short term mortality and hospital admission after departure from emergency department: population based cohort study from Ontario, Canada.
        Br Med J. 2011; 342: D2983
        • Kurowski E.M.
        • Byczkowski T.
        • Timm N.
        Return visit characteristics among patients who leave without being seen from a pediatric ED.
        Am J Emerg Med. 2012; 30: 1019-1024
        • Rootman D.B.
        • Mustard R.
        • Kalia V.
        Ahmed NPerceptions and realities of testing for alcohol and other drugs in trauma patients.
        J Trauma. 2007; 63: 1370-1373
        • Reijman M.
        • Hazes J.M.
        • Koes B.W.
        • Verhagen A.P.
        • Bierma-Zeinstra S.M.
        Validity, reliability, and applicability of seven definitions of hip osteoarthritis used in epidemiological studies: a systematic appraisal.
        Ann Rheum Dis. 2004; 63: 226-232
        • Hobbs D.
        • Kunzman S.C.
        • Tandberg D.
        • Sklar D.
        Hospital factors associated with emergency center patients leaving without being seen.
        Am J Emerg Med. 2000; 18: 767-772
        • Cardin S.
        • Afilalo M.
        • Lang E.
        • Collet J.P.
        • Colacone A.
        • Tselios C.
        • et al.
        Intervention to decrease emergency department crowding: does it have an effect on return visits and hospital readmissions?.
        Ann Emerg Med. 2003; 41: 173-185
        • Geelhoed G.C.
        • de Klerk N.H.
        Emergency department overcrowding, mortality and the 4-hour rule in Western Australia.
        Med J Aust. 2012; 196: 122-126