Advertisement

Prognostic utilization of models based on the APACHE II, APACHE IV, and SAPS II scores for predicting in-hospital mortality in emergency department

  • Author Footnotes
    1 Both contributed equally to this study as the first author.
    Zahra Rahmatinejad
    Footnotes
    1 Both contributed equally to this study as the first author.
    Affiliations
    Department of Medical Informatics, Faculty of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran
    Search for articles by this author
  • Author Footnotes
    1 Both contributed equally to this study as the first author.
    Fariba Tohidinezhad
    Footnotes
    1 Both contributed equally to this study as the first author.
    Affiliations
    Department of Medical Informatics, Faculty of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran
    Search for articles by this author
  • Hamidreza Reihani
    Affiliations
    Department of Emergency Medicine, Faculty of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran
    Search for articles by this author
  • Fatemeh Rahmatinejad
    Affiliations
    Department of Health Information Technology, Faculty of Paramedical Sciences, Mashhad University of Medical Sciences, Mashhad, Iran
    Search for articles by this author
  • Ali Pourmand
    Affiliations
    Department of Emergency Medicine, The George Washington University, School of Medicine and Health Sciences, Washington, DC, United States
    Search for articles by this author
  • Ameen Abu-Hanna
    Affiliations
    Department of Medical Informatics, Amsterdam UMC - Location AMC, University of Amsterdam, the Netherlands.
    Search for articles by this author
  • Saeid Eslami
    Correspondence
    Corresponding author at: Department of Medical Informatics, Faculty of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran.
    Affiliations
    Department of Medical Informatics, Faculty of Medicine, Mashhad University of Medical Sciences, Mashhad, Iran

    Department of Medical Informatics, Amsterdam UMC - Location AMC, University of Amsterdam, the Netherlands.

    Pharmaceutical Research Center, Pharmaceutical Research Institute, Mashhad University of Medical Sciences, Mashhad, Iran
    Search for articles by this author
  • Author Footnotes
    1 Both contributed equally to this study as the first author.

      Abstract

      Background

      This study was designed to evaluate and compare the prognostic value of the APACHE II, APACHE IV, and SAPSII scores for predicting in-hospital mortality in the ED on a large sample of patients. Earlier studies in the ED setting have either used a small sample or focused on specific diagnoses.

      Methods

      A prospective study was conducted to include patients with higher risk of mortality from March 2016 to March 2017 in the ED of Emam Reza Hospital, northeast of Iran. Logistic regression was used to develop three models. Evaluation was performed in terms of the overall performance (Brier Score, BS, and Brier Skill Score, BSS), discrimination (Area Under the Curve, AUC), and calibration (calibration graph).

      Results

      A total of 2205 patients met the study criteria (53% male and median age of 64, IQR: 50–77). In-hospital mortality amounted to 19%. For APACHE II, APACHE IV, and SAPS II the BS was 0.132, 0.125 and 0.133 and the BSS was 0.156, 0.2, and 0.144, respectively. The AUC was 0.755 (0.74 to 0.779) for APACHE II, 0.794 (0.775 to 0.818) for APACHE IV, and 0.751 (0.727 to 0.776) for SAPS II. The APACHE IV showed significantly greater AUC in comparison to the APACHE II and SAPS II. The graphical evaluation revealed good calibration of the APACHE IV model.

      Conclusion

      APACHEIV outperformed APACHEII and SAPSII in terms of discrimination and calibration. More validation is needed for using these models for decision-making about individual patients, although they would perform best at a cohort level.

      Keywords

      To read this article in full you will need to make a payment
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'

      Subscribe:

      Subscribe to The American Journal of Emergency Medicine
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect

      References

        • Yarmohammadian M.H.
        • Rezaei F.
        • Haghshenas A.
        • Tavakoli N.
        Overcrowding in emergency departments: a review of strategies to decrease future challenges.
        J Res Med Sci. 2017; 22: 23
        • Chan S.S.
        • Cheung N.K.
        • Graham C.A.
        • Rainer T.H.
        Strategies and solutions to alleviate access block and overcrowding in emergency departments.
        Hong Kong Med J. 2015; 21: 345-352
        • Oredsson S.
        • Jonsson H.
        • Rognes J.
        • et al.
        A systematic review of triage-related interventions to improve patient flow in emergency departments.
        Scand J Trauma Resusc Emerg Med. 2011; 19: 43
        • Jin B.
        • Zhao Y.
        • Hao S.
        • et al.
        Prospective stratification of patients at risk for emergency department revisit: resource utilization and population management strategy implications.
        BMC Emerg Med. 2016; 16: 10
        • Berge K.H.
        • Maiers D.R.
        • Schreiner D.P.
        • et al.
        Resource utilization and outcome in gravely ill intensive care unit patients with predicted in-hospital mortality rates of 95% or higher by APACHE III scores: the relationship with physician and family expectations.
        Mayo Clin Proc. 2005; 80: 166-173
        • Gao H.
        • McDonnell A.
        • Harrison D.A.
        • et al.
        Systematic review and evaluation of physiological track and trigger warning systems for identifying at-risk patients on the ward.
        Intensive Care Med. 2007; 33: 667-679
        • Knaus W.A.
        • Draper E.A.
        • Wagner D.P.
        • Zimmerman J.E.
        APACHE II: a severity of disease classification system.
        Crit Care Med. 1985; 13: 818-829
        • Baltussen A.
        • Kindler C.H.
        Citation classics in critical care medicine.
        Intensive Care Med. 2004; 30: 902-910
        • Zimmerman J.E.
        • Kramer A.A.
        • McNair D.S.
        • Malila F.M.
        Acute Physiology and Chronic Health Evaluation (APACHE) IV: hospital mortality assessment for today’s critically ill patients.
        Crit Care Med. 2006; 34: 1297-1310
        • Le Gall J.R.
        • Lemeshow S.
        • Saulnier F.
        A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study.
        JAMA. 1993; 270: 2957-2963
        • Goldstein R.S.
        Management of the critically ill patient in the emergency department: focus on safety issues.
        Crit Care Clin. 2005; 21 (viii-ix): 81-89
        • Brier G.W.
        Verification of forecasts expressed in terms of probability.
        Mon Weather Rev. 1950; 78: 1-3
        • Murphy A.H.
        A new vector partition of the probability score.
        J Appl Meteorol Climatol. 1973; 12: 595-600
        • DeLong E.R.
        • DeLong D.M.
        • Clarke-Pearson D.L.
        Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach.
        Biometrics. 1988; 44: 837-845
        • Steyerberg E.W.
        • Vickers A.J.
        • Cook N.R.
        • et al.
        Assessing the performance of prediction models: a framework for traditional and novel measures.
        Epidemiology. 2010; 21: 128-138
        • R Core Team
        R: a language and environment for statistical computing.
        R Foundation for Statistical Computing, Vienna, Austria2018 (URL)
        • Chen C.
        • Chong C.
        • Liu Y.
        • Chen K.
        • Wang T.
        Risk stratification of severe sepsis patients in the emergency department.
        Emerg Med J. 2006; 23: 281-285
        • Ozaydin M.G.
        • Guneysel O.
        • Saridogan F.
        • Ozaydin V.
        Are scoring systems sufficient for predicting mortality due to sepsis in the emergency department?.
        Turk J Emerg Med. 2017; 17: 25-28
        • Williams J.M.
        • Greenslade J.H.
        • Chu K.
        • Brown A.F.
        • Lipman J.
        Severity scores in emergency department patients with presumed infection: a prospective validation study.
        Crit Care Med. 2016; 44: 539-547
        • Innocenti F.
        • Bianchi S.
        • Guerrini E.
        • et al.
        Prognostic scores for early stratification of septic patients admitted to an emergency department-high dependency unit.
        Eur J Emerg Med. 2014; 21: 254-259
        • Nguyen H.B.
        • Rivers E.P.
        • Havstad S.
        • et al.
        Critical care in the emergency department: a physiologic assessment and outcome evaluation.
        Acad Emerg Med. 2000; 7: 1354-1361
        • Saadat S.
        • Yousefifard M.
        • Asady H.
        • Moghadas Jafari A.
        • Fayaz M.
        • Hosseini M.
        The most important causes of death in Iranian population; a retrospective cohort study.
        Emergency (Tehran, Iran). 2015; 3: 16-21
        • Aminiahidashti H.
        • Bozorgi F.
        • Montazer S.H.
        • Baboli M.
        • Firouzian A.
        Comparison of APACHE II and SAPS II scoring systems in prediction of critically ill patients’ outcome.
        Emergency. 2017; 5
        • Labaf A.
        • Zarei M.
        • Jalili M.
        • Talebian M.
        • Hoseyni H.
        • Mahmodi M.
        Evaluation of the Modified Acute Physiology and Chronic Health Evaluation scoring system for prediction of mortality in patients admitted to an emergency department.
        Hong Kong J Emerg Med. 2010; 17: 464
        • Polita J.R.
        • Gomez J.
        • Friedman G.
        • Ribeiro S.P.
        Comparison of APACHE II and three abbreviated APACHE II scores for predicting outcome among emergency trauma patients.
        Rev Assoc Med Bras. 2014; 60: 381-386
        • Yang B.
        • Wang X.
        • Li Y.
        • et al.
        A newly established severity scoring system in predicting the prognosis of patients with severe fever with thrombocytopenia syndrome.
        Tohoku J Exp Med. 2017; 242: 19-25
        • Seak C.J.
        • Ng C.J.
        • Yen D.H.
        • Wong Y.C.
        • Hsu K.H.
        • Seak J.C.
        • Seak C.K.
        Performance assessment of the Simplified Acute Physiology Score II, the Acute Physiology and Chronic Health Evaluation II score.
        Am J Emerg Med. 2014; 32: 1481-1484
        • Olsson T.
        • Lind L.
        Comparison of the rapid emergency medicine score and APACHE II in nonsurgical emergency department patients.
        Acad Emerg Med. 2003; 10: 1040-1048
        • Jones A.E.
        • Fitch M.T.
        • Kline J.A.
        Operational performance of validated physiologic scoring systems for predicting in-hospital mortality among critically ill emergency department patients.
        Crit Care Med. 2005; 33: 974-978
        • Rahmatinejad Z.
        • Reihani H.
        • Tohidinezhad F.
        • Rahmatinejad F.
        • Peyravi S.
        • Pourmand A.
        • Abu-Hanna A.
        • Eslami S.
        Predictive performance of the SOFA and mSOFA scoring systems for predicting in-hospital mortality in the emergency department.
        Am J Emerg Med. 2019; 37: 1237-1241