Emergency Medicine

Modified PRIEST score for identification of very low-risk COVID patients

a b s t r a c t

Background: COVID-19 transmission remains high around the world, and severe local outbreaks continue to occur. Prognostic tools may be useful in crisis conditions as risk stratification can help determine resource alloca- tion. One published tool, the Pandemic Respiratory Infection Emergency System Triage Severity Score, seems par- ticularly promising because of its Predictive ability and ease of application at the bedside. We sought to understand the performance of a modified version of this score (mPRIEST) in our institution for identifying pa- tients with a greater than minimal risk for adverse outcome (death or organ support) at 30 days after index visit. Methods: Consecutive visits at two northern Manhattan EDs with a new diagnosis of symptomatic COVID-19 were identified between November and December of 2020. Demographic variables and clinical characteristics were obtained from chart review. Outcomes were obtained from chart review and follow-up phone call.

Results: Outcomes were available on 306 patients. The incidence of death or mechanical ventilation at 30 days for patients in patients with mPRIEST above the threshold value was 43/181 (23.8%), and for patients below 1/125 (0.8%). The sensitivity of the score for adverse outcome was 97.7% (95% CI: 93.3% to 100%).

Conclusions: This data suggests the mPRIEST score, which can be calculated from clinical variables alone, has po- tential for use in EDs to identify patients at Very low risk for adverse outcomes within 30 days of COVID diagnosis. This should be confirmed in larger formal validation studies in diverse settings.

(C) 2021

  1. Introduction

Emergency Departments (EDs) in New York City operated under de facto crisis standards of care during the first surge of COVID-19 infection in the spring of 2020. [1] Our own ED deployed a crisis clinical pathway that attempted to stratify care of patients with COVID-19 by their risk of deterioration based on the best available evidence and simple clinical criteria in order to more effectively allocate resources. [2]

Since that time, numerous prediction models for COVID-19 have been published. [3] We desired to improve our approach to prognostic stratification to use as part of a crisis pathway to use if we again reached disaster conditions during a second wave. We found one tool, the Pan- demic Respiratory Infection Emergency System Triage (PRIEST) Severity Score, which seemed promising in its Predictive power and applicability at the bedside. This score, which consists of the NEWS2 score with the addition of age, sex, and a measure of activity level, can be calculated without laboratory tests or diagnostic imaging. The authors reported that PRIEST scores >4 predicted 30 day mortality or need for organ sup- port with 98% sensitivity. [4]

* Corresponding Author.

E-mail addresses: [email protected] (E.H. Suh), [email protected] (K.J. Lang), [email protected] (L.M. Zerihun).

We undertook a project to validate the performance of this for iden- tifying patients with a greater than minimal risk for adverse outcome (death or organ support) at 30 days after index visit.

  1. Methods

Consecutive patients with a new diagnosis of symptomatic COVID- 19 were collected from two northern Manhattan EDs from November 21st to December 15th, 2020. These EDs, part of the same academic medical center and hospital system, serve a largely urban Hispanic pop- ulation and see 100,000 and 45,000 visits per year. The larger ED sees adult patients and is staffed with residents; the smaller ED sees both adult and pediatric patients; the same attending physicians cover both. Potential subjects were identified by a report in the electronic medical record that identified all ED patients who had a positive nucleic acid amplification test for SARS-CoV2 collected on that visit with no his- tory of prior positive SARS-CoV2 test result in the system. Patients were included if they were adults (age 18 years or older) who had at least one documented symptom compatible with active infection, such as fever, cough, dyspnea, sore throat, malaise, vomiting, diarrhea, weakness, or fatigue. asymptomatic patients (such as those tested after an exposure or as standard screening for hospital admission for other reasons) and patients who presented in cardiopulmonary arrest were excluded.

https://doi.org/10.1016/j.ajem.2021.04.063

0735-6757/(C) 2021

This was initially conceived and conducted as a quality and operations project and no formal sample size estimation was performed a priori. We requested and received institutional-review board approval to ana- lyze and report our data as research.

Baseline data was obtained by chart review. Abstraction was per- formed by the principal investigator and the two co-investigators, both of whom received over an hour of training; abstractors were not blinded to the objectives. All abstractors had significant previous expe- rience with chart review for research. Variables and outcomes were de- fined in a data dictionary and a written protocol for both chart abstraction and structured follow-up phone calls was produced in ad- vance. Data was abstracted into a structured worksheet on a secure cloud-based platform. Each investigator received one set of charts for review and there was no performance monitoring, but testing of 10% of the charts for inter-rater reliability between the co-investigators and the principal investigator revealed complete agreement. This con- cordance is not surprising as the majority of the variables were taken from structured fields in the medical record.

The component variables of the original PRIEST Severity Score are given in Fig. 1. As one variable in the original score, “performance sta- tus”, could not be reliably obtained from chart review, this was excluded in advance to create a modified PRIEST score (mPRIEST). There were no missing clinical variables otherwise in the data set.

The outcomes of interest were death or mechanical ventilation within 30 days of diagnosis or documented clinical resolution of acute illness after 10 days of diagnosis. Outcomes were largely determined from chart review, as both outpatient visit notes and post-discharge clinical follow-up program notes were often available. When a defini- tive outcome could not be ascertained from the EMR, follow-up phone calls were attempted using the phone number listed in the chart. Charts were re-reviewed for patients who had non-functional numbers or were otherwise unreachable; patients were considered lost to follow- up if no further visits were noted.

Statistical analysis was conducted using R, version 3.6.0 (R Core Team, Vienna, Austria). Confidence intervals were calculated with a publically available package, ‘pROC’, using DeLong’s method for AUC

Image of Fig. 1

Fig. 1. PRIEST score components.

Table 1

Baseline clinical and demographic characteristics.

Admitted

Discharged

Total

N

200

164

365a

Age, mean (IQR)

67.4 (58.5, 79.5)

50.7 (35, 62)

59.8 (47, 75)

Sex

105 Male (52.5%)

65 Male (39.6%)

171 Male (46.8%)

Ethnicityb

53.5% Hispanic, 29.5% White, 15.5% Black

54.2% Hispanic, 18.3% White, 16.5% Black

54% Hispanic, 24.4% White, 15.9% Black

or African American, 16% Declined

or African American, 14.6% Declined

or African American, 15.3% Declined

Temperature, mean (IQR)

37.4C (36.8, 37.9)

37.2C (36.9, 37.4)

37.4C (36.8, 37.6)

Heart rate, mean (IQR)

97 bpm (83, 109)

93 bpm (85, 101)

96 bpm (85, 106)

Systolic blood pressure, mean (IQR)

129 mmHg (114, 145)

131 mmHg (115, 145)

130 mmHg (114, 145)

Respiratory rate, mean (IQR)

20 (18, 22)

18 (18, 18)

19 (18, 20)

Oxygen saturation, mean (IQR)

91% (90, 96)

97% (96, 99)

94% (93, 98)

Supplemental oxygen use, n (%)

121 (60.5%)

2 (1.2%)

123 (33.7%)

Abnormal mental status, n (%)

26 (13%)

0 (0%)

26 (7.1%)

mPRIEST score, median (IQR)

8 (5, 11)

3 (2, 4)

5 (3, 9)

a One patient left without completing evaluation.

b Non-exclusive designations for race & ethnicity. The predominant race indicated was “Other Combination Not Described” at 43.5% of all patients; the vast majority of those who in- dicated “Other” chose “Hispanic or Latino or Spanish Origin” as ethnicity. There were only 3 total patients who indicated their race as Asian, Native Hawaiian, or other Pacific Islander.

Image of Fig. 2

Fig. 2. Adverse outcome by mPRIEST score and diagram.

comparison and stratified bootstrap at 2000 replicates for estimation at the pre-determined mPRIEST threshold. [5]

  1. Results

A total of 392 visits were identified and screened. 27 patients were excluded – 4 patients who presented in cardiopulmonary arrest, 22 pa- tients who had no active COVID symptoms, and 1 child; this left 365 subjects who met inclusion and exclusion criteria. Baseline characteris- tics for all patients are available in Table 1. Primary outcomes could be ascertained for 306 patients, 200 patients who were admitted and 106 patients who were discharged. The discriminatory performance of mPRIEST was excellent, with an AUC of 0.86 (95% CI: 0.81 to 0.91). The rate of adverse outcome in patients with mPRIEST >4 was 23.8% (43/181), with 36 deaths and 7 patients on mechanical ventilation but still alive at 30 days. For mPRIEST <= 4, the rate of adverse outcome was 0.8% (1/125) with no deaths at 30 days. Fig. 2: The sensitivity of mPRIEST >4 for 30-day mortality or mechanical ventilation was 97.7% (CI: 93.2% to 100%). Given the specificity of 47.2% (95% CI: 41.1% to 53.2%) at that threshold, the likelihood ratios positive and negative are

1.86 (1.64 to 2.10) and 0.05 (0.01 to 0.33), respectively.

  1. Discussion

Efficient Prognostic systems are always of great interest to EDs but more so in the setting of a COVID surge. An instrument that can quickly identify patients who are unlikely to progress to severe adverse out- come would be very useful in a crisis situation.

Many prognostic instruments have been proposed in the past year. Several are promising; and a few draw from tens of thousands of pa- tient encounters and show significant ability to predict clinical deterio- ration. [6] Many models were derived only from patients who clinicians had already decided to hospitalize, and often employ laboratory mea- surements that are unlikely to be collected on every patient with COVID-like illness. However, the PRIEST score was developed out of an ED populations and has wide adaptability, as it does not rely on di- agnostic tests or special software. Although the NEWS2 score, from which Goodacre et al. derived their tool, also shows fair correlation with outcomes, the additional elements of the PRIEST score improve performance. [7]

We found that a mPRIEST score at <= 4 had similar sensitivity in our population as Goodacre et al. reported for PRIEST <=4 in their derivation and validation cohorts, 97.7% versus 98%. We found a higher specificity,

47.2% compared to 34%. Significantly, 27% of patients in the PRIEST co- hort fell under the threshold; but over 40% of patients in our study did. In normal conditions, the direct applicability of this information is limited, as the 30-day timeframe is longer than the usual acute course of illness that is the pressing concern for most ED clinicians, and because the particular outcomes measured do not take into account the poten- tial benefit of admission on outcomes through therapeutics interven- tions or more timely identification of deterioration. However, in a situation where crisis standards of care are being considered, we believe being able to estimate such a low rate of moderate-term mortality for a significant portion of potential patients would be of immense utility in

resource allocation decisions.

This represents to our knowledge the first description of attempted external validation of the performance of this score, and the first de- scription of its potential use outside of the United Kingdom where it was originally derived. Strengths of this data include the inclusion of consecutive visits and following outcomes of both admitted and discharged patients in an American, urban, largely Hispanic patient pop- ulation. The unknown effect of emerging variants and widespread vac- cination, the necessary modification made by removal of performance status due to the study design, as well as the relatively small sample size and substantial loss to follow-up represent the major limitations of our study. Our findings should be confirmed in formal validation studies in a variety of ED settings among diverse patient populations. The performance of the tool in the setting of coronavirus variants of con- cern should also be evaluated.

Presentations

None.

Sources of support

None.

Declaration of Competing Interest

None.

References

  1. Toner E, Mukherjee V, Hanfling D, et al. Crisis standards of care: Lessons from New York City Hospitals’ COVID-19 Experience – The Emergency Medicine Perspective [In- ternet]. The Johns Hopkins Center for Health Security; 2021 [cited 2021 Mar 4]. Avail- able from: https://www.centerforhealthsecurity.org/our-work/pubs_archive/pubs- pdfs/2021/210223-NYC-CSC-ER.pdf.
  2. Suh EH, Bodnar DJ, Melville LD, Sharma M, Farmer BM. Crisis clinical pathway for COVID-19. Emerg Med J. 2020;37(11):700-4.
  3. Wynants L, Van Calster B, Collins GS, et al. Prediction models for diagnosis and prog- nosis of covid-19 infection: systematic review and critical appraisal. BMJ. 2020;369: m1328.
  4. Goodacre S, Thomas B, Sutton L, et al. Derivation and validation of a clinical severity score for acutely ill adults with suspected COVID-19: the PRIEST observational cohort study. PLoS One. 2021;16(1):e0245840.
  5. Robin X, Turck N, Hainard A, et al. pROC: an open-source package for R and S+ to an- alyze and compare ROC curves. BMC Bioinformatics. 2011;12:77.
  6. Knight SR, Ho A, Pius R, et al. Risk stratification of patients admitted to hospital with covid-19 using the ISARIC WHO clinical characterisation protocol: development and validation of the 4C Mortality score. BMJ. 2020;370:m3339.
  7. Richardson D, Faisal M, Fiori M, et al. Use of the first national early warning score re- corded within 24 hours of admission to estimate the risk of in-hospital mortality in unplanned COVID-19 patients: a retrospective cohort study. BMJ Open. 2021;11(2): e043721.

Leave a Reply

Your email address will not be published. Required fields are marked *