Predictive performance of SOFA and qSOFA”>American Journal of Emergency Medicine 38 (2020) 2074-2080

Contents lists available at ScienceDirect

American Journal of Emergency Medicine

journal homepage:

Predictive performance of SOFA and qSOFA for in-hospital mortality in severe novel coronavirus disease

Sijia Liu, MD ^a,b, Ni Yao, MD ^b,c,d,e, Yanru Qiu, MD ^e,f, Chengqi He, MD ^a,?

^a Department of Rehabilitation Medicine, West China Hospital, Sichuan University, Sichuan, PR China

^b China International Emergency Medical Team, Sichuan, PR China

^c Department of Critical Care Medicine, West China Hospital, Sichuan University, PR China

^d COVID19 Medical Team (Hubei) of West China Hospital, Sichuan University, PR China

^e COVID-19 Ward of Renmin Hospital of Wuhan University, PR China

^f Oncology Department of Renmin Hospital of Wuhan University, East Campus, PR China

a r t i c l e i n f o

Article history:

Received 15 April 2020

Received in revised form 15 June 2020 Accepted 5 July 2020

Keywords:

Sequential organ failure assessment Quick sequential organ failure assessment Novel coronavirus disease

Mortality

a b s t r a c t

Objectives: The assessment of illness severity at admission can contribute to decreased mortality in patients with the coronavirus disease (COVID-19). This study was conducted to evaluate the effectiveness of the Sequential Organ Failure Assessment and Quick Sequential Organ Failure Assessment scoring systems at ad- mission for the prediction of mortality risk in COVID-19 patients.

Methods: We included 140 critically ill COVID-19 patients. Data on demographics, clinical characteristics, and lab- oratory findings at admission were used to calculate SOFA and qSOFA against the in-hospital outcomes (survival or death) that were ascertained from the medical records. The predictive accuracy of both scoring systems was evaluated by the Receiver operating characteristic curve analysis.

Results: The area under the ROC curve for SOFA in predicting mortality was 0.890 (95% CI: 0.826-0.955), which was higher than that of qSOFA (0.742, 95% CI 0.657-0.816). An optimal cutoff of >=3 for SOFA had sensitivity, spec- ificity, positive predictive value, and negative predictive value of 90.00%, 83.18%, 50.00%, and 97.80%, respectively. Conclusions: This novel report indicates that SOFA could function as an effective adjunctive Risk-stratification tool at admission for critical COVID-19 patients. The performance of qSOFA is accepted but inferior to that of SOFA.

(C) 2020

Introduction

The coronavirus disease (COVID-19), caused by the severe acute re- spiratory syndrome coronavirus 2 (SARS-CoV-2), emerged in December 2019 and has rapidly spread worldwide [1]. Globally, the onslaught of the Covid-19 outbreak on the public health system continues despite unprecedented mobilization. The mortality of critical ill patients with COVID-19 has been reported variously as low as 11% and as high as 61% [2-5]. Bhatraju et al. [5] reported that the ratio of patients with COVID-19 over 65 years of age had died greater than patients less than 65 years of age. In a 43 patients’ study, Saurabh Aggarwal et al. re- ported three patients aged 70 years or older died [6].

In critically ill COVID-19 patients, early medical intervention to re- duce mortality depended on early effective assessment [7]. The

Abbreviations: AUC, area under the curve; COVID-19, novel coronavirus disease; GCS, Glasgow Coma Scale; NPV, negative predictive value; PPV, positive predictive value; ROC, receiver operating characteristic; SOFA, Sequential Organ Failure Assessment; qSOFA, Quick Sequential Organ Failure Assessment.

* Corresponding author at: West China Hospital, Sichuan University, Sichuan 610000, PR China.

E-mail address: [email protected] (C. He).

application of scoring systems can facilitate effective evaluation by emergency or critical care physicians to screen severe patients. At pres- ent, however, there are no specific scoring systems for the evaluation of COVID-19 patients. Another option is to adopt existing scoring systems that are used for predicting the mortality of severe patients.

Scoring systems, such as the Sequential organ failure assessment and the more recent Quick Sequential Organ Failure Assessment , can help emergency or critical care physicians for prognosis and predicting mortality. The SOFA scoring system was developed in 1996 by an international group of experts [8]. SOFA describes the condi- tion of Multiple organ dysfunction through several parameters, includ- ing the oxygenation index (arterial oxygen tension [PaO₂]/fraction of inspiration oxygen [FiO₂]), mean arterial pressure, Glasgow Coma Scale(GCS), creatinine or urine volume, bilirubin, and platelets, for the respiratory, circulatory, neurologic, renal, hepatogenic, and coagulation systems, respectively. The function of each organ systems is scored from 0 to 4, and the individual SOFA scores are then summed to a total score that is between 0 and 24. A higher score infers aggravating conditions in the patients. Although SOFA was not initially designed to predict mor- tality, several studies revealed that the SOFA score could predict mor- bidity and mortality and this specific application has been validated in

https://doi.org/10.1016/j.ajem.2020.07.019

0735-6757/(C) 2020

severely ill patients [9-13].Therefore, it is of interest to determine whether the SOFA score could predict mortality in the COVID-19 population.

The qSOFA score was introduced in the Third International Consensus Definitions for Sepsis and Septic Shock (Sepsis-3) and is composed of three clinical parameters: systolic blood pressure <= 100 mmHg, respiratory rate >= 22 breaths/min, and altered mental status. The qSOFA score was created initially for the evaluation of septic patients, but several recent studies have described its effectiveness for predicting mortality in pa- tients with different diseases [14,15].

Both SOFA and qSOFA have widespread application to screen pa- tients with life-threatening conditions in the emergency or critical care department [16]. Several studies [17-19] have suggested that the SOFA and qSOFA were high-performance assessments to evaluate the risk of mortality in critically ill patients. These scoring systems can assist emergency or critical care physicians to predict mortality of such pa- tients, especially in resource-constrained scenarios.

Table 1

The criterion? of severe/critical COVID-19.

Classification

Minor The confirmed patient has mild symptoms,without radiography results of pneumonia.

Common The confirmed patient has fever and respiratory symptoms,with

radiography results of pneumonia.

Severe the confirmed COVID-19 patients who meet any one of the follows:

Shortness of breath, with RR >= 30 times/min

Oxygen saturation <= 93%

Oxygenation index <= 300 mmHg^?

Chest radiographic images showed that the lesion progressed more than 50% within 24-48 h

Critical the severe confirmed COVID-19 patients who meet any one of the follows

Respiratory failure needs mechanical ventilation

Shock

Combining other organ failure needs ICU admission

Inclusion and exclusion criteria

The COVID-19 epidemic is a scenario of shortage of medical re- sources. However, to our knowledge, no study has validated the perfor- mance of SOFA and qSOFA for severely/critically ill COVID-19 patients. This study was conducted with an aim to evaluate the prognostic value of SOFA and qSOFA models with regard to the in-hospital mortal-

Inclusion

criteria

Exclusion criteria

Age > 18 years

Diagnosed with severe/critical COVID-19

Admitted to the COVID-19 ward between January 31, 2020 and March 7, 2020

Cases with missing data

Pregnant women

ity of COVID-19 patients.

Material and methods

Study design

This study was undertaken through a retrospective analysis of elec- tronic medical records of patients managed by an emergency medical team that was deployed to Wuhan. The team independently managed the COVID-19 ward in a local hospital that was temporarily constructed for COVID-19 patients and comprised 80 beds.

Study subjects and setting

The Diagnosis and Treatment Plan of Novel Coronavirus [20], issued by the National Health Commission of China, was adopted as a criterion for the diagnosis, classification, and treatment of COVID patients in the COVID-19 ward.

We searched for all adult patients (age > 18 years) who had been di- agnosed with severe/critical COVID-19 and admitted to the ward be- tween January 31, 2020 and March 7, 2020 through the computerized registration system of the ward. The criterion of severe/critical COVID- 19 is followed the Diagnosis and Treatment Plan of Novel Coronavirus, which can be download in http://www.nhc.gov.cn (Table 1) [20]. We identified 140 records that were eligible for study inclusion. Then, all the cases was divided into sub-group of young (age < 65 years) and se- nior (age >= 65 years) for analysis. This study was conducted with the ap- proval of the local institutional review board and was undertaken in accordance with the principles of the Declaration of Helsinki for research on human participants.

Data collection

From the medical records, we extracted data that were collected at the time of admission, including details of the demographics, clinical manifestations, and laboratory results, to calculate the SOFA and qSOFA scores and recorded the patient outcomes.

Statistical analysis

Descriptive data are reported as mean +- standard deviation or me- dian [25%quartile, 75%quartile] for continuous variables, and percent- ages for categorical data, as appropriate. We divided the study sample intosurvivorsandnon-survivorsand carriedoutintergroupcomparisons

COVID-19: novel coronavirus disease; RR: respiratory rate; ICU: intensive care unit.

* The criterion of severe/critical COVID-19 is followed the Diagnosis and Treatment Plan of Novel Coronavirus, which can be download in http://www.nhc.gov.cn.

^? High altitude areas need correction.

of variables, including demographics, clinical features, laboratory find- ings, and both SOFA and qSOFA scores. Continuous data were compared by the Student t-test or the Mann-Whitney U test; categorical variables were tested by using either the chi-square test or Fisher exact test (when the expected value was <5 in one cell), as appropriate.

The receiver operating characteristic (ROC) curve analysis was ap- plied for a discriminatory evaluation of the performance of the SOFA and qSOFA scores. The classification performance of both scoring systems to discriminate between survivors and non-survivor was eval- uated by calculating the area under the curve (AUC) of the ROC and its 95% confidence intervals (95% CI). We conducted the Delong test [21] for a comparison of the AUCs of SOFA and qSOFA.

For each rapid scoring system, the score with the largest Youden Index was defined as the optimal cutoff value for predicting COVID-19 mortality. Based on the optimal cutoff value, we calculated the sensitiv- ity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for both scoring systems. A p-value < 0.05 was accepted as statistically significant.

Data were entered into the IBM Statistical Product and Service Solutions (SPSS) version 20.0 for Windows and were analyzed by MedCalc(R) 12.7.

Results

Baseline analysis

We initially included 140 severely or critically ill patients who were admitted to the COVID-19 ward. However, the final analysis dataset in- cluded 127 patients, after excluding 13 patients with missing data (Fig. 1). In total, the mean age of the patients in this study population was 61.25 (SD +- 15.53) years, and 75 patients (70.09%)were male. The baseline characteristics and comorbidities, including diabetes, hy- pertension, cardiovascular disease, chronic pulmonary disease, cerebral vascular disease, and malignant tumor, of the survivors and non- survivors in this study sample are listed in Table 2.

From among the 140 cases that were initially identified, we excluded 13 cases because of missing data. The final analysis dataset included 127 cases.

Fig. 1. Disposition of the study population.

Table 2

Comparison of the baseline characteristics of survivors and non-survivors.

Variable	Survivors	Non-survivors	p value
	N = 107	N = 20
Male (number)	60 (56.07%)	15 (75.00%)
Female (number)	47 (43.92%)	5 (25.00%)	0.090
Age (years)	58.62 +- 14.68	75.36 +- 12.14	0.000?
Pre-hospital day (day)	10 [7, 14]	7 [4, 14]	0.308
Underlying disease
Diabetes	10 (9.35%)	0 (0.00%)	0.361
Hypertension	26 (24.30%)	5 (25.00%)	0.999
Cardiovascular disease	5 (4.67%)	2 (10.00%)	0.304
Chronic pulmonary disease	5 (4.67%)	2 (10.00%)	0.304
Cerebral vascular disease	0 (0.00%)	3 (15.00%)	0.003?
Malignant tumor	4 (3.74%)	1 (5.00%)	0.582
Symptom and Sign
Fever	87 (81.31%)	15 (75.00%)	0.544
Cough	64 (59.81%)	15 (75.00%)	0.221
Sore throat	1 (0.93%)	2 (10.00%)	0.064
Diarrhea	19 (17.76%)	6 (30.00%)	0.222
Fatigue	46 (42.99%)	12 (60.00%)	0.226
Temperature	36.72 +- 0.63	36.9 +- 0.9	0.302
PR (/min)	88.45 +- 19.05	89.2 +- 13.48	0.867
RR (/min)	20.50 +- 3.79	23.50 +- 5.16	0.003?
HR (/min)	88.45 +- 19.05	89.2 +- 13.48	0.867
SBP	130.57 +- 17.87	141.75 +- 25.34	0.018?
DBP	79.44 +- 11.56	83.8 +- 13.96	0.137
MBP	96.48 +- 12.26	103.12 +- 16.03	0.037?
SpO2 (%)	95.03 +- 4.25	85.5 +- 12.6	0.003?
GCS (score)	15 [15, 15]	15 [15, 15]	0.121
laboratory characteristics White blood cell count (109/l)	5.60 [4.49, 6.77]	8.97 [5.63, 12.68]	0.009?
Red blood cell count (1012/l)	3.90 [3.62, 4.24]	3.91 [3.34, 4.19]	0.569
lymphocyte count (109/l)	1.28 [0.81, 1.65]	0.47 [0.36, 0.92]	0.000?
Platelets count (109/l)	233.36 +- 96.22	139.25 +- 72.85	0.000?
Bilirubin (umol/l)	13.02 +- 9.32	24.33 +- 21.14	0.029?
Creatinine (umol/l)	65.25 +- 28.47	100.05 +- 55.87	0.013?
PaO2 (mmHg)	108.07 +- 32.75	76.65 +- 34.96	0.000?
FiO2 (%)	21 [21, 29]	29 [29, 43.25]	0.027?
Chest radiographic images showed pneumonia	104 (97.20%)	20 (100.00%)	0.999
Ventilator applying	8 (7.48%)	8 (40.00%)	0.001?
Length of hospital stay?	20 [10, 33.5]	6 [3.75, 9]	0.000?
SOFA (score)	1 [0, 2]	4 [3, 5]	0.000?
qSOFA (score)	0 [0, 0]	1 [0, 1]	0.000?

PR: pulse rate; HR: heart rate; RR: respiratory rate; SBP: systolic blood pressure; DBP: diastolic blood pressure; MAP: mean arterial pressure;SpO₂: peripheral oxygen saturation; GCS: Glasgow Coma Scale; PaO₂: arterial oxygen tension; FiO₂: fraction of inspiration oxygen; SOFA: Sequential Organ Failure Assessment; qSOFA: Quick Sequential Organ Failure Assessment.

* p < 0.05.

^? The length of hospital stay of survivors is the duration from admission to discharge, while the length of hospital stay of non-survivors is the duration from admission to death.

Fig. 2. Classification performances of the SOFA and qSOFA scores.

Variables that showed significant differences between the survivors and non-survivors included age, respiratory rate, systolic blood pres- sure, mean arterial pressure, peripheral oxygen saturation, platelet count, bilirubin, creatinine, PaO₂, and FiO₂. The median [25% quartile, 75% quartile] values of the SOFA and qSOFA scores of survivors versus non-survivors were 1[0,2] versus 4[3,5] and 0[0,0] versus 1[0,1], respectively.

Overall analysis

The ROC curves of SOFA and qSOFA scoring systems for predicting in-hospital mortality are depicted in Part A of Fig. 2, and their AUCs were 0.915 (95% CI: 0.852-0.957) and 0.742 (95% CI: 0.657-0.816), re-

spectively; there was significant difference between the two scoring systems(p = 0.007; Table 3).

(A)The ROC of SOFA and qSOFA models for overall cases; (B) The ROC of SOFA and qSOFA models in the subgroup of persons aged

>=65 years; (C) The ROC of SOFA and qSOFA models in the subgroup of persons aged <65 years. The area under the curve of the solid line (for SOFA) was greater than the area under the curve of the dotted line (for qSOFA) for overall and subgroup analysis. SOFA: Sequential Organ Failure Assessment; qSOFA: Quick Sequential Organ Failure Assess- ment; ROC: receiver operating characteristics curve.

Based on the largest Youden Index, an optimum cutoff value of 3 was used to predict in-hospital mortality by using the SOFA score, with a sensitivity, specificity, PPV, and NPV of 90.00%, 83.18%, 50.00%, and

Table 3

The AUC of SOFA and qSOFA models in predicting in-hospital mortality.

97.80%, respectively. Similarly, an optimum cutoff value of 1 was used to predict in-hospital mortality by using the qSOFA score, with a sensi- tivity, specificity, PPV, and NPV of 70.00%, 80.37%, 40.00%, and 93.48%,

respectively (Table 4).

The sensitivity and the specificity of SOFA were considerably higher than those of qSOFA. Therefore, the SOFA score can better predict mor- tality in critically ill COVID-19 patients than the qSOFA score.

For verify importance of SOFA as predictor in COVID-19, we conduct univariable and multivariable logistic regression analysis. In univariable logistic analysis, the age, SOFA and qSOFA were predictors in COVID-19 (p < 0.05); while in multivariable logistic regression analysis, only the age and SOFA were predictors in COVID-19 (Table 5).

Subgroup analysis

The ROC of SOFA and qSOFA scores of subgroups in predicting in- hospital mortality is depicted in Fig. 2(B and C). The AUC of SOFA and qSOFA scores in subgroup of persons aged <65 years were 0.912 and 0.703, and the difference between the two was no statistically signifi- cant (p = 0.316). And the AUC of SOFA and qSOFA scores in the sub- group of persons aged >=65 years were 0.921 and 0.773, and the difference between the two was found to be statistically significant (p = 0.033) (Table 3).

According to the largest Youden index, an optimum cut-off value of 3 was used to predict in-hospital mortality using the SOFA, with sensitiv- ity, specificity, PPV, and NPV of 100.00%, 81.36%, 21.43%, and 100.00% in the subgroup of persons aged <65 years. Likewise, an optimum cut-off value of 3 was used, with sensitivity, specificity, PPV, and NPV of 88.24%, 85.42%, 68.18%, and 95.35% in the subgroup of persons aged

>=65 years (Table 4).

According to the largest Youden index, an optimum cut-off value of 1

Models	AUC of ROC	95%C.I. of AUC	?AUC	z statistic	p value	was used to predict in-hospital mortality using the qSOFA, with sensi-
Overall SOFA	0.890	0.826 to 0.955	0.148	2.612	0.009?	tivity, specificity, PPV, and NPV of 66.67%, 76.27%, 12.50%, and 97.83% in the subgroup of persons aged <65 years. Likewise, an optimum cut-

Age >= 65 years

SOFA	0.921	0.827 to 0.973	0.148	2.131	0.033?
qSOFA	0.773	0.652 to 0.867

qSOFA Subgroup	0.742	0.657 to 0.816
Age < 65 years
SOFA	0.912	0.813 to 0.969
qSOFA	0.703	0.574 to 0.813

0.209 1.002 0.316

off value of 1 was used, with sensitivity, specificity, PPV, and NPV of

70.59%, 85.42%, 63.16%, and 89.13% in the subgroup of persons aged

>=65 years. (Table 4).

Discussion

AUC: area under the curve of the Receiver Operating Characteristic; 95%C.I.: 95% confi- dence interval; ?AUC: difference between AUCs; SOFA: Sequential Organ Failure Assess- ment; qSOFA: Quick Sequential Organ Failure Assessment.

* p < 0.05.

Given the medical resource constraints during the COVID-19 pan- demic, [22] the early evaluation of severely or critically ill COVID-19 pa- tients is paramount to ensure early medical monitoring and interventions for these patients. Thus, an important task of the

Table 4

Sensitivities, specificities, and accuracy rates of SOFA and qSOFA for predicting in-hospital mortality.

Models	Cutoff value	Youden Index	Sen. (%)	Spe. (%)	Accuracy (%)	PPV (%)	NPV (%)
Overall
SOFA	1	0.50	100.00	50.47	58.27	27.40	100.00
	2	0.67	95.00	71.96	75.59	38.78	98.72
	3	0.73?	90.00	83.18	84.25	50.00	97.80
	4	0.58	70.00	87.85	85.04	51.85	94.00
	5	0.49	55.00	94.39	88.19	64.71	91.82
	6	0.23	25.00	98.13	86.61	71.43	87.50
	7	0.18	20.00	98.13	85.83	66.67	86.78
	8	0.20	20.00	100.00	87.40	100.00	86.99
qSOFA	1	0.50?	70.00	80.37	78.74	40.00	93.48
	2	-0.03	0.00	97.20	81.89	0.00	83.87
	3	0.00	0.00	100.00	84.25	N/A	84.25
Subgroup
Age < 65 years SOFA	1	0.54	100.00	54.24	56.45	10.00	100.00
	2	0.71	100.00	71.19	72.58	15.00	100.00
	3	0.81?	100.00	81.36	82.26	21.43	100.00
	4	0.53	66.67	86.44	85.48	20.00	98.08
	5	0.27	33.33	93.22	90.32	20.00	96.49
	6	0.32	33.33	98.31	95.16	50.00	96.67
	7	0.32	33.33	98.31	95.16	50.00	96.67
	8	0.33	33.33	100.00	96.77	100.00	96.72
qSOFA	1	0.43?	66.67	76.27	75.81	12.50	97.83
	2	-0.03	0.00	96.61	91.94	0.00	95.00
	3	0.00	0.00	100.00	95.16	N/A	95.16
Age >= 65 years
SOFA	1	0.46	100.00	45.83	60.00	39.53	100.00
	2	0.67	94.12	72.92	78.46	55.17	97.22
	3	0.74?	88.24	85.42	86.15	68.18	95.35
	4	0.60	70.59	89.58	84.62	70.59	89.58
	5	0.55	58.82	95.83	86.15	83.33	86.79
	6	0.21	23.53	97.92	78.46	80.00	78.33
	7	0.16	17.65	97.92	76.92	75.00	77.05
	8	0.18	17.65	100.00	78.46	100.00	77.42
qSOFA	1	0.56?	70.59	85.42	81.54	63.16	89.13
	2	-0.02	0.00	97.92	72.31	0.00	73.44
	3	0.00	0.00	100.00	73.85	N/A	73.85

Sen.: sensitivity; Spe.: specificity; PPV: positive predictive value; NPV: negative predictive value; SOFA: Sequential Organ Failure Assessment; qSOFA: Quick Sequential Organ Failure Assessment; N/A: cannot calculate due to the denominator is zero.

* Largest Youden Index.

emergency or critical care physicians is to screen patients with in- creased mortality from among the severely or critically ill COVID-19 pa- tients [23]. However, this task has become particularly difficult due to the shortage of medical resources after the COVID-19 outbreak [24]. A scoring system can help emergency and critical care physicians to more quickly and accurately identify patients with increased mortality. Each scoring system has their own advantages and disadvantages. The SOFA score, with higher accuracy, has been widely validated as a tool for this purpose across a range of healthcare settings and environments [25]. However, SOFA is complex and time-consuming, because it needs six parameters and four of them are from laboratory results. While the superiorities of qSOFA are simple, rapid and practical, but some studies suggests that qSOFA has a low sensitive for in-hospital mortality in hos- pitalized patients with suspected infection [26]. In the absence of a

specific risk-scoring system for COVID-19, we studied the performance of the SOFA and qSOFA in the evaluation of severely or critically ill patients in order to identify an effective risk-scoring system for COVID-19 patients.

To our knowledge, this is the first research to evaluate the perfor- mance of the SOFA and qSOFA in predicting the mortality risk of se- verely ill COVID-19 patients. This retrospective study has investigated the performance of both SOFA and qSOFA scores as predictors of prog- nosis in severely or critically ill COVID-19 patients. Initially, we deter- mined that a SOFA score of >=3 and a qSOFA score of >=1 were associated with mortality in severely ill COVID-19 patients. Further- more, the study showed that the SOFA score is a highly sensitive marker of in-hospital mortality in COVID-19 patients and is prognostically supe- rior to qSOFA in this setting.

Table 5

Univariate and multivariate logistic analysis for inpatient death of COVID-19.

Variables Univariate analysis Multivariate analysis

	?2	p value	OR	95% C.I.		?2	p value	OR	95% C.I.
Age	17.509	0.000?	1.140	1.072-1.213	11.061		0.001?	1.129	1.051-1.212
Gender	5.919	0.051	3.830	1.298-11.297	1.736		0.188	2.648	0.622-11.270
SOFA	22.647	0.000?	2.070	1.534-2.793	9.917		0.002?	1.867	1.266-2.753
qSOFA	11.925	0.001?	4.699	1.952-11.309	0.511		0.475	1.618	0.432-6.055

OR: odds ratio; 95% C.I.: 95% confidence intervals for OR; SOFA: Sequential Organ Failure Assessment; qSOFA: Quick Sequential Organ Failure Assessment.

* p < 0.05.

In our analysis, the SOFA illustrated acceptable predictive perfor- mance for in-hospital mortality on ROC curve analysis in both overall and subgroup analysis. At the time of admission, a SOFA score <= 3 was highly predictive of spontaneous survival in COVID-19 patients. The high NPV in overall analysis of 97.80% for SOFA functions as a gate keeper by accurately identifying low-risk patients.

The greater predictive advantage of the SOFA is attributable to the higher number of clinical parameters and inclusion of six important variables in the scoring system, which facilitated more accurate pa- tient stratification. On univariate analysis, we found that five (mean arterial pressure, platelets count, bilirubin, creatinine, PaO₂, and FiO₂) of the six variables of the SOFA differed significantly between survivors and non-survivors. Similar findings have been reported by other observational studies. Mo et al. reported [27] that severely ill pa- tients had a lower platelet count. In a meta-analysis conducted by Lippi and colleagues, [28] low platelet counts were associated with in- creased risk of mortality in patients with COVID-19. Zhang et al. [29] reported that a higher bilirubin level and lower oxygenation index were observed in severely or critically ill COVID-19 patients; further- more, higher serum Creatinine levels at admission were a predictive factor of disease severity in their patients. The pathogenic mechanisms underlying changes in the abovementioned clinical indicators in COVID-19 patients need to be further studied, although these indica- tors can be perceived, at the very least, as predictive factors of the mortality risk in critically ill patients [30].

The AUCs of qSOFA of both overall and subgroup analysis were lower than that of SOFA; therefore, we considered the accuracy of SOFA to be higher. The prerequisite parameters for SOFA can be determined through medical assessment and laboratory examination. Nevertheless, there is a need to recognize that, as qSOFA is simple, fast, and acceptable accuracy, it can be used in the emergency room or at admission if the parameters of the SOFA cannot be acquired in time.

This novel study explored the effectiveness of the SOFA and qSOFA scoring systems for predicting mortality risk of severely or critically ill COVID-19 patients and ascertained their practical value in identifying severely or critically ill COVID-19 patients with a higher mortality risk. However, some limitations of this study need to be considered. This study is limited by the small sample size. Studies with a larger number of severely or critically ill COVID-19 patients who fulfill the inclusion criteria would en- able a more effective evaluation of the SOFA and qSOFA scoring systems. This was a single-center study and may have been af- fected by a selection bias with regard to the study population. Mul- ticenter studies are required to prospectively validate the clinical utility of the SOFA and qSOFA scores in severely or critically ill COVID-19 patients. Moreover, the prognostic value of the scoring systems among patient subgroups that were stratified by gender groups or underlying diseases was not analyzed owing to the lim- ited sample size Furthermore, because of the limited availability of information; details of out-of-hospital therapy and nursing care were not captured. Finally, there are several risk-scoring tools, such as the Acute Physiology and Chronic Health Evaluation scor- ing systems, that find widespread application in the critical care unit and emergency departments. However, in this retrospective study based on medical records, the information that could be col- lected could only permit a calculation of SOFA and qSOFA scores. Future studies are needed to comparatively explore the perfor- mance of other scoring systems in the COVID-19 patient population.

Conclusions

The SOFA scoring systems at admission have high classification per- formance in predicting the mortality risk of severely or critically ill COVID-19 patients and have acceptable NPV for screening these

patients. The performance in predicting such patients of qSOFA is ac- cepted but inferior to that of SOFA.

Funding sources

None.

CRediT authorship contribution statement Sijia Liu: Software, Methodology, Formal analysis, Writing – original

draft, Writing – review & editing. Ni Yao: Investigation, Formal analysis, Writing – original draft, Writing – review & editing. Yanru Qiu: Investi- gation, Validation, Writing – review & editing. Chengqi He: Conceptual- ization, Supervision.

Declaration of competing interest

Sijia Liu, Ni Yao, YanruQiu, and Chengqi He declared there are no conflicts of interest.

Acknowledgements

We would like to thank all the physicians and nurses who worked in the COVID-19 ward for helping us collect the data. And we would like to thank Editage (www.editage.cn) for English language editing.

References

1. Liu W, Yue XG, Tchounwou PB. Response to the COVID-19 epidemic: the Chinese ex- perience and implications for other countries. Int J Environ Res Public Health. 2020 Mar 30;17(7):E2304. https://doi.org/10.3390/ijerph17072304.
2. Huang C, Wang Y, Li X, et al. Clinical features of patients infected with 2019 novel coronavirus in Wuhan, China. Lancet. 2020;395(10223):497-506. https://doi.org/ 10.1016/S0140-6736(20)30183-5.
3. Yang X, Yu Y, Xu J, et al. Clinical course and outcomes of critically ill patients with SARS-CoV-2 pneumonia in Wuhan, China: a single-centered, retrospective, observa- tional study. Lancet Respir Med. 2020;24. https://doi.org/10.1016/S2213-2600(20) 30079-5 S2213-2600(20)30079-5.
4. Chen Nanshan, Zhou Min, Dong Xuan, Jieming Qu, Gong Fengyun, Yang Han, et al. Epidemiological and clinical characteristics of 99 cases of 2019 novel coronavirus pneumonia in Wuhan, China: a descriptive study. Lancet. 2020;395(10223): 507-13. https://doi.org/10.1016/S0140-6736(20)30211-7.
5. Bhatraju PK, Ghassemieh BJ, Nichols M, et al. Covid-19 in critically ill patients in the Seattle region — case series. N Engl J Med. 2020. https://doi.org/10.1056/ NEJMoa2004500 (in press).
6. Aggarwal Saurabh, Garcia-Telles Nelson, Aggarwal Gaurav, et al. Clinical features, laboratory characteristics, and outcomes of patients hospitalized with coronavirus disease 2019 (COVID-19): early report from the United States. Diagnosis (Berl). 2020;7(2):91-6. https://doi.org/10.1515/dx-2020-0046.
7. Rong XM, Yang L, Chu HD, Fan M. Effect of Delay in diagnosis on transmission of COVID-19. Math Biosci Eng. 2020;17(3):2725-40. https://doi.org/10.3934/mbe. 2020149.
8. Vincent JL, Moreno R, Takala J, Willatts S, Mendonca A, Bruining H. The SOFA (Sepsis- related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-related Problems of the European Society of Intensive Care Medicine. Intensive Care Med. 1996;22(7):707-10. https://doi. org/10.1007/BF01709751.
9. Schoe Abraham, Bakhshi-Raiez Ferishta, de Keizer Nicolette, et al. Mortality predic- tion by SOFA score in ICU patients after cardiac surgery; comparison with traditional prognostic-models. BMC Anesthesiol. 2020;20:65-9. https://doi.org/10.1186/ s12871-020-00975-2.
10. Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, Bellomo R, Bernard GR, Chiche JD, Coopersmith CM, Hotchkiss RS, Levy MM, Mar- shall JC, Martin GS, Opal SM, Rubenfeld GD, van der Poll T, Vincent JL, Angus DC. The third international consensus definitions for sepsis and septic shock (Sepsis- 3). JAMA 2016; 315: 801-810.
11. Ferreira FL, Bota DP, Bross A, Melot C, Vincent JL. Serial evaluation of the SOFA score to predict outcome in critically ill patients. JAMA. 2001;286:1754-8.
12. Craig DG, Zafar S, Reid TWDJ, Martin KG, Davidson JS, Hayes PC, et al. The Sequential Organ Failure Assessment score is an effective triage marker following stag- gered paracetamol (acetaminophen) overdose. Aliment Pharmacol Ther. 2012;35 (12):1408-15. https://doi.org/10.1111/j.1365-2036.2012.05102.x.
13. Tranca S, Petrisor C, Hagau N, Ciuce C. Can APACHE II, SOFA, ISS, and RTS severity scores be used to predict septic complications in Multiple trauma patients? J Crit Care Med (TarguMures). 2016;2(3):124-30. https://doi.org/10.1515/ jccm-2016-0019.
14. Jiawen Dai, Wu Jian, Bin Gu, Jianquan You, Mingdong Ding, Fei Qian, et al. Value of TCC and qSOFA score on the early diagnosis of severe trauma. Chin J Emerg Med. 2019;28(2):185-9.
15. Jiang Jianjun, Yang Jin, Mei Jing, Jin Yongmei, Lu Youjin. Head-to-head comparison of qSOFA and SIRS criteria in predicting the mortality of infected patients in the emer- gency department: a meta-analysis. Scand J Trauma Resusc Emerg Med. 2018;26(1):

    56. https://doi.org/10.1186/s13049-018-0527-9.

    Maitra S, Som A, Bhattacharjee S. Accuracy of quick sequential organ failure assess- ment (qSOFA) score and Systemic Inflammatory Response Syndrome criteria for predicting mortality in hospitalized patients with suspected infection: a meta- analysis of observational studies. Clin Microbiol Infect. 2018 Nov;24(11):1123-9. https://doi.org/10.1016/j.cmi.2018.03.032.

16. de Freitas Garbero Rodrigo, Simoes Analice Alves, Martins Gabriela Alves, da Cruz Ludmilla Vale, von Zuben Vinicius Gabriel Monteiro. SOFA and qSOFA at admission to the emergency department: diagnostic sensitivity and relation with prognosis in patients with suspected infection. Turk J Emerg Med. 2019;19(3):106-10. https://doi.org/10.1016/j.tjem.2019.05.002.
17. Schlapbach LJ, Straney L, Bellomo R, MacLaren G, Pilcher D. Prognostic accuracy of age-adapted SOFA, SIRS, PELOD-2, and qSOFA for in-hospital mortality among chil- dren with suspected infection admitted to the intensive care unit. Intensive Care Med. 2018;44(2):179-88. https://doi.org/10.1007/s00134-017-5021-8.
18. Probst L, Schalk E, Liebregts T, Zeremski V, Tzalavras A, von Bergwelt-Baildon M, et al. Prognostic accuracy of SOFA, qSOFA and SIRS criteria in hematological cancer patients: a retrospective multicenter study. J Intensive Care. 2019 Aug 7;7:41. https://doi.org/10.1186/s40560-019-0396-y.
19. National Health Commission. The diagnosis and treatment plan of novel coronavirus

    (version 6). http://www.nhc.gov.cn/yzygj/s7653p/202002/8334a8326dd94d32 9df351d7da8aefc2/files/b218cfeb1bc54639af227f922bf6b817.pdf. Accessed Feb 19, 2020.

    DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Bio- metrics. 1988;44(3):837.

20. Ji Y, Ma Z, Peppelenbosch MP, Pan Q. Potential association between COVID-19 mor- tality and health-care resource availability. Lancet Glob Health. 2020;8(4):e480. https://doi.org/10.1016/S2214-109X(20)30068-1.
21. Swiss Academy Of Medical Sciences. COVID-19 pandemic: triage for intensive-care treatment under resource scarcity. Swiss Med Wkly. 2020;3. https://doi.org/10. 4414/smw.2020.20229 w20229.
22. She-yu Li, Wen-zhi Huang, Liao Xue-lian, et al. Emergency recommendation of West China hospital for the prevention and control of novel coronavirus pneumonia in hospitals. Chin J Evid Based Med. 2020;20(2):125-33.
23. Lambden Simon, Laterre Pierre Francois, Levy Mitchell M, Francois Bruno. The SOFA score-development, utility and challenges of accurate assessment in clinical trials. Crit Care. 2019;23(1):374. https://doi.org/10.1186/s13054-019-2663-7.
24. Maitra S, Som A, Bhattacharjee S. Accuracy of quick sequential organ failure assess- ment (qSOFA) score and Systemic Inflammatory Response Syndrome criteria for predicting mortality in hospitalized patients with suspected infection: a meta- analysis of observational studies. Clin Microbiol Infect. 2018;24(11):1123-9. https://doi.org/10.1016/j.cmi.2018.03.032.
25. Mo Pingzheng, Xing Yuanyuan, Yu Xiao, Deng Liping, Zhao Qiu, Wang Hongling, et al. Clinical characteristics of refractory COVID-19 pneumonia in Wuhan, China. Clin Infect Dis. 2020;3. https://doi.org/10.1093/cid/ciaa270 ciaa270.
26. Lippi G, Plebani M, Henry BM. Thrombocytopenia is associated with severe corona- virus disease 2019 infections: a meta-analysis. Clin Chim Acta. 2020 Mar 13;506:145-8. https://doi.org/10.1016/j.cca.2020.03.022.
27. Zhang X, Cai H, Hu J, Lian J, Gu J, Zhang S. Epidemiological, clinical characteristics of cases of SARS-CoV-2 infection with abnormal imaging findings. Int J Infect Dis. 2020; 3:S1201-9712(20) 30172-7 https://doi.org/10.1016/j.ijid.2020.03.040.
28. Yang W, Cao Q, Qin L, Wang X, Cheng Z, Pan A. Clinical characteristics and imaging manifestations of the 2019 novel coronavirus disease (COVID-19): a multi-center study in Wenzhou city, Zhejiang, China. J Infect. 2020;80(4):388-93. https://doi. org/10.1016/j.jinf.2020.02.016.