Article

Added value of inflammatory markers to vital signs to predict mortality in patients suspected of severe infection

a b s t r a c t

Objective: To evaluate the added value of inflammatory markers to vital signs to predict mortality in patients suspected of severe infection.

Methods: This study was conducted at an acute care hospital (471-bED capacity). Consecutive adult patients suspected of severe infection who presented to either ambulatory care or the emergency department from April 2015 to March 2017 were retrospectively evaluated. A prognostic model for predicting 30-day in- hospital mortality based on previously established vital signs (systolic blood pressure, respiratory rate, and men- tal status) was compared with an extended model that also included four inflammatory markers (C-reactive pro- tein, Neutrophil-lymphocyte ratio, mean platelet volume, and Red cell distribution width). Measures of interest were model fit, discrimination, and the net percentage of correctly reclassified individuals at the pre-specified threshold of 10% risk.

Results: Of the 1015 patients included, 66 (6.5%) died. The extended model including inflammatory markers per- formed significantly better than the vital sign model (likelihood ratio test: p b 0.001), and the c-index increased from 0.69 (range 0.67-0.70) to 0.76 (range 0.75-0.77) (p = 0.01). All included markers except C-reactive Protein Showed significant contribution to the model improvement. Among those who died, 9.1% (95% CI -2.8-21.8) were correctly reclassified by the extended model at the 10% threshold.

Conclusions: The inflammatory markers except C-reactive protein showed added predictive value to vital signs. Future studies should focus on developing and validating Prediction models for use in individualized predictions including both vital signs and the significant markers.

(C) 2020 The Authors. This is an open access article under the CC BY-NC-ND license (http://

creativecommons.org/licenses/by-nc-nd/4.0/).

Introduction

Sepsis, defined by Sepsis-3 as “life-threatening organ dysfunction caused by a dysregulated host response to infection” [1], is associated with high morbidity and mortality [2]. To improve the prognosis of pa- tients with sepsis, early detection and treatment are crucial [3].

Abbreviations: qSOFA, quick Sequential Organ function Assessment; TRIPOD, Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis; CRP, C-reactive protein; NLCR, neutrophil-lymphocyte ratio; MPV, mean plate- let volume; RDW, red cell distribution width; IQR, interquartile range; CI, confidence interval.

* Corresponding author at: Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Stratenum 6.131, PO Box 85500, 3508GA Utrecht, the Netherlands.

E-mail address: [email protected] (T. Takada).

Since changes in vital signs are often an early warning sign in critically ill patients [4,5], several Screening tools of vital sign pa- rameters have been used for early identification of infected patients at risk of death (e.g., the quick Sequential Organ Function Assess- ment [qSOFA] and National Early Warning Score 2) [6-10]. These tools also have been used as predictors to predict mortality in pa- tients suspected of infection [7,9,11]. Compared with tools that in- clude laboratory tests [12,13], those tools have the advantage of being able to immediately check on a patient’s vital signs on arrival and help physicians initiate appropriate management at a very early stage [7].

Besides using vital signs, there have been several attempts to predict the prognosis of patients with Infectious conditions using biomarkers like lactate and other inflammatory markers [14-28]. When evaluating the prognostic value of biomarkers, the interest is in the value that can be added to already available clinical information (e.g., history and

https://doi.org/10.1016/j.ajem.2019.11.030

0735-6757/(C) 2020 The Authors. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

physical examination) [29]. That is, to be useful in clinical practice, a bio- marker should add prognostic information to easily available existing measures, and whether they have predictive value by themselves is not the main focus [30].

Thus, in patients suspected of acute severe infection, the prognostic performance of biomarkers should be assessed in addition to at least vital signs, which are parameters commonly used for the screening of those patients, as discussed above [6-10]. Although the added value of lactate to qSOFA has been often evaluated [20,26,27], most inflamma- tory markers have not been adequately assessed in a sequential process in clinical practice.

We therefore quantified the added value of inflammatory markers to vital sign parameters in the prediction of poor outcome in patients suspected of severe infection. We used the Transparent Reporting of a Multivariable Prediction Model for Individual Prognosis or Diagnosis (TRIPOD) for transparent reporting in our study [31,32].

Materials and methods

This study was designed as a retrospective observational study at an acute care hospital (471-bed capacity). Approval was granted by the ethics committees of the hospital. Using a structured collection form, data were collected from electronic medical records by the authors. An- other author reviewed all the data and any disagreement was resolved by discussion among the authors.

Patients

We included consecutive outpatients aged 18 years or older who presented to either ambulatory care or the emergency department of our hospital with suspicion of acute severe infection, from April 2015 to March 2017, and in whom at least two sets of blood culture were or- dered. We included patients who presented not only to the emergency department but also to ambulatory care since walk-in patients with acute illness in our hospital are seen in ambulatory care during the day- time. As in previous studies, we used the physicians’ decision to order blood cultures as a surrogate marker for a patient at risk of severe infection [33-36]. We focused on this target population since the prediction of poor prognosis is more important in those highly suspected of severe infection than in those less suspected of severe infection. Exclusion criteria were as follows: duration of illness was unknown or longer than 1 week (because the target population was patients suspected of acute severe infection), and a past his- tory of blood disorders (some of the inflammatory markers used as candidate predictors were indices of blood cells, and were there- fore not reliable in these patients).

Candidate predictors

We a priori determined to study the three well known vital sign parameters of systolic blood pressure, respiratory rate, and mental status at presentation, which are all included in the qSOFA score [1].

Further, we studied the added prognostic value of the following blood biomarkers which are previously described as predictors of the prognosis of patients with infectious conditions: C-reactive protein (CRP) [25], neutrophil-lymphocyte ratio (NLCR) [15,17], Mean platelet volume [18,19], and Red Cell Distribution Width [14,16,21,24]. CRP levels were measured using an automated analyzer (7700; Hitachi High-Technologies Corporation, Tokyo, Japan). Complete blood count with differential, RDW and MPV were measured using an automated hematology system (XN-3100; Sysmex, Hyogo, Japan). NLCR was calculated as absolute Neutrophil count divided by absolute lymphocyte count [15,17]. We explicitly did not study the value of in- flammatory markers that were not routinely measured in our hospital,

such as procalcitonin and other newly developed markers such as mid-regional pro-adrenomedullin [28].

Outcomes

The primary outcome of this study was 30-day in-hospital mortality [37].

Statistical analysis

There were some missing values in the predictors. As shown in Table 1, these were not missing completely at random. Since ignoring these missing data can lead to biased results [38,39], missing values were multiply imputed using chained equations [40,41]. Missing data on predictors were imputed using all available information including the outcome [42]. Twenty-five imputed datasets were created and sub- sequently analyzed in accordance with methodological recommenda- tions [41,43].

To assess the predictive value of vital signs combined, we fitted a logistic regression model including systolic blood pressure, respira- tory rate, and mental status as predictors and 30-day in-hospital mortality (yes/no) as the outcome (vital sign model). Next, we fitted an extended logistic regression model by adding the four inflamma- tory markers (CRP, NLCR, MPV and RDW) simultaneously to the vital sign model (extended model). The functional form of all continuous variables (systolic blood pressure, respiratory rate, and all inflamma- tory markers) was evaluated using restricted cubic splines with three knots (two degrees of freedom), and incorporated as such in case of significant non-linearity [44,45]. The vital sign model and the extended model were compared by means of a likelihood ratio test using a p-value of 0.05.

Although the aim of this study was to quantify the added predictive value of inflammatory markers to vital signs and not to develop a novel prediction model to be used for individualized predictions in future pa- tients, we did assess the calibration and discrimination of the vital sign and extended models. Calibration plots were constructed and discrimi- nation was assessed using the c-index [32]. Also, we estimated the net percentage of correctly reclassified individuals after adding the inflam- matory markers to the vital sign model, at the cut-off point of risk prob- ability of 10%. This cut-off was predefined in accordance with the optimum threshold for starting immediate management in patients with suspected infection in Sepsis-3 [1]. For sensitivity analysis, we also assessed reclassification using a threshold of 5%, since a 10% risk of 30-day mortality could be considered too high for certain patients (e.g., when informing on relatively non-invasive Treatment decisions). Because of the high mortality of severe infection and the relatively low risk of treatment, it is considered more important to correctly re- classify those who died than those who were alive. The confidence in- terval (CI) of the net percentage of correctly reclassified individuals was obtained using the percentile method with 2000 bootstrap samples.

All analyses were performed with R statistical software (version 3.4.4; R foundation for Statistical Computing, www.R-project.org) [46].

Results

Patient characteristics

Of the 1256 potentially eligible patients, we excluded 45 with un- known illness duration, 177 with illness duration longer than 1 week, and 19 with past history of blood disorders, leaving a total of 1015 in- cluded study patients. The basic characteristics of these 1015 patients are shown in Table 1. Median age was 81 years (interquartile range [IQR] 66-87) and 48.8% were men. Respiratory infection was the most common clinical diagnosis (37.9%). Sixty-six patients (6.5%) died in

Table 1

Demographic characteristics, vital signs, and inflammatory markers

Patients characteristics

The number of patients

Patients with at least one

Complete cases pa Overallb

with missing

missing value

(n = 820)

(n = 1015)

information, n (%)

(n = 195)

Age (year), median (IQR)

0 (0.0)

77 (63, 86)

82 (67, 88)

0.004

81 (66, 87)

Male sex, n (%)

0 (0.0)

86 (44.1)

409 (49.9)

0.171

495 (48.8)

Presented to the emergency department, n (%)

0 (0.0)

62 (31.8)

373 (45.5)

0.001

435 (42.9)

Clinical diagnosis, n (%)

0 (0.0)

0.01

Respiratory

53 (27.2)

332 (40.5)

385 (37.9)

Urinary

21 (10.8)

99 (12.1)

120 (11.8)

Abdominal

40 (20.5)

90 (11.0)

130 (12.8)

Cutaneous

18 (9.2)

34 (4.1)

52 (5.1)

Neurological

2 (1.0)

5 (0.6)

7 (0.7)

Bone and joints

3 (1.5)

4 (0.5)

7 (0.7)

Others

137 (29.8)

564 (31.2)

701 (69.0)

Systolic blood pressure, (mmHg), median (IQR)

11 (1.0)

126.0 (107, 146.3)

123.0 (107.0, 141.3)

0.304

123.0 (107.0, 143.0)

Diastolic blood pressure, (mmHg), median (IQR)

11 (1.0)

71.0 (60.8, 81.0)

70.0 (59.8, 81.0)

0.576

70.0 (60.0, 81.0)

Heart rate (beats/min), median (IQR)

13 (1.3)

96.0 (80.0, 110.0)

95.0 (82.0, 109.0)

0.916

95.0 (82.0, 109.0)

Respiratory rate (breaths/min), median (IQR)

162 (16.0)

24.0 (20.0, 25.0)

22.0 (19.0, 25.0)

0.138

22.0 (20.0, 25.0)

Consciousness disturbance, n (%)

0 (0.0)

31 (15.9)

210 (25.6)

0.006

241 (23.7)

CRP (mg/dL), median (IQR)c

7 (0.7)

5.3 (1.3, 14.4)

5.6 (1.5, 13.0)

0.715

5.5 (1.4, 13.2)

NLCR, median (IQR)

33 (3.3)

8.8 (5.2, 15.4)

8.6 (4.6, 16.3)

0.808

8.6 (4.6, 16.2)

MPV (fL), median (IQR)

8 (0.8)

9.8 (9.0, 10.8)

9.9 (9.2, 10.6)

0.371

9.9 (9.2, 10.6)

RDW, median (IQR)

6 (0.6)

13.5 (12.8, 15.1)

13.5 (12.8, 14.5)

0.299

13.5 (12.8, 14.6)

Death, n (%)

0 (0.0)

14 (7.2)

52 (6.3)

0.791

66 (6.5)

IQR = interquartile range, CRP = C-reactive protein, NLCR = neutrophil-lymphocyte ratio, MPV = mean platelet volume, RDW = red cell distribution width.

a Comparison between patients with at least one missing value and complete cases.

b Data includes imputed data for missing values.

c To convert CRP to nmol/L, multiply values by 9.524.

the hospital within 30 days. The mortality rate in those presented to am- bulatory care, walk-in patients in the emergency department, and those taken to the emergency department by ambulance was 3.3% (8/243), 4.5% (15/337), and 9.9% (43/435), respectively.

Vital sign model and extended model

The vital sign model and extended model are shown in Table 2. Sys- tolic blood pressure and RDW were incorporated into the models using restricted cubic splines with three knots to account for the non-linear relationship with the outcome. In the vital sign model, respiratory rate and consciousness disturbance were significant, while systolic blood pressure was not. In the extended model, all inflammatory markers

Table 2

Formula of the vital sign model and the extended model.

except CRP were significant. Accordingly, model fit improved when adding all inflammatory markers (likelihood ratio test p b 0.001). The vital sign model showed slight over-prediction at lower predicted prob- abilities (below 0.05), which improved when extending the model with inflammatory markers (Fig. 1). Fig. 2 shows the change in probability between the vital sign model and the extended model. The improve- ment in estimated probability was relatively large in the higher deciles among patients who died, while it was small among patients who were alive after 30 days. The c-index of the vital sign model and the extended model was 0.69 (range 0.67-0.70) and 0.76 (range 0.75-0.77), respec- tively (p = 0.01).

Reclassification

Among 66 patients who died within 30 days, 9.1% (95% CI

-2.8-21.8) were correctly reclassified by the extended model at a risk threshold of 10%, while 1.3% (95% CI -1.2-3.6) of the 949 patients who were alive at 30 days were correctly reclassified (Table 3). At the

Intercept and predictors in the model

Vital sign model Extended model Coefficient Standard p Coefficient Standard p

error error

threshold of 5%, the corresponding values of those who died within 30 days and those who were alive at 30 days were 15.2% (95% CI 1.7-27.3) and 1.8% (95% CI -1.2-5.6), respectively (Table A.1).

Analysis per inflammatory marker

Intercept

-2.580

1.258

0.041

-13.352

4.066

0.001

Systolic blood

-0.013

0.010

0.344a

-0.012

0.011

0.501a

pressure 1

Systolic blood

0.011

0.013

0.012

0.014

pressure 2

Respiratory rate

0.043

0.020

0.037

0.037

0.022

0.091

Consciousness

1.029

0.264

b0.001

0.862

0.274

0.002

disturbance

CRP

0.006

0.013

0.650

NLCR

0.019

0.008

0.018

MPV

0.330

0.101

0.001

RDW 1

0.525

0.278

0.004a

RDW 2

-0.450

0.347

CRP = C-reactive protein, NLCR = neutrophil-lymphocyte ratio, RDW = red cell distribu- tion width, MPV = mean platelet volume.

a Since the variable was transformed using restricted cubic splines to account for the nonlinearity, there were two estimated coefficients for the variable. The p value is the pooled estimate of the two coefficients.

The models in which each inflammatory marker was separately added to the vital sign model are shown in Table A.2 and showed similar results. All inflammatory markers other than CRP were significant; CRP did not show a significant contribution even when the other inflamma- tory markers were not included in the model. However, when added as a single marker, none of the markers significantly improved the c-index of the vital sign model (Table A.3).

Discussion

We quantified the added value of inflammatory markers to vital signs in the prediction of 30-day in-hospital mortality in patients suspected of severe infection. When adding the four inflammatory

Vital sign model

Extended model

Predicted probability

Observed frequency

Fig. 1. Calibration plots of the vital sign model and the extended model. Ideally, all groups of predicted probabilities fit close to the dashed diagonal line (perfect calibration). Vertical lines in each group represent 95% confidence intervals of estimated probability.

markers to the vital sign model, the model improved significantly. While NLCR, MPV, and RDW contributed to the improvement of the model, CRP did not. These findings were consistent when each marker was separately added to the vital sign model. More accurate prediction of poor outcome can be expected by adding NLCR, MPV, and RDW to vital sign parameters.

We also estimated the net percentage of correctly reclassified individuals after adding the inflammatory markers to the vital sign model. Among patients who died within 30 days, the net per- centage of those correctly reclassified was 9.1% at the thresholds of 10% (this prevented misclassification of 91 per 1000 patients who died within 30 days). At the threshold of 5%, more patients who died were correctly reclassified (15.2%). Among those who were alive at 30 days, very few patients were correctly reclassified: however, the improvement among those who died is more crucial in clinical practice of patients suspected of severe infection, a fatal condition.

When assessing the utility of inflammatory markers in patients with suspected severe infection, the fact that it requires time to

obtain the results of the markers should be considered. turnaround time for a complete blood count including NLCR, MPV, and RDW is around 30 min, and it takes longer for quantitative measurement of CRP [47]. Given this and the fact that our study showed nonsignifi- cant predictive contribution of CRP, we recommend the use of markers included in a complete blood count, and not CRP. The 2018 updated version of the Surviving Sepsis Campaign has integrated its 3-hour and 6-hour bundles into a single-hour bundle [3]. It em- phasizes starting treatment immediately in patients with sepsis and septic shock. Thus, 30 min is precious in the management of sep- tic patients. As our analyses did not incorporate this negative aspect of inflammatory markers, the effect of delaying the treatment by waiting for the result should be considered separately.

Among the inflammatory markers, CRP showed much poorer perfor- mance than the others, both in isolation and in combination with other markers. This is consistent with the results of previous studies that showed NLCR and RDW predict mortality better than CRP in patients with infectious conditions [16,17]. However, these studies compared the performance of each marker as a sole predictor, not in the sequential

Change of probability for mortality between the extended model and the vital sign model

Patients who died

Patients who were alive

Decile of probability based on the vital sign model

Fig. 2. Change of probability for in-hospital mortality between the vital sign model and the extended model within decile of probability estimated by the vital sign model. The left panel is for those who died within 30 days, and the right panel is for those who were alive at 30 days. It is preferable that the change of probability is positive (N0) for those who died within 30 days, while it is negative (b0) for those who were alive at 30 days.

Table 3 Reclassification by adding the inflammatory markers to the vital sign model at the thresh- old of 10%.

Vital sign model Extended model

b10% risk >=10% risk

In 66 patients who died

b10% risk 30 12

>=10% risk 6 18

In 949 patients who were alive

b10% risk 719 62

>=10% risk 74 94

The net percentage of correctly reclassified individuals was calculated as (12-6)/66 = 9.1% for patients who died within 30 days, and (74-62)/949 = 1.3% for patients who were alive at 30 days.

process of clinical practice. Since patient examinations usually start with History taking and physical examination, the usefulness of subsequent tests that include inflammatory markers should be assessed by quanti- fying the added value of the test to the information obtained beforehand [29].

Our study had several limitations. First, to evaluate the added value of the inflammatory markers, we derived the vital sign model and extended model. While those models were used to quantify the added value of the markers, they were not developed for actual implementation in clinical practice. Additional research is required for the aim of developing an optimum prediction model that incorporates the inflammatory markers. Second, we did not capture data on treatments received and could therefore not include this information in the models. Since treatment could have been chosen based on vital sign parameters and the inflamma- tory markers, the Predictive performance of those variables could have been underestimated [48]. Third, as a rule of thumb, a sample size of at least 10 patients with the outcome events per candidate predictor is recommended to build a reliable logistic regression model [32]. Since there were nine parameters included in the ex- tended model (seven candidate predictors, of which two continu- ous predictors were modeled flexibly using an extra degree of freedom), it was desirable to have 90 patients with an event: how- ever, there were only 66 events in our study. This also caused the wide confidence intervals of the net percentage of reclassification for the patients who died within 30 days. Also, this issue of small sample size could explain nonsignificant effect of systolic blood pressure in both the vital sign and extended models. Thus, our find- ings should be further validated in studies with a larger sample size. Fourth, we could not evaluate the performance of lactate and newly developed markers since we did not routinely measure them in all patients who underwent blood cultures. It has been reported that lactate improves the predictive performance of qSOFA in patients

with suspected sepsis in emergency department settings [26]. Eval- uation of the added value of lactate compared to the studied inflam- matory markers remains an interesting topic of further investigation. On the other hand, among newly developed markers, mid-regional pro-adrenomedullin has been reported to improve the predictive performance of qSOFA in older patients with infec- tious conditions [22]. Although this study was conducted in a very small cohort and was limited to older subjects, such newly devel- oped markers have potential to support physicians’ decision mak- ing. Finally, we did not integrate patients’ history as predictors into the model. This was because the aim of this study was to focus on the additive value of inflammatory markers to vital sign parameters, that are commonly advocated and used for screening in patients with suspected infectious conditions. In future studies, it would be also relevant to evaluate the added value of inflamma- tory markers to physicians’ judgement based on the information available prior to blood tests.

Conclusions

Of the investigated inflammatory markers, NLCR, MPV, and RDW showed significantly added value to vital sings in the prediction of mor- tality in patients suspected of severe infection, while CRP did not. Future studies should focus on developing and validating prediction models for individualized predictions including both vital signs and the significant markers from our study.

Author contributions

T Takada had full access to all of the data in the study and takes responsibility for integrity of the data and the accuracy of the data analysis. T Takada contributed to study design, data collection, in- terpretation of data, and writing the manuscript. JH and KGMM contributed to data analysis, interpretation of data, and writing the manuscript. TY, KF, RF contributed to study design, data collec- tion, interpretation of data, and writing the manuscript. JM, T Takeshima, MH, TA contributed to study design, data analysis, inter- pretation of data, and writing the manuscript. All authors read and approved the final manuscript.

Funding

This study received no funding. Takada was supported by the Uehara Memorial Foundation.

Declaration of competing interest

None.

Appendix A

Table A.1

Reclassification by adding the inflammatory markers to the vital sign model at a threshold of 5%

Vital sign model

Extended model

b5% risk

>=5% risk

In 66 patients who died

b5% risk

8

15

>=5% risk

5

38

In 949 patients who were alive

b5% risk

445

109

>=5% risk

126

269

The net percentage of correctly reclassified individuals was calculated as (15-5)/66 = 15.2% for patients who died within 30 days, and (126-109)/949 = 1.8% for patients who were alive at 30 days.

Table A.2

Formula of the model with each inflammatory marker added separately

Intercept and predictors in

Model CRP

Model NLCR

Model MPV

Model RDW

the model

Coefficient

Standard p Coefficient error

Standard p Coefficient error

Standard p Coefficient Standard p

error error

Intercept

-2.810

1.274

0.028

-2.737

1.280

0.033

-6.384

1.709

b0.001

-9.770

3.801

0.010

Systolic blood pressure 1

-0.012

0.010

0.471a

-0.014

0.010

0.399a

-0.010

0.010

0.464a

-0.014

0.010

0.328a

Systolic blood pressure 2

0.011

0.013

0.014

0.013

0.008

0.013

0.013

0.013

Respiratory rate

0.038

0.021

0.066

0.037

0.021

0.076

0.038

0.021

0.068

0.044

0.020

0.031

Consciousness disturbance

1.031

0.265

b0.001

0.944

0.268

b0.001

1.012

0.266

b0.001

0.943

0.268

b0.001

CRP

0.019

0.012

0.107

NLCR

0.022

0.008

0.004

MPV

RDW 1

0.358

0.102

b0.001

0.535

0.270

0.006a

RDW 2

-0.484

0.334

CRP = C-reactive protein, NLCR = neutrophil-lymphocyte ratio, MPV = mean platelet volume, RDW = red cell distribution width

a Since the variable was transformed using restricted cubic splines to account for the

nonlinearity, there were two estimated coefficients for the variable. The p value is the pooled estimate of the two coefficients.

Table A.3

Performance of the model with each inflammatory marker added separately

The p value of the likelihood ratio test

AUC (range)

The p value for comparison of AUC with the vital sign model

CRP

0.118

0.701 (0.691, 0.707)

0.345

NLCR

0.007

0.707 (0.694, 0.715)

0.276

MPV

b0.001

0.714 (0.724, 0.724)

0.197

RDW

0.006

0.717 (0.701, 0.726)

0.243

AUC = Area under the curve, CRP = C-reactive protein, NLCR = neutrophil-lymphocyte ratio, MPV = mean platelet volume, RDW = red cell distribution width.

References

  1. Singer M, Deutschman CS, Seymour CW, Shankar-Hari M, Annane D, Bauer M, et al. The third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA 2016;315(8):801-10. https://doi.org/10.1001/jama.2016.0287.
  2. Liu V, Escobar GJ, Greene JD, Soule J, Whippy A, Angus DC, et al. Hospital deaths in patients with sepsis from 2 independent cohorts. JAMA 2014;312(1):90-2. https:// doi.org/10.1001/jama.2014.5804.
  3. Levy MM, Evans LE, Rhodes A. The surviving sepsis campaign bundle: 2018 update. Crit Care Med 2018;46(6):997-1000. https://doi.org/10.1097/CCM. 0000000000003119.
  4. Franklin C, Mathew J. Developing strategies to prevent Inhospital CArdiac arrest: an- alyzing responses of physicians and nurses in the hours before the event. Crit Care Med 1994;22(2):244-7.
  5. Schein RM, Hazday N, Pena M, Ruben BH, Sprung CL. Clinical antecedents to in- hospital cardiopulmonary arrest. Chest 1990;98(6):1388-92.
  6. Corfield AR, Lees F, Zealley I, Houston G, Dickie S, Ward K, et al. Utility of a single early warning score in patients with sepsis in the emergency department. Emerg Med J 2014;31(6):482-7. https://doi.org/10.1136/emermed-2012-202186.
  7. Freund Y, Lemachatti N, Krastinova E, Van Laer M, Claessens YE, Avondo A, et al. Prognostic accuracy of Sepsis-3 criteria for in-hospital mortality among patients with suspected infection presenting to the emergency department. JAMA 2017; 317(3):301-8. https://doi.org/10.1001/jama.2016.20329.
  8. Rudd KE, Seymour CW, Aluisio AR, Augustin ME, Bagenda DS, Beane A, et al. Associ- ation of the quick sequential (Sepsis-related) organ failure assessment (qSOFA) score with excess hospital mortality in adults with suspected infection in Low- and middle-income countries. JAMA 2018;319(21):2202-11. https://doi.org/10.1001/ jama.2018.6229.
  9. Serafim R, Gomes JA, Salluh J, Povoa P. A comparison of the quick-SOFA and systemic inflammatory response syndrome criteria for the diagnosis of sepsis and prediction of mortality: a systematic review and meta-analysis. Chest 2018;153(3):646-55. https://doi.org/10.1016/j.chest.2017.12.015.
  10. Seymour CW, Liu VX, Iwashyna TJ, Brunkhorst FM, Rea TD, Scherag A, et al. Assess- ment of clinical criteria for sepsis: for the third international consensus definitions for sepsis and septic shock (Sepsis-3). JAMA 2016;315(8):762-74. https://doi.org/ 10.1001/jama.2016.0288.
  11. Churpek MM, Snyder A, Han X, Sokol S, Pettit N, Howell MD, et al. Quick sepsis- related organ failure assessment, systemic inflammatory response syndrome, and early warning scores for detecting clinical deterioration in infected patients outside the intensive care unit. Am J Respir Crit Care Med 2017;195(7):906-11. https://doi. org/10.1164/rccm.201604-0854OC.
  12. Bone RC, Sibbald WJ, Sprung CL. The ACCP-SCCM consensus conference on sepsis and organ failure. Chest 1992;101(6):1481-3.
  13. Knaus WA, Wagner DP, Draper EA, Zimmerman JE, Bergner M, Bastos PG, et al. The APACHE III prognostic system. risk prediction of hospital mortality for critically ill hospitalized adults. Chest 1991;100(6):1619-36.
  14. Bazick HS, Chang D, Mahadevappa K, Gibbons FK, Christopher KB. Red cell distribu- tion width and all-cause mortality in critically ill patients. Crit Care Med 2011;39(8): 1913-21. https://doi.org/10.1097/CCM.0b013e31821b85c6.
  15. Cataudella E, Giraffa CM, Di Marca S, Pulvirenti A, Alaimo S, Pisano M, et al. Neutro- phil-to-lymphocyte ratio: an emerging marker predicting prognosis in elderly adults with community-acquired pneumonia. J Am Geriatr Soc 2017;65(8):1796-801. https://doi.org/10.1111/jgs.14894.
  16. Chen CK, Lin SC, Wu CC, Chen LM, Tzeng IS, Chen KF. STARD-compliant article: the utility of red cell distribution width to predict mortality for septic patients visiting the emergency department. Medicine 2016;95(24):e3692. https://doi.org/10.1097/ MD.0000000000003692.
  17. de Jager CP, Wever PC, Gemen EF, Kusters R, van Gageldonk-Lafeber AB, van der Poll T, et al. The neutrophil-lymphocyte count ratio in patients with community- acquired pneumonia. PLoS One 2012;7(10):e46561. https://doi.org/10.1371/jour- nal.pone.0046561.
  18. Gao Y, Li Y, Yu X, Guo S, Ji X, Sun T, et al. The impact of various platelet indices as Prognostic markers of septic shock. PLoS One 2014;9(8):e103761. https://doi.org/ 10.1371/journal.pone.0103761.
  19. Golcuk Y, Golcuk B, Bilge A, Irik M, Dikmen O. Combination of mean platelet volume and the CURB-65 score better predicts 28-day mortality in patients with community-acquired pneumonia. Am J Emerg Med 2015;33(5):648-52. https:// doi.org/10.1016/j.ajem.2015.02.001.
  20. Ho KM, Lan NS. Combining quick Sequential Organ Failure Assessment with plasma lactate concentration is comparable to standard Sequential Organ Failure Assess- ment score in predicting mortality of patients with and without suspected infection. J Crit Care 2017;38:1-5. https://doi.org/10.1016/j.jcrc.2016.10.005.
  21. Jo YH, Kim K, Lee JH, Kang C, Kim T, Park HM, et al. Red cell distribution width is a prognostic factor in severe sepsis and septic shock. Am J Emerg Med 2013;31(3): 545-8. https://doi.org/10.1016/j.ajem.2012.10.017.
  22. Julian-Jimenez A, Yanez MC, Gonzalez-Del Castillo J, Salido-Mota M, Mora-Ordonez B, Arranz-Nieto MJ, et al. Prognostic power of biomarkers for short-term mortality in the elderly patients seen in Emergency Departments due to infections. Enferm Infecc Microbiol Clin 2019;37(1):11-8. https://doi.org/10.1016/j.eimc.2017.11.017.
  23. Kelly BJ, Lautenbach E, Nachamkin I, Coffin SE, Gerber JS, Fuchs BD, et al. Combined biomarkers predict acute mortality among critically ill patients with suspected Sep- sis. Crit Care Med 2018;46(7):1106-13. https://doi.org/10.1097/CCM. 0000000000003137.
  24. Ku NS, Kim HW, Oh HJ, Kim YC, Kim MH, Song JE, et al. Red blood cell distribution width is an independent predictor of mortality in patients with gram-negative bac- teremia. Shock 2012;38(2):123-7. https://doi.org/10.1097/SHK.0b013e31825e2a85.
  25. Povoa P. C-reactive protein: a valuable marker of sepsis. Intensive Care Med 2002;28

    (3):235-43. https://doi.org/10.1007/s00134-002-1209-6.

    Shetty A, MacDonald SP, Williams JM, van Bockxmeer J, de Groot B, Esteve Cuevas LM, et al. Lactate N/=2 mmol/L plus qSOFA improves utility over qSOFA alone in emergency department patients presenting with suspected sepsis. Emerg Med Australas 2017;29(6):626-34. https://doi.org/10.1111/1742-6723.12894.

  26. Song H, Moon HG, Kim SH. Efficacy of quick Sequential Organ Failure Assessment with lactate concentration for predicting mortality in patients with community- acquired pneumonia in the emergency department. Clin Exp Emerg Med 2019;6

    :1-8. https://doi.org/10.15441/ceem.17.262.

  27. Spoto S, Cella E, de Cesaris M, Locorriere L, Mazzaroppi S, Nobile E, et al. Procalcitonin and MR-proadrenomedullin combination with SOFA and qSOFA scores for sepsis diagnosis and prognosis: a diagnostic algorithm. Shock 2018;50(1):44-52. https://doi.org/10.1097/SHK.0000000000001023.
  28. Moons KG, de Groot JA, Linnet K, Reitsma JB, Bossuyt PM. Quantifying the added value of a diagnostic test or marker. Clin Chem 2012;58(10):1408-17. https://doi. org/10.1373/clinchem.2012.182550.
  29. Kibe S, Adams K, Barlow G. Diagnostic and Prognostic biomarkers of sepsis in critical care. J Antimicrob Chemother 2011;66(Suppl. 2):ii33-40. https://doi.org/10.1093/ jac/dkq523.
  30. Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent Reporting of a multivar- iable prediction model for Individual Prognosis or Diagnosis (TRIPOD): the TRIPOD statement. Ann Intern Med 2015;162(1):55-63. https://doi.org/10.7326/M14-0697.
  31. Moons KG, Altman DG, Reitsma JB, Ioannidis JP, Macaskill P, Steyerberg EW, et al. Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD): explanation and elaboration. Ann Intern Med 2015;162

    :W1-73. https://doi.org/10.7326/M14-0698.

  32. Churpek MM, Snyder A, Sokol S, Pettit NN, Edelson DP. Investigating the impact of different suspicion of infection criteria on the accuracy of quick sepsis-related organ failure assessment, systemic inflammatory response syndrome, and early warning scores. Crit Care Med 2017;45(11):1805-12. https://doi.org/10.1097/CCM. 0000000000002648.
  33. Marwick CA, Guthrie B, Pringle JE, McLeod SR, Evans JM, Davey PG. Identifying which septic patients have increased mortality risk using severity scores: a cohort study. BMC Anesthesiol 2014;14(1). https://doi.org/10.1186/1471-2253-14-1.
  34. Shapiro NI, Wolfe RE, Moore RB, Smith E, Burdick E, Bates DW. Mortality in Emer- gency Department Sepsis (MEDS) score: a prospectively derived and validated clin- ical prediction rule. Crit Care Med 2003;31(3):670-5. https://doi.org/10.1097/01. CCM.0000054867.01688.D1.
  35. Yamamoto S, Yamazaki S, Shimizu T, Takeshima T, Fukuma S, Yamamoto Y, et al. prognostic utility of serum CRP levels in combination with CURB-65 in patients

    with clinically suspected sepsis: a decision curve analysis. BMJ Open 2015;5(4): e007049. https://doi.org/10.1136/bmjopen-2014-007049.

    Graham PL, Cook DA. Prediction of risk of death using 30-day outcome: a practical end point for quality auditing in intensive care. Chest 2004;125(4):1458-66.

  36. Donders AR, van der Heijden GJ, Stijnen T, Moons KG. Review: a gentle introduction to imputation of missing values. J Clin Epidemiol 2006;59(10):1087-91. https://doi. org/10.1016/j.jclinepi.2006.01.014.
  37. Janssen KJ, Donders AR, Harrell Jr FE, Vergouwe Y, Chen Q, Grobbee DE, et al. Missing covariate data in medical research: to impute is better than to ignore. J Clin Epidemiol 2010;63(7):721-7. https://doi.org/10.1016/j.jclinepi.2009.12.008.
  38. Vergouwe Y, Royston P, Moons KG, Altman DG. Development and validation of a prediction model with missing predictor data: a practical approach. J Clin Epidemiol 2010;63(2):205-14. https://doi.org/10.1016/j.jclinepi.2009.03.017.
  39. van Buuren S. Flexible imputation of missing data. . 2nd edBoca Raton: CRC Press; 2012.
  40. Moons KG, Donders RA, Stijnen T, Harrell Jr FE. Using the outcome for imputation of missing predictor values was preferred. J Clin Epidemiol 2006;59(10):1092-101. https://doi.org/10.1016/j.jclinepi.2006.01.009.
  41. Marshall A, Altman DG, Holder RL, Royston P. Combining estimates of interest in prognostic modelling studies after multiple imputation: current practice and guide- lines. BMC Med Res Methodol 2009;9:57. https://doi.org/10.1186/1471-2288-9-57.
  42. Styerberg EW. clinical prediction model: a practical approach to development, vali- dation, and updating. New York: Springer; 2009.
  43. Harrel FE. Regression modeling strategies: with applications to linear models, logis- tic regression, and survival analysis. . 2nd edNew York: Springer; 2015.
  44. R Core Team. R: a language and environment for statistical computing. Available at http://www.R-project.org/.
  45. Hawkins RC. Laboratory turnaround time. Clin Biochem Rev 2007;28(4):179-94.
  46. Groenwold RH, Moons KG, Pajouheshnia R, Altman DG, Collins GS, Debray TP, et al. Explicit inclusion of treatment in prognostic modeling was recommended in obser- vational and randomized settings. J Clin Epidemiol 2016;78:90-100. https://doi.org/ 10.1016/j.jclinepi.2016.03.017.

Leave a Reply

Your email address will not be published. Required fields are marked *