Two-step predictive model for early detection of emergency department patients with prolonged stay and its management implications
predictive model for early det”>American Journal of Emergency Medicine 40 (2021) 148-158
Contents lists available at ScienceDirect
American Journal of Emergency Medicine
journal homepage:
Two-step predictive model for early detection of emergency department patients with prolonged stay and its management implications
James P. d’Etienne, M.D. a, Yuan Zhou, Ph.D. b, Chen Kan, Ph.D. b, Sajid Shaikh, M.S. c, Amy F. Ho, M.D. a, Eniola Suley, M.S. b, Erica C. Blustein, M.D. a, Chet D. Schrader, M.D. a,d,
Nestor R. Zenarosa, M.D. a,d, Hao Wang, M.D., Ph.D. a,d,?
a Department of Emergency Medicine, John Peter Smith Health Network, 1500 S. Main St., Fort Worth, TX 76104, USA
b Department of Industrial, Manufacturing, and Systems Engineering, The University of Texas at Arlington, 701 S. Nedderman Dr., Arlington, TX 760199, USA
c Department of Information Technology, John Peter Smith Health Network, 1500 S. Main St., Fort Worth, TX 76104, USA
d Integrative Emergency Services, 4835 LBJ Fwy Suite 900, Dallas, TX 75244, USA
a r t i c l e i n f o
Article history:
Received 25 August 2019
Received in revised form 20 January 2020
Accepted 27 January 2020
Keywords:
Emergency department Length of stay Management
Predict Model
a b s t r a c t
Objective: To develop a novel model for predicting Emergency Department (ED) prolongED length of stay patients upon triage completion, and further investigate the benefit of a targeted intervention for patients with prolonged ED LOS.
Materials and methods: A two-stEP model to predict patients with prolonged ED LOS (N16 h) was constructed. This model was initially used to predict ED resource usage and was subsequently adapted to predict patient ED LOS based on the number of ED resources using binary logistic regressions and was validated internally with accuracy. Finally, a Discrete event simulation was used to move patients with predicted prolonged ED LOS directly to a virtual clinical decision unit . The changes of ED crowding status (Overcrowding, Crowding, and Not-Crowding) and savings of ED bed-hour equivalents were estimated as the measures of the efficacy of this intervention.
Results: We screened a total of 123,975 patient visits with final enrollment of 110,471 patient visits. The overall accuracy of the final model predicting prolonged patient LOS was 67.8%. The C-index of this model ranges from
0.72 to 0.82. By implementing the proposed intervention, the simulation showed a 12% (1044/8760) reduction of ED overcrowded status – an equivalent savings of 129.3 ED bed-hours per day. Conclusions: Early prediction of prolonged ED LOS patients and subsequent (simulated) early CDU transfer could lead to more efficiently utilization of ED resources and improved efficacy of ED operations. This study provides evidence to support the implementation of this novel intervention into real healthcare practice.
(C) 2020 Published by Elsevier Inc.
Approximately 1.4 billion patients visited the Emergency Depart- ment (ED) in the US between 2006 and 2016 [1]. ED patient length of stay (LOS) varies among different EDs across the nation [2-4]. Previous studies have shown that prolonged ED LOS could reduce the quality of care and increase adverse events among ED patients [3,5,6]. With each additional hour in the ED, patient satisfaction scores and their likelihood
* Corresponding author at: John Peter Smith Health Network, 1500 S. Main St., Fort Worth, TX 76104, USA.
E-mail addresses: [email protected] (J.P. d’Etienne), [email protected] (Y. Zhou), [email protected] (C. Kan), [email protected] (S. Shaikh),
[email protected] (A.F. Ho), [email protected] (E. Suley), [email protected] (E.C. Blustein), [email protected] (C.D. Schrader), [email protected] (N.R. Zenarosa), [email protected] (H. Wang).
to recommend the treatment hospital trend downward while the pa- tient left-without-been-seen rate trends upward [5].
Many factors can affect patients’ ED LOS. Previous studies investi- gated the variables that are correlated with prolonged ED LOS [7-9], re- vealing that prolonged length of stay may be due to patient factors (such as advanced age, patient comorbidity, or transferred/referred pa- tients [7,10]), systematic factors (such as high patient volume, ED over- crowding, and day/night shifts [8,11]), or factors relating to patient care (such as different levels of acuity assigned, need for service consultation, or need for Advanced diagnostic imaging [9,12]). Most of these factors that predicted patient ED LOS were either directly or indirectly linked to the availability of resources that a certain ED provided. However, less is known about whether a predictive algorithm can accurately pre- dict the number and types of resources needed for a patient after triage completion. This subsequently affects the ability to accurately predict patient ED LOS. If patient ED LOS could be predicted very early in a
https://doi.org/10.1016/j.ajem.2020.01.050 0735-6757/(C) 2020 Published by Elsevier Inc.
ED resource use predictio”>visit, it would yield significant ED administrative value: allowing for rapid implementation of interventions to reduce patient ED LOS, mini- mization of ED crowding, and redistribution of limited ED resources [13].
Currently, several interventions have been studied in the literature to improve ED LOS with diverse outcomes [13-15]. In New Zealand, the ED LOS target for discharging patients is 6 h, however, setting a tar- get ED LOS failed to improve ED flow [16]. A provider-in-triage model allows for initiation of ED work-up while the patients remained in the waiting room; however, results are controversial. Some studies revealed a shorter ED LOS [14,17] with a provider-in-triage, while others had op- posite findings [18,19]. Recently, triage-related interventions have shown promising outcomes to improve patient flow in the ED [13]. Fast track and split flow models have been used to triage patients and have been proven to effectively reduce patient ED LOS, but these studies are limited to patients with relatively low Acuity levels [13,20]. With the increased use of the Clinical Decision Unit (CDU) (also known as ED ob- servation units, ED short stay unit, etc.) in most High-volume ED settings [21,22], transferring those patients with projected prolonged ED LOS to the CDU for further evaluation and treatment becomes a potential inter- vention to reduce ED LOS [23]. However, the effectiveness of such an in- tervention would rely on the ability to accurately predict ED LOS upon the completion of patient triage. Unfortunately, there is no currently available tool capable of predicting patient LOS with limited triage information.
Therefore, we aim to derive a novel model to predict patient ED LOS
upon their triage completion, and furthermore, to investigate the bene- fit of intervention by diverting prolonged ED LOS patients to the CDU using a computer Simulation model. Moreover, we hypothesize that this novel intervention can help mitigate ED crowding situations and improve ED patient flow efficiency.
This is a single-center retrospective study with data prospectively collected. The study hospital is a tertiary referral center with 573 li- censed beds. It is a publicly funded hospital, regional level 1 trauma cen- ter, chest pain and comprehensive stroke center. The study ED has 53 licensed beds with an annual volume of N120,000 patient visits. This study was approved by the John Peter Smith Health Network Institu- tional Review Board with waived inform consent.
The data used in this study were extracted from patients’ Electronic medical records by designated persons from the department of Information Technology. internal validations were performed to ensure the accuracy of the extracted EMR data. The accuracy of the data re- trieval process was separately validated three times. Each time, EMRs of 20 randomly selected patients were manually verified against all of the variables in the data collection sheet.
All patients who registered for the study ED from January 1, 2017 to December 31, 2017 were initially considered for inclusion. Patients with missing values or with extreme values of their vitals were excluded. The detailed criteria used for data inclusion and exclusion is provided in Appendix-1.
Fig. 1 depicts the two-step modeling framework for predicting prolonged ED LOS. Using the data collected from patients at and prior
to triage, the first step predicts the use of a set of pre-specified resources by individual patients during their ED visit. Then, the second step fur- ther predicts patients’ LOS as prolonged or not, based on the predicted resource use derived from the previous step and the ED crowding level in real time. Details of all the outcome and predictor variables are discussed in the next two subsections.
The proposed two-step ED LOS prediction model has two important traits. First, this model conducts prospective prediction upon triage completion. This allows for the detection of patients who have the highest likelihood of a prolonged ED LOS at an early stage of their ED en- counter, and therefore allows for interventions that can help improve the Efficiency of ED operations while addressing the specific needs of those patients. Second, the two-step model should achieve a better ED LOS prediction. According to the literature as well as our preliminary study (unpublished), we found that the LOS was generally highly asso- ciated with the use of resources (e.g., X-ray, procedure, consulting ser- vice) rather than the data collected at and prior to triage (e.g., demographics, arrival time, ESI, chief complaint, vitals, etc. [24]). Indeed, these data were found to be more closely correlated to the use of specific resources based on the same preliminary study. Motivated by these two observed relationships, this two-step model was con- structed to achieve better prediction accuracy.
-
-
- Step 1: ED resource use prediction
-
In a typical ED setting, patients often consume a variety of resources due to their heterogeneous conditions. By adapting and modifying known literature resources on Emergency Severity Index estima- tion, this study selected 10 specific types of resources as the outcome variables, including intravenous therapy (IV), point-of-care (POC) lab tests, non-point-of-care (NPOC) lab tests, urine (UR), ultrasound (U/ S), computed tomography (CT), X-ray (XR), contrast imaging (CTRST), procedure (PROC), and consultations (CONLT). These outcome variables were dichotomous (yes = resource used, no = resource is not used), and predicted by the same set of pre-specified predictor variables (see Table 2), respectively. To predict patients’ resource use, only those var- iables that were collected before or during the nurse triage evaluations were considered as predictor candidates. Specifically, these predictor candidates included patient demographics (age, gender, ethnicity, mar- ital status), administrative variables (arrival time, transfer mode), and clinical conditions (chief complaint, ESI, and initial vital signs at triage
– blood pressure, temperature, heart rate, respiratory rate, and oxygen
saturation). In particular, the patient chief complaints were classified into 12 illness-related categories according to the Reason for Visit Clas- sification System used by the National Hospital Ambulatory Medical Care Survey [25], such as cardiovascular diseases, digestive system dis- eases, nervous diseases, etc. ESI is a well-established triage tool (ESI-1
= most emergent, ESI-5 = least emergent) to determine the acuity level of patients based on the urgency of their chief complaint and esti- mated number of ED resources by the patients.
-
-
- Step 2: ED LOS prediction
-
The single outcome variable, patient ED LOS, was calculated in hours beginning at the time of patient arrival at the ED and ending at the time when the patient physically left the ED as documented in the medical record. In this study, LOS was categorized into two classes: regular (<= 16 h) and prolonged (N 16 h) according to a sensitivity analysis. The model performances (i.e. how accurately the model predicted LOS) were compared based on different cutoff values used to define Prolonged LOS, including 6, 8, 12, 16, and 23 h (Appendix-2). Obviously, the model showed the best performance (i.e., highest accuracy, sensitiv- ity and specificity) when the cutoff value was 16 h. To predict ED LOS prospectively at triage, the predictor variables used in this study in- cluded the predicted resource use of patients from the initial prediction modeling (individual-level) and the crowding level of the ED (system- level). Here, resource use included the 10 binary variables predicted in step 1, corresponding respectively to the 10 specific resources. ED
Fig. 1. General framework of the two-phase prediction model for patient length of stay.
validation data sets”>crowding level was considered as another influential factor that may af- fect patient total ED LOS. Each patient, upon their arrival at ED triage, was also characterized by an ED crowding score. A detailed explanation of ED crowding score assignment is addressed in Appendix-3.
-
-
- performance measuremen”>Binary logistic regression
-
Binary (or binomial) logistic regression models were constructed to predict the probability of an outcome falling into one of the two classes (e.g., the probability that a patient would have prolonged LOS). Binary logistic regression (BLR) is a widely used statistical technique desig- nated to measure the relationships between a set of predictor variables
-
-
- Training and validation data sets
-
The final data set was randomly divided into a 70% training (devel- opment) data set and a 30% test (validation) data set. The same training and test data sets were used for resource use and length of stay predic- tion to remain data consistency.
-
-
- Prediction performance measurement
-
The performance of each machine learning algorithm used in this analysis was evaluated using a range of measures including accuracy, sensitivity specificity, and c-statistics. The accuracy is defined as
and a dichotomous outcome variable for the purpose of predicting its future value. The results of BLR are easy to interpret (odds ratios) com-
TN + TP
TN + FP + FN + TP
, where TN, TP, FP, and FN represent true negative,
paring to several other more advanced data mining techniques. The ap-
true positive, false positive, and false negative, respectively. The speci-
plications of BLR can be found in many clinical studies [26-29]. Mathematically, the BLR models are formulated based on a logistic func-
ficity is given as TN
TN + FP
, and sensitivity is TP
TP + FN
. In this study, TP
tion, as shown in Eq. (1). The probability of outcome is denoted as p(x) in Eq. (1). The logic of the outcome is modeled as a linear combination of different independent predictors, these predictor variables are listed as x1, x2, …, xd in this formula where their corresponding coefficients are denoted by ?1, ?2, …, ?d, respectively. The coefficient of the intercept is denoted by ?0.
e?0+?1 x1+…+?d xd
p(x) = 1 + e?0+?1 x1+…+?d xd (1)
was defined as: a resource was used by a patient and it was predicted
as used (step 1 models) or a patient had prolonged ED LOS and pre- dicted LOS was also classified as prolonged (step 2 model). C-index is equal to the area under the Receiver Operating Characteristic curve for the binary outcome in logistic regression. All these measures were taken into consideration in determining the suitability and efficacy of a model. When interpreting the C-index, values of between 0.7 and
0.8 can be interpreted as having good discrimination ability, and models with c-index of N0.8 can be interpreted as having excellent discrimina- tion ability, with values above 0.9 indicating outstanding ability.
-
- outcome evaluations”>Intervention and evaluation
- Interventions
- outcome evaluations”>Intervention and evaluation
Patients with Prolonged ED stay often exhaust a large amount of ED resources and have been known as one of the major bottlenecks imped- ing ED operations. Therefore, we hypothesize that transferring those prolonged patients to a special unit [Clinical Decision Unit (CDU)] im- mediately after triage for further emergency care could potentially im- prove ED operational outcomes. Furthermore, in General practice, since the CDU functions as an extension of the ED and includes patients requiring prolonged evaluation, observation, and treatment. The CDU can be treated as one section of the ED or as a separate unit outside of the ED. In order to gain a comprehensive understanding of this CDU intervention’s effectiveness prior to the implementation in practice, we consider three intervention scenarios and evaluate them against a baseline scenario – i.e., status quo of ED operation – in a simulated set- ting (Table 1). After consulting with experts on ED operational manage- ment using a modified Delphi method [30], consensus reached that an extra 6 beds could be used for accommodating ED prolonged stay pa- tients without affecting study ED/CDU routine operations. These scenar- ios are generated based on two important considerations: (1) retaining or expanding the current bED capacity; (2) having the ability to identify prolonged patients at triage or not. Details of the baseline and imple- mentation scenarios are described in Table 1.
-
-
- ED operational outcomes
-
Two ED operational outcomes were investigated: crowding level and Time Savings (in hours). The ED crowding level is calculated for each hour based on NEDOCS and categorized by three crowding levels: not crowded, crowded, and overcrowded (Appendix-3). The percentage of overcrowded hours are calculated and compared among different scenarios. Further, an assumption was made that one ED bed occupied for one day is equal to 24 h of an ED patient stay. The total number of hours is then divided by 24 to determine the change in bed-hours by implementing the three intervention scenarios.
In this study, a DES model is built to assess the effectiveness of CDU interventions based on a general ED patient flow from arrival, treat- ment, to departure [31]. To represent different patient arrival and LOS patterns, the DES model categorizes patients into two types: regular and prolonged. Probability distributions are generated by fitting the re- corded EMR data using the goodness-of-fit technique to capture the stochasticity embedded in patient arrivals and treatment durations. The capacity of beds (the main resource in this simulation) varies ac- cording to the baseline and intervention scenarios described above (see Table 1). Further, for intervention scenarios with the LOS prediction model (scenario A & C), Monte Carlo (MC) simulation is used to
Descriptions of baseline and intervention scenarios.
determine whether a patient is prolonged or not. Specifically, the MC simulation involves using a set of computer-generated random num- bers (between 0 and 1) as the cutoff values to determine the prediction class of patient ED LOS. This inevitably changes whether a patient’s LOS was predicted as not prolonged or prolonged. Then, through repeated sampling (each random number generates a sample), the stochastic properties in this classification process can be obtained (e.g., number of times patients are classified into the prolonged class) and the final class of patients will be determined by the class that is predicted in ma- jority of the samples.
- Results
The final data set includes a total of 110,471 patients, accounting for about 89% of the total of 123,975 patient visits in 2017. The general characteristics of these patients are shown in Table 2.
-
- Prediction performance and final models
Table 3 shows the performance measures (accuracy, sensitivity, specificity, C-index) of resource use and LOS prediction models. The overall accuracy of model prediction is 67.8% (Table 3). The C-index ranges from 0.72 to 0.82, indicating good discrimination ability of the models. Appendix-4 summarizes the odds ratios for individual predictor variables derived from the 10 resource use prediction models, respectively.
-
- Intervention outcome evaluations
To improve efficiency of the main ED, prolonged ED LOS patients in the model were diverted to a virtual CDU after their triage evaluations. As described earlier, two intervention outcomes, ED crowding level and time savings, were estimated by using the discrete event simulation. ED crowding level was determined by NEDOCS and calculated on an hourly basis. A 12% (1044/8760) reduction of ED overcrowded statuses can be achieved by scenario C (e.g. transferring maximally 6 predicted prolonged ED LOS patients to an outside CDU and maintaining same ED beds, Fig. 2 and Appendix-5,p b 0.001). An 8% of ED overcrowding reduction can be observed by scenario A (e.g. take 6 ED beds and con- verted to CDU beds, with the remained 47 beds functioned as ED beds) and a 7% of reduction by scenario B (simply expanding 6 more ED beds with a total of 59 ED beds).
Based on the scenario C intervention, a daily savings of nearly 130 ED bed-hours can be achieved, whereas, a far lower daily savings are achievable with the implementation of scenario A (17 ED bed-hours) or B (27 ED bed-hours) separately (Fig. 2). Fig. 2 depicts the ED crowding statuses changes and average daily hour saved with the im- plementation of these interventions in computer Simulated models. In addition, we also found ED crowding and bed saving had no significant changes when scenario A and B are compared. This indicates that either randomly choosing 6 patients moving out of the ED (scenario B) or cut- ting dedicated 6 beds from the ED for holding ED prolong stay patients
prediction (no information about patient LOS at triage)
ED LOS
prediction (identify prolonged patients at triage)
Current bed capacity (Total 53 beds)
Baseline scenario
Bed allocation: 53 regular bed; Patient assignment: patients will be seen in order based on their arrival time and severity
Intervention scenario A
Bed allocation: 47 regular ED beds and 6 ED-CDU beds Patient assignment: predicted prolonged patients will be assigned to either ED-CDU or regular beds depending on bed availability
Expanded bed capacity (Total 59 beds)
Intervention scenario B Bed allocation: 59 regular beds;
Patient assignment: patients will be seen in order based on their arrival time and severity Intervention scenario C
Bed allocation: 53 regular beds and additional 6 CDU beds Patient assignment: predicted prolonged patients will be assigned to either CDU or regular beds depending on bed availability
(scenario A) might not efficiently improve ED crowding.
- Discussion
The overall prediction performance of the model using patient triage information is acceptable with an overall accuracy of 67.8%. Meanwhile, a Computer simulation was developed to show the efficacy of an ED in- tervention that directly moves predicted prolonged ED stay patients to the CDU for evaluation and treatment. With this innovative interven- tion, we can potentially decrease over 10% of ED overcrowding. To the best of our knowledge, neither this novel ED LOS Prediction tool nor the outcomes of ED intervention have been reported in the literature. Using this model, early recognition of potential prolonged ED LOS pa- tients and subsequent transfer of those patients to the CDU for
Distribution of patients’ characteristics and frequency of resource use and prolonged length of stay in overall study population.
Number of patients
Percent of resource use Endpoints
Variable |
N |
% |
IVa |
POC |
NPOC |
UR |
U/S |
CT |
XR |
CTRST |
PROC |
CONLT |
PLOSb |
|||||
Overall |
||||||||||||||||||
Total |
110,471 |
100 |
43.3 |
30.6 |
66.3 |
35.6 |
6.2 |
26.9 |
45.0 |
13.5 |
4.1 |
16.8 |
2.3 |
|||||
Age |
||||||||||||||||||
b18 |
3281 |
3.0 |
0.7 |
0.5 |
1.3 |
0.9 |
0.1 |
0.3 |
0.8 |
0.2 |
0.2 |
0.2 |
0.0 |
|||||
18-64 |
97,172 |
88.0 |
38.1 |
26.0 |
57.8 |
31.4 |
5.7 |
22.7 |
38.5 |
11.8 |
3.5 |
13.7 |
1.8 |
|||||
>=65 |
10,018 |
9.1 |
4.5 |
4.1 |
7.2 |
3.3 |
0.4 |
3.8 |
5.7 |
1.6 |
0.4 |
2.9 |
0.5 |
|||||
Gender |
||||||||||||||||||
Male |
55,302 |
49.9 |
21.4 |
15.4 |
31.3 |
14.1 |
1.9 |
13.9 |
23.8 |
6.7 |
2.6 |
9.4 |
1.2 |
|||||
Female |
55,169 |
50.1 |
21.9 |
15.2 |
35.0 |
21.5 |
4.3 |
13.0 |
21.2 |
6.8 |
1.5 |
7.4 |
1.1 |
|||||
Single |
61,453 |
55.6 |
22.2 |
14.8 |
34.6 |
19.6 |
3.2 |
13.2 |
22.9 |
6.5 |
2.5 |
8.3 |
1.1 |
|||||
Married |
25,210 |
22.8 |
10.9 |
8.2 |
16.5 |
8.5 |
1.8 |
7.3 |
11.2 |
3.9 |
0.9 |
4.1 |
0.5 |
|||||
Divorced |
16,363 |
14.8 |
6.9 |
5.0 |
10.3 |
5.0 |
0.8 |
4.3 |
7.3 |
2.1 |
0.5 |
2.8 |
0.4 |
|||||
Widowed |
5128 |
4.6 |
2.3 |
2.0 |
3.6 |
1.7 |
0.2 |
1.7 |
2.9 |
0.7 |
0.1 |
1.3 |
0.2 |
|||||
Significant other |
2317 |
2.1 |
0.9 |
0.6 |
1.4 |
0.8 |
0.2 |
0.5 |
0.9 |
0.3 |
0.1 |
0.3 |
0.0 |
|||||
Ethnicity |
||||||||||||||||||
Hispanic or Latino |
30,317 |
27.4 |
12.4 |
9.3 |
18.6 |
10.5 |
2.5 |
7.7 |
11.9 |
4.1 |
1.2 |
4.3 |
0.6 |
|||||
Not Hispanic or Latino |
80,154 |
72.6 |
30.9 |
21.3 |
47.7 |
25.1 |
3.7 |
19.2 |
33.1 |
9.4 |
2.9 |
12.5 |
1.8 |
|||||
Transfer mode |
||||||||||||||||||
Not transferred |
101,740 |
92.1 |
39.5 |
28.5 |
5.8 |
33.1 |
5.7 |
24.7 |
41.4 |
12.4 |
3.8 |
15.3 |
0.2 |
|||||
Transferred |
8731 |
7.9 |
3.8 |
2.1 |
60.5 |
4.5 |
0.5 |
2.2 |
3.9 |
1.1 |
0.3 |
1.5 |
2.1 |
|||||
Blood pressure |
||||||||||||||||||
Abnormal |
88,128 |
79.8 |
34.2 |
24.5 |
52.7 |
27.5 |
4.6 |
21.6 |
36.6 |
10.9 |
3.2 |
13.0 |
1.7 |
|||||
Normal |
22,343 |
20.2 |
9.1 |
6.1 |
13.6 |
8.1 |
1.6 |
5.3 |
8.4 |
2.6 |
0.9 |
3.8 |
0.6 |
|||||
Temperature |
||||||||||||||||||
Abnormal |
3834 |
3.5 |
2.6 |
2.5 |
3.0 |
2.0 |
0.2 |
1.4 |
2.2 |
0.8 |
0.3 |
1.2 |
0.2 |
|||||
Normal |
106,637 |
96.5 |
40.7 |
28.1 |
63.3 |
33.6 |
6.0 |
25.5 |
42.8 |
12.7 |
3.8 |
15.6 |
2.1 |
|||||
Heart rate |
||||||||||||||||||
Abnormal |
27,513 |
24.9 |
14.5 |
10.2 |
18.3 |
10.2 |
1.6 |
7.6 |
12.3 |
4.3 |
1.3 |
5.8 |
0.9 |
|||||
Normal |
82,958 |
75.1 |
28.8 |
20.4 |
48.0 |
25.4 |
4.6 |
19.2 |
32.8 |
9.2 |
2.8 |
10.9 |
1.4 |
|||||
Respiratory rate |
||||||||||||||||||
Abnormal |
60,231 |
54.5 |
25.7 |
18.0 |
37.1 |
19.5 |
3.3 |
15.5 |
26.3 |
8.0 |
2.3 |
10.5 |
1.5 |
|||||
Normal |
50,240 |
45.5 |
17.7 |
12.6 |
29.2 |
16.1 |
2.9 |
11.4 |
18.8 |
5.5 |
1.8 |
6.3 |
0.8 |
|||||
Pox |
||||||||||||||||||
Abnormal |
69,357 |
4.8 |
3.0 |
2.3 |
3.9 |
1.8 |
0.3 |
1.9 |
3.2 |
1.0 |
0.3 |
1.6 |
0.3 |
|||||
Normal |
51,176 |
95.2 |
40.3 |
28.3 |
62.4 |
33.8 |
5.9 |
25.0 |
41.8 |
12.5 |
3.8 |
15.2 |
2.0 |
|||||
Chief complaint |
||||||||||||||||||
Cardiovascular system |
2307 |
2.1 |
0.9 |
0.7 |
1.6 |
0.7 |
0.1 |
0.4 |
1.1 |
0.1 |
0.1 |
0.5 |
0.1 |
|||||
Digestive system |
15,690 |
14.2 |
9.3 |
4.6 |
12.6 |
8.9 |
2.9 |
5.5 |
3.5 |
4.5 |
0.1 |
2.5 |
0.4 |
|||||
General symptoms |
22,835 |
20.7 |
10.0 |
7.4 |
16.3 |
7.5 |
1.0 |
4.9 |
12.0 |
2.8 |
0.4 |
3.9 |
0.6 |
|||||
Injury |
12,619 |
11.4 |
4.4 |
3.4 |
5.0 |
2.4 |
0.1 |
5.9 |
6.8 |
2.3 |
1.7 |
1.5 |
0.1 |
|||||
Nervous system |
8364 |
7.6 |
4.2 |
2.6 |
5.7 |
2.6 |
0.2 |
3.3 |
2.3 |
0.5 |
0.2 |
1.2 |
0.2 |
|||||
10,059 |
9.1 |
3.8 |
2.1 |
6.3 |
4.4 |
0.2 |
1.9 |
2.2 |
0.4 |
0.3 |
2.1 |
0.3 |
||||||
Skin, nails and hair |
2686 |
2.4 |
0.7 |
0.6 |
1.0 |
0.3 |
0.1 |
0.3 |
0.4 |
0.2 |
0.4 |
0.3 |
0.0 |
|||||
Eyes and ears |
2220 |
2.0 |
0.3 |
0.4 |
0.5 |
0.2 |
0.0 |
0.3 |
0.2 |
0.1 |
0.1 |
0.2 |
0.0 |
|||||
4348 |
3.9 |
1.2 |
0.9 |
3.1 |
2.8 |
1.2 |
0.5 |
0.4 |
0.3 |
0.0 |
0.4 |
0.0 |
||||||
13,888 |
12.6 |
3.0 |
2.5 |
4.7 |
2.5 |
0.2 |
1.6 |
8.1 |
0.6 |
0.5 |
1.3 |
0.1 |
||||||
Procedures related |
4588 |
4.2 |
1.1 |
1.1 |
2.2 |
1.0 |
0.1 |
0.5 |
1.0 |
0.3 |
0.1 |
0.8 |
0.1 |
|||||
10,867 |
9.8 |
4.4 |
4.3 |
7.2 |
2.3 |
0.2 |
1.7 |
7.1 |
1.3 |
0.2 |
2.1 |
0.3 |
||||||
ESI |
||||||||||||||||||
1 |
2348 |
2.1 |
1.8 |
1.6 |
2.0 |
1.1 |
0.0 |
1.4 |
1.7 |
0.8 |
0.5 |
0.8 |
0.1 |
|||||
2 |
25,749 |
23.3 |
14.2 |
10.0 |
19.3 |
8.8 |
0.9 |
8.2 |
13.4 |
3.8 |
1.1 |
7.3 |
1.1 |
|||||
3 |
66,606 |
60.3 |
25.8 |
16.9 |
41.2 |
23.2 |
5.1 |
16.5 |
25.2 |
8.5 |
2.0 |
8.1 |
1.1 |
|||||
4 |
14,492 |
13.1 |
1.4 |
1.9 |
3.6 |
2.4 |
0.1 |
0.8 |
4.7 |
0.4 |
0.5 |
0.4 |
0.0 |
|||||
5 |
1276 |
1.2 |
0.1 |
0.1 |
0.2 |
0.1 |
0.0 |
0.0 |
0.1 |
0.0 |
0.0 |
0.0 |
0.0 |
|||||
ED crowding |
||||||||||||||||||
Not crowded (<=100) |
28,151 |
25.5 |
11.5 |
7.9 |
16.7 |
9.0 |
1.6 |
7.1 |
11.4 |
3.6 |
1.1 |
4.4 |
0.1 |
|||||
Crowded (101-140) |
28,041 |
25.4 |
11.1 |
7.6 |
16.8 |
9.0 |
1.6 |
6.9 |
11.4 |
3.5 |
1.1 |
4.2 |
0.4 |
|||||
Overcrowded (N140) |
54,279 |
49.1 |
20.7 |
15.1 |
32.9 |
17.6 |
2.9 |
12.9 |
22.3 |
6.4 |
1.9 |
8.2 |
1.8 |
|||||
Disposition |
||||||||||||||||||
In-patient |
17,013 |
15.4 |
12.8 |
8.6 |
14.7 |
8.9 |
1.4 |
8.0 |
10.2 |
4.4 |
1.1 |
10.1 |
1.6 |
|||||
Observation |
12,559 |
11.4 |
7.3 |
4.7 |
10.7 |
4.6 |
0.7 |
4.4 |
7.1 |
2.5 |
0.2 |
4.2 |
0.6 |
|||||
Discharged |
80,899 |
73.2 |
23.3 |
17.3 |
41.0 |
22.1 |
4.0 |
14.4 |
27.8 |
6.6 |
2.7 |
2.5 |
0.2 |
comprehensive evaluation and treatment made more efficient use of ED resources, which would subsequently improve ED operations. This study provides some evidence for this model, an essential step prior to implementing these interventions on real patients.
We excluded patients with extreme values of their vital signs (see Appendix-1) – critically ill patients who are treated as a “priority alert” in the study hospital and handled immediately by multidisciplin- ary teams including ED and ICU physicians. In general, these patients will take priority to move to the ICU, and there is less operational value to predict patient ED LOS upon their arrival. In addition, based on the severity of their condition, they are often contraindicated to move to the ED observation unit. Therefore, we only included relatively stable patients for ED LOS predictions. After a literature survey, we found several prediction tools that have been used to determine LOS. Some only predicted patient hospital LOS [32,33], while others pre- dicted patient ED LOS with varying accuracies [7,12]. Most studies re- ported the associations between prolonged ED LOS and patient acuity levels, but failed to develop a ED LOS prediction tool [34-36]. Other stud- ies reported ED LOS prediction limited to only certain diseases or condi- tions including syncope, crowding, or patients with critical care conditions which make such predictions less applicable in the general patient population [37,38]. Most studies predicted patient LOS only upon the completion of patient care. Very few studies can predict LOS in very early stages of patient care (e.g. triage) [7,8,12]. In addition, ei- ther or both steps of our prediction tool can be used depending on the needs of the ED. Some may only need to predict the number of ED re- sources to implement appropriate interventions accordingly, while others might follow the same innovative intervention from this study which would require a two-step prediction model.
In this study, two-step predictive modeling was performed. The ini-
tial step predicted the amount of ED resources used by a patient. Then, ED prolonged LOS based on the initial prediction of ED resources was predicted. This is similar to ED triage using ESI (Emergent Severity Index) system [39]. However, this predictive model is an extension of the ESI system including the resource prediction among higher acuity patients with the addition of taking other factors into account (such as patient demographics, ED crowding, time of arrival, etc.).
Among all of the independent variables used to predict ED LOS, we identified some variables associated with prolonged ED LOS that are consistent with previous reports in the literature (such as some ED re- sources – e.g. consultation, Urine sample, crowding) [9,11,40]. However, others such as POC (point-of-care) testing were still controversial al- though extensively studied [41,42]. Our results found that the sensitiv- ity and specificity of prediction for most of the resources are very close except the “procedure” variable which has a relatively larger gap be- tween sensitivity and specificity. This is due to a significant shortage of samples that used the resource (~4000) versus samples that did not use the resource (~90,000). When the two-step prediction tool was de- rived, our intent was not to accurately predict the exact number of hours that patients might stay at the ED, since such predictions lack clin- ical significance. Instead, we sought to accurately predict the patients who were likely to stay in the ED significantly longer. This information has more potential to be meaningful in the field of ED operational man- agement. Therefore, with the derivation and validation of this two-step ED LOS prediction tool, we added some evidence to the literature pool on the feasibility of predicting ED LOS among general patients at a very early triage stage. With early prediction of prolonged ED LOS pa- tients, ED/hospital managers or administrators could have enough time to initiate Effective interventions.
Simply implementing interventions based on a novel prediction tool without estimating the outcomes could result in a financial loss if such intervention proved to be less efficient. Therefore, it is vital to perform a computer simulation to measure the potential outcomes. Discrete even simulation (DES) was mainly used to predict study outcomes (e.g. ED crowding reduction) by using different simulated interventions. Since the ED flow can change dynamically at any time with different in- terventions, DES is a better simulation model to evaluate Outcome changes. Such methods have previously been used to predict ED crowding in the literature [43].
Reducing ED crowding, saving ED resources, and eventually de- creasing Healthcare costs are major outcome measurements in the current ED management [44]. With already limited healthcare re- sources, ED or Hospital beds cannot be expanded endlessly. One al- ternative is to reallocate patients within current available resources. Our ED administration team manages the CDU, and the in- tent of placing patients into the CDU is to provide opportunity for prolonged observations. We believe expanding our CDU patient en- rollment and including prolonged ED stay patients might maximize CDU operational functions. Furthermore, converting some ED beds to function as CDU beds in advance will help ease staff resources, which might potentially improve ED operational managements. However, such studies are rarely reported in the literature. Given an uncertain outcome and less historical findings in the literature, such outcome change evaluations might be better measured with the initiation of simulation models. Though such results are not from real patients, these findings suggested that simply expanding the ED might not be an optimal intervention to overcome ED crowding [45]. Therefore, it could provide useful evidence and set as the foundation for ED management on real patients in the future. However, we did not compare the accuracy differences between ED providers’ perceptions of patient LOS and computer predictive model in this study, our future study will be focused on such comparisons.
Our study has its limitations: 1) This is a single-center study, as such, patient selection bias or potential missing data cannot be ruled out; 2) Only internal validation was performed on this novel prediction tool and is still lacking external validation; 3) Very early and limited information obtained at ED triage is being utilized, which could result in lesser accuracy of this scoring tool derivation;
4) A multivariate logistic regression model is used for model predic- tion, which requires a significant number of variables to avoid pa- tient selection bias. Nevertheless, all potential confounders cannot be included in the final analysis, which might decrease the accuracy of this prediction tool. Therefore, multi-center prospective compari- son studies to external validate the study findings are warranted be- fore adoption in a clinical setting.
- Conclusion
In summary, an innovative two-step prediction tool with fair accu- racy was derived. This tool can be used after the completion of patient triage to predict patient ED LOS. With early patient LOS prediction, pa- tients with prolonged ED stay could potentially be transferred to the CDU for further evaluation and treatment. With such intervention, over 10% of overcrowded ED statuses were converted to less crowded ones in a computer simulated model. However, an external validation with a large sample size is required before full-scale implementation of this prediction tool.
a IV – Intravenous therapy; POC – Point-of-Care; NPOC – Non-Point-of-Care; UR – Urine Testing; U/S – Ultrasound; CT – CT scan; XR – X-ray; CTRST – Contrast Imaging; PROC – Pro- cedure; CONLT – Consultant.
b PLOS refers to prolonged length of stay, i.e., LOS is longer than 16 h.
Performance measures of resource use and prolongED length of stay prediction models.
analysis. YZ, CK, AFH, EB, CDS, NRZ, and HW drafted the manuscript. YZ, CK, AFH, CDS, NRZ, and HW contributed to Data interpretation, critical re- view and revisions of the manuscript for important intellectual content.
Accuracy |
Sensitivitya |
Specificity |
c-index |
All authors approved the final version of the submitted manuscript and |
Resource use (use/not use) |
agree to be accountable for all aspects of the work. |
a Class “Yes” = Use.
IV |
68.8% |
58.4% |
76.8% |
0.75 |
POC |
72.7% |
27.6% |
92.3% |
0.73 |
NPOC |
76.8% |
87.5% |
55.8% |
0.81 |
UR |
70.6% |
40.9% |
87.2% |
0.73 |
U/S |
93.7% |
0.24% |
99.9% |
0.82 |
CT |
75.7% |
26.3% |
94.1% |
0.74 |
XR |
70.8% |
67.3% |
73.7% |
0.78 |
CTRST |
86.5% |
3.6% |
99.3% |
0.75 |
PROC |
96.1% |
0.3% |
99.9% |
0.79 |
CONLT |
83.4% |
9.8% |
98.4% |
0.74 |
Length of stay (prolonged/not prolonged) |
||||
LOS |
67.8% |
72.7% |
67.7% |
0.72 |
Author contributions statement
JPD, YZ, NRZ, and HW conceived the research and designed the study. SS and HW performed data collection. YZ, CK, ES, and HW performed
Ethical approval statement
This study was approved by the John Peter Smith Health Network In- stitutional Review Board.
Funding
N/A.
Data availability statement
Data available by request to the corresponding author.
Declaration of competing interest
Authors have no conflict of interests.
50.0%
(a)
40.0%
46%
41%
40%
41%
34% 33% 34%
29%
25%
26%
25%
25%
30.0%
20.0%
10.0%
0.0%
Baseline Scenario A Scenario B Scenario C Not Crowded Crowded Overcrowded
(b)
200
129.3
17.7
27.1
160
120
80
40
0
Scenario A Scenario B Scenario C
Fig. 2. Outcomes of different interventions.
N/A.
Appendix 1. Data exclusion/inclusion criteria
*Number of patient encounters with detail missing information: Gender (n = 13), Marital Status (n = 1803), Ethnicity (n = 1248), Chief complaint (n = 1109), ESI (n = 962), Blood Pressure (n = 3202), Temperature (n = 5240), Heart Rate (n = 2576), Respiratory Rate (n = 2583), Oxygen level (n = 2859). The total number added exceeds 13,178 because one patient encounter may have multiple missing items.
**Patient encounters with extreme values of vital signs informed by clinical professionals that was excluded from this prediction model: Systolic blood pressure b 60 mmHg (n = 46), diastolic blood pressure b 40 mmHg (n = 173), temperature b 80 ?F (n = 30), Heart rate (b20 or N 300) (n
= 48), Respiratory rate (b4 or N 40) (n = 256), and Oxygen level b 50% (n = 92). The total number added exceeds 326 because one patient encounter may have multiple extreme values of vital signs.
Appendix 2. . Model performance comparison by Patient Length-Of-Stay cutoff value
Overall Accuracy Sensitivity Specificity
0.85
0.80
0.75
0.70
0.65
0.60
0.55
0.50
6-HR
8-HR
12-HR
16-HR
23-HR
Prolonged LOS Cutoff Value
Appendix 3. . ED crowding measurement In this study, we used NEDOCS (National Emergency Department OverCrowding Study) to measure ED crowding [1]. NEDOCS is one of the most com- mon ED crowding estimation tools and has been externally validated in other studies [2-4]. To simplify comparisons, NEDOCS was modified to three- tier classification schemes in this study. NEDOCS scores <=100 were considered "not-crowded", scores N100 and <= 140 were classified as "crowded", and scores N140 were classified as "overcrowded". ED crowding was measured at the top of each hour throughout the study period. Under this cir- cumstance, we are able to measure the percentage of ED hours deemed to be "overcrowded", "crowded", or "not-crowded" during study period [5]. To determine daily ED crowdedness, we score each crowding statuses as the following: overcrowded scores 3, crowded scores 2, and not-crowded scores 1. ED daily crowd status was deemed to be "overcrowded" if total daily score is above 48 (e.g. at least more than half day under crowded/over- crowded statuses). Not-crowded was defined as daily score equal or b36 (e.g. at least more than half day under not-crowded statuses). Whereas, crowded was defined as daily score above 36 but equal or b48. In addition, each patient arriving at the ED was assigned a crowding score determined by NEDOCS. Crowding scores estimated at the beginning of each hour were assigned to all subjects who arrived throughout the given hourly interval. Patients who arrived at the end of the hour were assigned the same crowding score as measured at the start of the hour (e.g., patients who arrived from 0801 to 0859 were assigned to the crowding score measured at 0800).
Reference list
- Weiss SJ, Derlet R, Arndahl J, Ernst AA, Richards J, Fernandez-Frackelton M et al.: Estimating the degree of emergency department overcrowding in academic medical centers: results of the National ED Overcrowding Study (NEDOCS). Acad Emerg Med 2004, 11: 38-50.
- Weiss SJ, Ernst AA, Nick TG: Comparison of the National Emergency Department Overcrowding Scale and the Emergency Department Work Index for quantifying emergency department crowding. Acad Emerg Med 2006, 13: 513-518.
- Jones SS, Allen TL, Flottemesch TJ, Welch SJ: An independent evalu- ation of four quantitative emergency department crowding scales. Acad Emerg Med 2006, 13: 1204-1211.
- Todisco C: Overcrowding and clinical risk in Emergency Depart- ments. A model for the reduction in NEDOCS: preliminary results. Acta Biomed 2015, 86: 170-175.
- Wang H, Ojha RP, Robinson RD, Jackson BE, Shaikh SA, Cowden CD et al.: Optimal Measurement Interval for Emergency Depart- ment Crowding Estimation Tools. Ann Emerg Med 2017, 70: 632-639.
Appendix 4. . Estimated odds ratios for resource use prediction models
Predictor variable |
Odds Ratios |
(95% C.I.) |
|||||||||
IV |
POC |
NPOC |
UR |
U/S |
CT |
XR |
CTRST |
PROC |
CONLT |
||
Age |
1.01 |
1.02 |
1.02 |
1.00 |
0.99 |
1.02 |
1.02 |
1.01 |
0.99 |
1.03 |
|
(1.01,1.01) |
(1.02,1.02) |
(1.02,1.02) |
(1.00,1.00) |
(0.99,0.99) |
(1.02,1.02) |
(1.02,1.02) |
(1.01,1.01) |
(0.99, 1.00) |
(1.03,1.03) |
||
Gender |
|||||||||||
Male |
0.99 |
1.03 |
0.78 |
0.56 |
0.56 |
1.02 |
1.11 |
0.98 |
1.49 |
1.31 |
|
(0.96,1.02) |
(1,1.07) |
(0.75,0.8) |
(0.54,0.57) |
(0.52,0.6) |
(0.99,1.06) |
(1.07,1.14) |
(0.94,1.03) |
(1.37,1.61) |
(1.26,1.36) |
||
Female |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
|
Marital status: |
|||||||||||
Single |
0.89 |
0.93 |
0.97 |
1.09 |
0.93 |
1.02 |
0.98 |
0.93 |
1.10 |
1.05 |
|
(0.85,0.93) |
(0.88,0.97) |
(0.91,1.02) |
(1.04,1.15) |
(0.84,1.03) |
(0.97,1.08) |
(0.93,1.03) |
(0.87,0.99) |
(0.97,1.24) |
(0.99,1.12) |
||
Married |
1.08 |
1.10 |
1.19 |
1.15 |
1.17 |
1.24 |
1.07 |
1.22 |
1.11 |
1.04 |
|
(1.02,1.14) |
(1.04,1.16) |
(1.12,1.26) |
(1.09,1.21) |
(1.06,1.31) |
(1.18,1.32) |
(1.02,1.13) |
(1.14,1.31) |
(0.98,1.27) |
(0.97,1.11) |
||
Widowed |
0.97 |
0.97 |
1.07 |
1.05 |
0.90 |
1.02 |
1.17 |
0.91 |
1.05 |
1.09 |
|
(0.9,1.06) |
(0.89,1.05) |
(0.96,1.18) |
(0.97,1.15) |
(0.74,1.08) |
(0.93,1.12) |
(1.07,1.28) |
(0.82,1.02) |
(0.84,1.31) |
(0.99,1.19) |
||
Significant other |
1.00 |
1.07 |
1.03 |
1.16 |
1.04 |
1.03 |
1.11 |
1.02 |
1.03 |
1.04 |
|
(0.89,1.12) |
(0.95,1.21) |
(0.9,1.17) |
(1.03,1.31) |
(0.83,1.28) |
(0.9,1.17) |
(0.99,1.26) |
(0.87,1.20) |
(0.77,1.37) |
(0.89,1.22) |
||
Divorced |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
|
Ethnicity: |
|||||||||||
Not Hispanic or Latino |
0.89 |
0.78 |
0.89 |
0.91 |
0.68 |
0.95 |
0.98 |
0.95 |
0.86 |
1.02 |
|
(0.86,0.92) |
(0.76,0.81) |
(0.86,0.93) |
(0.87,0.94) |
(0.64,0.73) |
(0.91,0.99) |
(0.94,1.01) |
(0.90,0.99) |
(0.79,0.94) |
(0.97,1.07) |
||
Hispanic or Latino |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
|
Transfer mode: |
|||||||||||
Not transferred |
0.84 |
1.43 |
1.09 |
1.37 |
0.93 |
1.47 |
1.34 |
1.97 |
1.06 |
0.71 |
|
(0.72,0.98) |
(1.19,1.72) |
(0.92,1.3) |
(1.17,1.61) |
(0.59,1.55) |
(1.21,1.78) |
(1.12,1.6) |
(1.38,2.93) |
(0.75,1.55) |
(0.6,0.84) |
||
Transferred |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
|
Month |
0.99 |
0.95 |
1.00 |
0.98 |
0.99 |
1.00 |
0.99 |
1.00 |
0.99 |
1.03 |
|
(0.99,1.00) |
(0.94,0.95) |
(0.99,1.00) |
(0.98,0.99) |
(0.98,1.00) |
(0.99,1.00) |
(0.98,0.99) |
(0.99,1.01) |
(0.98,1.00) |
(1.02,1.03) |
||
Day of week: |
|||||||||||
Mon |
0.93 |
0.98 |
1.05 |
1.02 |
0.90 |
0.98 |
1.02 |
0.96 |
0.95 |
1.12 |
|
(0.88,0.98) |
(0.93,1.04) |
(0.99,1.12) |
(0.97,1.09) |
(0.80,1.01) |
(0.92,1.04) |
(0.96,1.09) |
(0.88,1.04) |
(0.83,1.09) |
(1.04,1.20) |
||
Tue |
0.93 |
0.92 |
1.08 |
0.99 |
0.93 |
0.97 |
1.01 |
0.96 |
0.96 |
1.12 |
|
(0.88,0.98) |
(0.87,0.98) |
(1.01,1.16) |
(0.93,1.05) |
(0.83,1.05) |
(0.91,1.04) |
(0.95,1.07) |
(0.89,1.04) |
(0.83,1.10) |
(1.04,1.21) |
||
Wed |
0.93 |
0.94 |
1.10 |
1.00 |
0.94 |
1.03 |
1.02 |
0.96 |
1.00 |
1.08 |
|
(0.87,0.98) |
(0.89,1.00) |
(1.03,1.17) |
(0.94,1.06) |
(0.84,1.05) |
(0.97,1.10) |
(0.96,1.09) |
(0.89,1.05) |
(0.87,1.15) |
(1.00,1.16) |
||
Thu |
0.98 |
1.03 |
1.08 |
1.09 |
0.91 |
1.03 |
0.97 |
0.96 |
0.88 |
1.02 |
|
(0.92,1.04) |
(0.97,1.09) |
(1.01,1.16) |
(1.02,1.15) |
(0.81,1.02) |
(0.96,1.09) |
(0.92,1.03) |
(0.88,1.04) |
(0.76,1.01) |
(0.95,1.10) |
||
Sat |
1.08 |
1.03 |
0.88 |
0.95 |
0.90 |
1.03 |
0.90 |
0.97 |
1.11 |
0.86 |
|
(1.02,1.15) |
(0.97,1.10) |
(0.82,0.94) |
(0.89,1.01) |
(0.80,1.01) |
(0.97,1.10) |
(0.85,0.96) |
(0.89,1.05) |
(0.97,1.27) |
(0.80,0.94) |
||
Sun |
1.05 |
0.95 |
0.90 |
0.98 |
0.87 |
1.04 |
0.94 |
0.96 |
1.04 |
0.97 |
|
(0.99,1.12) |
(0.90,1.02) |
(0.84,0.96) |
(0.93,1.05) |
(0.77,0.99) |
(0.97,1.11) |
(0.88,1.00) |
(0.89,1.05) |
(0.91,1.19) |
(0.90,1.05) |
||
Fri |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
|
Hour |
0.99 |
0.99 |
0.99 |
1.00 |
1.00 |
0.99 |
0.99 |
0.99 |
0.99 |
0.99 |
|
(0.99,0.99) |
(0.99,1.00) |
(0.99,1.00) |
(1.00,1.00) |
(0.99,1.00) |
(0.99,1.00) |
(0.99,1.00) |
(0.99,1.00) |
(0.99,1.00) |
(0.99,0.99) |
||
Chief complaint: |
(continued)
Predictor variable Odds Ratios (95% C.I.)
IV |
POC |
NPOC |
UR |
U/S |
CT |
XR |
CTRST |
PROC |
CONLT |
||
Digestive system |
4.50 |
1.36 |
3.61 |
3.78 |
7.33 |
5.02 |
0.44 |
10.5 |
0.27 |
1.40 |
|
(4.01,5.06) |
(1.21,1.53) |
(3.13,4.14) |
(3.37,4.24) |
(5.56,9.91) |
(4.37,5.77) |
(0.39,0.49) |
(8.61,12.9) |
(0.20,0.39) |
(1.22,1.61) |
||
Eyes & ears |
0.48 |
1.11 |
0.23 |
0.25 |
0.11 |
2.20 |
0.14 |
2.04 |
1.71 |
0.93 |
|
(0.40,0.58) |
(0.93,1.31) |
(0.19,0.27) |
(0.20,0.30) |
(0.04,0.25) |
(1.81,2.66) |
(0.11,0.17) |
(1.53,2.72) |
(1.18,2.50) |
(0.74,1.16) |
||
General symptoms |
1.79 |
1.37 |
1.59 |
1.23 |
1.58 |
1.90 |
1.74 |
2.84 |
0.61 |
1.21 |
|
(1.60,2.00) |
(1.22,1.54) |
(1.39,1.81) |
(1.10,1.38) |
(1.19,2.15) |
(1.66,2.19) |
(1.56,1.94) |
(2.33,3.50) |
(0.46,0.83) |
(1.06,1.39) |
||
Genitourinary system |
1.09 |
0.92 |
2.37 |
6.27 |
11.97 |
1.22 |
0.15 |
1.76 |
0.33 |
0.85 |
|
(0.95,1.25) |
(0.80,1.07) |
(2.02,2.78) |
(5.47,7.20) |
(8.99,16.3) |
(1.03,1.45) |
(0.13,0.18) |
(1.38,2.25) |
(0.20,0.52) |
(0.71,1.02) |
||
Injury, poisoning & AEs |
1.09 |
1.09 |
0.26 |
0.56 |
0.30 |
7.99 |
1.98 |
4.77 |
4.91 |
0.75 |
|
(0.97,1.23) |
(0.96,1.23) |
(0.23,0.30) |
(0.50,0.63) |
(0.21,0.44) |
(6.96,9.21) |
(1.76,2.21) |
(3.90,5.88) |
(3.75,6.54) |
(0.65,0.87) |
||
Musculoskeletal system |
0.95 |
0.87 |
0.36 |
0.66 |
0.58 |
1.32 |
3.40 |
1.42 |
1.56 |
1.05 |
|
(0.84,1.07) |
(0.77,0.99) |
(0.32,0.41) |
(0.58,0.74) |
(0.42,0.81) |
(1.14,1.53) |
(3.03,3.81) |
(1.14,1.78) |
(1.17,2.11) |
(0.91,1.22) |
||
Nervous system |
2.55 |
1.34 |
1.14 |
1.05 |
0.60 |
5.27 |
0.48 |
1.40 |
0.91 |
1.05 |
|
(2.26,2.88) |
(1.19,1.52) |
(0.99,1.31) |
(0.93,1.19) |
(0.43,0.85) |
(4.58,6.09) |
(0.42,0.54) |
(1.12,1.75) |
(0.67,1.25) |
(0.91,1.22) |
||
Procedures, therapy & exams |
1.15 |
1.26 |
0.76 |
0.93 |
1.27 |
1.33 |
0.46 |
2.53 |
0.54 |
1.85 |
|
(1.00,1.32) |
(1.09,1.45) |
(0.66,0.89) |
(0.82,1.07) |
(0.90,1.81) |
(1.12,1.57) |
(0.40,0.53) |
(2.01,3.21) |
(0.36,0.80) |
(1.57,2.17) |
||
Psychological & mental disorders |
0.96 |
0.65 |
0.76 |
1.94 |
0.46 |
1.53 |
0.34 |
0.67 |
0.94 |
1.34 |
|
(0.85,1.09) |
(0.57,0.74) |
(0.66,0.88) |
(1.72,2.18) |
(0.33,0.66) |
(1.32,1.77) |
(0.30,0.38) |
(0.53,0.84) |
(0.71,1.28) |
(0.71,1.28) |
||
Respiratory system |
1.25 |
1.68 |
1.11 |
0.61 |
0.60 |
1.12 |
3.23 |
2.42 |
0.61 |
1.16 |
|
(1.11,1.41) |
(1.49,1.90) |
(0.97,1.27) |
(0.54,0.69) |
(0.44,0.84) |
(0.97,1.30) |
(2.87,3.63) |
(1.97,3.00) |
(0.45,0.84) |
(1.01,1.34) |
||
Skin, nails & hair |
1.25 |
1.32 |
0.49 |
0.40 |
1.23 |
1.49 |
0.41 |
2.90 |
7.50 |
1.31 |
|
(1.07,1.45) |
(1.13,1.54) |
(0.41,0.57) |
(0.34,0.48) |
(0.85,1.80) |
(1.23,1.81) |
(0.35,0.48) |
(2.25,3.75) |
(5.60,10.2) |
(1.08,1.59) |
||
Cardiovascular system |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
Ref |
|
ESI |
0.37 |
0.50 |
0.34 |
0.70 |
0.81 |
0.50 |
0.52 |
0.51 |
0.64 |
0.41 |
|
Blood pressure |
(0.36,0.38) |
(0.49,0.51) |
(0.33,0.35) |
(0.68,0.72) |
(0.76,0.86) |
(0.49,0.52) |
(0.51,0.54) |
(0.49,0.53) |
(0.61,0.68) |
(0.40,0.43) |
|
Systolic blood pressure |
1.00 |
1.01 |
1.00 |
1.00 |
1.00 |
1.00 |
1.00 |
1.00 |
1.00 |
1.00 |
|
(1.00,1.00) |
(1.00,1.01) |
(1.00,1.00) |
(1.00,1.00) |
(1.00,1.00) |
(1.00,1.00) |
(1.00,1.00) |
(1.00,1.00) |
(0.99,1.00) |
(1.00,1.00) |
||
Diastolic blood pressure |
1.01 |
0.99 |
1.00 |
1.00 |
1.00 |
1.00 |
1.00 |
1.01 |
1.00 |
1.00 |
|
(1.00,1.01) |
(0.99,0.99) |
(1.00,1.00) |
(1.00,1.00) |
(0.99,1.00) |
(1.00,1.01) |
(1.00,1.00) |
(1.00,1.01) |
(1.00,1.01) |
(1.00,1.00) |
||
Temperature |
1.15 |
1.29 |
1.11 |
1.15 |
1.00 |
1.04 |
1.08 |
1.06 |
0.94 |
1.12 |
|
(1.12,1.18) |
(1.26,1.32) |
(1.08,1.14) |
(1.12,1.18) |
(0.95,1.04) |
(1.01,1.06) |
(1.05,1.10) |
(1.03,1.09) |
(0.89,0.98) |
(1.09,1.15) |
||
Heart rate |
1.02 |
1.02 |
1.01 |
1.01 |
1.00 |
1.00 |
1.00 |
1.01 |
1.01 |
1.01 |
|
(1.02,1.02) |
(1.02,1.02) |
(1.01,1.01) |
(1.01,1.01) |
(1.00,1.00) |
(1.00,1.01) |
(1.00,1.00) |
(1.01,1.01) |
(1.01,1.01) |
(1.01,1.01) |
||
Respiratory rate |
1.04 |
1.00 |
1.00 |
1.00 |
1.01 |
1.02 |
1.03 |
1.01 |
1.02 |
1.04 |
|
(1.03,1.05) |
(1.00,1.01) |
(0.99,1.01) |
(0.99,1.01) |
(0.99,1.02) |
(1.01,1.02) |
(1.03,1.04) |
(1.01,1.02) |
(1.01,1.03) |
(1.03,1.05) |
||
Oxygen level |
0.98 |
0.98 |
1.00 |
1.01 |
1.02 |
0.98 |
0.97 |
0.98 |
0.95 |
0.99 |
|
(0.97,0.99) |
(0.98,0.99) |
(0.99,1.01) |
(1.00,1.01) |
(1.00,1.04) |
(0.98,0.99) |
(0.96,0.98) |
(0.97,0.99) |
(0.94,0.97) |
(0.98,1) |
Appendix 5. . ED crowding condition changes hourly with the different intervention scenarios
- Singer AJ, Thode Jr HC, Viccellio P, Pines JM. The association between length of emer- gency department boarding and mortality. Acad Emerg Med 2011 Dec;18(12): 1324-9.
- Wang H, Ojha RP, Robinson RD, et al. Optimal measurement interval for emer- gency department crowding estimation tools. Ann Emerg Med 2017;70(5): 632-9 Nov.
- Handel DA, Fu R, Vu E, et al. Association of emergency department and hospital char- acteristics with elopements and length of stay. J Emerg Med 2014;46(6):839-46 Jun.
- Phillips JL, Jackson BE, Fagan EL, et al. Overcrowding and its association with patient outcomes in a median-low volume emergency department. J Clin Med Res 2017;9
- Chang AM, Lin A, Fu R, McConnell KJ, Sun B. Associations of emergency department length of stay with publicly reported quality-of-care measures. Acad Emerg Med 2017;24(2):246-50 Feb.
- Horwitz LI, Green J, Bradley EH. US emergency department performance on wait time and length of visit. Ann Emerg Med 2010;55(2):133-41 Feb.
- Chaou CH, Chen HH, Chang SH, et al. Predicting length of stay among patients discharged from the emergency department-using an accelerated failure time model. PLoS One 2017;12(1):e0165756.
- Chaou CH, Chiu TF, Yen AM, Ng CJ, Chen HH. Analyzing factors affecting emergency department length of stay-using a competing risk-accelerated failure time model. Medicine (Baltimore) 2016;95(14):e3263 Apr.
- van d V, Remeijer C, Fogteloo AJ, Heringhaus C, de GB. Independent determinants of prolonged emergency department length of stay in a tertiary care centre: a prospec- tive cohort study. Scand J Trauma Resusc Emerg Med 2018 Sep 20;26(1):81.
- Hofer KD, Saurenmann RK. Parameters affecting length of stay in a pediatric emer- gency department: a retrospective observational study. Eur J Pediatr 2017;176(5): 591-8 May.
- Derose SF, Gabayan GZ, Chiu VY, Yiu SC, Sun BC. Emergency department crowding predicts admission length-of-stay but not mortality in a large health system. Med Care 2014;52(7):602-11 Jul.
- Zhang X, Kim J, Patzer RE, Pitts SR, Chokshi FH, Schrager JD. Advanced diagnostic im- aging utilization during emergency department visits in the United States: a predic- tive modeling study for emergency department triage. PLoS One 2019;14(4): e0214905.
- Oredsson S, Jonsson H, Rognes J, et al. A systematic review of triage-related interven- tions to improve patient flow in emergency departments. Scand J Trauma Resusc Emerg Med 2011 Jul 19(19):43.
- Pierce BA, Gormley D. Are split flow and provider in triage models in the emergency department effective in reducing discharge length of stay? J Emerg Nurs 2016;42
- Wallingford Jr G, Joshi N, Callagy P, Stone J, Brown I, Shen S. Introduction of a hori- zontal and vertical split flow model of emergency department patients as a response to overcrowding. J Emerg Nurs 2018;44(4):345-52 Jul.
- Tenbensel T, Chalmers L, Jones P, Appleton-Dyer S, Walton L, Ameratunga S. New Zealand’s emergency department target – did it reduce ED length of stay, and if so, how and when? BMC Health Serv Res 2017;17(1):678 Sep 26.
- Day TE, Al-Roubaie AR, Goldlust EJ. Decreased length of stay after addition of healthcare provider in emergency department triage: a comparison between computer-simulated and real-world interventions. Emerg Med J 2013;30(2):134-8 Feb.
- Weston V, Jain SK, Gottlieb M, et al. Effectiveness of resident physicians as triage li- aison providers in an academic emergency department. West J Emerg Med 2017;18
- Nestler DM, Halasy MP, Fratzke AR, et al. Patient throughput benefits of triage liaison providers are lost in a resource-neutral model: a prospective trial. Acad Emerg Med 2014;21(7):794-8 Jul.
- Arya R, Wei G, McCoy JV, Crane J, Ohman-Strickland P, Eisenstein RM. Decreasing length of stay in the emergency department with a split emergency severity index 3 patient flow model. Acad Emerg Med 2013;20(11):1171-9 Nov.
- Rydman RJ, Roberts RR, Albrecht GL, Zalenski RJ, McDermott M. Patient satisfaction with an emergency department asthma observation unit. Acad Emerg Med 1999;6
- Shetty AL, Teh C, Vukasovic M, Joyce S, Vaghasiya MR, Forero R. Impact of emergency department discharge stream short stay unit performance and hospital bed
occupancy rates on access and patient flowmeasures: a single site study. Emerg Med Australas 2017;29(4):407-14 Aug.
- Williams J, Aurora T, Baker K, Thompson J, Smallheer B. Triage to observation: a quality improvement initiative for chest pain patients presenting to the emergency department. Crit Pathw Cardiol 2019;18(2):75-9 Jun.
- Tanabe P, Gimbel R, Yarnold PR, Adams JG. The Emergency Severity Index (version 3) 5-level triage system scores predict ED resource consumption. J Emerg Nurs 2004;30(1):22-9 Feb.
- U.S.Department of Healt EaW. A reason for visit classification for ambulatory care, series 2, number 78 (DHEW Publication No. (PHS) 79-1352); 2019.
- Patton HM, Lavine JE, Van Natta ML, Schwimmer JB, Kleiner D, Molleston J. Clinical correlates of histopathology in pediatric nonalcoholic steatohepatitis. Gastroenterol- ogy 2008;135(6):1961-71 Dec.
- Harrell Jr FE, Lee KL, Pollock BG. Regression models in clinical studies: determining relationships between predictors and response. J Natl Cancer Inst 1988;80(15): 1198-202 Oct 5.
- Prestigiacomo CJ, He W, Catrambone J, et al. Predicting aneurysm rupture probabil- ities through the application of a computed tomography angiography-derived bi- nary logistic regression model. J Neurosurg 2009;110(1):1-6 Jan.
- Turin TC, Kita Y, Rumana N, et al. Time lag to hospitalisation and the associated de- terminants in patients with acute myocardial infarction: the Takashima AMI Regis- try, Japan. Emerg Med J 2011;28(3):239-41 Mar.
- Rodger M, Hills J, Kristjanson L. A Delphi study on Research priorities for emergency nurses in Western Australia. J Emerg Nurs 2004;30(2):117-25 Apr.
- Connelly LG, Bair AE. Discrete event simulation of emergency department activity: a platform for system-level operations research. Acad Emerg Med 2004;11(11): 1177-85 Nov.
- Liu V, Kipnis P, Gould MK, Escobar GJ. Length of stay predictions: improvements through the use of automated laboratory and comorbidity variables. Med Care 2010;48(8):739-44 Aug.
- Cai X, Perez-Concha O, Coiera E, et al. Real-time prediction of mortality, readmission, and length of stay using electronic health record data. J Am Med Inform Assoc 2016; 23(3):553-61 May.
- Yoon P, Steiner I, Reinhardt G. Analysis of factors influencing length of stay in the emergency department. CJEM 2003;5(3):155-61 May.
- Casalino E, Wargon M, Peroziello A, et al. Predictive factors for longer length of stay in an emergency department: a prospective multicentre study evaluating the impact of age, patient’s clinical acuity and complexity, and Care pathways. Emerg Med J 2014;31(5):361-8 May.
- Lowthian JA, Curtis AJ, Jolley DJ, Stoelwinder JU, McNeil JJ, Cameron PA. Demand at the emergency department front door: 10-year trends in presentations. Med J Aust 2012;196:128-32 Feb 6.
- Lin M, Wolfe RE, Shapiro NI, Novack V, Lior Y, Grossman SA. Observation vs admis- sion in syncope: can we predict short length of stays? Am J Emerg Med 2015;33 (11):1684-6 Nov.
- Davis B, Sullivan S, Levine A, Dallara J. Factors affecting ED length-of-stay in surgical critical care patients. Am J Emerg Med 1995;13(5):495-500 Sep.
- Hocker MB, Gerardo CJ, Theiling BJ, et al. NHAMCS validation of emergency severity index as an indicator of emergency department resource utilization. West J Emerg Med 2018 Sep;19(5):855-62.
- van d V, Heringhaus C, de GB. Appropriateness, reasons and independent predictors of consultations in the emergency department (ED) of a Dutch Tertiary Care Center: a prospective cohort study. PLoS One 2016;11(2):e0149079.
- Li L, McCaughey E, Iles-Mann J, Sargeant A, Westbrook JI, Georgiou A. Does point-of- care testing impact length of stay in emergency departments (EDs)?: a before and after study of 26 rural and remote EDs. Stud Health Technol Inform 2018;252:99-104.
- Pines JM, Zocchi MS, Carter C, Marriott CZ, Bernard M, Warner LH. Integrating point- of-care testing into a community emergency department: a mixed-methods evalua- tion. Acad Emerg Med 2018;25(10):1146-56 Oct.
- Hoot NR, LeBlanc LJ, Jones I, et al. Forecasting emergency department crowding: a discrete event simulation. Ann Emerg Med 2008;52(2):116-25 Aug.
- April MD, Murray BP. cost-effectiveness analysis appraisal and application: an emer- gency medicine perspective. Acad Emerg Med 2017;24(6):754-68 Jun.
- Mumma BE, McCue JY, Li CS, Holmes JF. Effects of emergency department ex- pansion on emergency department patient flow. Acad Emerg Med 2014;21