Predicting 72-hour emergency department revisits
a b s t r a c t
Objectives: To develop a predictive model that hospitals or healthcare systems can use to identify patients at high risk of revisiting the ED within 72 h so that appropriate interventions can be delivered.
Methods: This study employed multivariate logistic regression in developing the predictive model. The study data were from four Veterans medical centers in Upstate New York; 21,141 patients in total with ED visits were in- cluded in the analysis. Fiscal Year (FY) 2013 data were used to predict revisits in FY 2014. The predictive variables were patient demographics, prior year healthcare utilizations, and comorbidities. To avoid overfitting, we vali- dated the model by the split-sample method. The Predictive power of the model is measured by c-statistic.
Results: In the first model using only patient demographics, the c-statistics were 0.55 (CI: 0.52-0.57) and 0.54 (95% CI: 0.51-0.56) for the development and validation samples, respectively. In the second model with prior year utilization added, the c-statistics were 0.70 (95% CI: 0.68-0.72) for both samples. In the final model where comorbidities were added, the c-statistics were 0.74 (CI: 0.72-0.76) and 0.73 (95% CI: 0.71-0.75) for the development and validation samples, respectively.
Conclusions: Reducing ED revisits not only lowers healthcare cost but also shortens wait time for those who crit- ically need ED care. However, broad intervention for every ED visitor is not feasible given limited resources. In this study, we developed a predictive model that hospitals and healthcare systems can use to identify “frequent flyers” for early interventions to reduce ED revisits.
Introduction
The cost of providing care in the Emergency Department (ED) is rel- atively high. The average cost of ED visits is $1038 compared to $176 for primary care visits in the United States [1]. Moreover, ED use is on the rise; according to the National Health Statistics Reports from CDC, the number of ED visits increased from 119.2 million (40.5 visits per 100 persons) in 2006 to 136.3 million (44.5 visits per 100 persons) in 2011 [2,3]. Further, a systematic review revealed that frequent-flyer patients constitute a key factor of ED crowding, resulting in treatment delays and excessive mortality [4,5]. Thus, reducing unnecessary ED use, especially repeated visits, should be a key part of the solution to the problem.
However, compared to Hospital readmissions, which have been used by the Centers for Medicare & Medicaid Services (CMS) since October 1, 2012 to reduce payments to the hospitals with excessive readmissions [6], ED revisits have received less attention [7]. To develop interventions
? Funding source: this material is based upon work supported in part by the Office of research and development, Department of Veterans Affairs.
* Corresponding author.
E-mail addresses: [email protected] (G. Pellerin), [email protected] (K. Gao), [email protected] (L. Kaminsky).
to reduce ED revisits, accurate predictive modeling that can identify high risk patients is needed. Although there is some published literature analyzing factors influencing ED revisits, research on predictive model- ing of ED revisits is limited [8-17]. There are a few studies intended to predict 30-day or 6-month ED revisits [18-20]; however, we have not been able to find any published studies designed to predict 72-hour ED revisits [1,21-23].
In this study, we developed a statistical model that predicts patient risk of revisiting the ED within 72 h of discharge, which can be used to identify high risk frequent-flyers for appropriate intervention. With rapid adaptation of Meaningful Use of Health information technology, administrative data have been becoming increasingly closer to real time and offering greater potentials for improving patient care. Our model, based on administrative data and publically available case-mix schemes, could offer a valuable tool for the field.
Method
Data source and study variables
In the present study, we analyzed fiscal years (FY) 2013 and 2014 ED visit data from Veterans Healthcare Network Upstate New York (VISN 2 Upstate), which is one of the 21 Networks through which the US De- partment of Veterans Affairs delivers care to its 5.8 million patients
http://dx.doi.org/10.1016/j.ajem.2017.08.049 0735-6757/
annually. VISN 2 Upstate, with five medical centers and 31 outpatient clinics across upstate New York, serves approximately 140,000 patients with an annual budget of one billion dollars (starting from FY16, VISN 2 was restructured to include NY downstate VA hospitals). In FY 2014 for VISN 2 Upstate, 21,141 patients had ED visits in four of the medical cen- ters which provide ED services.
VA National Patient Care Database (NPCD) hosted at the VA Informa- tion and Computing Infrastructure (VINCI) was the primary data source for this study. We used outpatient care File (OPC) and clinical stop code 130 to identify index ED visits and revisits. In addition to encounter in- formation such as visit dates and ICD-9 CM codes, OPC also contains pa- tient demographic and socioeconomic variables such as age, gender, race and income. NPCD, including OPC, is the gold standard for VA oper- ational analysis and research. Most of the data fields such as visit dates and clinical information like ICD-9 CM codes are routinely and rigorous- ly validated with strict business rules. Its income information is means tested. One exception is that its race information is often incomplete be- cause the VA does not mandate veterans to report race status. However, for the last several years, VA has systematically gathered race informa- tion from other data sources such as Medicare and Department of De- fense (DOD); as a result, the updated race data is deemed accurate and reliable [24,25].
We also used Decision support system (DSS) files that contain actual patient care costs rather than claims or paid as in private health plans. DSS costs are the primary financial data for internal operations and con- gressional inquiries. For case-mix or patient risk, we used a publically available and widely used algorithm, Clinical Classifications Software (CCS), developed by Agency for Healthcare Research and Quality (AHRQ) [26], which classifies patients into 285 homogeneous groups based on ICD-9 codes.
The dependent variable in this study is dichotomous and indicates whether a patient had any ED revisits within 72 h in FY14. The indepen- dent or predictive variables used in this study were from FY13 and were grouped into four categories: (1) demographics: age, sex, marital status, race, period of military service, and disability rating; (2) socioeconomic variables: patient income, homeless (equals 1, otherwise 0), and patient insurance status, i.e. not covered by any insurance (equals 1, otherwise 0), enrolled in Medicare (equals 1, otherwise 0), enrolled in Medicaid (equals 1, otherwise 0), and covered by private insurance (equals 1, oth- erwise 0); and (3) prior year utilization and cost: ED revisit within 72 h (yes/no), ED revisit within 30 days (yes/no), the number of ED revisits within 30 days, total number of ED visits, the number of primary care visits, the number of tele-health encounters, the total outpatient visits, the number of hospitalizations, and the Total cost; and (4) patient risk or comorbidities: 285 clinically homogeneous groups produced by Clin- ical Classifications Software (CCS) developed by Agency for Healthcare Research and Quality (AHRQ) [26].
The present study did not require or use any identifiable patient pri- vate information and therefore had expedited IRB review.
Modeling and analysis
We adopted logistic regression to predict the probability or risk of 72-hour ED revisit. Logistic regression has been the most extensively used model in predicting outcomes where the dependent variable is bi- nary, i.e., equals 1 if the event happened, otherwise equals 0. The model’s predictive or discriminative ability is measured by the c-statis- tic, which is defined as the proportion of times the model correctly dis- criminates a random pair of individuals with or without the event. It is also equivalent to the area under the receiver operating characteristic curve. A c-statistic of 0.5 indicates that the model is no better than flip- ping a coin; a c-statistic of 0.7-0.8 suggests that the model has good dis- criminative ability; and a c-statistic of 0.8 or greater suggests great discriminative ability [27].
To prevent model over-fit, we only included variables with p-values
b 0.05 (by stepwise) in the final regression analysis, and we also
calculated shrinkage coefficient, an indicator of over-fit [28]. We further validated the model by the split-sample method [28,29]. With this method, the full sample (after merging the dependent variable from 2014 and the independent variables from 2013) was randomly split into a derivation sample (2/3) and a validation sample (1/3) [30]. The model was fitted on the derivation sample and then the estimated coef- ficients were applied to the validation sample to produce the risk score (probability) and the model fit statistics. The split-sample method is widely used to prevent predictive models from fitting random noises rather than a true trend or pattern. The analyses were conducted by using PROC LOGISTIC of SAS 9.3.
To demonstrate the predictive power of different independent vari- ables, we configured and fitted three models from basic to comprehen- sive. Model 1: only demographic, socioeconomic variables are included in the regression as the explanatory variables, i.e., age, sex, marital sta- tus, race, income, enrolled in Medicare, enrolled in Medicaid or covered by other private insurance (no insurance status was omitted in the re- gression as reference). We also used three dummy variables (one is omitted as reference) as the fixed effect to take into account the poten- tially different practice patterns among the four medical centers. Model 2: variables in model 1, and prior year utilizations, i.e., ED revisit within 72 h (yes/no), ED revisit within 30 days (yes/no), the number of ED re- visits within 30 days, total number of ED visits, the number of primary care visits, the number of tele-health encounters, the total outpatient visits, the number of hospitalizations, and the total cost. Model 3: vari- ables in model 2, and patient comorbidities by CCS. The inclusion or ex- clusion of the variables in the final regression depends on the p-values in the stepwise procedure.
Results
All 21,141 patients who had ED visits in FY 2014 were included in this study. Among the 21,141 patients, 2346 returned to the EDs within 72 h; the overall 72-hour revisit rate was 11%. The indepen- dent variables and their descriptive statistics are reported in Table
1. The CCSs (285 indicator variables) are not reported in Table 1; instead, those 20 CCS indictor variables (representing 591 distinct ICD-9-CM codes) that were statistically significant in the final model are reported along with other variables in Table 3 showing the parameter estimates, odds ratios, and confidence intervals. Table 2 shows the top 20 most frequent diagnoses of patients with 72-hour ED revisits.
In predicting ED 72-hour revisits, the first model only included de- mographics, socioeconomic characteristics and the fixed effect of the medical centers, in which nine variables were statistically significant (p-values b 0.05) and kept in the model. The c-statistics were 0.55 (CI: 0.52-0.57) and 0.54 (95% CI: 0.51-0.56) for the development and vali- dation samples, respectively. In the second model, 12 variables were statistically significant and kept in the model. The c-statistics were
0.70 (95% CI: 0.68-0.72) for both samples. In the final model, 32 vari-
ables that were statistically significant were kept in the model, and the c-statistics were 0.74 (CI: 0.72-0.76) and 0.73 (95% CI: 0.71-0.75) for the development and validation samples, respectively. The receiver operating characteristic curves of all three models based on the valida- tion sample are reported in Fig. 1.
The parameter estimates of the full model are reported in Table 3. Note that for the age groups, we omitted aged 75 or older as the baseline in the regression and kept the three age groups that were statistically in- significant in the model as a convention.
To further examine the prediction accuracy, we graphed the ob-
served 72-hour revisit rate against the five estimated risk categories (quintiles) in Fig. 2; as shown, the revisit rate was 1.7% among the pa- tients in the lowest risk quintile and 26.2% in the highest risk quintile. In addition, we also estimated the shrinkage coefficient which yielded a value of 0.89, indicating no over-fit [28].
Descriptive statistics of the independent variables.
Variables Patients
without ED revisit (n = 21,141)
Patients with ED revisit (n = 1593)
p-Value
Mean, SD Mean, SD |
|||||
Age |
59.7 |
17.3 |
61.2 |
16.9 |
0.0009 |
Age b 35 |
0.12 |
0.33 |
0.09 |
0.29 |
0.0005 |
35 <= Age b 45 |
0.08 |
0.27 |
0.08 |
0.27 |
0.7890 |
45 <= Age b 55 |
0.14 |
0.35 |
0.13 |
0.34 |
0.4401 |
55 <= Age b 65 |
0.23 |
0.42 |
0.26 |
0.44 |
0.0052 |
65 <= Age b 75 |
0.23 |
0.42 |
0.23 |
0.42 |
0.5232 |
Age >= 75 |
0.19 |
0.39 |
0.21 |
0.40 |
0.1661 |
Sex (male = 0, female = 1) |
0.10 |
0.29 |
0.09 |
0.28 |
0.1737 |
Marital status (married) |
0.41 |
0.49 |
0.37 |
0.48 |
0.0015 |
Race status (Black) |
0.12 |
0.33 |
0.14 |
0.35 |
0.0135 |
Patient income (1000) |
24.27 |
41.37 |
22.80 |
25.14 |
0.1628 |
Served Vietnam War |
0.37 |
0.48 |
0.40 |
0.49 |
0.0150 |
Disability rating (%) |
26.54 |
36.35 |
27.64 |
37.63 |
0.2469 |
Disability rating N 70% |
0.22 |
0.42 |
0.24 |
0.43 |
0.0854 |
Homeless |
0.001 |
0.03 |
0.003 |
0.06 |
0.0079 |
Not covered by any health insurance |
0.40 |
0.49 |
0.36 |
0.48 |
0.0036 |
Enrolled in Medicaid |
0.03 |
0.17 |
0.03 |
0.18 |
0.1505 |
Enrolled in Medicare |
0.47 |
0.50 |
0.53 |
0.50 |
0.0000 |
Covered by private insurance |
0.09 |
0.28 |
0.07 |
0.25 |
0.0100 |
Prior year ED revisit within 72 h (yes/no) |
0.04 |
0.19 |
0.11 |
0.31 |
0.0000 |
Prior year ED revisit within 30 days (yes/no) |
0.12 |
0.32 |
0.25 |
0.43 |
0.0000 |
Prior year total outpatient visits |
13.31 |
24.44 |
23.59 |
33.64 |
0.0000 |
Prior year total cost (1000) |
25.84 |
39.80 |
48.32 |
59.65 |
0.0000 |
Prior year number of hospitalizations |
0.28 |
0.83 |
0.61 |
1.48 |
0.0000 |
Medical center A |
0.25 |
0.44 |
0.25 |
0.43 |
0.7386 |
Medical center B |
0.04 |
0.20 |
0.03 |
0.18 |
0.0433 |
Medical center C |
0.39 |
0.49 |
0.41 |
0.49 |
0.0352 |
Medical center D |
0.31 |
0.46 |
0.30 |
0.46 |
0.3073 |
Discussion
Across the country, waiting lines in EDs remain long. Centers for Dis- ease Control and Prevention (CDC) reported the average wait time in
U.S. EDs increased by 25% from 46.5 min to 58.1 min during the period of 2003 through 2009 [31]. In 2015, American College of Emergency
systematic review found repeated ED visits as a key contributor to ED crowding, which causes treatment delays and excessive mortality [4,5]. To free ED capacity for true emergencies, reducing unnecessary ED use, especially repeated visits, is desired. However, unlike the rate of hospital readmissions that have been used by CMS to penalize the hos- pitals with excessive readmissions, ED revisits, and particularly short- term revisits, have not received much attention [7]. Interventions to re- duce ED revisits require accurate predictive modeling that can identify the patients at high risk for short-term readmissions, but we have not
found any fully risk-adjusted predictive model in the literature.
Prior year number of ED revisit within 30 |
0.22 |
0.83 |
0.91 |
3.27 |
0.0000 |
Fig. 1. Receiver operating characteristic curves for 72-hour ED revisits. |
days |
||||||
Prior year total number of ED visit |
0.93 |
1.62 |
2.12 |
3.93 |
0.0000 |
|
Prior year number of primary care visits |
1.83 |
3.36 |
2.92 |
4.12 |
0.0000 |
Physicians (ACEP) released a survey report showing ED visits went up |
Prior year number of tele-health |
1.62 |
3.75 |
2.92 |
5.54 |
0.0000 |
since implementation of Affordable Care Act [32]. Consequentially, |
encounters |
waiting for hours before seeing a doctor is not uncommon [2,3,33]. A |
In this study, we developed a statistical model that predicts which patients are at risk for ED revisits within 72 h with good accuracy. The strength of the model for broad application lies in three features: (1)
Top 20 diagnoses with frequent ED revisits.
Diagnosis |
Patients with 72_hour ED revisit |
Patients with any ED visit |
Percent of patients with 72_hour ED revisit (%)? |
CCS0197: skin and subcutaneous tissue infections |
122 |
1131 |
11% |
CCS0205: spondylosis; intervertebral disc disorder |
119 |
1878 |
6% |
CCS0660: alcohol related disorders |
102 |
727 |
14% |
CCS0102: nonspecific chest pain |
98 |
1672 |
6% |
CCS0251: abdominal pain |
92 |
1061 |
9% |
CCS0127: chronic obstructive pulmonary disease |
77 |
1197 |
6% |
CCS0163: genitourinary symptoms |
75 |
543 |
14% |
CCS0204: other nontraumatic joint disorders |
66 |
1418 |
5% |
CCS0257: other aftercare |
65 |
352 |
18% |
CCS0133: other lower respiratory disease |
61 |
1062 |
6% |
CCS0661: substance related disorders |
55 |
402 |
14% |
CCS0651: anxiety disorders |
54 |
420 |
13% |
CCS0211: other connective tissue disease |
53 |
1208 |
4% |
CCS0155: other gastrointestinal disorders |
49 |
681 |
7% |
CCS0126: other upper Respiratory infections |
48 |
1411 |
3% |
CCS0259: residual codes; unclassified |
44 |
689 |
6% |
CCS0084: headache; including migraine |
43 |
484 |
9% |
CCS0106: cardiac dysrhythmias |
39 |
611 |
6% |
CCS0122: pneumonia |
39 |
676 |
6% |
CCS0134: other upper respiratory disease |
38 |
292 |
13% |
Logistic regression parameter estimates.
Variable |
Parameter estimate |
p-Value |
Odds ratio |
95% CI: low limit |
95% CI: up limit |
Intercept |
-7.056 |
b0.0001 |
|||
Age b 35 |
0.297 |
0.067 |
1.345 |
0.979 |
1.848 |
35 <= Age b 45 |
0.436 |
0.008 |
1.546 |
1.121 |
2.132 |
45 <= Age b 55 |
0.193 |
0.154 |
1.213 |
0.930 |
1.581 |
55 <= Age b 65 |
0.106 |
0.356 |
1.112 |
0.888 |
1.391 |
65 <= Age b 75 |
0.020 |
0.848 |
1.021 |
0.829 |
1.257 |
Sex (male = 0, female = 1) |
-0.271 |
0.030 |
0.762 |
0.597 |
0.974 |
Disability rating |
-0.003 |
0.004 |
0.997 |
0.995 |
0.999 |
Prior year total cost |
0.356 |
b0.0001 |
1.428 |
1.333 |
1.529 |
Prior year total number of emergence department visits |
0.132 |
b0.0001 |
1.141 |
1.104 |
1.181 |
Prior year total number of hospitalizations |
-0.126 |
0.001 |
0.882 |
0.819 |
0.950 |
CCS0007: viral infection |
0.396 |
0.004 |
1.486 |
1.137 |
1.942 |
CCS0085: coma; stupor; and brain damage |
0.584 |
0.013 |
1.793 |
1.131 |
2.841 |
CCS0093: conditions associated with dizziness or vertigo |
0.429 |
b0.0001 |
1.536 |
1.253 |
1.883 |
CCS0102: nonspecific chest pain |
0.225 |
0.006 |
1.252 |
1.066 |
1.471 |
CCS0109: acute cerebrovascular disease |
0.462 |
0.000 |
1.587 |
1.248 |
2.020 |
CCS0154: noninfectious gastroenteritis |
0.399 |
0.015 |
1.491 |
1.080 |
2.058 |
CCS0159: urinary tract infections |
0.489 |
b0.0001 |
1.630 |
1.326 |
2.004 |
CCS0163: genitourinary symptoms and ill-defined conditions |
0.231 |
0.010 |
1.260 |
1.058 |
1.501 |
CCS0197: skin and subcutaneous tissue infections |
0.438 |
b0.0001 |
1.550 |
1.287 |
1.867 |
CCS0231: other fractures |
0.399 |
0.009 |
1.490 |
1.103 |
2.013 |
CCS0236: open wounds of extremities |
0.370 |
0.010 |
1.448 |
1.093 |
1.918 |
CCS0239: superficial injury; contusion |
0.321 |
0.001 |
1.378 |
1.134 |
1.673 |
CCS0250: nausea and vomiting |
0.478 |
b0.0001 |
1.614 |
1.286 |
2.025 |
CCS0251: abdominal pain |
0.322 |
0.000 |
1.380 |
1.163 |
1.637 |
CCS0253: Allergic reactions |
0.396 |
0.000 |
1.485 |
1.201 |
1.838 |
CCS0254: rehabilitation care; fitting of prostheses; and adjustment of devices |
-0.216 |
0.004 |
0.806 |
0.696 |
0.933 |
CCS0661: substance-related disorders |
0.293 |
0.002 |
1.340 |
1.110 |
1.618 |
CCS0662: suicide and intentional self-inflicted injury |
0.492 |
0.000 |
1.636 |
1.257 |
2.129 |
CCS2619: external cause codes: Other specified; NEC |
0.698 |
0.001 |
2.009 |
1.361 |
2.968 |
CCS2621: E codes: place of occurrence |
0.310 |
0.001 |
1.363 |
1.128 |
1.647 |
Medical center A vs D |
0.199 |
0.011 |
0.977 |
0.819 |
1.166 |
Medical center B vs D |
-0.657 |
0.000 |
0.415 |
0.262 |
0.657 |
Medical center C vs D |
0.236 |
0.001 |
1.013 |
0.863 |
1.190 |
it is based on administrative data that are readily available in nearly all hospitals or health systems; (2) the grouping algorithm of patient risks or comorbidities is publicly available from AHRQ; and (3) the model is based on logistic regression, which can be readily implemented with many common software packages, including MS Excel.
Unlike predicting hospital readmissions [34], diagnoses or comor- bidities only added limited predictive power to demographics and prior year utilization: the c-statistic jumped up by 0.17 from model 1 (only demographics are included) to model 2 (prior year utilization are added), while only increased 0.028 from model 2 to model 3 (diag- noses are added). Similarly, our sensitivity analysis revealed that prior year utilization did not add much predictive power either if the patient demographics and comorbidities were entered into the model first, which indicates prior year utilization and comorbidities are correlated. On a special note, female patients in this study were less likely to ex- perience ED revisits given all other things being equal, but further stud- ies are needed to ascertain the reasons. In any event, the finding should
Fig. 2. Observed 72-hour ED revisit rate by predicted risk level.
not be generalized to the general public because VA patients are pre- dominantly male. It is also interesting to see that a higher disability rat- ing was associated with low risk of ED revisits. Although we do not know the exact reason(s), it could be due to the fact that medical needs for disability can be better anticipated and planned. It is paradox- ical to see the number of prior year hospitalizations was inversely corre- lated with ED revisits. Again, without knowing the exact reason(s) we speculate inpatient care rather than repeated ED visits may be more ap- propriate or justified for the patients who were hospitalized in prior years. In other words, prior year hospitalizations, an indicator of the dis- ease severity, warrant inpatient care rather than repeated ED use. Obvi- ously, more research is needed but this is a study focusing on prediction rather than on the effect of each variable in the model.
This study has several limitations. First, the data used in this study were from one region (Upstate New York), and a Veteran population (fewer female and more mental health patients). The predictive power could be improved if a larger population is used in the analysis. Second, veterans in this study may have received care from other health systems such as Medicare and Medicaid, which may also have de- creased the predictive power of the model. Third, for the approved study period, ICD-10 codes were not available and therefore ICD-9 codes were used to identify patient diseases or comorbidities. However, the modeling process remains same when ICD-10 codes are used [35].
Conclusions
Reducing ED revisits will not only lower healthcare cost but also shorten wait time for those who critically need ED care. However, broad intervention to every ED visitor is not feasible given limited re- sources. In this study, we developed a predictive model based on ad- ministrative data, i.e., patient demographics (e.g., age and gender), prior year health care utilization (e.g., total cost and ED visits), and diag- noses. The model has a good predictive power and produces a risk score
(the probability of revisiting ED within 72 h) for each patient. The model can be readily implemented by hospitals and healthcare systems with medical health records, and the resultant risk score can be used to iden- tify patients at high risk of returning to the ED for proactive interven- tions to reduce ED revisits.
References
- White D. Seventy-two hour emergency department returns. College of Nursing, Washington state University; May 2010.
- Pitts SR, Niska RW, Xu J, Burt CW. National Hospital Ambulatory Medical Care Sur- vey: 2006 emergency department summary. Natl Health Stat Rep 2008;7:1-38.
- National Hospital Ambulatory Medical Care Survey: 2011 emergency department summary tables. Available from: http://www.cdc.gov/nchs/data/ahcd/nhamcs_ emergency/2011_ed_web_tables.pdf.
- Gaieski DF, Agarwal AK, Mikkelsen ME, Drumheller B, Cham Sante S, Shofer FS, et al. The impact of ED crowding on early interventions and mortality in patients with se- vere sepsis. Am J Emerg Med 2017;35(7):953-60.
- Hoot NR, Aronsky D. Systematic review of emergency department crowding: causes, effects, and solutions. Ann Emerg Med 2008 Aug;52(2):126-36.
- Pizer SD. Should hospital readmissions be reduced through payment penalties? Med Care 2013;51(1):20-2.
- Committee on the Future of Emergency Care in the United States Health System. Hospital-based emergency care: at the breaking point. Washington, DC: National Academies Press; 2006.
- Hastings SN, Oddone EZ, Fillenbaum G, Sloane RJ, Schmader KE. Frequency and pre- dictors of adverse health outcomes in older Medicare beneficiaries discharged from the emergency department. Med Care 2008;46(8):771-7.
- McCusker J, Cardin S, Bellavance F, Belzile E. Return to the emergency department among elders: patterns and predictors. Acad Emerg Med 2000;7(3):249-59.
- Friedmann PD, Jin L, Karrison TG, et al. Early revisit, hospitalization, or death among older persons discharged from the ED. Am J Emerg Med 2001;19(2):125-9.
- Yee JL, Hasson NK, Schreiber DH. Drug-related emergency department visits in an el- derly veteran population. Ann Pharmacother 2005;39(12):1990-5.
- Hastings SN, Schmader KE, Sloane RJ, Weinberger M, Goldberg KC, Oddone EZ. Ad- verse health outcomes after discharge from the emergency department – incidence and risk factors in a veteran population. J Gen Intern Med 2007;22(11):1527-31.
- Doran KM, Raven MC, Rosenheck RA. What drives frequent emergency department use in an integrated health system? National data from the Veterans Health Admin- istration. Ann Emerg Med 2013;62(2):151-9.
- Goodman RM. Emergency department use associated with primary care office man- agement. Am J Manag Care 2013;19(5):e185-6.
- Rising KL, Padrez KA, O’Brien M, et al. Return visits to the emergency department: the patient perspective. Ann Emerg Med 2015;65(4):377-86.
- Sauvin G, Freund Y, Saidi K, et al. UnschedulED return visits to the emergency de- partment: consequences for triage. Acad Emerg Med 2013;20(1):33-9.
- Hasegawa K, Tsugawa Y, Tsai CL, et al. Frequent utilization of the emergency depart- ment for acute exacerbation of chronic obstructive pulmonary disease. Respir Res 2014;15:40.
- Hastings SN, et al. Emergency department visits in veterans affairs medical facilities. Am J Manag Care 2011;17(6):e215-23.
- Hao S, Jin B, Shin AY, et al. risk prediction of emergency department revisit 30 days post discharge: a prospective study. PLoS One 2014;9(11):e112944.
- Jin B, Zhao Y, Hao S, et al. Prospective stratification of patients at risk for emergency department revisit: resource utilization and population Management strategy impli- cations. BMC Emerg Med 2016;16:10. http://dx.doi.org/10.1186/s12873-016-0074- 5 [Published online 2016 Feb 3. PMCID: PMC4739399].
- Wu CL, Wang FT, Chiang YC, et al. Unplanned emergency department revisits within 72 hours to a secondary teaching referral hospital in Taiwan. J Emerg Med 2010; 38(4):512-7.
- Chan AH, Ho SF, Fook-Chong SM, et al. Characteristics of patients who made a return visit within 72 hours to the emergency department of a Singapore tertiary hospital. Singapore Med J 2016;57(6):301-6.
- Tsai IT, Sun CK, Chang CS, et al. Characteristics and outcomes of patients with emer- gency department revisits within 72 hours and subsequent admission to the inten- sive care unit. Tzu Chi Med J 2016;28(4):151-6.
- Stroupe KT, Tarlov E, Zhang Q, Haywood T, Owens A, Hynes DM. Use of Medicare and DoD data for improving VA race Data quality. J Rehabil Res Dev 2010;47(8): 781-95.
- Trivedi AN, Crebia RC, Wright SM, Washinton DL. Despite improved quality of care in the veterans affairs health system, Racial disparity persists for important clinical out- comes. Health Aff 2011;4:707-15.
- Available from: https://www.hcup-us.ahrq.gov/toolssoftware/ccs/ccs.jsp.
- Kansagara D, Englander H, Salanitro A, Kagen D, Theobald C, Freeman M, et al. Risk prediction models for hospital readmission, a systematic review. JAMA 2011; 306(15):1688-98.
- Harrell FE, Lee KL, Mark DB. Tutorial in biostatistics multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 1996;15:361-87.
- Efron B. Estimating the error rate of a prediction rule: improvement on cross-valida- tion. J Am Stat Assoc 1983;78:316-31.
- Wang L, Porter B, Maynard C, et al. Predicting risk of hospitalization or death among patients receiving primary care in the Veterans Health Administration. Med Care 2012;51:368-73.
- Available from: https://www.cdc.gov/nchs/products/databriefs/db102.htm.
- Available from: http://newsroom.acep.org/statistics_and_reports?item=30102.
- Horwitz LI, Green J, Bradley EH. United States emergency department performance on wait time and length of visit. Ann Emerg Med 2010;55(2):133-41.
- Shulan M, Gao K, Moore CD. Predicting 30-day all-cause hospital readmissions. Health Care Manag Sci 2013;16(2):167-75.
- Available from: https://www.hcup-us.ahrq.gov/toolssoftware/ccs10/ccs10.jsp.