a b s t r a c t

Background: Involuntary mental health detainments should only be utilized when less restrictive alternatives are unavailable and should be discontinued as soon as safety can be ensured. The study objective was to determine if child and adolescent psychiatrists discontinue a greater proportion of Involuntary holds than general psychia- trists for similar pediatric patients.

Methods: Retrospective analysis of consecutive patients under 18 years placed on an involuntary hold in the prehospital setting presenting over a 1-year period to one high-volume emergency department (ED) where youth on involuntary holds are seen by child and adolescent psychiatrists when available and general psychia- trists otherwise. The primary outcome of interest was hold discontinuation after initial Psychiatric consultation. The key predictor of interest was psychiatrist specialty (child and adolescent vs. general). We conducted multi- variate logistic regression modeling adjusting for patient characteristics and time of arrival.

Results: Child and adolescent psychiatrists discontinued 27.4% (51/186) of prehospital holds while general psy- chiatrists discontinued only 10.6% (22/207). After adjusting for observable confounders, holds were over 3 times as likely to be discontinued in patients evaluated by child and adolescent psychiatrists rather than general emergency psychiatrists (adjusted OR 3.2, 95% CI 1.7-5.9, p b 0.001).

Conclusions: Child and adolescent psychiatrists are much more likely to discontinue prehospital involuntary men- tal health holds compared with general emergency psychiatrists. While inappropriate hold discontinuation places patients at risk of harm, prolonged hold continuation limits patients’ rights and potentially increases psy- chiatric boarding in EDs. Earlier access to child and adolescent psychiatry may facilitate early hold discontinua- tion and standardize patient care.

(C) 2019

Introduction

While involuntary mental health holds are an important tool for en- suring safety in youth with severe mental health crises, their use signif- icantly infringes on patient rights. As such, involuntary holds should only be used when less restrictive alternatives have failed or are unsafe and should be discontinued as soon as feasible [1,2]. Early hold discon- tinuation, when possible without compromising safety, can benefit both patients and EDs.

* Corresponding author.

E-mail addresses: [email protected] (G. Santillanes), [email protected] (K. Rowland), [email protected] (M. Demarest), [email protected] (C.N. Lam), [email protected] (M.P. Wilson).

Emergency responders in prehospital settings sometimes initiate in- voluntary holds in relatively uncontrolled situations and then transport patients to emergency departments (EDs) for “medical clearance” which is generally an evaluation to rule out medical disorders, stabilize conditions arising from attempts at self-harm, and, when necessary, treat severe agitation. After the initial ED evaluation, patients are often calmer and collateral information may be obtained from the patient, family, outpatient mental health clinicians and other sources. In this context, qualified specialists may discontinue holds in the ED allowing for discharge to outpatient rather than inpatient care. A decision to con- tinue a hold may result in the patient “boarding,” or remaining in the ED after the decision is made to admit or transfer, if a psychiatric inpatient bed is not available. Shorter hold durations have been associated with shorter ED lengths of stay [3]. Prolonged lengths of stay and patient

https://doi.org/10.1016/j.ajem.2019.06.002

0735-6757/(C) 2019

boarding impact ED operations generally [2,4-8] and contribute to ED crowding which is associated with decreased quality of care [9-14]. Fur- thermore, ED settings are typically poorly aligned with patients’ mental health needs and time spent boarding in the ED may not be therapeutic as EDs often lack resources to provide Optimal care for patients with mental health emergencies [6,15-17]. The commonly noisy and chaotic ED environment may worsen underlying symptoms including anxiety and agitation [6,8,18-20] and involuntary hospitalizations can be psy- chologically harmful to patients [21-23].

Availability of ED psychiatric consultation has been identified as a best practice with potential to reduce the need for inpatient psychiatric hospitalization, thereby reducing psychiatric boarding in EDs [24]. However, guidelines do not specifically address the evaluation of youth on involuntary mental health holds, particularly whether pediat- ric patients should be seen by specialists in child and adolescent psychi- atry. Mental health emergencies in youth and adults differ along multiple dimensions including underlying disease processes, lethality of threats, available social supports and applicable laws and general psy- chiatry residency may provide as little as 2 months of child and adoles- cent training while child and adolescent psychiatrists train in the care of children and adolescents for 24 months, including training in the man- agement of pediatric psychiatric emergencies [25,26].

Our study objective was to determine if the probability of discontin- uation of pediatric prehospital involuntary mental health holds after ini- tial psychiatric consultation varied based on the specialty of the initial treating psychiatrist. We hypothesized that for similar patients, psychi- atrists board certified in child and adolescent psychiatry discontinue proportionally more holds at the conclusion of the initial consultation than general psychiatrists.

Methods

We conducted a retrospective chart review of all patients under 18 years old presenting to a large urban ED for initial evaluation follow- ing placement of an involuntary hold in a pre-hospital setting from July 1, 2016 to June 30, 2017. The University of Southern California IRB ap- proved the study with waiver of consent.

Study setting and patients

LAC+USC Medical Center is a general public hospital with a 24-h onsite emergency psychiatry service, but no pediatric inpatient psychi- atric beds. Annual ED volume is 150,000 patients, including 21,000 in a dedicated pediatric ED. In Los Angeles County, multiple agencies includ- ing law enforcement and mobile mental health emergency response teams initiate prehospital holds. Patients who arrive to LAC+USC on an involuntary mental health hold are evaluated by psychiatry after being deemed medically stable.

Patients were identified via a query of an electronic administrative database. Patients transferred from a psychiatric urgent care or psychi- atric hospital were excluded. Our primary analysis was limited to pa- tients arriving from 3 pm to midnight weekdays and 9 am-6 pm weekends in order to observe encounters with a relatively comparable acuity and ease of obtaining collateral information while also observing variation in the subspecialty of psychiatrist evaluating the patient. At the study site youth are seen by a child and adolescent psychiatry at- tending physician or a general emergency psychiatry attending physi- cian based on the psychiatrist staffing the emergency consult service at the time that they present for treatment. As child and adolescent psy- chiatrists initially evaluated 92% of patients arriving weekday daytime hours and b3% of patients arriving on weekend evenings during the study period, we did not include patients arriving during these hours in the analysis (Appendix A). We did not include patients arriving after midnight because overnight hold discontinuation is especially challenging; social work services are limited and, frequently, important sources of collateral information cannot be contacted. Furthermore,

patients presenting overnight are potentially fundamentally different from patients presenting at other hours in level of acuity, social support, and other factors.

Key outcome measures

The primary outcome of interest was documentation of involuntary hold discontinuation documented in the psychiatry note at the conclu- sion of the initial consultation. The key independent variable of interest was the specialty of the initial psychiatry attending physician (child and adolescent vs. general psychiatry). Psychiatrists were defined as child and adolescent psychiatrists if board certified in child and adolescent psychiatry at any point during the 1-year study period.

Data collection

Two medical students blinded to study hypothesis were trained as abstractors by the primary investigator (GS) and directly supervised for abstraction of at least 30 charts. Abstraction methods were devel- oped a priori and included a standardized abstraction form as well as a protocol driven abstraction method. Instructions were also included on the abstraction form as a prompt. The primary investigator con- ducted regular monitoring of data collection and abstractor questions were resolved by discussion with the primary investigator. Variables in- cluding demographic information, agency initiating the hold, reason for hold, initial psychiatrist evaluating patient and initial action on hold (continued or discontinued) were abstracted from the medical record into a standardized abstraction form using REDCap electronic data cap- ture tools hosted at University of Southern California [27].

10% of charts were independently abstracted by both abstractors. A weighted Cohen’s kappa coefficient was calculated for 4 key variables to measure inter-rater agreement. When abstractors were compared, the weighed kappa was 0.94 for agreement on whether inclusion criteria were met, 0.95 for agency placing hold, 0.95 for initial psychia- trist evaluating patient and 1.0 for initial action on hold (continued or discontinued).

Data analysis

Descriptive statistics were used to characterize patients. We used a t-test for continuous variables and chi-square statistics for categorical variables to compare differences between patients seen by child and ad- olescent psychiatry and general emergency psychiatry. Statistical tests were conducted in Stata 13 (Statacorp, College Station, TX).

We analyzed the association between factors potentially affecting hold discontinuation using a multivariate logistic regression model. In- cluded covariates were patient age, gender, primary language, hold in- dication, agency initiating hold, and time and day of arrival. The regression was driven by theory and included covariates were not de- termined by automated statistical procedures. We were concerned that patients presenting at different hours might differ in severity but there is no validated measure available to adjust for case severity. As such we included arrival time and weekend versus weekday arrival as a covariate in an effort to capture some of this case mix variety. We tested for multicollinearity of the independent variables using the

-collin- command in Stata and the independent variables were not col- linear. We also conducted a sensitivity analysis of a subgroup of patients seen from 3 pm to 7:59 pm weekdays theorizing that there is a lower chance of case mix variation during this narrow time window. We also conducted the multivariate logistic regression model with the addi- tion of one additional covariate: psychiatrist years of experience (de- fined as years from medical school graduation to date of visit). Finally, because we hypothesized that general emergency psychiatrists might be less comfortable assessing suicidal youth and young children (e.g. b10 years), we analyzed interaction effects between psychiatrist

specialty and reason for hold (e.g. suicidal ideation) and between psy- chiatrist specialty and patient age.

Results

901 ED visits for pediatric patients on involuntary mental health holds were identified (Fig. 1). 716 were for initial evaluation of holds placed in the pre-hospital setting, of which 393 presented during times of interest. Of the 393, 186 (47.3%) were initially evaluated by child and adolescent psychiatry and 207 (52.7%) by general emergency psychiatry. Mean patient age was 12.9 years and the majority of holds were for danger to self (Table 1).

Overall, 18.6% (73/393) of holds were discontinued after initial psychiatry evaluation. Child and adolescent psychiatrists discontinued 27.4% of holds (51 of 186) while general emergency psychiatrists discontinued only 10.6% of holds (22 of 207) (p b 0.001).

After adjusting for observable confounders in a multivariate logistic regression, patients evaluated by child and adolescent psychiatrists had N3 times the odds of hold discontinuation as patients evaluated by gen- eral emergency psychiatrists (adjusted OR 3.16, 95% CI 1.69-5.90, p b 0.001) (Table 2). Patient age and time and day of arrival were associated with significant differences in proportion of holds discontinued. Pa- tients under age 10 years had 3 times the odds of hold discontinuation as patients aged 15-17 years. Holds were less likely to be discontinued in patients arriving later in the evening than those arriving earlier and in patients arriving weekend afternoons as compared to weekday afternoons.

The interaction between psychiatrist subspecialty and patient age and psychiatric subspecialty and reason for hold were not significant. A sensitivity analysis of the 187 patients arriving 3-7:59 pm weekdays also found that patients evaluated initially by child and adolescent psy- chiatry were significantly more likely to have their holds discontinued than patients evaluated by general emergency psychiatry (39.5% vs 16.8%, p = 0.001) (Appendix B). When the regression was performed additionally controlling for psychiatrist years of experience, patients evaluated by child and adolescent psychiatrists had nearly 4 times the odds of hold discontinuation as patients evaluated by general emer- gency psychiatrists (OR 3.97, 95% CI 1.70-9.28, p = 0.001) (Appendix C).

Discussion

In this study, nearly 1 in 5 involuntary mental health holds were discontinued after initial psychiatry evaluation. Critically, we found sig- nificant variation in the proportion of holds that were rescinded de- pending on the subspecialty of initial consulting psychiatrist. Youth evaluated by child and adolescent psychiatrists were N3 times as likely to have involuntary holds discontinued compared with youth seen by general emergency psychiatrists.

This variation in hold disposition between child and adolescent psychiatrists and general psychiatrists is concerning, and is particu- larly relevant because the majority of youth with mental health emergencies are seen in general EDs not staffed by child and adoles- cent psychiatrists. Inappropriate hold discontinuation may result in death or serious harm to the patient or others, but unnecessary

146 visits- holds placed by psychiatry in ED

901 visits for <18 year olds on involuntary mental health holds.

4 visits- patients not on involuntary hold

ED visit for initial evaluation of hold placed in pre-hospital setting?

33 visits- transfer from a psychiatric urgent care

Yes

716 ED visits

No

Total Excluded: 185 visits

1 visit- patient initially thought to be >18 years old and treated in adult psychiatric ED

Arrival during times of interest? (see text)

No

323 visits

Yes

393 visits

1 visit- patient represented after eloping from placement

186 Visits Initially Evaluated by Child/Adolescent Psychiatry

207 Visits Initially evaluated by General Emergency Psychiatry

Fig. 1. Flow diagram: pediatric emergency department patients on involuntary mental health holds.

Table 1 Characteristics of patients evaluated by child and adolescent psychiatry and general emer- gency psychiatry.

Table 2

Results of multivariate logistic regression model of factors predicting hold discontinuation.

Consulting psychiatrist specialty

	Patients seen by child & adolescent psychiatry	Patients seen by general emergency psychiatry	p-Valueb
	n = 186	n = 207
Age (years) (mean, SD)	12.9 (+-2.84)	12.9 (+-3.01)	0.865
	n (%)	n (%)
Hold discontinued Sex	51 (27.4)	22 (10.6)	b0.001

n Adjusted OR (95% CI)^a

p-Value^a

General emergency psychiatrist 207 Reference

Child and adolescent psychiatrist 186 3.16 (1.69-5.90) b0.001

Patient sex

Female 196 Reference

Male 197 1.26 (0.69-2.33) 0.452

Patient age

15-17 years 361 Reference

10-14 years	452	1.15 (0.60-2.20)	0.678
b10 years	88	3.23 (1.33-7.81)	0.009

Race/ethnicity

Male	105 (56.5)	92 (44.4)	0.017
Female	81 (43.5)	115 (55.6)

Hispanic/Latino	105 (56.5)	134 (64.7)	0.286
Non-Hispanic White	5 (2.7)	9 (4.4)
African-American	23 (12.4)	20 (9.7)
Asian/Pacific Islander	5 (2.7)	6 (2.9)
Other/unknown	48 (25.8)	38 (18.4)

Primary language

English	147 (79.0)	153 (73.9)	0.224
Spanish	33 (17.7)	51 (24.6)
Other	5 (2.7)	3 (1.4)
Not listed	1 (0.5)	0 (0)

Reason for hold^a

DTS Only	120 (64.5)	126 (60.9)	0.400
DTO Only	14 (7.5)	18 (8.7)
DTS & DTO	42 (22.6)	58 (28.0)
GD +- DTS/DTO	3 (1.6)	3 (1.4)
Hold not scanned/legible	7 (3.8)	2 (1.0)

Agency initiating hold

Police/sheriffs	31 (16.7)	35 (16.9)	b0.001
School police	37 (19.9)	13 (6.3)
Mental health response team	107 (57.5)	153 (73.9)
Other	11 (5.9)	6 (2.9)
Day and time of arrival Weekday 3 pm-5:59 pm	65 (34.9)	59 (28.5)
Weekday 6 pm-8:59 pm	31 (16.7)	66 (31.9)	b0.001
Weekday 9 pm-11:59 pm	44 (23.7)	69 (33.3)
Weekend 9 am-11:59 am	9 (4.8)	2 (1.0)
Weekend 12 pm-2:59 pm	16 (8.6)	4 (1.9)
Weekend 3 pm-5:59 pm	21 (11.3)	7 (3.4)

Categorical variables analyzed using Chi-square statistics. Continuous variables analyzed using t-test.

^a DTS = Danger to self, DTO = Danger to others, GD = Gravely disabled.

^b Significant p-values in bold type.

emergency detainment also has the potential to harm patients [21- 23] and should occur only when less restrictive means are not possi- ble. Additionally, unnecessary detainment and boarding of patients who could be treated as outpatients contributes to ED crowding, straining the ability of EDs to provide high quality care to all patients. In this context, tolerance for variation in care should be minimal and practice standardization is needed.

Our findings are consistent with studies documenting significant variation in admission decisions for a variety of medical conditions [28-30]. Generally, variation is driven by the interaction of physician, patient, local culture and health system factors. A significant strength of this study is that, because it takes place at a single site, we isolate effect to psychiatrist factors, particularly subspecialty. We demonstrated that child and adolescent psychiatrists were able to discontinue a significantly higher proportion of involuntary holds than general emergency psychiatrists. Based on their experi- ence and specialized knowledge, child and adolescent psychiatrists may be better able to determine which patients are sufficiently low risk for discharge. Physicians who mainly treat pediatric patients may be more comfortable discharging patients requiring observation to parents, while physicians treating mostly adults often cannot dis- charge patients to responsible caregivers and might be more likely to hospitalize marginal cases. Assessing a youth’s level of risk and

Primary language

English	300	Reference
Spanish	84	1.22 (0.61-2.43)	0.577
Other/not listed	9	1.41 (0.22-9.06)	0.720

Agency initiating involuntary hold

Police	66	Reference
School police	50	0.87 (0.30-2.51)	0.804
Mental health response team	260	1.29 (0.54-3.11)	0.564
Other or hold not scanned/legible	17	2.38 (0.38-14.75)	0.352

Reason for involuntary hold

Danger to self only	246	Reference
Danger to others only	32	0.49 (0.14-1.71)	0.265
Danger to self and others	100	0.58 (0.28-1.21)	0.146
Gravely disabled +- danger to self/others	6	0.56 (0.05-6.53)	0.646
Hold not scanned or reason not legible	9	0.08 (0.01-1.21)	0.068

Day and time of arrival

Weekday 3 pm-5:59 pm	124	Reference
Weekday 6 pm-8:59 pm	97	0.53 (0.25-1.09)	0.083
Weekday 9 pm-11:59 pm	113	0.14 (0.06-0.35)	b0.001
Weekend 9 am-11:59 am	11	1.35 (0.31-5.94)	0.688
Weekend 12 pm-2:59 pm	20	0.88 (0.28-2.69)	0.816
Weekend 3 pm-5:59 pm	28	0.28 (0.08-0.92)	0.036

^a Significant differences in bold type.

determining appropriate disposition requires not only an assess- ment of the patient, but also the parent’s judgement, insight and ability to provide adequate supervision. Child and adolescent psychi- atrists may be more comfortable evaluating caregivers and assessing interactions between youth and families. They are likely to be more aware of available resources and more experienced in interacting with child protective services.

In the current study, most holds were for danger to self. Estimat- ing suicide risk and determining appropriate disposition is challeng- ing in all ages [31], but evaluations for suicidal ideation may differ somewhat between youth and adults. From an epidemiologic per- spective, the lethality of suicidal ideation in youth differs from sui- cidal ideation in adults. Suicidal ideation is quite common in adolescence. In 2017, 17.2% of high school students responding to the National Youth Risk Behavior Survey reported seriously consid- ering attempting suicide and 13.6% reported making a suicide plan in the prior year [32]. While not to be minimized, suicide attempts by adolescents are less likely to be lethal than suicide attempts in adults [31,33]. Although suicide is a leading cause of death in adoles- cents, adolescent suicide deaths are in fact fairly rare. In 2015, there were 2061 deaths by suicide in 15-19 year olds, with a suicide rate for 15-19 year olds of 14.2 per 100,000 population for males and

5.1 per 100,000 for females [34]. In contrast, the suicide rate for

45-64 year olds was approximately twice as high at 29.7 per 100,000 population for males and 9.8 per 100,000 for females in 2014 [33]. While the number of adolescent suicides is significant, and suicidal ideation requires intervention, outpatient treatment is sometimes appropriate for youth with suicidal ideation [35]. How- ever, there are no validated criteria for use in the assessment of level of risk of subsequent suicide and appropriate level of care for youth with suicidal ideation [35]. These factors, combined with the high stakes of youth suicide, may result in general psychiatrists being more reluctant to discontinue holds for danger to self.

In theory, practice variation could be decreased if all youth on in- voluntary holds were evaluated by child and adolescent psychiatrists on ED arrival. However, profound workforce shortages often make it impossible for youth with mental health emergencies to be evalu- ated by general psychiatrists much less child and adolescent psychi- atrists [36-39]. As an example, there are only 3 practicing child and adolescent psychiatrists in South Dakota and 8 in Wyoming [40]. Recognizing resource limitations, alternative means of improving ac- cess to this service are necessary. Models such as regionalized psy- chiatric emergency services [41-43] and telepsychiatry [44-46] could potentially provide earlier access to child and adolescent spe- cialists and merit further investigation.

Currently, the majority of pediatric mental health emergencies are managed by ED physicians, pediatricians and general psychiatrists with comparatively little formal training in pediatric psychiatric emer- gencies [41]. Given workforce shortages, general psychiatrists will con- tinue to evaluate youth with mental health emergencies. Further study is needed to determine the reasons that child and adolescent psychia- trists discontinue a greater proportion of holds than general psychia- trists. A deeper understanding of factors considered by child and adolescent psychiatrists when determining risk level is necessary in order to develop guidelines to increase standardization of care.

As the number of youth presenting to EDs for acute mental health care continues to increase, the current patchwork system of psychiatric emergency care is not sustainable. We describe a disparity in care be- tween patients evaluated initially by child and adolescent psychiatrists and general emergency psychiatrists. Given the serious negative conse- quences to patients both of unnecessary hold continuation and discon- tinuation of necessary holds, accepted standards of care are needed. For EDs, earlier discontinuation of involuntary holds has the additional po- tential benefit of decreased psychiatric boarding. Validated criteria to aid in risk assessment and determination of appropriate level of care may improve standardization of care for youth with mental health emergencies.

Limitations

This retrospective design is a study limitation, but we followed best practice guidelines for chart reviews to reduce potential limita- tions of the chart review method [47-49]. The key outcome measure was objective and we had almost perfect inter-rater agreement. While systematic differences between patients treated by general and child and adolescent psychiatrists could confound our findings, we limited the sample to patients arriving during specific hours in order to minimize confounding by severity. As there is no reliable quantitative measure of severity of a mental health crisis, we consid- ered that patients arriving during the selected hours likely had sim- ilar severity of illness and that psychiatrists had similar ability to obtain collateral information. Outcomes after the ED visit were not assessed. Absent an objective method of determining patients’ risk of harming themselves or others, we have no reason to believe that evaluations and risk assessments of children and adolescents by a child and adolescent psychiatrist would be less accurate than those performed by a general psychiatrist.

Findings of a single-center study are less generalizable and further study in additional settings is warranted. For this pilot study we felt that a single-center study ensured that the underlying legal, cultural and health system context was the same for patients seen by general psychiatry and child and adolescent psychiatry. A multicenter study would introduce additional sources of variation based on local culture and resources which might obscure the relationship between psychia- trist subspecialty and Disposition decision. We believe that characteris- tics of the study site could actually lead to an underestimation of differences in hold discontinuation decisions between subspecialties. The ED treats a high volume of pediatric patients with mental health emergencies and available mental health resources are likely more

robust than in smaller volume hospitals. The hospital’s general emer- gency psychiatrists have significant experience evaluating pediatric mental health emergencies, work closely with child and adolescent psy- chiatrists, and evaluate more pediatric patients on involuntary holds than most general psychiatrists.

Conclusions

We demonstrated significant variation in care of youth on invol- untary holds based on consulting psychiatrist subspecialty in a high-volume ED with 24-hour psychiatry coverage. Youth evaluated by child and adolescent psychiatrists had N3 times the odds of hold discontinuation compared to youth evaluated by general emergency psychiatrists. While further validation of these findings in other set- tings is needed, our analysis suggests that strategies allowing earlier access to child and adolescent psychiatry may facilitate early hold discontinuation with potential benefits to patients and EDs.

Financial support

This research did not receive any specific grant from funding agen- cies in the public, commercial, or not-for-profit sectors.

Presentations

An abstract based upon preliminary data was presented at the American College of Emergency Physicians Annual Meeting in October 2018.

Contributions

Dr. Santillanes has full access to all of the data presented in the study and takes responsibility for the integrity of the data and accuracy of data analysis.

Concept and design: Santillanes and Menchine.

Acquisition, analysis, or interpretation of data: Santillanes, Rowland and Demarest and Menchine.

Drafting of the manuscript: Santillanes. Critical revision of the manuscript for important intellectual content:

Wilson, Claudius, Rowland, Demarest, Lam and Menchine.

Statistical analysis: Lam.

All authors declare that they have no conflicts of interest.

Acknowledgements

The authors greatly appreciate the assistance in manuscript prepara- tion provided by Carla Martinez. The authors also gratefully acknowl- edge the support of the UAMS Clinician Scientist program, which generously supported Dr. Wilson’s time.

Appendix A. Initial evaluating psychiatrist by time and day of arrival

	Child & adolescent psychiatry	Adult psychiatry
Weekdays (n = 595) 12:00 am-8:59 am	46.2% (49/106)	53.8% (57/106)
9:00 am-2:59 pm	92.3% (143/155)	7.7% (12/155)
3:00 pm-5:59 pm	52.4% (65/124)	47.6% (59/124)
6:00 pm-8:59 pm	32.0% (31/97)	68.0% (66/97)
9:00 pm-11:59 pm	38.9% (44/113)	61.1% (69/113)
Weekends (n = 121) 12:00 am-8:59 am	20.0% (5/25)	80.0% (20/25)
9:00 am-11:59 am	81.8% (9/11)	18.2% (2/11)
12:00 pm-2:59 pm	80.0% (16/20)	20.0% (4/20)
3:00 pm-5:59 pm	75.0% (21/28)	25.0% (7/28)
6:00 pm-11:59 pm	2.7% (1/37)	97.3% (36/37)

Appendix B. Sensitivity analysis of patients arriving 3 pm-7:59 pm weekdays and 3 pm-5:59 pm weekends

(continued)

n Adjusted OR (95% CI)^a

Danger to self and others	100	0.58 (0.27-1.21)	0.143
Gravely disabled +- danger to self/others	6	0.57 (0.05-6.70)	0.656
Hold not scanned or reason not legible Day and time of arrival	9	0.08 (0.00-1.14)	0.062
Weekday 3 pm-5:59 pm	124	Reference
Weekday 6 pm-8:59 pm	97	0.54 (0.26-1.11)	0.093
Weekday 9 pm-11:59 pm	113	0.14 (0.05-0.35)	b0.001
Weekend 9 am-11:59 am	11	1.36 (0.30-6.09)	0.686

p-Value^a

Patients seen by child & adolescent psychiatry n = 186

Patients seen by general emergency psychiatry n = 207

p-Value^b

Age (years) (mean, SD)

	n (%) n (%)			Weekend 12 pm-2:59 pm	20	0.84 (0.27-2.64)	0.767
Hold discontinued	34 (39.5)	17 (16.8)	0.001	Weekend 3 pm-5:59 pm	28	0.28 (0.08-0.93)	0.037
Sex				aSignificant differences in bold type.

12.5 (+-3.13) 13.2 (+-2.75) 0.095

R

Male	50 (58.1)	37 (36.6)	0.003
Female	36 (41.9)	64 (63.4)
ace/ethnicity Hispanic/Latino 50 (58.1) 69 (68.3)
Non-Hispanic White	1 (1.2)	5 (5.0)	0.167
African-American	7 (8.1)	8 (7.9)
Asian/Pacific Islander Other/unknown	4 (4.7) 24 (27.9)	2 (2.0) 17 (16.8)

Primary language

English 67 (77.9) 72 (71.3) 0.127

Spanish 17 (19.8) 29 (28.7)

Other 2 (2.3) 0 (0)

Not listed Reason for hold^a

DTS only	57 (66.3)	68 (67.3)
DTO only	7 (8.1)	7 (6.9)
DTS & DTO	16 (18.6)	23 (22.8)	0.502
GD +- DTS/DTO	1 (1.2)	1 (1.0)
Hold not	5 (5.8)	2 (2.0)
scanned/legible

Agency initiating hold

Police/sheriffs 15 (17.4) 13 (12.9)

School police 22 (25.6) 11 (10.9) 0.012

References

1. Hedman LC, Petrila J, Fisher WH, Swanson JW, Dingman DA, Burris S. State laws on emergency holds for mental health stabilization. Psychiatr Serv 2016;67(5):529-35.
2. Chun TH, Mace SE, Katz ER. Evaluation and management of children and adolescents with acute mental health or behavioral problems. Part I: common clinical challenges of patients with mental health and/or behavioral emergencies. Pediatrics 2016;138 (3).
3. Wilson MP, Brennan JJ, Modesti L, et al. Lengths of stay for involuntarily held psychi- atric patients in the ED are affected by both patient characteristics and medication use. Am J Emerg Med 2015;33(4):527-30.
4. The Joint Commission. Quick safety: Alleviating ED boarding of Psychiatric patients. The Joint Commission, Division of Health Care Improvement; 2015.
5. Alakeson V, Pande N, Ludwig M. A plan to reduce emergency room ‘boarding’ of psy- chiatric patients. Health Aff (Millwood) 2010;29(9):1637-42.
6. Abid Z, Meltzer A, Lazar D, Pines J. Psychiatric boarding in U.S. EDs: a multifactorial problem that requires multidisciplinary solutions. Policy Brief 2014;1(2) [Accessed 10/11/18].
7. Nolan JM, Fee C, Cooper BA, Rankin SH, Blegen MA. Psychiatric boarding incidence, duration, and associated factors in United States emergency departments. J Emerg Nurs 2015;41(1):57-64.
8. Bender D, Pande N, Ludwig M. Literature review: psychiatric boarding. U.S. Depart- ment of Health and Human Services, Assistant Secretary for Planning and Evalua- tion, Office of Disability, Aging and Long-Term Care Policy; 2008.
9. Wu D, Zhou X, Ye L, Gan J, Zhang M. Emergency department crowding and the per-

   Mental health response team

   42 (48.8) 72 (71.3)

   formance of Damage control resuscitation in major trauma patients with hemor- rhagic shock. Acad Emerg Med 2015;22(8):915-21.

   Other 7 (8.1) 5 (5.0)

   ^aDTS = danger to self, DTO = danger to others, GD = gravely disabled.

   ^bSignificant p-values in bold type.

   ?Continuous variables analyzed using t-test.

   Categorical variables analyzed using Chi-square statistics.

   Appendix C. Results of multivariate logistic regression model of fac- tors predicting hold discontinuation

   Sills MR, Fairclough D, Ranade D, Kahn MG. Emergency department crowding is as-

   sociated with decreased quality of care for children with acute asthma. Ann Emerg Med 2011;57(3):191-200 [e191-197].

   Pines JM, Hollander JE. Emergency department crowding is associated with poor care for patients with severe pain. Ann Emerg Med 2008;51(1):1-5.

10. Schull MJ, Vermeulen M, Slaughter G, Morrison L, Daly P. Emergency department crowding and thrombolysis delays in acute myocardial infarction. Ann Emerg Med 2004;44(6):577-85.
11. Sun BC, Hsia RY, Weiss RE, et al. Effect of emergency department crowding on out- comes of admitted patients. Ann Emerg Med 2013;61(6):605-611.e606.
12. Reznek MA, Murray E, Youngren MN, Durham NT, Michael SS. Door-to-imaging time for acute stroke patients is adversely affected by emergency department crowding. Stroke 2017;48(1):49-54.
13. Baraff LJ, Janowicz N, Asarnow JR. Survey of California emergency departments

    Consulting psychiatrist specialty

    n Adjusted OR (95% CI)^a

    p-Value^a

    about practices for management of suicidal patients and resources available for their care. Ann Emerg Med 2006;48(4):452-8 [458.e451-452].

    Stone A, Rogers D, Kruckenberg S, Lieser A. Impact of the mental healthcare delivery system on California emergency departments. West J Emerg Med 2012;13(1):51-6.

    gency medical services system. Pediatrics 2011;127(5):e1356-66.

    General emergency psychiatrist	207	Reference	[17] American College of Emergency Physicians. ACEP psychiatric and substance abuse
    Child and adolescent psychiatrist Patient sex Female	186 196	3.97 (1.70-9.28) Reference	0.001 survey 2008 Irving, TX ; 2008. [18] Doupnik SK, Esposito J, Lavelle J. Beyond mental health crisis stabilization in emer- gency departments and acute care hospitals. Pediatrics 2018;141(5).
    Male Patient age	197	1.27 (0.69-2.34)	0.452 [19] Nordstrom K, Berlin J, Schmelzer N, Shah S, Gitlin D. Position statement on emer- gency boarding of patients with acute Mental illness. American Psychiatric Associa-
    15-17 years	361	Reference	tion; 2016.
    10-14 years	452	1.15 (0.60-2.21)	0.673 [20] Dolan MA, Fein JA. Pediatric and adolescent mental health emergencies in the emer-
    b10 years Primary language English	88 208	3.22 (1.32-7.82) Reference	0.010 [21] Sibitz I, Scheutz A, Lakeman R, Schrank B, Schaffer M, Amering M. Impact of coercive measures on life stories: qualitative study. Br J Psychiatry 2011;199(3):239-44. [22] Katsakou C, Priebe S. Patient’s experiences of involuntary hospital admission and

    Spanish	158	1.41 (0.61-3.27)	0.421	treatment: a review of qualitative studies. Epidemiol Psichiatr Soc 2007;16(2):
    Other/not listed	27	1.22 (0.36-4.16)	0.746	172-8.

    Agency initiating involuntary hold

    Police 66 Reference

    School police 50 0.87 (0.30-2.51) 0.801

    Mental health response team 260 1.31 (0.54-3.14) 0.551

    Other or hold not scanned/legible 17 2.54 (0.39-16.35) 0.327 Reason for involuntary hold

    Danger to self only 246 Reference

    Danger to others only 32 0.50 (0.14-1.77) 0.285

    Rusch N, Muller M, Lay B, et al. Emotional reactions to involuntary psychiatric hos- pitalization and stigma-related stress among people with mental illness. Eur Arch Psychiatry Clin Neurosci 2014;264(1):35-43.

14. American College of Emergncy Physicians Emergency Medicine Practice Committee. Care of the Psychiatric Patient in the emergency department -a review of the liter- ature; 2014.
15. Accreditation Council for Graduate Medical Education. ACGME program require- ments for graduate medical education in psychiatry; 2017.
16. Accreditation Council for Graduate Medical Education. ACGME program require- ments for graduate medical education in child and adolescent psychiatry; 2015.
17. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)–a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 2009;42 (2):377-81.
18. Gaucher N, Bailey B, Gravel J. Impact of physicians’ characteristics on the admission risk among children visiting a pediatric emergency department. Pediatr Emerg Care 2012;28(2):120-4.
19. Abualenain J, Frohna WJ, Shesser R, Ding R, Smith M, Pines JM. Emergency depart- ment physician-level and hospital-level variation in admission rates. Ann Emerg Med 2013;61(6):638-43.
20. Guterman JJ, Lundberg SR, Scheib GP, et al. Wide variability in emergency physician admission rates: a target to reduce costs without compromising quality. West J Emerg Med 2016;17(5):561-6.
21. Practice guideline for the assessment and treatment of patients with suicidal behav- iors, Am J Psychiatry 2003;160(11 Suppl):1-60.
22. Centers for Disease Control. Youth risk behavior surveillance system: Trends in the prevalence of suicide-related behaviors. National Youth Risk Behavior Surveillance: 1991-2017. [Accessed 06/19/2018].
23. Curtin SC, Warner M, Hedegaard H. Increase in suicide in the United States, 1999-2014. NCHS Data Brief 2016;241:1-8.
24. QuickStats. Suicide rates for teens aged 15-19 years, by sex — United States, 1975-2015. MMWR Morb Mortal Wkly Rep 2017;66:816.
25. Chun TH, Katz ER, Duffy SJ, Gerson RS. Challenges of managing pediatric mental health crises in the emergency department. Child Adolesc Psychiatr Clin N Am 2015;24(1):21-40.
26. Thomas CR, Holzer 3rd CE. The continuing shortage of child and adolescent psychi- atrists. J Am Acad Child Adolesc Psychiatry 2006;45(9):1023-31.
27. Mian AI, Milavic G, Skokauskas N. Child and adolescent psychiatry training: a global perspective. Child Adolesc Psychiatr Clin N Am 2015;24(4):699-714.
28. Kim WJ. Child and adolescent psychiatry workforce: a critical shortage and national challenge. Acad Psychiatry 2003;27(4):277-82.
29. Child and adolescent psychiatry workforce crisis: Solutions to improve early inter- vention and access to care. Washington, DC: American Academy of Child & Adoles- cent Psychiatry; 2013.
30. American Academy of Child & Adolescent Psychiatry. Workforce maps by state web site. https://www.aacap.org/aacap/Advocacy/Federal_and_State_Initiatives/ Workforce_Maps/Home.aspx, Accessed date: 19 September 2018.
31. Gerson R, Havens J. The child and adolescent emergency: a public health challenge. Psychiatr Times 2015;32(11).
32. Sisson P. Rady Children’s to create unique psychiatric emergency department for kids. San Diego Tribune; 2017.
33. Grover P, Lee T. Dedicated pediatric behavioral health unit: serving the unique and individual needs of children in behavioral health crisis. Pediatr Emerg Care 2013; 29(2):200-2.
34. Telepsychiatry program eases patient crowding in the ED, expedites mental health services to patients and providers, ED Manag 2013;25(11):121-4.
35. Narasimhan M, Druss BG, Hockenberry JM, et al. Impact of a telepsychiatry program at emergency departments statewide on the quality, utilization, and costs of mental health services. Psychiatr Serv 2015;66(11):1167-72.
36. Seidel RW, Kilgus MD. Agreement between telepsychiatry assessment and face-to- face assessment for Emergency Department psychiatry patients. J Telemed Telecare 2014;20(2):59-62.
37. Kaji AH, Schriger D, Green S. Looking through the retrospectoscope: reducing bias in emergency medicine chart review studies. Ann Emerg Med 2014;64(3):292-8.
38. Gearing RE, Mian IA, Barber J, Ickowicz A. A methodology for conducting retrospec- tive chart review research in child and adolescent psychiatry. J Can Acad Child Adolesc Psychiatry 2006;15(3):126-34.
39. Auten J, Ishimine P. How to design a study that everyone will believe: retrospective reviews. In: Wilson MP, Guluma KK, Hayden S, editors. Doing research in emergency and acute care: Making order out of chaos. John Wiley & Sons, Ltd.; 2015