Diagnostic performance of emergency physician-performed point-of-care ultrasonography for acute appendicitis: A meta-analysis
Contents lists available at ScienceDirect
American Journal of Emergency Medicine
journal homepage:
Diagnostic performance of emergency physician-performed point-of- care ultrasonography for acute appendicitis: A meta-analysis
Sun Hwa Lee, MD, PhD a, Seong Jong Yun, MD, PhD b,?
a Department of Emergency Medicine, Sanggye Paik Hospital, Inje University College of Medicine, 1342 Dongil-ro, Nowon-gu, Seoul, 01757, Republic of Korea
b Department of Radiology, Kyung Hee University Hospital at Gangdong, Kyung Hee University School of Medicine, 892 Dongnam-ro, Gangdong-gu, Seoul, 05278, Republic of Korea
a r t i c l e i n f o
Article history:
Received 9 June 2018
Accepted 11 July 2018
Keywords:
Meta-analysis Acute appendicitis
Point-of-care ultrasonography Pediatric
Emergency physician
a b s t r a c t
Objective: To assess the sensitivity and specificity of emergency physician-performed point-of-care ultrasonogra- phy (EP-POCUS) for diagnosing Acute appendicitis .
Material and methods: The PubMed and EMBASE databases were searched, and the diagnostic performance of EP- POCUS was evaluated using bivariate modeling and hierarchical summary receiver operating characteristic curves. Subgroup analysis was performed for pediatric patients to compare EP-POCUS and radiologist- performed ultrasonography (RADUS). Meta-regression analyses were performed according to patient and study characteristics.
Results: In 17 studies (2385 patients), EP-POCUS for diagnosing AA exhibited a pooled sensitivity of 84% (95% con- fidence interval [CI]: 72%-92%) and a pooled specificity of 91% (95% CI: 85%-95%), with even better diagnostic performance for pediatric AA (sensitivity: 95%, 95% CI: 75%-99%; specificity: 95%, 95% CI: 85%-98%). A direct comparison revealed no significant differences (p = 0.18-0.85) between the diagnostic performances of EP- POCUS (sensitivity: 81%, 95% CI: 61%-90%; specificity: 89%, 95% CI: 77%-95%) and RADUS (sensitivity: 74%, 95% CI: 65%-81%; specificity: 97%, 95% CI: 93%-98%). The meta-regression analyses revealed that study location, AA proportion, and mean age were sources of heterogeneity. Higher sensitivity and specificity tended to be associ- ated with an appendix diameter cut-off value of 7 mm and the EP as the initial operator.
Conclusion: The diagnostic performances of EP-POCUS and RADUS were excellent for AA, with EP-POCUS having even better performance for pediatric AA. Accurate diagnoses may be achieved when the attending EP is the ini- tial POCUS operator and uses a 7-mm cut-off value.
(C) 2018
Introduction
acute appendicitis is the most common acute abdominal disor- der that requires surgery, with a lifetime incidence of 7-8% [1, 2]. Early diagnosis and management are crucial because of this condition’s high Morbidity and mortality rates. The diagnosis of AA is based on History taking, a physical examination, and laboratory testing, although it re- mains challenging because the findings are often atypical and overlap with those of other conditions [3, 4]. Although the diagnosis of AA de- pends on a clinical suspicion and physical examination, imaging before surgery plays an important role in confirming or ruling out the diagnosis [5], with Ultrasonography , computed tomography (CT), and mag- netic resonance imaging being common diagnostic modalities. Previous reports [6, 7] have recommended US as the initial imaging modality for adult and pediatric patients, as there is no requirement for a contrast agent or radiation exposure. This is important, as there are increasing concerns regarding ionizing radiation exposure, particularly for young
E-mail address: [email protected] (S.J. Yun).
adults and pediatric patients, with 0.7-2% of cancer cases being attrib- uted to CT-related radiation exposure and a multiplicative and cumula- tive effect observed after multiple CT examinations [8, 9].
The limitations of US are its highly operator-dependent interpreta- tion and the fact that radiologists are not available 24 h per day for on-demand US in many emergency departments (ED), which can delay the diagnosis and prolong the ED stay. Thus, emergency physi- cians performing point-of-care ultrasonography (EP-POCUS) at the patient’s bedside are gradually becoming important in the ED. Further- more, the American College of Emergency Physicians’ 2008 revision of their Emergency Ultrasound Guideline policy statement supports an ex- panded role for POCUS.
Previous reports and meta-analyses have examined clinician- performed POCUS, which included emergency physicians (EP), sur- geons, pediatricians, and gastroenterologists [10, 11]. However, those studies did not focus on EP-POCUS for diagnosing AA. In addition, the data regarding the diagnostic performance of EP-POCUS have revealed broad ranges of sensitivity and specificity. Therefore, the relative value of EP-POCUS for AA should be fully explored, and high-level evidence should be gathered through the quantitative synthesis of data from
https://doi.org/10.1016/j.ajem.2018.07.025
0735-6757/(C) 2018
Fig. 1. The study Selection process for the meta-analysis.
the existing studies. The pooled results will also be interesting, as the published studies have used different study designs, patient ages, and cut-off values.
This systematic review and meta-analysis aimed to assess the diag- nostic performance of EP-POCUS for AA, with a subgroup analysis of pe- diatric AA, and to compare the results to those for radiologist-performed US (RADUS).
Methods
This meta-analysis followed the revised guidelines of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses of Diagnos- tic Accuracy Studies (PRISMA-DTA) statement [12].
Data sources
The Pubmed and EMBASE databases were searched up to April 20, 2018 to identify English-language reports regarding EP-POCUS for diag- nosing AA. Search terms that were related to “acute appendicitis”, “emergency physician”, or “ultrasound” were combined with “appen- dectomy” and “diagnosis”, “sensitivity”, “specificity”, or “receiver oper- ating characteristic” as follows: ((appendicitis)) AND ((emergency physician) OR (emergency physicians) OR (emergency medicine resi- dent) OR (emergency medicine residency)) AND ((“POCUS”) OR (point-of-care) OR (bedside) OR (“US”) OR (“USG”) OR (ultrasound) OR (ultrasonography) OR (sonogram) OR (sonography)) AND ((pathol- ogy) OR (pathological) OR (histology) OR (histopathological) OR (ap- pendectomy) OR (surgery) OR (surgical)) AND ((diagnosis) OR
The included patients’ demographic characteristics
Author |
No. of patients |
No. of AA |
AA (%) |
Mean age |
Age range |
No. of male |
No. of female |
Male (%) |
Chen SC, et al. [36] |
147 |
110 |
74.8 |
37.1 |
8-84 |
82 |
65 |
55.8 |
Fathi M, et al. [39] |
97 |
43 |
44.3 |
34.4 |
9-82 |
56 |
41 |
57.7 |
Fox JC1, et al. [40] |
155 |
69 |
44.5 |
NR |
NR |
85 |
70 |
54.8 |
Fox JC2, et al. [41] |
126 |
57 |
45.2 |
NR |
2-81 |
NR |
NR |
NR |
Gungor F,et al. [42] |
264 |
169 |
64.0 |
30 |
N18 |
151 |
113 |
57.2 |
Kang B, et al. [43] |
166 |
40 |
24.1 |
10.6 |
b19 |
94 |
72 |
56.6 |
Karimi E, et al. [44] |
108 |
37 |
34.3 |
23.9 |
NR |
66 |
42 |
61.1 |
Kim C, et al. [45] |
115 |
36 |
31.3 |
10.6 |
b19 |
65 |
50 |
56.5 |
Lam SH, et al. [46] |
116 |
46 |
39.7 |
20.2 |
4- |
59 |
57 |
50.9 |
Mallin M, et al. [48] |
97 |
34 |
35.1 |
28 |
10-51 |
NR |
NR |
NR |
Sivitz AB, et al. [50] |
231 |
85 |
36.8 |
Median age, 10.2 |
2-20.9 |
60 |
171 |
26.0 |
Topin F, et al. [51] |
104 |
25 |
24.0 |
33.2 |
5-73 |
62 |
42 |
59.6 |
Unluer EE, et al. [52] |
100 |
45 |
45.0 |
32.9 |
NR |
43 |
57 |
43.0 |
Siu AYC, et al. [49] |
85 |
29 |
34.1 |
31.6 |
8-85 |
45 |
40 |
52.9 |
Elikashvili I, et al. [38] |
150 |
50 |
33.3 |
12 |
2-19 |
66 |
84 |
44.0 |
Lin WC, et al. [47] |
284 |
112 |
39.4 |
6 |
1-8 |
NR |
NR |
NR |
Doniger SJ, et al. [37] |
40 |
16 |
40.0 |
9.3 |
2-18 |
20 |
20 |
50.0 |
No., number; AA, acute appendicitis; NR, not reported.
Characteristics of the included studies
Author |
Year |
Locale |
Study period |
Study design |
Reference standard for AA |
Reference standard for No AA |
Blinding |
Initial US performer |
Separate US training for AA |
Chen SC, et al. [36] Fathi M, |
2000 2015 |
Taiwan Iran |
1997.9-1999.6 2013.5-2014.4 |
Prospective, consecutive Prospective, |
Surgical/patholog-ical finding Surgical/patholog-ical |
Clinical follow-up (2 weeks) Clinical follow-up |
Blinding Blinding |
Attending EP Resident/attending |
Yes Yes |
et al. [39] Fox JC1, |
2007 |
United |
2002.1-2004.1 |
consecutive Retrospective, |
finding Surgical/patholog-ical |
Radiologist-performed US or CT |
NR |
EP Resident/attending |
NR |
et al. [40] Fox JC2, |
2008 |
States United |
2004.2-2004.11 |
consecutive Prospective, |
finding Surgical/patholog-ical |
Clinical follow-up (2 weeks-3 |
Blinding |
EP Resident/attending |
Yes |
et al. [41] Gungor |
2017 |
States Turkey |
2014.5-2015.5 |
consecutive Prospective, |
finding Surgical/patholog-ical |
months) Surgical/pathological finding, CT, or |
Blinding |
EP NR |
Yes |
F,et al. [42] Kang B, |
2015 |
South |
2011.9-2013.2 |
consecutive Retrospective, |
finding Surgical/patholog-ical |
clinical follow-up Clinical follow-up (1-3 days) |
Blinding |
Attending EP |
NR |
et al. [43] |
Korea |
consecutive |
finding |
||||||
Karimi E, et al. [44] Kim C, |
2017 2015 |
Iran South |
2014-2015 2014.6-2015.2 |
Prospective, consecutive Prospective, |
Surgical/patholog-ical finding Surgical/patholog-ical |
Clinical follow-up (48 h) Clinical follow-up (at least 3 |
Blinding Blinding |
NR Resident with |
NR Yes |
et al. [45] Lam SH, |
2014 |
Korea United |
NR |
consecutive Prospective, |
finding Surgical/patholog-ical |
months) Surgical/pathological finding, CT, or |
Blinding |
supervision NR |
Yes |
et al. [46] Mallin M, |
2015 |
States United |
2011.8-2013.11 |
consecutive Prospective, |
finding Surgical/patholog-ical |
clinical follow-up Pathological finding |
Blinding |
Resident with |
Yes |
et al. [48] Sivitz AB, |
2014 |
States United |
2009.8-2012.5 |
consecutive Prospective, |
finding Surgical/patholog-ical |
Clinical follow-up (1 week- |
Blinding |
supervision Attending EP |
NR |
et al. [50] |
States |
consecutive |
finding |
6 months) |
|||||
Topin F, et al. [51] Unluer EE, |
2016 2016 |
Republic of Djibouti Turkey |
2010.11-2011.1 2015.1-2015.3 |
Prospective, consecutive Prospective, |
Surgical/patholog-ical finding Surgical/patholog-ical |
Clinical follow-up (2 days-3 months) Clinical follow-up (1 week-1 |
Blinding Blinding |
Attending EP Attending EP |
Yes Yes |
et al. [52] Siu AYC, |
2007 |
Hong |
2004.1-2006.6 |
consecutive Prospective, |
finding Surgical/patholog-ical |
month) Clinical follow-up via chart review |
Blinding |
Attending EP |
NR |
et al. [49] Elikashvili |
2014 |
Kong United |
2011.5-2012.10 |
consecutive Prospective, |
finding Surgical/patholog-ical |
Clinical follow-up (3 weeks) |
Blinding |
Attending EP |
Yes |
I, et al. [38] Lin WC, |
2013 |
States Taiwan |
2004.12-2006.6 |
consecutive Retrospective, |
finding Surgical/patholog-ical |
Clinical follow-up |
NR |
Attending EP |
NR |
et al. [47] |
consecutive |
finding |
|||||||
Doniger SJ, |
2018 |
United |
2009.6-2010.6 |
Prospective, |
Surgical/patholog-ical |
Surgical/pathological finding, CT, or |
Blinding |
Resident/attending |
Yes |
et al. [37] |
States |
consecutive |
finding |
clinical follow-up (2 weeks~12 months) |
EP |
No., number; AA, acute appendicitis; EP, emergency physician; NR, not reported.
(accuracy) OR (sensitivity) OR (specificity) OR (receiver operating char- acteristic) OR (ROC curve)). The bibliographies of the identified articles were also screened to identify additional relevant studies. Two investi- gators screened the titles and abstracts for potential eligibility, and dis- agreements were resolved through discussion.
Study selection
We included studies that fulfilled the following criteria: (1) patients with right-lower quadrant pain, (2) EP-POCUS was performed as the index test, (3) use of surgical or Pathological findings as the reference standard for AA, (4) availability of sufficient information to reconstruct 2 x 2 contingency tables regarding sensitivity and specificity, and
(5) original research article as the publication type.
The exclusion criteria were: (1) case reports, case series, review ar- ticles, guidelines, consensus statements, letters, editorials, clinical trials,
and conference abstracts; (2) studies that did not pertain to the field of interest; (3) insufficient data to create 2 x 2 tables; (4) the POCUS was not performed by EPs; and (5) studies that only used the radiologists’ final report.
Data extraction and quality assessment
Two investigators independently extracted data regarding the pa- tient and study characteristics. The same investigators evaluated meth- odological quality using the Quality Assessment of Diagnostic Accuracy Studies-2 tool [13]. Inconsistencies between the reviewers were re- solved through discussion.
A standardized form was used to extract data regarding (1) patient characteristics (number of patients, AA proportion, clinical features, mean age, age range, and sex), (2) study characteristics (study location, Publication year, study design, reference standard, interval between EP-
Technical parameters and interpretative characteristics of the included studies
Author Technical parameters Interpretation
Vendor Model Frequency (MHz)
Cut-off value of appendiceal diameter
Other findings for acute appendicitis
Chen SC,
et al. [36] Fathi M,
et al. [39] Fox JC1,
et al. [40]
Toshiba SSA-340A 3.75 7 Noncompressible appendix, obstructing appendicolith,discontinuity of the echogenic submucosa, periappendiceal fluid collection.
Medison SonoAce x8 7.5 7 Noncompressible, aperistaltic appendix, target sign, sonographic McBurney sign,
peri-appendicular fat infiltration/increased vascular flow/fluid collection BK medical Hawk 2102 5.5 6 Noncompressible appendix
Fox JC2,
et al. [41]
BK medical/ sonosite
Hawk 2102/Titan
5.5, 5 6 Noncompressible, aperistaltic appendix
Gungor F,et al. [42]
ESAOTE MYLAB CLASS-C US
3-13 6 Noncompressible, aperistaltic appendix
Kang B,
et al. [43] Karimi E,
et al. [44] Kim C, et al.
[45]
NR NR NR 6 Appendicolith with an obstructive dilatation, periappendiceal fat infiltration
Honda HS2000 5-7.5 6 Noncompressible, appendicolitis, loss of bowel movements, peri-appendiceal fluid collection NR NR NR 6 Periappendiceal fat infiltration, obstructing appendicolith
Lam SH,
et al. [46] Mallin M,
et al. [48]
Zonare/sonosite Z.
One/M-Turbo
Mindray M7 portable Ultraound
5-10 6 Appendix wall thickness N 2 mm, periappendiceal fluid, visualization of the appendix tip, sonographic Mcburney’s sign
4-10 7 Noncompressible appendix
Sivitz AB, et al. [50]
Sonosite M turbo 8-10 or 3-5
6 Noncompressible, aperistaltic appendix
Topin F,
et al. [51] Unluer EE,
et al. [52]
Hitachi EUB-525 Odyssey
Mindray M7 portable Ultraound
5-10 6 Enlarged and noncompressible appendix
5-10 6 Noncompressible and aperistaltic appendix, periappendiceal anechoic fluid collection, appendix wall thickness N 2 mm, obstructing appendicolith, sonographic McBurney sign.
Siu AYC,
et al. [49] Elikashvili I, et al. [38]
Lin WC,
et al. [47] Doniger SJ,
et al. [37]
NR NR 7.5 6 Noncompressible, localised ileus or local collection of peritoneal fluid Sonosite M Turbo 5-10 6 Noncompressible tubular structure
Sonosite Titan 5 NR Noncompressible tubular structure, target sign on short axis scan, obstructing appendicolith, pus
collection in the lumen of the appendix Sonosite M Turbo 8-10 6 Noncompressible, aperistaltic appendix
NR, not reported.
POCUS and reference standard, blinding to reference standard, and op- erator characteristics), and (3) POCUS characteristics (probe, technical parameters, and interpretation). Study outcomes were also extracted
to create the 2 x 2 tables (i.e., true-positive, true-negative, false- positive, and false-negative results). The 2 x 2 tables were calculated using the Bayesian method (data were back-calculated based on
Fig. 2. Grouped bar charts showing the risk of bias (left) and applicability concerns (right) for the 17 included studies, using the Quality Assessment of Diagnostic Accuracy Studies-2 domains.
Fig. 3. Coupled forest plots for the pooled sensitivity and specificity of emergency physician-performed point-of-care ultrasonography for diagnosing acute appendicitis. Dots in squares represent sensitivity and specificity. Horizontal lines represent the 95% confidence interval (CI) for each included study. The combined estimate (“Summary”) is based on the random- effects model and is indicated using diamonds. Corresponding heterogeneities (I2) with 95% CIs are provided in the bottom right corners: I2 = 100%x (Q – df)/Q, where Q is Cochran’s heterogeneity statistic and df is the degrees of freedom.
prevalence and sample size) if only sensitivity and specificity were pre- sented for an eligible study. If two or more EPs independently assessed the diagnostic accuracy, the result with the highest accuracy was extracted.
Data synthesis and analysis
The primary outcome of this meta-analysis was the diagnostic per- formance of EP-POCUS for AA in all included studies. The secondary out- comes were the diagnostic performance of EP-POCUS for pediatric AA and the relative performances of EP-POCUS and RADUS for diagnosing AA.
Patient demographic characteristics and extracted covariates were summarized using standard descriptive statistics. Continuous variables were expressed as means and 95% confidence intervals (CIs), while cat- egorical variables were expressed as frequencies or percentages, unless stated otherwise.
A bivariate random-effects model was used to analyze and pool the diagnostic performance measurements (sensitivity and specificity) across studies. Summary estimates of the diagnostic performance were created by plotting the estimates of the observed sensitivities and specificities for each test in the forest plots and hierarchical sum- mary receiver operating characteristic (HSROC) curves, which were de- rived from the individual study results [14-16]. These results were
plotted using HSROC curves with 95% confidence and prediction regions.
Heterogeneity was evaluated using Cochran’s Q test (p b 0.05 indi- cates the presence of heterogeneity) and the I2 test (0-40%: heterogene- ity might not be present, 30-60%: moderate heterogeneity, 50-90%: substantial heterogeneity, and 75%-100%: considerable heterogeneity) [17]. Spearman’s correlation coefficient for the sensitivity and false- positive rate was calculated to identify any threshold effect, with a coef- ficient of N0.6 indicating a considerable threshold effect [18]. We omit- ted Deeks’ funnel plots [19] of the individual studies to check for publication bias according to the PRISMA-DTA guidelines.
Meta-regression analyses using several covariates were performed to explore the Potential causes of heterogeneity: (1) study design (pro- spective vs. retrospective), (2) study location (United States vs. other countries), (3) total patients (>=150 vs. b150), (4) proportion of AA (>=40% vs. b40%), (5) proportion of male patients (>=50% vs. b50%),
(6) mean age (>=30 years vs. b30 years), (7) cut-off appendix diameter for diagnosing AA (7 mm vs. 6 mm), (8) initial US operator (attending EP vs. resident), (9) separate US training for diagnosing AA (yes vs. not reported), and (10) blinding (blinding vs. not reported).
All statistical analyses were performed by one author (S.J.Y.) who has 3 years of experience performing systematic reviews and meta- analyses. The statistical analyses were performed using the “midas” and “metandi” modules in Stata software (version 10.0; StataCorp LP, College Station, TX) and the “mada” package in R software (version
Fig. 4. Hierarchical summary receiver operating characteristic (HSROC) curve for using emergency physician-performed point-of-care ultrasonography to diagnose acute appendicitis. The summary point (red box) indicates that the summary sensitivity was
0.84 (95% CI: 0.72-0.92) and the summary specificity was 0.91 (95% CI: 0.85-0.95). The 95% confidence region represents the 95% CIs of summary sensitivity and specificity, and the 95% prediction region represents the 95% CIs of sensitivity and specificity for each included study. The study estimates indicate the sensitivity and specificity estimated using the data from each study. The size of the marker is scaled according to the total number of patients in each study.
3.4.1; R Foundation for Statistical Computing, Vienna, Austria). Results were considered statistically significant at a p-value of b0.05.
Results
Literature search
Fig. 1 shows the study selection process. The initial search identified 385 reports, although 36 duplicates were removed before reviewing the titles and abstracts of 349 reports. A total of 316 studies were excluded because they were case reports, letters, editorials, or conference ab- stracts (n = 177); review articles, guidelines, or consensus statements (n = 55); or were not related to the field of interest (n = 84). After reviewing the full texts of 33 eligible reports, 16 reports were subse- quently excluded because of insufficient data to construct the 2 x 2 table (n = 1) [20], the POCUS was not performed by EPs (n = 13) [21-33], and the study used the radiologist’s final report (n = 2) [34, 35]. Thus, 17 studies [36-52] were included that evaluated the diagnos- tic performance of EP-POCUS in 2385 patients.
Patient and study characteristics
The patient characteristics are summarized in Table 1. The numbers of patients ranged from 40 to 284. The mean age range was 6-37.1 years, and the mean proportions of male patients were 26-61.1%.
The study characteristics are summarized in Table 2. Fourteen stud- ies used a prospective design [36-39, 41, 42, 44-46, 48-52] and 3 studies
used a retrospective design [40, 43, 47]. All studies [36-52] were single- center studies, included consecutive patients, and used the surgical/ pathological findings as the reference standard for AA. The appendix di- ameter cut-off values for diagnosing AA were 7 mm in 3 studies [36, 39, 48] and 6 mm in 13 studies [37, 38, 40-46, 49-52]. The technical and in- terpretative characteristics are summarized in Table 3.
Quality assessment
Fig. 2 shows the risk of bias and applicability concerns for the 17 in- cluded studies. No studies were considered seriously flawed based on the Quality Assessment of Diagnostic Accuracy Studies-2 tool, and all studies satisfied >=4 of the 7 items. All studies [36-52] were considered to have a low risk of bias regarding patient selection and the reference standard. The risk of bias regarding the index test domain was consid- ered high in 16 studies [36-46, 48-52] because a specific appendix diam- eter cut-off value was used to diagnose AA, while the risk of bias was unclear for one study [47] because the blinding from the reference stan- dard was not explicitly described. The risk of the flow and timing do- main was considered high in all studies [36-52] because the assessment of the reference standard was not the same (e.g., surgical/ pathological specimens, clinical follow-up, or CT). All studies had lim- ited concerns regarding the applicability to our research question in the patient selection and reference standard domains. However, there were substantial concerns regarding the index test domain for 16 stud- ies [36-46, 48-52] because the diagnostic performance might vary ac- cording to the appendix diameter cut-off value for diagnosing AA.
Overall diagnostic performance of EP-POCUS
The 17 studies [36-52] had sensitivity values that ranged from 0.39 to 1.00 and specificity values that ranged from 0.67 to 1.00. The pooled sensitivity and specificity values were 0.84 (95% CI: 0.72-0.92) and 0.91 (95% CI: 0.85-0.95), respectively. The Q test revealed significant hetero- geneity (Q = 100.575, p b 0.001), with considerable heterogeneity de- tected for sensitivity (I2 = 93.50%) and specificity (I2 = 89.37%). A threshold effect was observed in the coupled forest plot of sensitivity and specificity (Fig. 3) and in the correlation between sensitivity and the false-positive rate (-0.041, 95% CI: -0.512, 0.448). The area under the HSROC curve was 0.95 (95% CI: 0.92-0.96) (Fig. 4).
Diagnostic performance of EP-POCUS for pediatric AA
We identified 7 studies [37, 38, 41, 43, 45, 47, 50] that analyzed the diagnostic performance of EP-POCUS for pediatric AA. Six studies [37, 38, 43, 45, 47, 50] only included pediatric patients, and 1 study [41] in- cluded both adult and pediatric patients but separately described the di- agnostic performance for pediatric AA. The pooled sensitivity and specificity values were 0.95 (95% CI: 0.75-0.99) and 0.95 (95% CI: 0.85-0.98), respectively. The Q test revealed significant heterogeneity (Q = 40.369, p b 0.001), and the Higgins I2 statistic revealed consider- able heterogeneity for sensitivity (I2 = 94.91%) and specificity (I2 = 94.51%). A threshold effect was observed in the coupled forest plot (Fig. 5) and in the correlation between sensitivity and the false- positive rate (-0.081, 95% CI: -0.786, 0.716). The area under the
HSROC curve was 0.98 (95% CI: 0.97-0.99) (Fig. 6).
Direct comparison of the diagnostic performances of EP-POCUS and RADUS
Five studies [37, 38, 42, 44, 52] directly compared the performances of EP-POCUS and RADUS for diagnosing AA. Although one study [41] de- scribed the numbers of true-positive, false-positive, false-negative, and true-negative cases, that study was excluded because it combined the cases with RADUS or CT as the reference standard and it did not focus on comparing EP-POCUS to RADUS. In the remaining studies, the pooled
Fig. 5. Coupled forest plots of pooled sensitivity and specificity of emergency physician-performed point-of-care ultrasonography for diagnosing pediatric acute appendicitis. Dots in squares represent sensitivity and specificity. Horizontal lines represent the 95% confidence interval (CI) for each included study. The combined estimate (“Summary”) is based on the random-effects model and is indicated using diamonds. Corresponding heterogeneities (I2) with 95% CIs are provided in the bottom right corners.
sensitivity and specificity of EP-POCUS were 0.81 (95% CI: 0.61-0.90) and 0.89 (95% CI: 0.77-0.95), respectively. For RADUS, the pooled sensi- tivity and specificity were 0.74 (95% CI: 0.65-0.81) and 0.97 (95% CI: 0.93-0.98), respectively. The results were not significantly different for sensitivity (p = 0.18) or specificity (p = 0.85), and the diagnostic performances were also not significantly different in a joint model (p
= 0.24) (Table 4).
Meta-regression analyses
The results of the meta-regression analyses are summarized in Table 5. The only significant source of heterogeneity in terms of sensitiv- ity was study location (p = 0.03), with higher sensitivity reported in countries other than the United States. The significant sources of hetero- geneity in terms of specificity were the proportion of AA (p b 0.01, higher specificity at proportions of b40% vs. >=40%) and mean age (p = 0.02, higher specificity at ages of b30 years vs. >=30 years).
Discussion
The present meta-analysis revealed that EP-POCUS was excellent for diagnosing AA (sensitivity: 84%, specificity: 91%) and even better for di- agnosing pediatric AA (sensitivity: 95%, specificity: 95%). In addition,
similar diagnostic performances were observed for EP-POCUS (sensitiv- ity: 81%, specificity: 89%) and RADUS (sensitivity: 74%, specificity: 97%).
Strengths of our analyses
The present study has three useful attributes. The first attribute is that it focused on EP-POCUS, and revealed that it can be used clinically as the initial method for diagnosing AA. This is important because POCUS is a non-invasive, real-time, and repeatable diagnostic tool that is faster than CT and can be rapidly performed and interpreted at the bedside. These characteristics can improve decision-making, shorten ED stays, improve prognosis, lower costs, and reduce the cumulative ra- diation dose. The second attribute is the inclusion of a subgroup analysis for pediatric AA, as these patients are more vulnerable to radiation haz- ards than adults [53], and approximately 60% of American children who are diagnosed with acute appendicitis undergo preoperative CT [54, 55]. Although the small number of studies (n = 7) that included pediatric patients may limit the interpretation of our findings, the diagnostic per- formance in pediatric studies was higher than that in studies of adults. Therefore, EP-POCUS should be used for children with suspected appen- dicitis, rather than CT, to limit the radiation-related cancer risk. The third attribute is the fact that EP-POCUS and RADUS did not have signif- icantly different diagnostic performances, although EP-POCUS had bet- ter sensitivity than RADUS. It is possible that the EP’s awareness of the
Fig. 6. Hierarchical summary receiver operating characteristic (HSROC) curve for using emergency physician-performed point-of-care ultrasonography to diagnose pediatric acute appendicitis. The summary point (red box) indicates that the summary sensitivity was 0.95 (95% CI: 0.75-0.99) and the summary specificity was 0.95 (95% CI: 0.85-0.98). The 95% confidence region represents the 95% CIs of summary sensitivity and specificity, and the 95% prediction region represents the 95% CI of sensitivity and specificity for each included study.
specific point of tenderness provided better sensitivity, although the sensitivity and specificity values for RADUS were lower than in previous meta-analyses [56, 57]. This may be related to the fact that radiology residents typically perform US for diagnosing AA during night and weekend shifts [38, 42].
Differences from previous meta-analyses related to POCUS
Two previous meta-analyses have evaluated the diagnostic perfor- mance of POCUS [10, 11], although those studies had several limitations. First, they did not use a hierarchical model (e.g., the bivariate model and the HSROC model), which are recommended statistical tools for the meta-analysis of studies regarding diagnostic accuracy [14, 15]. Second, they did not perform a thorough analysis of the potential sources of het- erogeneity, as they did not distinguish between sensitivity and specific- ity for the covariates’ effects, which precluded any recommendations regarding methods to increase the diagnostic performance of POCUS for AA. Third, they only included a limited number of studies, which is likely related to missing or restricted search terms. For example,
Benabbas et al. [10] omitted 3 studies [41, 43, 47] and one study [40] was mis-quoted because it did not report sensitivity and specificity for pediatric AA, while Matthew et al. [11] omitted 2 studies [40, 43]. Fourth, Matthew et al. [11] evaluated the diagnostic performance with- out distinguishing between EPs and other clinicians, and did not clearly describe the analyses that were used to evaluate the diagnostic perfor- mance of EP-POCUS. Fifth, the meta-analysis by Matthew et al. [11] in- cluded Inaccurate data regarding the 2 x 2 tables for the studies by Lam et al. [46] and Lin et al. [47] Finally, the technique of 2 x 2 table evaluation when two or more reviewers independently assessed their diagnostic accuracy remains unclear. In our meta-analysis, the result with the highest accuracy was extracted.
Sources of heterogeneity and suggestion for EP-POCUS
Our Meta-regression analysis revealed that study location, AA pro- portion, and mean age were sources of heterogeneity. In particular, the pooled specificity was higher in studies with a low proportion of AA than in studies with a high proportion of AA. The pooled sensitivity and specificity were also higher in studies that used an appendix diam- eter cut-off of 7 mm than in studies with a cut-off of 6 mm, although the difference was not statistically significant. Similarly, the pooled sensitiv- ity and specificity were higher in studies with the attending EP as the initial operator, rather than a resident, although this difference was also not statistically significant. Thus, we recommend that the attending EP perform the initial US using an appendix diameter cut-off value of 7 mm to more accurately diagnose AA.
Weakness of our analyses
This meta-analysis only examined studies where the diagnostic per- formance of EP-POCUS was based on conclusive cases, as the eligible studies did not include cases with equivocal or inconclusive findings. Moreover, the studies using POCUS emphasized its diagnostic perfor- mance alone and did not compare it to other modalities. Thus, a com- prehensive study that includes equivocal and inconclusive cases and more advanced methodology (e.g., comparison to low-dose CT or mag- netic resonance imaging) may be needed to confirm the usefulness of EP-POCUS as an initial diagnostic tool in routine clinical practice.
Limitations
The present study has several limitations. The first limitation is the relatively small number of included studies. Nevertheless, we were able to draw several important conclusions regarding the diagnostic performance of EP-POCUS and related factors (diameter cut-off value and initial US operator), which we believe provides a useful overview because we used broad search terms and only included easily accessible studies (published in English and available in the PubMed and EMBASE databases). The second limitation is that all included studies revealed positive results, and that fact could be attributed to publication bias, which impossible to quantify. Although we omitted Deeks’ funnel plots according to the PRISMA-DTA guidelines, we observed a low prob- ability of publication bias (p = 0.62), which suggests that this factor did not undermine our results. The third limitation is the methodological differences between the included studies, and the extensive meta-
Diagnostic performance of emergency physician-performed point-of-care ultrasonography (EP-POCUS) and radiologist-performed ultrasonography (RADUS)
Sensitivity (95% CI) |
p-value |
Specificity (95% CI) |
p-value |
Area under the ROC curve (95% CI) |
PLR (95% CI) |
NLR (95% CI) |
p-value (joint model) |
|
EP-POCUS |
0.81 (0.61-0.90) |
0.18 |
0.89 (0.77-0.95) |
0.85 |
0.92 (0.89-0.94) |
7.0 (3.2-15.3) |
0.22 (0.12-0.42) |
0.24 |
RADUS |
0.74 (0.65-0.81) |
0.97 (0.93-0.98) |
0.94 (0.91-0.95) |
21.6 (10.4-44.7) |
0.27 (0.20-0.36) |
CI, confidence interval; ROC, receiver operating characteristic; PLR, positive likelihood ratio; NLR, negative likelihood ratio.
Meta-regression analyses for potential source of the heterogeneity
Covariate |
No. of studies |
Sensitivity (95% CI) |
p-value |
Specificity (95% CI) |
p-value |
Study design Prospective |
14 |
0.82 (0.70-0.94) |
0.37 |
0.91 (0.85-0.96) |
0.21 |
Retrospective Locale |
3 |
0.92 (0.79-1.00) |
0.03 |
0.94 (0.86-1.00) |
0.15 |
United States |
7 |
0.71 (0.50-0.91) |
0.92 (0.86-0.98) |
||
Country other than United States No. of patients >= 150 |
10 6 |
0.90 (0.82-0.98) 0.88 (0.75-1.00) |
0.99 |
0.91 (0.84-0.97) 0.94 (0.89-0.99) |
0.40 |
b 150 AA (%) >= 40% |
11 7 |
0.82 (0.69-0.95) 0.79 (0.61-0.97) |
0.19 |
0.89 (0.82-0.96) 0.86 (0.77-0.96) |
b0.01 |
b 40% Male (%) >= 50% |
10 11 |
0.87 (0.77-0.98) 0.87 (0.76-0.98) |
0.37 |
0.94 (0.89-0.98) 0.92 (0.87-0.98) |
0.89 |
b 50% Mean age (years) >= 30 |
3 6 |
0.74 (0.40-1.00) 0.82 (0.65-0.99) |
0.21 |
0.88 (0.74-1.00) 0.87 (0.75-0.98) |
0.02 |
b 30 Cut-off 7-mm |
8 3 |
0.90 (0.80-0.99) 0.77 (0.49-1.00) |
0.60 |
0.94 (0.89-0.99) 0.88 (0.74-1.00) |
0.18 |
6-mm Initial US performer Attending EP |
12 8 |
0.84 (0.72-0.95) 0.93 (0.76-1.00) |
0.77 |
0.93 (0.88-0.97) 0.99 (0.87-1.00) |
0.16 |
Resident Separate US training for AA Yes |
2 11 |
0.89 (0.78-1.00) 0.84 (0.71-0.96) |
0.56 |
0.90 (0.82-0.98) 0.91 (0.86-0.97) |
0.13 |
Not reported Blinding Blinding |
6 15 |
0.85 (0.69-1.00) 0.85 (0.75-0.95) |
0.63 |
0.91 (0.83-0.99) 0.92 (0.88-0.96) |
0.66 |
Not reported |
2 |
0.80 (0.46-1.00) |
0.84 (0.63-1.00) |
Boldface type indicates statistical significance (p b 0.05). CI, confidence interval; No., number; AA, acute appendicitis; US, ultrasound; EP, emergency physician.
regression analysis revealed that these variables were also significant sources of heterogeneity. This methodological diversity might affect the pooled estimates, especially as the POCUS technical parameters were not assessed in the meta-regression analysis because not all stud- ies reported the values for gain, dynamic range, and mechanical index. Further studies with larger sample sizes are needed to determine the optimal parameters for POCUS.
Conclusion
In conclusion, the present meta-analysis revealed that EP-POCUS had excellent performance for diagnosing AA, which was similar to that of RADUS, and even better diagnostic performance for pediatric
AA. We recommend that the attending EP perform the initial POCUS using a diameter cut-off value of 7 mm to more accurately diagnose AA.
Declaration of interest“>Declaration of interest
There is no potential conflict of interest related to this article.
This research did not receive any specific grant from funding agen- cies in the public, commercial, or not-for-profit sectors.
Grant or other financial support.
None declared.
Acknowledgments
None.
References
- Callahan MJ, Rodriguez DP, Taylor GA. CT of appendicitis in children. Radiology 2002;224(2):325-32.
- Hale DA, Molloy M, Pearl RH, Schutt DC, Jaques DP. Appendectomy: a contemporary appraisal. Ann Surg 1997;225(3):252-61.
- Birnbaum BA, Wilson SR. Appendicitis at the millennium. Radiology 2000;215(2): 337-48.
- Irvin TT. Abdominal pain: a surgical audit of 1190 emergency admissions. Br J Surg 1989;76(11):1121-5.
- Yun SJ, Ryu CW, Choi NY, Kim HC, Oh JY, Yang DM. Comparison of low- and standard-dose CT for the diagnosis of acute appendicitis: a meta-analysis. AJR Am J Roentgenol 2017;208(6):W198-207.
- Mostbeck G, Adam EJ, Nielsen MB, Claudon M, Clevert D, Nicolau C, et al. How to di- agnose acute appendicitis: ultrasound first. Insights into imaging 2016;7(2):255-63.
- Parker L, Nazarian LN, Gingold EL, Palit CD, Hoey CL, Frangos AJ. Cost and radiation savings of partial substitution of ultrasound for CT in appendicitis evaluation: a na- tional projection. AJR Am J Roentgenol 2014;202(1):124-35.
- Brenner DJ, Hall EJ. Computed tomography-an increasing source of radiation expo- sure. N Engl J Med 2007;357(22):2277-84.
- Hall EJ, Brenner DJ. Cancer risks from diagnostic radiology: the impact of new epide- miological data. Br J Radiol 2012;85(1020):e1316-7.
- Benabbas R, Hanna M, Shah J, Sinert R. Diagnostic accuracy of history, physical ex- amination, laboratory tests, and point-of-care ultrasound for pediatric acute appen- dicitis in the emergency department: a systematic review and meta-analysis. Academic emergency medicine : official journal of the Society for Academic Emer- gency Medicine 2017;24(5):523-51.
- Matthew Fields J, Davis J, Alsup C, Bates A, Au A, Adhikari S, et al. Accuracy of point- of-care ultrasonography for diagnosing acute appendicitis: a systematic review and meta-analysis. Academic emergency medicine : official journal of the Society for Ac- ademic Emergency Medicine 2017;24(9):1124-36.
- McInnes MDF, Moher D, Thombs BD, McGrath TA, Bossuyt PM. The P-DTAG, et al. preferred reporting items for a systematic review and meta-analysis of diagnostic test accuracy studies: the PRISMA-DTA statement. JAMA 2018;319(4):388-96.
- Whiting PF, Rutjes AW, Westwood ME, Mallett S, Deeks JJ, Reitsma JB, et al. QUADAS-2: a revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 2011;155(8):529-36.
- Kim KW, Lee J, Choi SH, Huh J, Park SH. Systematic review and meta-analysis of stud- ies evaluating Diagnostic test accuracy: a practical review for clinical researchers- part I. General guidance and tips. Korean J Radiol 2015;16(6):1175-87.
- Lee J, Kim KW, Choi SH, Huh J, Park SH. Systematic review and meta-analysis of stud- ies evaluating diagnostic test accuracy: a practical review for clinical researchers- part II. Statistical methods of meta-analysis. Korean J Radiol 2015;16(6):1188-96.
- Suh CH, Park SH. Successful publication of systematic review and meta-analysis of studies evaluating diagnostic test accuracy. Korean J Radiol 2016;17(1):5-6.
- Higgins J, Green S. Cochrane handbook for systematic reviews of interventions. Ver- sion 5.1.0. The Cochrane Collaboration http://handbook.cochrane.org/chapter_9/9_ 5_2_identifying_and_measuring_heterogeneity.htm.
- Deville WL, Buntinx F, Bouter LM, Montori VM, de Vet HC, van der Windt DA, et al. Conducting systematic reviews of diagnostic studies: didactic guidelines. BMC Med Res Methodol 2002;2:9.
- Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol 2005;58(9):882-93.
- Jang T, Chauhan V, Cundiff C, Kaji AH. Assessment of emergency physician- performed ultrasound in evaluating nonspecific abdominal pain. American Journal of Emergency Medicine 2014;32(5):457-60.
- Abo A, Shannon M, Taylor G, Bachur R. The influence of body mass index on the ac- curacy of ultrasound and computed tomography in diagnosing appendicitis in chil- dren. Pediatr Emerg Care 2011;27(8):731-6.
- Carney E, Kempf J, Decarvalho V, Yudd A, Nosher J. Preliminary interpretations of after-hours CT and sonography by radiology residents versus final interpretations by body imaging radiologists at a level I trauma center. Am J Roentgenol 2003;181 (2):367-73.
- Carrico CW, Fenton LZ, Taylor GA, Difiore JW, Soprano JV. Impact of sonography on the diagnosis and treatment of acute lower abdominal pain in children and young adults. AJR Am J Roentgenol 1999;172(2):513-6.
- Imler D, Keller C, Sivasankar S, Wang NE, Vasanawala S, Bruzoni M, et al. Magnetic resonance imaging versus ultrasound as the initial imaging modality for pediatric and young adult patients with suspected appendicitis. Acad Emerg Med 2017;24 (5):569-77.
- Leung YK, Chan CP, Graham CA, Rainer TH. Acute appendicitis in adults: diagnostic accuracy of Emergency doctors in a university hospital in Hong Kong. Emergency medicine Australasia : EMA 2017;29(1):48-55.
- Menoch M, Simon HK, Hirsh D, Shim Y, Baxter AL, Clifton M, et al. Imaging for suspected appendicitis: variation between academic and private Practice models. Pediatr Emerg Care 2017;33(3):147-51.
- Nasiri S, Mohebbi F, Sodagari N, Hedayat A. diagnostic values of ultrasound and the Modified Alvarado Scoring System in acute appendicitis. International Journal of Emergency Medicine 2012;5(1).
- Poonai N, Gregory J, Thompson G, Lim R, Van Osch S, Andrusiak T, et al. Is pelvic ul- trasound associated with an increased time to appendectomy in pediatric appendi- citis? Journal of Emergency Medicine 2014;47(1):51-8.
- Ramarajan N, Krishnamoorthi R, Gharahbaghian L, Pirrotta E, Barth RA, Wang NE. Clinical correlation needed: what do emergency physicians do after an equivocal ul- trasound for pediatric acute appendicitis? Journal of clinical ultrasound : JCU 2014; 42(7):385-94.
- Ross M, Selby S, Poonai N, Liu H, Minoosepehr S, Boag G, et al. The effect of a full bladder on proportions of diagnostic Ultrasound studies in children with suspected appendicitis. Canadian Journal of Emergency Medicine 2016;18(6):414-9.
- Talan DA, Krishnadasan A, Amii R, Pathmarajah K, Morim A, Moran GJ, et al. Antibiotics-First Versus Surgery for Appendicitis: A US Pilot Randomized Controlled Trial Allowing Outpatient Antibiotic Management. Annals of Emergency Medicine 2017;70(1):1-11.e9.
- Uzunosmanoglu H, Cevik Y, Corbacioglu SK, Akinci E, Bulus H, Agladioglu K. Diagnos- tic value of appendicular Doppler ultrasonography in acute appendicitis. Ulus Travma Acil Cerrahi Derg 2017;23(3):188-92.
- van Atta AJ, Baskin HJ, Maves CK, Rollins MD, Bolte RG, Mundorff MB, et al. Implementing an ultrasound-based protocol for diagnosingappendicitis while main- taining diagnostic accuracy. Pediatr Radiol 2015;45(5):678-85.
- Bal A, Anil M, Narturk M, Ozdemir T, Arikan A, Koyluoglu G, et al. Importance of clin- ical decision making by experienced pediatric surgeons when children are suspected of having acute appendicitis: the reality in a high-volume pediatric emergency de- partment. Pediatr Emerg Care 2017;33(9):e38-e42.
- Mittal MK, Dayan PS, Macias CG, Bachur RG, Bennett J, Dudley NC, et al. Performance of ultrasound in the diagnosis of appendicitis in children in a multicenter cohort.
Academic emergency medicine : official journal of the Society for Academic Emer- gency Medicine 2013;20(7):697-702.
Chen SC, Wang HP, Hsu HY, Huang PM, Lin FY. Accuracy of ED sonography in the di- agnosis of acute appendicitis. American Journal of Emergency Medicine 2000;18(4): 449-52.
- Doniger SJ, Kornblith A. Point-of-care ultrasound integrated into a staged diagnostic algorithm for Pediatric appendicitis. Pediatr Emerg Care 2018;34(2):109-15.
- Elikashvili I, Tay ET, Tsung JW. The effect of point-of-care ultrasonography on emer- gency department length of stay and computed tomography utilization in children with suspected appendicitis. Academic emergency medicine : official journal of the Society for Academic Emergency Medicine 2014;21(2):163-70.
- Fathi M, Hasani SA, Zare MA, Daadpey M, Hojati Firoozabadi N, Lotfi D. Diagnostic ac- curacy of emergency physician performed graded Compression ultrasound study in acute appendicitis: a prospective study. J Ultrasound 2015;18(1):57-62.
- Fox JC, Hunt MJ, Zlidenny AM, Oshita MH, Barajas G, Langdorf MI. Retrospective analysis of emergency department ultrasound for acute appendicitis. The California journal of emergency medicine 2007;8(2):41-5.
- Fox JC, Solley M, Anderson CL, Zlidenny A, Lahham S, Maasumi K. Prospective eval- uation of emergency physician performed bedside ultrasound to detect acute ap- pendicitis. Eur J Emerg Med 2008;15(2):80-5.
- Gungor F, Kilic T, Akyol KC, Ayaz G, Cakir UC, Akcimen M, et al. Diagnostic value and effect of bedside ultrasound in acute appendicitis in the emergency department. Acad Emerg Med 2017;24(5):578-86.
- Kang B, Kim C, Park JB, Ha YR. The use of clinician-performed ultrasonography to de- termine the treatment method for suspected paediatric appendicitis. Hong Kong Journal of Emergency Medicine 2015;22(1):31-40.
- Karimi E, Aminianfar M, Zarafshani K, Safaie A. The accuracy of emergency physi- cians in ultrasonographic screening of acute appendicitis; a Cross sectional study. Emergency 2017;5(1):117-23.
- Kim C, Kang BS, Choi HJ, Lim TH, Oh J, Chee Y. Clinical application of real-time tele- ultrasonography in diagnosing pediatric acute appendicitis in the ED. American Jour- nal of Emergency Medicine 2015;33(10):1354-9.
- Lam SH, Grippo A, Kerwin C, Konicki PJ, Goodwine D, Lambert MJ. Bedside ultraso- nography as an adjunct to routine evaluation of acute appendicitis in the emergency department. The western journal of emergency medicine 2014;15(7):808-15.
- Lin WC, Lin CH. Re-appraising the role of sonography in pediatric acute abdominal pain. Iran J Pediatr 2013;23(2):177-82.
- Mallin M, Craven P, Ockerse P, Steenblik J, Forbes B, Boehm K, et al. Diagnosis of ap- pendicitis by bedside ultrasound in the ED. Am J Emerg Med 2015;33(3):430-2.
- Siu AYC, Chung CH. The use of ultrasonography to assess patients with right lower quadrant pain in the emergency department. Hong Kong Journal of Emergency Medicine 2007;14(2):70-3.
- Sivitz AB, Cohen SG, Tejani C. Evaluation of acute appendicitis by pediatric emer- gency physician sonography. Ann Emerg Med 2014;64(4):358-64.e4.
- Topin F, Thierry AL, Catrevaux O, Barnoux T, Menguy P, Bertani A, et al. Diagnostic accuracy of emergency physician-performed ultrasound for acute appendicitis in a Remote location. J Emerg Med 2016;50(6):859-67.
- Unluer EE, Urnal R, Eser U, Bilgin S, Haciyanli M, Oyar O, et al. Application of scoring systems with point-of-care ultrasonography for bedside diagnosis of appendicitis. World J Emerg Med 2016;7(2):124-9.
- Donnelly LF, Emery KH, Brody AS, Laor T, Gylys-Morin VM, Anton CG, et al. Minimiz- ing radiation dose for pediatric body applications of single-detector helical CT: strat- egies at a large Children’s hospital. AJR Am J Roentgenol 2001;176(2):303-6.
- Martin AE, Vollman D, Adler B, Caniano DA. CT scans may not reduce the Negative appendectomy rate in children. J Pediatr Surg 2004;39(6):886-90; discussion -90.
- York D, Smith A, Phillips JD, von Allmen D. The influence of advanced radiographic imaging on the treatment of pediatric appendicitis. J Pediatr Surg 2005;40(12): 1908-11.
- Giljaca V, Nadarevic T, Poropat G, Nadarevic VS, Stimac D. Diagnostic accuracy of ab- dominal ultrasound for diagnosis of acute appendicitis: systematic review and meta- analysis. World J Surg 2017;41(3):693-700.
- Yu SH, Kim CB, Park JW, Kim MS, Radosevich DM. Ultrasonography in the diagnosis of appendicitis: evaluation by meta-analysis. Korean J Radiol 2005;6(4):267-77.