Article, Radiology

Retrospective comparison of the Low Risk Ankle Rules and the Ottawa Ankle Rules in a pediatric population

a b s t r a c t

Background: A recent multicenter prospective Canadian study presented prospective evidence supporting the Low Risk Ankle Rules (LRAR) as a means of reducing the number of ankle radiographs ordered for children pre- senting with an ankle injury while maintaining nearly 100% sensitivity. This is in contrast to a previous prospec- tive study which showed that this rule yielded only 87% sensitivity.

Objective: It is important to further investigate the LRAR and compare them with the already validated Ottawa Ankle Rules (OAR) to potentially curb Healthcare costs and decrease unnecessary radiation exposure without compromising diagnostic accuracy.

Methods: We conducted a retrospective chart review of 980 qualifying patients ages 12 months to 18 years pre- senting with ankle injury to a commonly staffed 310 bed children’s hospital and auxiliary site pediatric emergen- cy department.

Results: There were 28 high-risk fractures identified. The Ottawa Ankle Rules had a sensitivity of 100% (95% CI 87.7-100), specificity of 33.1% (95% CI 30.1-36.2), and would have reduced the number of ankle radiographs or- dered by 32.1%. The Low Risk Ankle Rules had a sensitivity of 85.7% (95% CI 85.7-96), specificity of 64.9% (95% CI 61.8-68), and would have reduced the number of ankle radiographs ordered by 63.1%. The latter rule missed 4 high-risk fractures.

Conclusion: The Low Risk Ankle Rules may not be sensitive enough for use in Pediatric Emergency Departments, while the Ottawa Ankle Rules again demonstrated 100% sensitivity. Further research on ways to implement the Ottawa Ankle Rules and maximize its ability to decrease wait times, healthcare costs, and improve patient satis- faction are needed.

(C) 2017

Introduction

With healthcare costs in the United States continuing to rise and emergency department becoming overcrowded [1,2] it becomes crucial to find ways to cut costs without compromising healthcare quality. Pe- diatric Emergency Departments (PEDs) are an important setting to cut costs while maintaining quality.

Roughly 85-100% of children presenting to United States PEDs with a history of ankle injury receive an ankle radiograph [3]. While the Otta- wa Ankle Rule (OAR) has been validated for use in the pediatric popula- tion [4], a less well-studied rule, the Low Risk Ankle Rule (LRAR), has also shown promising results. A large multicenter prospective study conducted in Canada and published in 2013 suggested that the LRAR could reduce the number of ankle x-rays performed in PEDs by up to 60%, while maintaining nearly 100% sensitivity [5]. While such results

* Corresponding author.

E-mail address: [email protected] (A.L. Ellenbogen).

are promising, further validation is needed prior to implementation. This is particularly true given that a smaller 272 subject prospective study performed several years earlier showed only 87% sensitivity for the LRAR, missing 6 clinically significant fractures, versus 100% sensitiv- ity for the OAR [6].

Thus, our aim was to further investigate the LRAR and to compare this clinical decision rule to the well-validated OAR in a pediatric population. To the best of our knowledge, there has never been a retrospective study comparing the two clinical decision rules, which could provide an- other perspective and eliminate possible expectation bias introduced by non-blinded clinicians in prior discordant prospective studies.

Methods

Definitions

The Low Risk Ankle Rules state that an ankle radiograph is not re- quired if an ankle examination reveals tenderness and swelling isolated to the Distal fibula and/or adjacent lateral ligaments distal to the tibial

http://dx.doi.org/10.1016/j.ajem.2017.03.058

0735-6757/(C) 2017

A.L. Ellenbogen et al. / American Journal of Emergency Medicine 35 (2017) 12621265 1263

anterior joint line [7]. The Ottawa Ankle Rules state that an ankle radio- graph is required if examination reveals pain in the malleolar zone and one of the following: 1) inability to bear weight immediately after the injury and in the Emergency Department for four steps or 2) bone ten- derness along the distal 6 cm of the posterior edge of the tibia or tip of the medial malleolus 3) bone tenderness along the distal 6 cm of the pos- terior edge of the fibula or tip of the Lateral malleolus [8]. A high-risk ankle injury is defined as any fracture of the foot, distal tibia, and fibula proximal to the distal physis; tibiofibular syndesmosis injury, or ankle dislocation, with increased risk of requiring surgical intervention [5].

Study design and data collection

We conducted an institutional review board (IRB) approved retro- spective chart review at a 310 bed children’s hospital and one auxiliary site. We used the radiology search engine Montage (Philadelphia, Penn- sylvania) to identify all ankle x-rays performed on patients between 12 months and 18 years of age at either PED between 1/1/2011 and 4/ 30/2014. Relevant data including patient gender, age, presence and type of fracture were accessed in January 2015 and manually entered into an Excel spreadsheet. Each ankle radiograph series had already been interpreted by an attending fellowship-trained pediatric radiolo- gist. If the radiographic report impression was indeterminate for the presence of a fracture, the subject was excluded from the study. Other- wise, the radiologic interpretation as to whether a fracture was present and if so, what type of fracture was entered into the study data spread- sheet. The accession numbers obtained from Montage were entered into our picture archiving and communication system (PACS, Synapse, Fujifilm) in order to obtain the patient’s medical record number (MRN). The MRN was then used to obtain the patient’s electronic medical re- cord (Cerner, Kansas City) note in order to determine if the patient met criteria for ankle x-ray under each the OAR and the LRAR criteria. If there was inadequate documentation in the patient’s EMR note to make this

determination for either rule, the subject was excluded.

Additional exclusion criteria included Inability to walk prior to ankle injury, physical deformity on exam, previous diagnosis of fracture, and underlying disease that could influence decision for x-ray (these condi- tions included history of bony neoplasm, Sickle cell disease, osteogene- sis imperfecta and osteopetrosis) (Table 1).

Statistical methods

The baseline characteristics of the study subjects were summarized in terms of counts and percentages, or means (standard deviation), and ranges. All data were analyzed using Stata IC/13.1 (College Station, TX). The same set of analyses were performed for both of the clinical de- cision rules (Ottawa Ankle Rules and the Low Risk Ankle Rules) and three different age groups of patients: 1) all patients (1-18 y), 2) pre- school-adolescents (3-16 y), and 3) toddlers (1-2 y). The age range for the second group was chosen because it is similar to what was

used in other published studies [5,8].

Table 1

Eligibility status of all subjects who received an ankle radiograph in the emergency department.

Sensitivity was calculated as the percentage of patients with a radio- graphically-confirmed high-risk fracture that would have been correctly identified by applying the clinical decision rule in the PED. Likewise, specificity was calculated as the percentage of patients without a radio- graphically confirmed high-risk fracture that would have been correctly identified by applying the clinical decision rule. MedCalc’s online diag- nostic test evaluation calculator was used to generate the estimates and 95% confidence intervals for sensitivity and specificity [9].

The potential reduction in ankle radiographs was expressed as a per- cent reduction and was calculated as follows: [Total number of radio- graphs actually performed – number of radiographs that would have been ordered solely based on the clinical decision rule] / total number of radiographs actually performed) * 100.

The number of radiographically confirmed high-risk fractures that

would have been missed by each of the clinical decision rules was re- corded along with the total number of radiographically confirmed high-risk fractures that actually occurred in each group.

Results

A total of 980 subjects with an average age of 11.7 years (range 1- 18) met the inclusion criteria (Table 2). A mere 21/980 (2%) reviewed charts mentioned the Ottawa Ankle Rules and 0/980 (0%) mentioned the Low Risk Ankle Rules as justification for obtaining an ankle x-ray. There were a total of 28 high-risk fractures within the study population. Systematically applying the OAR in the ER to these 980 patients would have identified all 28 high-risk ankle fractures, with 100% sensi- tivity and 33.1% specificity and reduced the number of ankle x-rays or- dered by 32.1%. Systematically applying the LRAR in the ER would have missed 4 high-risk ankle fractures, including a spiral fracture of the tibia and Salter Harris II, III, and IV fractures of the tibia. The LRAR had 85.7% sensitivity and 64.9% specificity. The LRAR would have decreased the

number of ankle x-rays ordered by 63.1% (Table 3).

Additional analyses performed after excluding subjects b 3 and N 16 years of age (in line with the age criteria used in the recent multi- center Canadian study investigating the LRAR [5] and many of the stud- ies on the OAR [6]) showed similar results as compared to the analysis that included all 980 subjects ages 1 through 18 years.

When the same analysis was performed on the very youngest group of children (1 to 2 years of age), both the OAR and LRAR had 100% sen- sitivity and 77.8% specificity. However, these are rather imprecise esti- mates (as evidenced by the wide confidence intervals) because they were based on a very small group of children (n = 29) with only 2 high-risk fractures.

Discussion

The sensitivity of the LRAR among children 1-18 years of age in our study was lower than that found in the 2013 study by Boutis et al. [5], but similar to those of the 2009 study by Gravel et al. [6]. As with the

Table 2

Characteristics of the study population (N = 980).

Characteristic

Age (y)a 11.7 (4.0)

Total number of age-eligible exams (>= 1 and <= 18 years) identified using Montage

Number of excluded cases

N = 1378

Sexb

Male 485 (49.5)

Female 495 (50.5)

Duration of symptomsb

X-ray report indeterminate for fracture 71

Exam data insufficient to determine if OAR criteria met 218

Already diagnosed with fracture, presenting for post-reduction 15

Underlying disease that could predispose to fracture (e.g. 18

osteogenesis imperfecta)

Obvious physical deformity on exam 6

Exam data insufficient to determine if LRAR criteria met 70

Total number of eligible exams N = 980

<= 24 h 720 (73.5)

N 24-<= 72 h 134 (13.6)

N 72 h 81 (8.3)

Unknown 45 (4.6)

High risk fracturesc 28

a Mean (SD).

b n(%).

c (n).

1264 A.L. Ellenbogen et al. / American Journal of Emergency Medicine 35 (2017) 12621265

Table 3

Performance of the Ottawa Ankle Rules compared to the Low Risk Ankle Rules.

Age group & rule

Sensitivity % (95% CI)

Specificity % (95% CI)

Potential reduction in radiographs

High-risk fractures (missed/total)

1-18 y (N = 980)

Ottawa Ankle Rules

100 (87.6, 100)

33.1 (30.1, 36.2)

32.1%

0/28

Low Risk Ankle Rules

85.7 (67.3, 96.0)

64.9 (61.8, 68.0)

63.4%

4/28

3 y-16 y (N = 857)

Ottawa Ankle Rules

100 (86.8, 100)

31.3 (28.2, 34.6)

30.3%

0/26

Low Risk Ankle Rules

84.6 (65.1, 95.6)

64.4 (61.0, 67.6)

62.9%

4/26

1-2 y (N = 29)

Ottawa Ankle Rules

100 (15.8, 100)

77.8 (57.7, 91.4)

72.4%

0/2

Low Risk Ankle Rules

100 (15.8, 100)

77.8 (57.7, 91.4)

72.4%

0/2

latter study, the specificity of the LRAR was greater than that of the OAR. In agreement with prior meta-analyses [8], the sensitivity of the OAR was 100%. Despite its greater specificity, the lower sensitivity of the LRAR and the fact that four high-risk ankle fractures were missed using this rule argues against its use in the PED setting.

A disadvantage of the prospective study performed by Boutis et al.

[5] was that patients who met criteria for the LRAR or the OAR did not receive x-rays. Only some of these patients were followed up by phone or physical examination and thus, there is a possibility that high-risk fractures were missed. In our study, imaging was available for all subjects. Additionally, only a portion of subjects in the study per- formed by Boutis et al. [5] had documented physical examinations, thus limiting reproducibility as compared to our study.

The study by Boutis et al. [5] found that the implementation of the ankle rules did not significantly change patient or Physician satisfaction or length of stay. Further research is needed to determine if this would hold true in the United States, with a healthcare system distinct from that of Canada. For example, Canadians use the Emergency Department more often and 31% of emergency department patients waited N 4 h versus a mere 13% of those in the United States [11]. Additionally, utili- zation of Malpractice litigation differs in the two countries.

Another potential application of the OAR that was not investigated in the study by Boutis et al. [5] and could potential curb PED wait times and thus increase parent satisfaction is use of these rules by Triage nurses. The accuracy and reproducibility with which nurses applied the OAR in one study performed in the United States including 185 five to nineteen year olds was 98% [12]. While the effect of triage nurse use of the OAR on length of stay and patient satisfaction has been studied in adult Emergency Departments with mixed results [13,14,15], there has been little research on such outcomes in PED. Thus, this is a promis- ing area for future research.

Though effects on wait times and patient satisfaction as a result of OAR implementation have not been satisfactorily studied in the pediat- ric population, OAR implementation has been found to decrease healthcare costs [16]. One study found savings between $614,226 and

$3,145,910 per 100,000 patients [16].

Limitations

One limitation of our study was that it was a retrospective chart re- view, which was dependent on EMR records to determine which criteria were met. A total of 288 (22.7%) subjects were excluded secondary to inadequate information included in the EMR. Changes or updates to the search engine Montage could also result in a slightly different pa- tient sample, reducing reproducibility. Additionally, the experience of clinicians whose notes were reviewed varied and included residents, nurse practitioners, fellow, and attending physicians. This also makes our results more generalizable, as most PEDs have clinicians at a variety of skill levels.

At the time of our study, there was no standardized method to eval- uate whether a child qualified for an ankle x-ray at our institution. At

times, the triage nurse would order a radiograph and in other instances, the resident, fellow, or attending physician would do so. Thus, the per- centage of patients with ankle injuries receiving x-rays varied depend- ing on the PED staff at any specific time. However, the majority of children (approximately 70-80% by the author’s estimation) with ankle injuries had x-rays performed. Given the retrospective nature of our study, it remains possible that fractures were missed simply by not being imaged. Part of justification of the LRAR, though, is that missing low-risk fractures is acceptable since they are treated conservatively.

With a mere 2% of clinicians’ notes mentioning these clinical deci- sion rules according to our data, the OAR may not have been widely uti- lized. It is possible, though, that more clinicians took the OAR or LRAR into consideration without specifying as such in the EMR. A study conducted in Canada [10] showed 87.5% of PED clinicians using the OAR. However, this study was conducted via a mail survey. An- other study [17] found that United States clinicians had much less positive attitudes towards and were much less likely to use the OAR as compared to clinicians from Canada and the United King- dom. Thus, more research is needed to determine what percentage of PED clinicians utilizes the OAR, what barriers to usage of the rules exist, and how to motivate more clinicians to adopt these clinical decision rules.

Conclusions

Our retrospective study further supports the use of the OAR in the pediatric population and suggests that the LRAR are not sensitive enough for implementation. With only a small number of clinicians documenting use of the OAR, it remains important to continue to inves- tigate methods of implementation of this clinical decision rule. Specifi- cally, research into the use of this rule by triage nurses and the effect such application of this rule would have on PED wait times, healthcare costs, and patient and parent satisfaction should remain important fu- ture research considerations.

References

  1. Timm NL, Ho ML, Luria JW. Pediatric emergency department overcrowding and im- pact on patient flow outcomes. Acad Emerg Med 2008;15(9):832-7.
  2. Hoot NR, Aronsky DA. Systematic review of emergency department crowding:

    causes, effects, and solutions. Ann Emerg Med 2008;52(2):126-36.

    Boutis K, Constatine E, Schuh S, Pecaric M, Stephens D, Narayanan UG. Pediatric emergency physician opinions on ankle radiograph clinical decision rules. Acad Emerg Med 2010;17(7):709-17.

  3. Dowling S, Spooner CH, Liang Y, Dryden DM, Friesen C, Klassen TP, et al. Accuracy of Ottawa Ankle Rules to exclude fractures of the ankle and midfoot in children: a meta-analysis. Acad Emerg Med 2009;16(4):277-87.
  4. Boutis K, Grootendorst P, William A, Plint AC, Babyn P, Brison RJ, et al. Effect of the Low Risk Ankle Rules on the frequency of radiography in children with ankle inju- ries. Can Med Assoc J 2013;185(15):731-8.
  5. Gravel J, Hedrei P, Grimard G, Gouin S. Prospective validation and head-to-head comparison of 3 ankle rules in a pediatric population. Ann Emerg Med 2009; 54(4):534-40.

    A.L. Ellenbogen et al. / American Journal of Emergency Medicine 35 (2017) 12621265 1265

    Boutis K, Komar L, Jaramillo D, Babyn P, Alman B, Synder B, et al. Sensitivity of a clin- ical examination to predict the need for radiography in children with ankle injuries: a prospective study. Lancet 2001;358(9299):2118-21.

  6. Plint AC, Bullock B, Osmond MH, Stiell I, Dunlap H, Reed M, et al. Validation of the Ottawa Ankle Rules in children with ankle injuries. Acad Emerg Med 1999;6(10): 1005-9.
  7. MedCalc Online Diagnostic Test Evaluation Calculator. accessed March 9, 2016. Available from: https://www.medcalc.org/calc/diagnostic_test.php.
  8. Dowling SK, Wishart I. Use of the Ottawa Ankle Rules in children: a survey of physi- cians’ practice patterns. Can J Emerg Med 2011;3(5):333-8.
  9. Pines JM, Hilton JA, Weber EJ, Alkemade AJ, Al Shabanah H, Anderson PD, et al. Inter- national perspective on emergency department crowding. Acad Emerg Med 2011; 18(12):1358-69.
  10. Karpas A, Hennes H, Walsh-Kelly CM. Utilization of the Ottawa Ankle Rules by nurses in a pediatric emergency department. Acad Emerg Med 2002;9(2):130-3.
  11. Fan J, Woolfrey K. The effect of triage-applied Ottawa Ankle Rules on the length of stay in a Canadian urgent care department: a randomized controlled trial. Acad Emerg Med 2006;13(2):153-7.
  12. Sorensen E, Keeling A, Snyder A, Syverud S. Decreasing ED length of stay with use of the Ottawa Ankle Rules among nurses. J Emerg Nurs 2012;38(4):350-2.
  13. Rowe BH, Villa-Roel C, Guo X, Bullard MJ, Ospina M, Vandermeer B, et al. The role of triage nurse ordering on mitigating overcrowding in emergency departments: a sys- tematic review. Acad Emerg Med 2011;18(12):1349-57.
  14. Anis AH, Stiell IG, Stewart DG, Laupacis A. cost-effectiveness analysis of the Ottawa Ankle Rules. Ann Emerg Med 1995;26(4):433-8.
  15. Graham ID, Stiell IG, Laupacis A, McAuley L, Howell M, Clancy M, et al. Awareness and use of the Ottawa Ankle and Knee Rules in 5 countries: can publication alone be enough to change practice? Ann Emerg Med 2001;37(3):259-66.

Leave a Reply

Your email address will not be published. Required fields are marked *