Airway Checklists in the ED

A checklist is a physical, cognitive aid that lists essential actions required for a task to compensate for limitations in human memory. While checklists have been around forever, the concept of checklists in medicine is much newer. Medical checklists have been gaining popularity and appearing on social media, but is there actually any evidence to suggest we should be using them? Let’s take a quick look through the history and evidence for checklists as they pertain to the ED.

History of safety checklists

In 1935 the US military was looking to purchase new long-range bombers and the Boeing Model 299, nicknamed ‘The Flying Fortress’, was the frontrunner for purchase.¹ During its first military test flight, Boeing’s chief test pilot and 2 experienced military pilots took off in the Model 299, climbing for a few seconds before stalling and crashing. Unfortunately, the pilots died of their injuries. An investigation was launched to determine the cause of the crash.

And they found… Nothing.

In the end, it was decided that pilot error caused the crash. The pilots had made a simple mistake. They left the rudder and elevator controls locked. How could such experienced pilots make such a simple error? The answer was that there were just too many little things that had to be done to properly operate the Model 299 that the pilots forgot one. The initial reaction was that the plane was just too complicated to operate. The military was determined to make these planes work, however, since the Model 299s were superior to the other new bomber candidates. They sat down and looked for solutions. Their solution was a simple one: A pilot’s checklist that listed all the important steps to operating the aircraft during critical moments. The next 2 million test miles occurred without incident. Over time checklists became cemented as the gold standard in aviation. All pilots began being trained on the use of checklists to engrain their importance in critical situations.

Boeing Model 299, stock photo

Checklists in Surgery

Several studies have been performed looking at shortcomings, preventable errors, and complications in medicine. One field that has carefully examined these aspects is surgery. The more people studied errors, the more patterns emerged.  Some of the most notable errors in surgery were found to be wrong site surgeries, post-operative infections, intra- and post-operative bleeding, technique related issues, and incorrect procedure selection. These were not new problems in surgery, but people were starting to realize that there may be ways to mitigate these predictable errors. They began to study the types of errors made perioperatively and set out to try to rectify them. From this research surgical checklists emerged.

WHO Surgical Safety Checklist²

Important papers describing before-after studies began to emerge in the late 2000s showing considerable improvements in morbidity and mortality.^3,4,5 Literature was being published showing dramatic reductions in complication rates that went beyond the actual items checked in the checklist.  This seemed to indicate that checklists improved more than just what was written down on the paper. Many suspected that it was the improvement in communication between the allied health professionals in the ORs that resulted in this improvement. These findings led to sweeping policy change across the surgical world. Safety checklists became mandatory in many countries.

Then a paper was published that gave people pause.

In 2014 Urbach et al.⁶ published a retrospective study examining the implementation of surgical safety checklists in 101 hospitals across Ontario, Canada. They found that although adherence to the checklists was very high, there was no statistically significant difference in complication rates or mortality across the entire population. So why did this happen when there was a mountain of global evidence showing that safety checklists were effective? Well, the studies on checklists are all uncontrolled, and the methods are widely variable, which may play a role. More critically, this study noted that the implementation of these mandatory checklists were left to the discretion of the individual hospitals. Implementation strategies are likely the most important factor in determining the success of a checklist. This is supported by subsequent literature reviews noting improved success with more rigorous implementation strategies.⁷

Airway – the issues

Definitive airway control is often considered one of the higher risk procedures we do as ED physicians. It can be described as a low-frequency high-risk situation, where failure can be devastating. A few studies have looked at issues around intubation in the ED and, it turns out, errors and failures are common.

Kerrey et al.⁸ performed a retrospective video review of 123 intubation attempts in a pediatric ED. They found that the success rate on video was lower than reported. They found a first pass success rate of only 52% (26% requiring 3+ attempts); 61% of RSIs experienced an adverse event, including a 33% desaturation rate. Of the patients who desaturated, 76% dropped below 80% and 29% dropped below 60%. Rinderknecht et al.⁹ from the Kerrey group performed a retrospective study in 2015 on factors associated with desaturation in a pediatric ED. In this study, they analyzed video of 114 intubation attempts on children and noted that desaturation occurred in half the children after only 30 seconds of laryngoscopy. Total laryngoscopy time rather than individual attempt time was also associated with increased desaturation. Other factors associated with desaturation were patients under 2 years old and esophageal intubation.

For further information on pediatric airway management, please see Dr. Rebecca Brown’s grand rounds post.

Wong et al.¹⁰ performed a prospective observational study in 2008 to record the percentage of difficult or failed airways to see what factors were associated with them. They noted 4% of airways were considered difficult, which they defined as difficult BVM, 3+ orotracheal intubation attempts, or cricothyrotomy performed. They noted that ED physicians had a 68.8% success rate with difficult airways, with anesthesia or surgery performing the other rescues. The majority of rescues were by direct laryngoscopy + bougie. In the failed group, most of the patients died. This really hammers home what’s at stake during intubation. The authors conclude that increased use of adjuncts, notably earlier bougie, by the ED would improve our success rate in difficult airways.

Cook et al.¹¹ performed a retrospective study in the ED + ICU looking at major airway complications. They found 184 major airway complications, resulting in 31% death or significant neurologic injury. The highest risk of these outcomes was with junior learners, lack of equipment, failure to identify at-risk patients, failure to interpret capnography, and inadequate airway plans. This is notable because the factors listed here are predictable and correctable issues that can be addressed.

So you can see that the issues in airway management are recurrent and potentially deadly. From these studies, we can see the importance of identifying patient-specific airway concerns, building complete airway plans, having adjuncts, avoiding multiple or longer attempts as possible. It is also important to remember that our failure rate and the required number of attempts is likely higher than reported in the literature, as was noted in the video review studies. Luckily, there are cognitive aids that may help us mitigate these errors.

Airway Checklists

There has been a fair amount of research regarding airway checklists. However, the studies themselves are not of the highest methodological quality. Below you will find summaries of selected studies from the airway checklist literature.

Pediatric ED

Long et al. 2016¹²

• Randomized control trial on the use of cognitive aids in airway equipment preparation
• 21 participants in each arm: picture template, checklist, control
• Every participant in the control group had at least 1 error
• Higher error rate in control vs. template and checklist
• No difference between template and checklist
• Control groups often omitted oral airways, bougie, LMA

Checklist used in Long et al. study¹²

Caruso et al. 2017¹²

• Retrospective pre/post study examining a checklist based intervention on selection and administration of RSI meds in peds ED
• No effect on RSI medication selection
• 32% increase in participants waiting 45 seconds after neuromuscular blockade to start laryngoscopy
• 40% increase in patients receiving both NMB and sedation within 30 seconds of each other
• Limited as video laryngoscopy (VL) was simultaneously implemented
• Limited to experienced providers

Kerrey et al. 2015¹³

• Retrospective pre/post implementation video study of pediatric intubation checklist
• 92% uptake of the checklist
• Decrease in desaturation 33% → 16%, reduced rate of multiple desaturations
• Limited as 4 interventions simultaneously implemented: RSI checklist, pilot-copilot model,  use of VL, and restriction of intubation to PED fellows, PED staff, anesthesia or senior EM residents

Adult ED

Conroy et al. 2014¹⁴

• Retrospective pre-post implementation of a peri-intubation checklist for trauma patients
• RSI was greater in the checklist group 75.6% →90.7%
• No differences in the number of attempts, saturation, post-intubation anxiolytics, mortality, and ICU LOS
• Limitations
  • 1/3 of intubations were performed by anesthesiologists in the control group while the checklist group was almost exclusively ED physicians
    • The change in provider alone can explain the difference in RSI use
  • Did not monitor checklist compliance

Smith et al. 2015¹⁵

• Retrospective pre-post implementation of preprocedural intubation checklist for trauma
• Absolute risk reduction of 7.7% for composite complication outcome
• Reduced paralysis to intubation time by 12s
• Improvement of adherence to safety measures by 52%
• Limited by use of composite outcome, as some less clinically significant outcomes than others

Janz et al. 2018¹⁶

• Multicentre randomized trial studying intubation checklist in 262 critically ill adults
• No difference in lowest saturation, number of intubation attempts, complication rates, or BP
• Limitation: checklist items bled into control group attempts

So as we just discussed there’s very mixed evidence that doesn’t definitively answer the questions around checklists. There is a potential reduction of adverse events with checklists, but the evidence overall is quite weak. This body of literature is more hypothesis-generating than it is practice changing.

Crafting a checklist

To make an effective checklist, you need to select a target task that is complicated but can be broken down into useful and actionable chunks. Ultimately, a good checklist has to be a cognitive aid that can address issues with human performance. This means it has to list critical steps that are actionable, at risk of being missed, and not checked by other mechanisms.

Emergent situation checklists are generally designed to be read out loud, often in a pilot-copilot style or a challenge-do-response style. This is a modification of the read-do checklist, meaning one provider reads and the other confirms as each item is done. This ensures that you do not miss any critical step. The people in these roles do not have to be physicians, so may involve any other healthcare provider. In emergencies, checklists have to be used in a timely fashion, meaning they have to be short and to the point.

Another form of the checklist is the do-check. With this style, you do it all by memory then confirm that nothing has been missed. This can be faster but results in a higher risk of missing a step. This is often used in set-up type tasks, like setting up your central line or airway equipment.

Implementation

I would argue that implementation is just as important, if not more important than the checklist itself. A good implementation strategy will almost guarantee you put out a good checklist. If you look through the studies with positive results and those with negative results one of the key factors in them is their implementation. There are a few key aspects of checklist implementation. You need organizational support, stakeholder buy-in, multidisciplinary involvement, responsiveness to feedback, and voluntary participation^17,18,19.

Diagram showing the steps to implementation of a checklist¹⁸

Setting up a proper implementation strategy takes time. A study by Davis et al.¹⁹ looking at airway bundle implementation in the ICU noted that it took over 500 days to develop and implement their airway checklist fully. There is a good reason why it takes so long. You first need the support of the organization. If there’s no support, you’ll never get enough money, time, or buy in to make it work. Next, you need stakeholder buy-in. Meaning you need people to acknowledge the need for a checklist and commit to developing and improving the cognitive aid. You also need to involve all parties. Nursing buy-in is just as important as physician buy-in. Having champions from all professions is a crucial part of rolling out any checklist, as they are the model for which people will evaluate their willingness to change their practice to accommodate the checklist.

With bringing in all these other health professionals, you need to be responsive to feedback. Across the board, studies show that checklists modified or built in-house using feedback by the intended audience via surveys, testing sessions, and focus groups were more effective than ones used without customization.

And finally and most importantly, it must be voluntary. Forcing someone to use a checklist they don’t believe in is a recipe for disaster. Looking back at surgical checklists, the most likely reason that surgical checklists seemed to work in so many places but failed miserably during the rollout across Ontario may have to do with this. The government made it mandatory in all surgeries, forcing those who had not bought in to use them. This strategy harbors resentment and undermines the communication benefits of the checklist. Voluntary checklists likely work by having non-believers converted by champions and early adopters as their outcomes improve and they describe satisfaction with the tool.

Take home points

• Checklists should be a concise list of critical elements not checked through other means, that if missed could result in failure
• Evidence for airway checklists in the ED is relatively weak with poor methodology, but probably work if properly crafted and implemented
• Implementation requires time, stakeholder buy-in, multidisciplinary champions, voluntary use, and local customization