Error Management Training in Medical Simulation

Function

Error management training derives from the cognitive theory that describes work-related actions to be based on action-oriented mental models.[1] An action-oriented mental model is a person’s understanding of how a machine or system functions; and the more accurate and substantive the mental model, the more successful performance will be. Errors act to provide feedback and identify specific areas that one’s mental model is not adequate. EMT sees errors not just as negative feedback indicating failed performance, but rather as an impetus for active learning and exploration to further develop one’s mental model.[1]

The two key elements of error management training based learning are 1) learners are “given only minimal guidance and otherwise, are encouraged to actively explore and experiment on their own,” and 2) “EMT creates a learning environment in which errors are likely to occur.”[1] Learners are informed from the start that errors will occur and the goal is to view these errors as positive learning opportunities and to reflect on the errors committed. This emphasis and positive framing of errors are unique to EMT and set it apart from solely exploratory training or procedure based training methods focused on error avoidance or correct task solutions. EMT has been demonstrated to not only improve task performance compared to more traditional EAT;[1] but has also resulted in improvements in emotional reactions to errors[16] and increased engagement in metacognitive actions including planning, monitoring, and evaluating one’s performance.[1]

Curriculum Development

Error management training in medical simulation has primarily been studied in procedural specialties, including surgery and intensive care settings.[13][5][9][12] By placing the trainees into the simulation-based environment, students may actively explore the simulation models, make procedural errors, identify strategies to mitigate those errors and assess their performance. To build a training curriculum centered on EMT, it is vital to understand the procedure being performed and the potential errors that can occur. This process can take place through obtaining expert opinion or structured needs assessment.[11][13] Expert opinion can be extremely useful in helping to obtain critical procedural information, including preferred approaches and most common errors.[11][17][5][17] Needs assessments can also help to underscore where there are specific deficits in knowledge and skill that are addressable with EMT.[13][18]

An example of the development of an EMT based curriculum has recently been published in the field of combat medicine. There were high rates of lower extremity fasciotomies requiring revision amongst combat-wounded patients when they reached a higher level of care, indicating a need for improvement in training surgeons to perform fasciotomies.[19] The Department of Defense (DoD) and Advanced Surgical Skills for Exposure in Trauma (ASSET) courses were employed to address this deficiency in performance. However, authors Kucera et al. (2019) raised concerns that these EAT courses (DoD and ASSET) may not adequately improve performance.[13] Thus these authors conducted a simulation-based needs assessment for the development of an EMT based training curriculum for 2-incisions, 4-compartment lower extremity fasciotomies amongst military surgeons undergoing routine pre-deployment training.[13] The authors created a list of 18 possible major errors and two possible minor errors related to the performance of fasciotomies on lower extremity models. They found that 90% of the lower extremity models had errors related to the performance of a four-compartment fasciotomy, and there was a mean of 4.8 errors per lower extremity model. When looking at the errors, 26% of the models had at least one completely missed compartment, which is a critical error in performing this procedure. The high rate of technical errors in the simulation environment mirrors the rate of errors seen in the actual clinical environment and indicates a need for improvement in training.[13] By conducting this needs assessment, the authors identified common errors while performing lower extremity fasciotomies and have used this information to create an EMT based curriculum currently underway at the Walter Reed National Military Medical Center.[13] They structured this curriculum to be simulation-based and focused on error identification and management with reassessment at six months to evaluate for knowledge retention.

When creating an error management training curriculum, it is important to evaluate what is being taught and who are the trainees. Prior work has demonstrated that trainees who had higher cognitive skills or were more open to new experiences learned the most from error encouragement training.[20][21] These students learn more from EMT through increasing their motivation to learn, gaining a deeper understanding of tasks, and learning technical and emotional management strategies for handling errors.[21] Achieving these learning outcomes requires training tasks to be designed such that trainees not only commit errors but also have the opportunity to learn from those errors and try new strategies. In the simulation environment, this requires the model to be a working representation of the task being taught. It does not mandate that the model be an exact replica, but rather it needs to provide high “psychological fidelity” such that it mirrors the relevant content for task performance (26173283). Additionally, the educators need to have a working representation of the requirements for successful performance, the type, and frequency of errors related to that procedure and the existing error management strategies. These three requirements provide a framework for giving feedback specifically directed towards errors and management of errors.

D’Angelo et al. created a classification scheme for evaluating errors made by residents performing a simulated surgical task. The classification scheme had its basis on prior work in the field of human factors engineering.[17][5] They categorized errors as technical or cognitive, and the error management steps of error occurrence, error detection, and error recovery were further broken down and recorded.[17][5] This line of research found that the total number of errors committed by senior residents during a simulated laparoscopic ventral hernia (LVH) repair increased from the first to the second day of a training course.[17] At first glance this would indicate a lack of learning, however, by investigating the types of error committed, the error management strategies utilized, and the timing of error occurrence, these authors found changes in participants’ performance. The participants progressed further in the procedure in the second simulation and made primarily technical errors at later stages in the procedure.[17] This result argues for the creation and incorporation of error based assessment tools to evaluate performance in the simulation environment, which will be particularly valuable for designing and implementing error management training based simulation training as it allows educators to understand trainee errors and how to best support learning to prevent or effectively recover from those errors in the actual clinical environment.

Procedural Skills Assessment

Traditional assessment of procedural skills has been through checklists, global rating scales, and final product analysis.[22] However, each of these assessment tools may only capture a component of performance and may not be sufficient to evaluate performance holistically without using other assessment modalities.[23][24] Prior work has demonstrated a variety of critical errors that were identified during the final product analysis but were not captured by the task-specific checklist or global rate scale.[23] This finding indicates that trainees are committing serious errors that are subsequently not reflected in their performance assessments. Error based assessment may be an additional method to evaluate trainees’ technical and decision-making skills. Forsyth et al. adapted Rasmussen’s model of human performance from the field of human factors engineering to create an error based checklist for a specific simulated surgical procedure, the LVH repair. Based on prior work, these authors identified common, procedure-specific errors and grouped them into skill-, rule-, and knowledge-based behavior according to Rasmussen’s model.[17][24] Skill-based behavior consists of subconscious or automated actions such as suturing or knot-tying, and errors of this type relate to inappropriate manual coordination or force. Existing rules or procedures guide rule-based behavior, and errors relate to applying the incorrect rule or not following the correct rule appropriately. Lastly, knowledge-based behavior applies to unfamiliar scenarios where there are no corresponding rules and errors related to the use of a wrong strategy in these situations.[24] In the work by Forsyth et al., residents predominantly committed rule-based errors while performing the simulated LVH repair. Additionally, poor performance on the error-based checklist significantly correlated with the quality of residents’ hernia repairs. This work demonstrates the feasibility of implementing error-based assessments of resident performance during a simulated procedure and offers evidence of validly for utilizing this method of assessment. This form of assessment applies to EMT as it provides a richer understanding of where trainees commonly make errors and what strategies they employ to recover from those errors.

Clinical Significance

It is now clear that clinician’s technical skills are associated with patient outcomes including post-procedure complications, and rates of reoperation, readmissions, and emergency department visits.[25] Additionally, it is not simply the number of errors that correlate with poor patient outcomes, but rather how providers detect and manage errors.[26] This provides a strong impetus to elevate our educational curriculum in medicine to produce healthcare providers that are competent and able to detect and manage errors in the clinical environment. This requires simulation-based training that can transfer to improvements in patient care. Work outside the discipline of medicine has demonstrated the success of error management training in promoting learning and transfer of learning.[1] The ability to incorporate EMT into medical education, especially in the simulation environment, may allow for improvements in training procedural skills and critical decision making.[27] Ultimately for EMT to be effective in medical training there must be evidence of improvement in performance, skill retention, and adaptive transfer to similar tasks and the clinical environment.

There is evidence that EMT can be utilized to teach skills to novices. Medical students without prior ultrasound experience were randomized to EMT or EAT for performing fetal sonographic evaluations.[8] The participants in the EMT group were instructed to anticipate errors, explore the simulated environment, and reflect on errors made when working though the simulated task of taking fetal measurements on an ultrasound simulator. In contrast, the EAT participants were instructed to follow the simulator instructions step by step to perform the fetal measurements. The EMT and EAT groups performed similarly on the simulated post-training test, however, the EMT group significantly outperformed the EAT group on the transfer assessment of fetal measurements in the clinical environment on pregnant patients. This indicates the participants that underwent EMT were better able to adapt the training they received on the simulator to that of the more complex clinical environment.

An important factor in medical training is the durability of the skills learned. General surgery interns were randomized into EMT and EAT groups for a simulation-based curriculum on internal jugular and subclavian central line placement.[12] Similar to the study previously discussed on sonographic skills,[8] the EMT and EAT groups performed similarly on the simulation-based post-test assessment. However, when assessed 30 days later, the EMT group performed significantly better than the EAT for internal jugular and subclavian line placement. The participants in the EAT demonstrated skill decay at 30 days, which did not occur with the EMT group. Additionally, the EMT performed better than the EAT on the transfer test (femoral line placement) at that same time. Conceptualizing errors as part of the training process and encouraging trainees to identify and reflect what went wrong in a procedure may encourage a deeper understanding of the procedure and thus improved skill retention and transfer to task-similar procedures.[12]

The encouraging results of these studies indicate a need to create a formalized curriculum for integrating EMT into medical training. Riefkohl-Ortiz et al. published their work creating an intensive EMT based curriculum for emergency medicine residents and fellows to learn how to manage iatrogenic critical care procedure complications.[9] Learners had the opportunity to engage in six simulation scenarios (propofol overdose, chest tube placed below the diaphragm, tracheal laceration during attempted intubation, pneumothorax after attempted central venous catheter placement, central venous catheter placement in the carotid artery, and pneumothorax on the ventilator) and were encouraged to make errors and then reflect on how mismanagement impacted their understanding of the scenario. The authors found that each participant demonstrated significant improvement postintervention for each assessment. Interestingly, they identified specific areas of deficits in knowledge and decisions making that were critical for appropriately managing patients after iatrogenic complications occurred, such as the need for intubation in the setting of airway compromise from an expanding neck hematoma.[9] This revealed substantive deficits in clinical knowledge that would otherwise not have been uncovered in an EAT curriculum. This pilot curriculum highlights the importance of educating medical trainees on errors and complications so they may decrease the potential harm to patients when errors occur.    

For healthcare practitioners to provide excellent patient care, they need a training method that supports clinicians progressing to independent practice who have the skills to recognize and mitigate errors while taking care of patients. Combining EMT and medical simulation may provide a means by which to achieve this goal more effectively than traditional training methods, such as EAT.

Pearls and Other Issues

In summary, the major teaching points are as follows:

1) The traditional assessment of procedural skill has through checklists, global rating scales, and final product analysis.[22] However, each of these assessment tools may only capture a component of performance and may not be sufficient to evaluate performance holistically without using other assessment modalities.[23][24]

2) Errors act to provide feedback and identify specific areas that one’s mental model is not adequate. Error management training sees errors not just as negative feedback indicating failed performance, but rather as an impetus for active learning and exploration to further develop one’s mental model.[1]

3) The two key elements of error management training based learning are that firstly learners are “given only minimal guidance and otherwise are encouraged to actively explore and experiment on their own”; and secondly “EMT creates a learning environment in which errors are likely to occur.”[1]

Enhancing Healthcare Team Outcomes

Although the majority of research on error management training in medical simulation has focused on surgical and procedural skill acquisition amongst individual trainees, the educational method applies to team-based training and multiple disciplines throughout medicine.[21] Learning in a simulated environment that encourages error management strategies and the discussion of errors within teams may promote communication and shared cognition among team members.[27] Additionally, EMT has been demonstrated to improve emotional regulation in the setting of errors,[16] and this type of training may be particularly crucial in high-stress fields such as medicine.[27] There is an excellent opportunity to integrate EMT into both individual and team-based training in the simulation environment for a wide array of healthcare providers and specialties. Both trainees and practicing clinicians could benefit from this training. Overall, EMT has great potential to change the culture of training and communication in healthcare, and more instructors should familiarize themselves with the theoretical underpinnings that support EMT-based curricula. [Level III]