Skip to main content
Advertisement
  • Neurology.org
  • Journals
    • Neurology
    • Clinical Practice
    • Education
    • Genetics
    • Neuroimmunology & Neuroinflammation
  • Online Sections
    • Neurology Video Journal Club
    • Diversity, Equity, & Inclusion (DEI)
    • Innovations in Care Delivery
    • Practice Buzz
    • Practice Current
    • Residents & Fellows
    • Without Borders
  • Collections
    • COVID-19
    • Disputes & Debates
    • Health Disparities
    • Infographics
    • Null Hypothesis
    • Patient Pages
    • Topics A-Z
    • Translations
  • Podcast
  • CME
  • About
    • About the Journals
    • Contact Us
    • Editorial Board
  • Authors
    • Submit New Manuscript
    • Submit Revised Manuscript
    • Author Center

Advanced Search

Main menu

  • Neurology.org
  • Journals
    • Neurology
    • Clinical Practice
    • Education
    • Genetics
    • Neuroimmunology & Neuroinflammation
  • Online Sections
    • Neurology Video Journal Club
    • Diversity, Equity, & Inclusion (DEI)
    • Innovations in Care Delivery
    • Practice Buzz
    • Practice Current
    • Residents & Fellows
    • Without Borders
  • Collections
    • COVID-19
    • Disputes & Debates
    • Health Disparities
    • Infographics
    • Null Hypothesis
    • Patient Pages
    • Topics A-Z
    • Translations
  • Podcast
  • CME
  • About
    • About the Journals
    • Contact Us
    • Editorial Board
  • Authors
    • Submit New Manuscript
    • Submit Revised Manuscript
    • Author Center
  • Home
  • Articles
  • Issues
  • Blog

User menu

  • My Alerts
  • Log in

Search

  • Advanced search
Neurology: Education
Home
An open access peer-reviewed journal in neurologic and neuroscience training
  • My Alerts
  • Log in
Site Logo
  • Home
  • Articles
  • Issues
  • Blog

Share

December 2022; 1 (2) EditorialOpen Access

Attaining Base Camp

Moving in the Right Direction, Miles Still to Go

Catherine S.W. Albin
First published December 9, 2022, DOI: https://doi.org/10.1212/NE9.0000000000200036
Catherine S.W. Albin
From the Department of Neurology, Emory University School of Medicine, Atlanta, GA.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Full PDF
Citation
Attaining Base Camp
Moving in the Right Direction, Miles Still to Go
Catherine S.W. Albin
Neurol Edu Dec 2022, 1 (2) e200036; DOI: 10.1212/NE9.0000000000200036

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
Permissions

Make Comment

See Comments

Downloads
79

Share

  • Article
  • Figures & Data
  • Info & Disclosures
Loading

Simulation is a highly engaging methodology for both teaching and assessment. Yet simulation frameworks, techniques, and procedures have largely been developed and investigated outside of the neurologic context, and much work remains in understanding how simulation can be best leveraged by neurology educators. In this issue of Neurology® Education, Tchopev et al. and Pergakis et al. report important findings from 2 acute neurology simulation curricula.1,2

In both studies, the learners found the experience meaningful and engaging, but beyond the shared enthusiasm of participants, these studies represent 2 disparate points on the continuum of how simulation can be applied in neurology. In the study by Tchopev et al., simulation is explored as an engaging method for teaching in a safe learning environment—simulation as a formative experience. In the study by Pergakis et al., simulation is used to assess trainee performance—simulation as a summative experience. Both provide continued evidence of the feasibility and educational value of simulation, with the study by Pergakis et al. also highlighting how we can establish validity. Both, however, underscore the heights we have yet to ascend to create evidence-based best practices for our field (Figure).

Figure
  • Download figure
  • Open in new tab
  • Download powerpoint
Figure Simulation From Base Camp to Summit

A representation of the steps necessary to demonstrate how simulation can be used for assessment and ultimately to have an effect on patient and health system outcomes.

Simulation as a Formative Experience

Tchopev et al. used self-assessments and a limited knowledge assessment before and after the curriculum to demonstrate learning. As expected, students performed better on most knowledge questions and reported higher confidence after the simulation. This work establishes that learners react well to simulation, but it invites many more questions—how can we assess long-term retention and bedside effect of simulation curriculum? As the authors note, self-assessment is subject to bias and is a low-level outcome in Kirkpatrick's model of curricular assessment.3,4 Improvement in knowledge-based questions provides an indication of factual acumen but may not translate to management at the bedside. Equally important to how we should measure learning is when we should measure it. Assessments delivered directly after a simulation measure only short-term learning. Evaluating outcomes too far removed from the simulation may confound the assessment with additional learning, maturation, or skill decay.

Furthermore, many of these formative curricula are used to develop “nontechnical” skills such as communication, triage, teamwork, and cognitive load reduction. These attributes are difficult to measure but are equally important in the practice of medicine. It will be important to determine when nontechnical skills should be emphasized or if there needs to be separate technical and nontechnical training. Directly assessing these skills, providing feedback to trainees, and conducting follow-up observation in a clinical setting may be needed to understand how practice of nontechnical skills improves bedside performance.

Finally, while the actual simulation is important in activating limbic pathways and engaging learners in decision-making, the formative experience of simulation is debriefing.5,-,8 Any study of simulation as a formative experience needs to explore the style and lens through which debriefing is accomplished. Educators can benefit from training in these techniques (such as Debriefing with Good Judgement,9 TeamGAINS,10 PEARLS,11 GIFT,12 etc.). Simulation studies should clearly define the approach to debriefing and the framework used (Figure).

Simulation as a Summative Experience

With the Accreditation Council for Graduate Medical Education milestones revision,13 there has been an emphasis on moving toward more objective methods to measure trainee performance. Pioneering simulation work has demonstrated that for status epilepticus, achieving a “ready to graduate” milestone had poor predictive value for how well they performed in a status epilepticus simulation.14 The study by Pergakis et al. provides further important validity evidence for simulation in acute neurology. By having many different levels of learners complete the same simulation case, they demonstrate that the trainees' level of experience had a significant effect on the critical action sum score (improving with increasing experience), suggesting that simulation can be used to benchmark achievement. Future studies have much to build on. If simulation is going to be used for evaluation, who sets the competency bar? Scores must include attaining technical milestones, but should we also aim to assess nontechnical skills: how well does the trainee triage, integrate new information, and provide closed-loop communication? If so, what are those benchmarks?

We will also need to qualitatively assess why trainees make mistakes. Pergakis et al. provide an important window into the mental frameworks of trainees: many had anchoring bias that prevented them from reimaging the situation as an infection. In this way, simulation uncovers 2 different kinds of mistakes: (1) “I didn't remember correctly” (best categorized as a low-level mistake in Blooms Taxonomy, e.g., the trainee forgot to give thiamine) and (2) “I didn't analyze correctly” (a higher-level Blooms Taxonomy mistake, e.g., the trainee failed to reconsider an alternative etiology of status). While the lower-level mistakes can be remedied by checklists and cognitive aids, the higher-level analytic mistake requires coaching trainees to avoid cognitive missteps; to essentially “think better.” This—understanding how trainee's think—is perhaps the most interesting application of simulation and as such deserves dedicated study. Certainly, remedying narrowed thinking or failed heuristics may not be possible over 1 simulation, but a thoughtful debrief may provide trainees with insight about their own bias and blind spots. We should assess the degree to which it does.

Finally, and of course most challengingly, educators must demonstrate that simulation results in patient-level outcome benefits. Demonstrating that central line training reduced central line–associated blood stream infections and saved healthcare dollars set the gold standard for simulation as a translational science.15 While we have evidence that simulation may improve door to needle times in acute stroke,16 we have yet to explore how simulation of procedural training might translate into tangible quality improvements in the bedside (e.g., brain death testing or endovascular stroke training). To demonstrate these higher Kirkpatrick feats will require multicenter collaboration and rigorous study.

For simulation in neurology, it is clear the steepest ascent is ahead. Undoubtably, though, there are educators up for the challenge.

Study Funding

No targeted funding reported.

Disclosure

The author reports no disclosures relevant to the manuscript. Go to Neurology.org/NE for full disclosures.

Footnotes

  • Received by Neurology: Education

  • Go to Neurology.org/NE for full disclosures. Funding information and disclosures deemed relevant by the authors, if any, are provided at the end of the editorial.

  • Research Articles Education Research: High-Fidelity Simulation to Evaluate Diagnostic Reasoning Reveals Failure to Detect Viral Encephalitis in Medical Trainees Page e200020

  • Curriculum Innovations: High-Fidelity Simulation of Acute Neurology Enhances Rising Resident Confidence: Results From a Multicohort Study Page e200022

  • Received October 25, 2022.
  • Accepted in final form October 31, 2022.
  • © 2022 American Academy of Neurology

This is an open access article distributed under the terms of the Creative Commons Attribution License 4.0 (CC BY), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

References

  1. 1.↵
    1. Tchopev ZN,
    2. Nelson AE,
    3. Hunninghake JC,
    4. Cacic K,
    5. Cook MK,
    6. Jordan MC
    . Curriculum innovations: high-fidelity simulation of acute neurology enhances rising resident confidence results from a multicohort study. Neurol Edu. 2022;1(2):e200022.
    OpenUrlAbstract/FREE Full Text
  2. 2.↵
    1. Pergakis MB,
    2. Chang WTW,
    3. Gutierrez CA, et al
    . Education research: high-fidelity simulation to evaluate diagnostic reasoning reveals failure to detect viral encephalitis in medical trainees. Neurol Edu. 2022;1(2):e200020.
    OpenUrlAbstract/FREE Full Text
  3. 3.↵
    1. Watson RS,
    2. Borgert AJ,
    3. CT OH, et al
    . A multicenter prospective comparison of the Accreditation Council for Graduate Medical Education milestones: clinical competency committee vs. resident self-assessment. J Surg Educ. 2017;74(6):e8-e14.
    OpenUrl
  4. 4.↵
    1. Kirkpatrick D
    . The Four Levels of Evaluation: Measurement and Evaluation. American Society for Training & Development, 2007.
  5. 5.↵
    1. Cheng A,
    2. Morse KJ,
    3. Rudolph J,
    4. Arab AA,
    5. Runnacles J,
    6. Eppich W
    . Learner-centered debriefing for health care simulation education: lessons for faculty development. Simul Healthc. 2016;11(1):32-40.
    OpenUrlPubMed
  6. 6.↵
    1. Raemer D,
    2. Anderson M,
    3. Cheng A,
    4. Fanning R,
    5. Nadkarni V,
    6. Savoldelli G
    . Research regarding debriefing as part of the learning process. Simul Healthc. 2011;6(suppl):S52-S57.
    OpenUrlCrossRefPubMed
  7. 7.↵
    1. Fanning RM,
    2. Gaba DM
    . The role of debriefing in simulation-based learning. Simul Healthc. 2007;2(2):115-125.
    OpenUrlCrossRefPubMed
  8. 8.↵
    1. Issenberg SB,
    2. McGaghie WC,
    3. Petrusa ER,
    4. Lee Gordon D,
    5. Scalese RJ
    . Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach. 2005;27(1):10-28.
    OpenUrlCrossRefPubMed
  9. 9.↵
    1. Rudolph JW,
    2. Simon R,
    3. Dufresne RL,
    4. Raemer DB
    . There's no such thing as “nonjudgmental” debriefing: a theory and method for debriefing with good judgment. Simul Healthc. 2006;1(1):49-55.
    OpenUrlCrossRefPubMed
  10. 10.↵
    1. Kolbe M,
    2. Weiss M,
    3. Grote G, et al
    . TeamGAINS: a tool for structured debriefings for simulation-based team trainings. BMJ Qual Saf. 2013;22(7):541-553.
    OpenUrlAbstract/FREE Full Text
  11. 11.↵
    1. Eppich W,
    2. Cheng A
    . Promoting Excellence and Reflective Learning in Simulation (PEARLS): development and rationale for a blended approach to health care simulation debriefing. Simul Healthc. 2015;10(2):106-115.
    OpenUrlCrossRefPubMed
  12. 12.↵
    1. Roze des Ordons AL,
    2. Eppich W,
    3. Lockyer J,
    4. Wilkie RD,
    5. Grant V,
    6. Cheng A
    . Guiding, Intermediating, Facilitating, and Teaching (GIFT): a conceptual framework for simulation educator roles in healthcare debriefing. Simul Healthc. 2022;17(5):283-292.
    OpenUrl
  13. 31.↵
    1. Dewey JJ,
    2. Chiota-McCollum N,
    3. Barratt D, et al
    . Introducing the neurology milestones 2.0. Neurology. 2022;98(9):366-372.
    OpenUrl
  14. 14.↵
    1. Mikhaeil-Demo Y,
    2. Holmboe E,
    3. Gerard EE, et al
    . Simulation-based assessments and graduating neurology residents' milestones: status epilepticus milestones. J Grad Med Educ. 2021;13(2):223-230.
    OpenUrl
  15. 15.↵
    1. Cohen ER,
    2. Feinglass J,
    3. Barsuk JH, et al
    . Cost savings from reduced catheter-related bloodstream infection after simulation-based education for residents in a medical intensive care unit. Simul Healthc. 2010;5(2):98-102.
    OpenUrlCrossRefPubMed
  16. 16.↵
    1. Ajmi SC,
    2. Advani R,
    3. Fjetland L, et al
    . Reducing door-to-needle times in stroke thrombolysis to 13 min through protocol revision and simulation training: a quality improvement project in a Norwegian stroke centre. BMJ Qual Saf. 2019;28(11):939-948.
    OpenUrlAbstract/FREE Full Text

Letters: Rapid online correspondence

No comments have been published for this article.
Comment

REQUIREMENTS

If you are uploading a letter concerning an article:
You must have updated your disclosures within six months: http://submit.neurology.org

Your co-authors must send a completed Publishing Agreement Form to Neurology Staff (not necessary for the lead/corresponding author as the form below will suffice) before you upload your comment.

If you are responding to a comment that was written about an article you originally authored:
You (and co-authors) do not need to fill out forms or check disclosures as author forms are still valid
and apply to letter.

Submission specifications:

  • Submissions must be < 200 words with < 5 references. Reference 1 must be the article on which you are commenting.
  • Submissions should not have more than 5 authors. (Exception: original author replies can include all original authors of the article)
  • Submit only on articles published within 6 months of issue date.
  • Do not be redundant. Read any comments already posted on the article prior to submission.
  • Submitted comments are subject to editing and editor review prior to posting.

More guidelines and information on eLetters

Compose Comment

More information about text formats

Plain text

  • No HTML tags allowed.
  • Web page addresses and e-mail addresses turn into links automatically.
  • Lines and paragraphs break automatically.
Author Information
NOTE: The first author must also be the corresponding author of the comment.
First or given name, e.g. 'Peter'.
Your last, or family, name, e.g. 'MacMoody'.
Your email address, e.g. higgs-boson@gmail.com
Your role and/or occupation, e.g. 'Orthopedic Surgeon'.
Your organization or institution (if applicable), e.g. 'Royal Free Hospital'.
Publishing Agreement
NOTE: All authors, besides the first/corresponding author, must complete a separate Publishing Agreement Form and provide via email to the editorial office before comments can be posted.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.

Vertical Tabs

You May Also be Interested in

Back to top
  • Article
    • Simulation as a Formative Experience
    • Simulation as a Summative Experience
    • Study Funding
    • Disclosure
    • Footnotes
    • References
  • Figures & Data
  • Info & Disclosures

SARS-CoV-2 Vaccination Safety in Guillain-Barré Syndrome, Chronic Inflammatory Demyelinating Polyneuropathy, and Multifocal Motor Neuropathy

Dr. Jeffrey Allen and Dr. Nicholas Purcell

► Watch

Related Articles

  • Education Research: High-Fidelity Simulation to Evaluate Diagnostic Reasoning Reveals Failure to Detect Viral Encephalitis in Medical Trainees
  • Curriculum Innovations: High-Fidelity Simulation of Acute Neurology Enhances Rising Resident ConfidenceResults From a Multicohort Study

Alert Me

  • Alert me when eletters are published
Neurology: Education: 2 (2)

Articles

  • Articles
  • Issues
  • Popular Articles

About

  • About the Journals
  • Ethics Policies
  • Editors & Editorial Board
  • Contact Us
  • Advertise

Submit

  • Author Center
  • Submit a Manuscript
  • Information for Reviewers
  • AAN Guidelines
  • Permissions

Subscribers

  • Subscribe
  • Sign up for eAlerts
  • RSS Feed
Site Logo
  • Visit neurology Template on Facebook
  • Follow neurology Template on Twitter
  • Visit Neurology on YouTube
  • Neurology
  • Neurology: Clinical Practice
  • Neurology: Education
  • Neurology: Genetics
  • Neurology: Neuroimmunology & Neuroinflammation
  • AAN.com
  • AANnews
  • Continuum
  • Brain & Life
  • Neurology Today

Wolters Kluwer Logo

Neurology: Education | Online ISSN: 2771-9979

© 2023 American Academy of Neurology

  • Privacy Policy
  • Feedback
  • Advertise