Curriculum Innovations: A Podcast-Based Neurologic Emergency Flipped Classroom Curriculum for Neurology Residents

Discussion

We present an FC curriculum for neurology residents on 3 emergency neurology topics featuring podcast-based instruction followed by expert-led discussion on clinical topics. The primary impact reported focuses on residents' reaction and changes in confidence. These results demonstrate preliminary findings of a positive impact of the curriculum and establish a rationale for further, more rigorous, assessment of this curriculum's impact on residents' knowledge or clinical skills.

Using the Kirkpatrick framework to evaluate educational interventions, the curricular sessions' impact was positive on residents' reaction and confidence (Kirkpatrick Level 1).^24,25 Level 1 impact is demonstrated via learners' responses or attitudes toward the session. Level 2 impact gauges knowledge, skills, and confidence, typically via measuring knowledge and skill improvement. While some studies have used self-reported confidence to demonstrate Level 2 impact, these reports of confidence often pertain to specific topics covered by the curriculum rather than overarching confidence.²⁵

Participants’ reported confidence increased after completing the sessions on status epilepticus and movement disorder emergencies (Figures 1 and 2). This effect was not found for acute stroke management (Figure 3). Baseline confidence ratings in acute stroke management were high, perhaps due to high volume of acute stroke management in neurology training. Thus, improvement in confidence may be limited by a ceiling effect.

Based on subgroup analysis, we suspect that actual experience in managing these neurologic emergencies modifies the sessions' effect. The session on movement disorder emergencies was associated with positive shifts (some large) in confidence across training levels. Movement disorder emergencies are estimated to be encountered only 1–2 times in the course of residency training.³ This lack of experience may result in a more robust change in confidence after education.

In further support of this finding, confidence gains in managing status epilepticus were seen in subgroup analysis only among PGY2 learners, again implicating that less experienced residents gain more confidence. Status epilepticus has an incidence of approximately 40 per 100,000 patients, and PGY4 neurology residents report having managed status epilepticus frequently.² A confidence ceiling effect may be observed among more experienced PGY3 and PGY4 residents in managing status epilepticus. The acute stroke session was associated with decreased confidence in PGY3 residents, though it is worth noting that no PGY3 participants reported feeling somewhat or extremely unconfident before or after the session. Further evaluation is needed, but we speculate that the more rare and nuanced clinical decisions discussed in the acute stroke podcast (e.g., thrombolysis in pregnancy or retinal artery occlusion) may have been humbling, emphasizing experience or knowledge gaps among PGY3 residents, which resulted in a mild reduction in confidence.²⁶ We take caution in drawing excessive conclusions from a small sample, but this could be an area of further study.

In addition to gains in confidence, enjoyment of the curriculum was high. Baseline data demonstrated that podcasts are already popular for learning neurology. At the end of the curriculum, most residents reported that podcasts were helpful in their education (94%) and that they were more likely to use podcasts in the future compared with their recollection before the curriculum (74%). However, the percent of residents reporting they are likely to use podcasts for neurology on the initial survey was 86% compared with 77% on the final survey. While this difference was not significant, the decrease from before to after curricular implementation contradicts the former figure and creates some uncertainty in these results. This may be potentially attributed to noise in the data from the differences in sample size between the baseline and final surveys; future studies could examine actual learning practices via surveys or metadata of podcast downloads/streams.

Preference data in this study repeatedly showed that podcasts are preferred over lectures and reading of text, while less preferred compared with simulation and case-based discussion. One comparison indicated that this curriculum was able to increase the preference of podcasts compared with medical simulation; however, this result fails to remain significant if correcting for multiple comparisons.²⁷

The curriculum is likely easy to implement, using a freely available podcast as instructional material. While our discussion sessions featured subspecialty experts in movement disorders, epilepsy, and vascular neurology, programs could potentially use nonfellowship-trained faculty if they feel equipped to discuss the neurologic emergency topics. The curriculum is presented as a 3-part series; however, each session can be implemented in isolation.

The curriculum is not without disadvantages. Residents spent more than 30 minutes before the discussion in podcast listening. Some respondents noted an advantage of podcast listening was the ability to take notes or to rewind and relisten, which requires time and can be difficult to do while multitasking. When coupled with the 60-minute discussion session, the time devoted to the topic may total more than 90 minutes, which educators may find inefficient or residents may consider burdensome. Future research could experiment with protecting 30 minutes of resident time for listening followed by a 30-minute discussion with an assessment of educational impact. In an era of busy clinical services and emphasis on work-life balance, the time commitments of these sessions should be considered.²⁸

The curriculum's impact on learning should not be overstated. While a growth in resident confidence may reflect a desired impact, further evaluation of knowledge or clinical skill development is needed. Future research could develop tests or other knowledge assessments for before and after the curriculum. Simulations of the neurologic emergencies taught could be developed to assess learners' skills in managing these clinical scenarios.

In describing changes in confidence, changes in proportions of learners reporting confidence had varying magnitudes. Some showed seemingly large magnitudes such as the 61% increase among those who participated in the movement disorder emergencies session where others were more modest such as the 19% increase within status epilepticus management. Ultimately, knowing whether any of these changes are meaningful is challenging. As mentioned earlier, future studies that better define educational impact through changes in clinical behavior or performance would better capture the impact of an educational intervention. For example, if 19% more trainees could achieve competence on a validated stroke simulation protocol, educators could likely be more confident in the meaningful impact their curriculum conferred.

We are cautious in deducing that an increase in confidence begets increased clinical competence. In status epilepticus, a simulation-based assessment of neurology residents demonstrated that, despite high levels of self-reported confidence and experience in recognizing and managing status epilepticus, standards of competence were difficult to achieve.² Other studies show that self-assessment of skill correlates poorly with objective measures of clinical abilities.²⁹ Future educational interventions could study the implementation of similar teaching methods for other neurologic emergencies (e.g., cerebral herniation) or for nonemergent clinical syndromes (e.g., headache). One could also look at residents' actual clinical practice changes to assess for higher-level impact.

This was an uncontrolled comparison among learners; everyone was exposed to the educational curriculum, and no control group was formed. Thus, it is possible that learners' confidence increased by experience over time rather than due to our intervention. Our post hoc analysis suggests that the timing of the baseline survey correlates with participants' reported confidence, with confidence growing over time. However these correlation coefficients were typically weak. Significant correlation was not found with postsession surveys, which suggests the effect on confidence seen was not only due to elapsed time but could be attributed to the curriculum intervention. We also acknowledge that the postsession subgroups are small and thus poorly powered such that we may have failed to detect a significant correlation that is actually present.

Survey participation was low at baseline and waned with subsequent survey administration. With 10 programs averaging 22 trainees per program, the baseline survey number of 110 indicates a relatively low response rate. Subsequent postsession response numbers may only represent approximately 10% of eligible residents. Given this sample size attrition, the confidence and attitudes of the remaining 90% of eligible but not surveyed residents may differ from those who responded. Attitudes toward podcast learning vs other modalities (lecture, simulation, etc) may be misrepresented among those who completed data collection when compared with those who did not. Data on education session administration were not collected, so the total number of participants exposed to the curriculum is not known. Quantitative data to explain this drop are not available. The first survey was not associated with session attendance, while subsequent surveys were. Attendance numbers at each session were not recorded. We have previously discussed that attendance at GME didactics can be challenging in the current training environment.^28,30 It is possible that residents who missed a session were less likely to open the survey link, reducing completion rates. Furthermore, while attendance at all sessions was not a requirement for completion of the final survey, some participants may have erroneously thought as much and thus not opened the final survey. Anecdotally, participating educators described other challenges to study and curricular execution. The impact of SARS-CoV-2 (COVID) on GME education has been well described.^31,-,34 Over the course of this study, surges in cases with COVID required rescheduling and reformatting of scheduled resident didactic sessions. Other sites commented that distributing the survey in the final minutes of the discussion session itself yielded higher response rates; however, this practice was not required and likely not applied across all sites.

Statistical inferences must be interpreted with caution. Resident participants were not tracked with unique identifiers; thus statistical tests for independent samples were used. In performing so, statistical power is lost when compared with a paired-participant design with comparisons for dependent samples. In addition, there may be bias in residents' decisions to complete the postsession surveys that could influence the postsession data. For example, attendance and survey participation may have been higher in residents with preexisting interest or confidence in the topics discussed, skewing the postsession results toward showing high confidence impact. Finally, the subgroups used in analyzing these sessions are small. Some of these subgroups still differed from one another at a significant level (e.g., analysis of confidence in movement disorder emergency management) indicating that power in these groups was adequate. However, among those analyses without statistical significance, inadequate power leading to Type II error is certainly possible. Power analysis was not performed before the study.

The theoretical background for the use of podcasts in an FC method is guided by the notion of a podcast as a Learning Object.¹⁹ The Learning Object is often conceptualized as a building block in a greater curriculum, analogous to Lego bricks; they can be reused or fit together with other learning objects to build curricula.³⁵ Typically, learning objects are described as a component of E-Learning in digital learning materials. This aligns well with the podcast, given its portable, reusable, and digital qualities.

Inherent in the definition of Learning Object is the need for context for the learner. To provide that context, the FC instructional method was used. Kolb experiential learning theory informs the use of FC-based instruction. In our model, the podcast/learning object serves as the concrete experience to deliver knowledge, while the discussion facilitates the reflective observation, abstract conceptualization, or even active experimentation components of Kolb cycle.^15,30 Of interest, some evidence indicates that dialog podcasts may begin to facilitate cognitive integration, where discussion between 2 people role models knowledge application.^36,37 While our survey indicated residents appreciated the dialog format, our study is not equipped to assess our podcast compared with a monolog format.

As discussed earlier, while Kolb theory points to experiential learning, the primary outcome measured in this study was confidence, which is an inadequate corollary. To further solidify the use of podcast-based FC instructional methods and to better integrate the Podcast as a Learning Object in this paradigm, future work should assess learning via measured knowledge or even clinical skill acquisition.

Resident educators and neurology learners have various teaching and learning methods at their disposal. Because residency programs find that attendance at in-person didactic sessions wanes, engaging self-directed educational material becomes increasingly valuable.^28,30 Neurology education experts advocate for “together time” as a vital educational component for learners to integrate the complexity of the material.^30,38

In conclusion, this study adds to a growing set of data on the use of podcasts in resident education, contextualized by theoretical frameworks surrounding learning objects and experiential learning. Future studies are needed to measure the educational impact of this or other podcast-based curricula via controlled comparison studies with objective assessment of skills or knowledge.

In GME, learners are shifting increasingly toward immediately accessible digital information.^39,-,41 Educators need to be increasingly agile and facile with educational techniques and methods that simultaneously fit into the learners' clinical schedule and maintain attention and engagement, while helping trainees master complex neurologic topics. Podcast-based FC education may have a role in this educational paradigm shift.