Education Research: Appraisal of Outpatient Clinical Experience During Neurology Residency

Background and Objectives Outpatient clinical experience is a key component of neurology residency. Understanding the educational environment for residents in the outpatient setting can inform educators to maximize teaching and learning opportunities, enhance resident exposure to subspecialty diagnoses and management, and deliver quality care. We studied the continuity clinic experience of 5 neurology residents over the course of their residency to determine the breadth of their ambulatory experience. Methods We used administrative health data from new and return patient visits scheduled with 5 neurology residents of the same class over 3 years of continuity clinic. International classi ﬁ - cation of disease codes pertaining to neurologic diagnoses and symptoms associated with these visits were analyzed. Frequency and proportions of the most commonly evaluated diagnoses and symptoms were tabulated. These were compared with previously published data about resident experience during training. We also analyzed resident experience over time.


Methods
We used administrative health data from new and return patient visits scheduled with 5 neurology residents of the same class over 3 years of continuity clinic.International classification of disease codes pertaining to neurologic diagnoses and symptoms associated with these visits were analyzed.Frequency and proportions of the most commonly evaluated diagnoses and symptoms were tabulated.These were compared with previously published data about resident experience during training.We also analyzed resident experience over time.

Discussion
The clinical experience of residents in the continuity clinic was diverse, but it was skewed toward headache, neuromuscular, and movement disorders, which constituted 54% of the workload.When compared with previous studies, the range of resident's outpatient clinical experience differed from that of inpatient experience.Based on the results of this study, we made changes to our outpatient curriculum by adding 2-month-long rotations in subspecialty clinics from postgraduate year 2 to 4 with the aim of boosting resident exposure to neurologic disorders in the outpatient setting.
Postgraduate neurology training takes place in the inpatient, outpatient, and emergency care settings; however, residents spend most time in the inpatient setting. 1 The relative educational value of training in one setting vs the other has not been evaluated. 2It is a commonly held notion that neurologic disorders encountered in the inpatient and outpatient settings are different from each other; however, there have been no direct comparisons.The few studies that assessed trainee experience during residency have shown that cerebrovascular disorders and epilepsy are the 2 most common neurologic disorders evaluated by residents in the inpatient setting. 3,4We studied the neurologic diagnoses and symptoms evaluated by 5 neurology residents in their continuity clinics over 3 years of neurology residency to understand the breadth of their outpatient experience.These data would guide educators in maximizing learning opportunities for residents in the outpatient setting.We described the number and type of neurologic diagnoses encountered in the outpatient setting and its change over the course of residency.We compared outpatient and inpatient clinical experience to explore similarities and differences between the 2 of them.

Structure of the Continuity Clinic
During this study (2015-2018), West Virginia University's neurology program had a complement of 20 residents (5 per year).In line with the Accreditation Council for Graduate Medical Education (ACGME) guidelines, 1 a weekly, half-day continuity clinic was scheduled for each of the 15 residents from postgraduate year (PGY)-2-PGY-4.Eight residents had continuity clinic on Tuesday afternoon, and the rest on Thursday afternoon.Specific faculty members were designated to supervise the continuity clinics.Time spent by the faculty in the continuity clinic represented 10% of their clinical duties.The supervising faculty were subspecialists in headache medicine, neuromuscular medicine, vascular neurology, and epilepsy.Two additional faculty members were available to cross-cover if designated faculty were unavailable.The supervising faculty-to-resident ratio was maintained at 1: 3 for every clinic.New patient visits (NPVs) were scheduled for 60 minutes and return patient visits (RPVs) for 30 minutes at all levels of training.Two NPVs and 3 RPVs were scheduled during each 3.5-hour continuity clinic.The clinic was cancelled for a resident when they were on night float rotation, on vacation, attending a national conference, and on hospital-designated holidays.In addition, for PGY-4 residents, continuity clinic was cancelled when they were on inpatient neurology services.To compensate for these cancelled clinics, 6 half-day clinics were added for each resident during their subspecialty clinic rotations.Therefore, each resident had 47, 50, and 47 half-day continuity clinics during PGY-2, PGY-3, and PGY-4, respectively.The neurology clinic is located on the health sciences campus of the West Virginia University in Morgantown, WV.The catchment area of the clinic consists of the entire state of West Virginia along with southwest Pennsylvania, western Maryland, and the upper Ohio valley.

Data Acquisition and Analysis
We queried the electronic medical record to obtain a list of NPVs and RPVs scheduled with 5 neurology residents of the same class between July 2015 and June 2018.We selected these 5 residents from a cohort of 20 residents-the full complement of residents in the program during the study.The visits represented all continuity clinic patient encounters over 3 years of adult neurology residency training (PGY-2 to PGY-4) for these 5 residents.The following variables were collected for each clinic visit: visit type (new or return), year of visit, patient gender, patient age, associated International Classification of Disease, Ninth and Tenth Revisions-Clinical Modification (ICD-9-CM and ICD-10-CM) disease codes.ICD-10 codes were exclusively used after January 1, 2016.
Using the ICD-9 and ICD-10, one of the authors (A.D.) obtained disease entries corresponding to each code.To facilitate meaningful analysis, each entry was initially categorized as a neurologic diagnosis, a neurologic symptom, or a comorbidity.Neurologic diagnoses and symptoms were the focus of this study and they were further classified.A neurologic diagnosis was placed in one of these 9 groups: cerebrovascular disorders, cognitive and behavioral disorders, demyelinating disorders, epilepsy, headache disorders, movement disorders, neuromuscular disorders, neuro-oncological disorders, and other neurologic disorders (congenital and structural disorders of the CNS, cranial nerve disorders, encephalopathy, injuries of the CNS, neurologic infections, psychiatric disorders, sleep disorders, spinal cord disorders, and miscellaneous neurologic disorders).These groups were based on the ACGME milestones project implemented during this study. 5A neurologic symptom was placed in one of these 11 groups: dizziness, headache, gait abnormality and falls, memory and cognitive symptoms, motor symptoms, pain (including back pain), sensory symptoms, seizure-like events and convulsions, tremor and involuntary movements, visual symptoms, and other neurologic symptoms (abnormal deep tendon reflexes, bladder and bowel symptoms, circadian symptoms, malaise and fatigue, psychiatric symptoms, sequelae of cerebrovascular disease, speech and cranial nerve symptoms, syncope, and miscellaneous symptoms).We also assessed resident experience over the course of their residency to determine trends over time.Deidentified data for the study was available by calendar year (and not by academic year or calendar month).First, we analyzed neurologic diagnoses by calendar year.In a second analysis for trends over time, we divided the 36-month-long continuity clinic experience into two 18-month-long epochs; "the first half" comprising PGY-2 and first 6 months of PGY-3 (July 2015-December 2016), and "the second half" comprising last 6 months of PGY-3 and PGY-4 (January 2017-June 2018).Discrete data were reported as frequencies and proportions (percentage).Epic Electronic Medical Record system (Epic Systems Corporation, Verona, WI) was used by all residents through the duration of this study.

Standard Protocol Approvals, Registrations, and Patient Consents
The institutional review board of West Virginia University approved the study.Participant consent requirement was waived for this retrospective study.

Data Availability
Anonymized data not published within this article will be made available by request from any qualified investigator.
During the first-half of continuity clinic experience, 3 residents (60%) entered more diagnosis-related codes than symptom codes.All 5 residents (100%) entered more diagnosis-related codes than symptom codes in the second half of continuity clinic.When compared with the second half, the resident cohort had more NPVs and total patient visits in the first half of continuity clinic training.The case mix evaluated by residents changed over the course of their training: cerebrovascular and neuromuscular disorders were less often encountered by residents as they advanced through training, whereas the proportion of headache, movement demyelinating, cognitive disorders, neuro-oncology, and epilepsy increased (Figures 2 and 3).

Discussion
In this study of outpatient resident experience during postgraduate training in the United States, a cohort of 5 neurology residents of the same class encountered a variety of neurologic diseases and symptoms in their continuity clinics.However, more than half of the clinical workload was constituted by just 3 neurologic disorders: headache, neuromuscular, and movement disorders.This salient finding is congruent with the natural history of these particular neurologic disorders, which rarely require inpatient management.It is of more interest that the clinical experience of residents in the outpatient setting appears to be noticeably distinct from the inpatient setting.Headache, neuromuscular, and movement disorders, which accounted for 54% of the outpatient diagnosis in our study, made up only 14%-17% of all diagnoses in previously published inpatient focused studies. 3,4On the contrary, cerebrovascular disorders and epilepsy, the most commonly encountered diseases evaluated by residents in the inpatient setting, comprised 19% of outpatient diagnoses in our study (Table 3).This contrast of case mix supports the common notion that outpatient and inpatient clinical experiences of neurologists (including trainees) are relatively different.Although certain neurologic disorders such as cerebrovascular disorders and epilepsy are likely to encountered with similar frequency in either setting, others, such as movement, headache, and neuromuscular disorders are near exclusively encountered in the outpatient setting.This finding demonstrates that a resident's outpatient experience is not just a replication of inpatient experience but adds to it in a significant manner.Similar to our results, a survey of 24 outpatient neurology practices in Colorado found neuromuscular and headaches disorders to be the most common neurologic disorder diagnosed in the clinic. 6This suggests that the range of outpatient clinical experience during residency mirrors real-life practice of neurology.
Over the course of residency, we found that the number of cerebrovascular and neuromuscular disorders encountered by the cohort decreased.However, it increased for all other neurologic disorders.We speculate this may be because the long-term care of these disorders was taken over by primary care providers after they were diagnosed and initially managed in a neurology clinic.
Residency training primarily occurs in the inpatient setting, which is in contrast with general neurology practice, which tends  to be primarily based in the office setting. 2,6Only 6 months of outpatient experience during 36 months of residency is mandated according to current requirements. 1The continuity clinic forms the core of this outpatient experience, with a minimum of 40 half-day clinics required per year throughout residency. 1esident and program director surveys have expressed a desire to increase the amount of time spent in the outpatient training, [7][8][9][10] perhaps reflecting the differences between general neurology practice and neurology training.However, there are several challenges in restructuring neurology residency programs, including an increased focus on revenue generation and not teaching, growth of inpatient neurology service lines leading to  pressure on time spent in the outpatient setting, and the need for direct supervision of trainees by faculty. 11Obtaining robust data that demonstrate the value of outpatient education in neurology education can overcome some of these barriers.
We used administrative health data to gauge resident clinical experience in the outpatient setting.][15] It allows a thorough assessment of resident's clinical experience by capturing the diagnoses and symptoms evaluated during patient encounters.Unlike previous studies, 3,4,16 which only analyzed NPVs, we analyzed both new and RPVs to assess the longitudinal experience of residents-the core goal of continuity clinic.Furthermore, we analyzed all neurologic diagnoses associated with a patient visit rather than focusing on the principal diagnosis or chief complaint.It is not uncommon for patients to experience comorbid neurologic disorders; a patient with a remote history of ischemic stroke who develops focal epilepsy and a patient with generalized epilepsy with comorbid migraine.Managing multiple neurologic disorders in a patient contributes to a trainee's experience and should be reflected in any such assessment.To obtain meaningful information from the data set, and when possible, we grouped diagnoses and symptoms into categories.When possible,  The number of patients evaluated by each resident over the course of residency was low and perhaps inadequate to prepare for independent practice Two subspecialty clinic rotations (1-month long) added each year to the curriculum from PGY-2 to PGY-4 to increase outpatient clinical exposure and supplement continuity clinic experience (implemented) Lack of exposure to primarily outpatient-based subspecialties such as neuro-oncology, sleep neurology, and neuro-ophthalmology we grouped neurologic diagnoses according to categories outlined in the ACGME milestones project during the study. 5Neurologic symptoms were grouped anatomically or according to sensory domain affected.For example, visual symptoms consisted of ICD codes for diplopia, vision loss, decreased vision, and anisocoria.The approach of grouping diagnoses and symptoms may have compensated for the variability among residents in choosing the ICD code for a given complaint; for example, a patient reporting tingling and numbness in the feet can be coded as "paresthesia," "anesthesia," and "disturbances of skin sensation," but enumerating it as a sensory symptom overcomes the differences.We assigned specific diagnoses such as migraine, migraine with aura, migraine without aura, and other primary headache disorders (ICD codes G43, G44) to the neurologic diagnosis category.However, ICD code for headache (R51) was placed in the symptom category.Similarly, we placed epilepsy-focal or generalized (ICD G40), in the diagnosis category, whereas unspecified convulsions (R56.9), was placed in the symptom category.We acknowledge that this system was somewhat arbitrary and may have led to overcounting or undercounting diagnoses and symptoms.
This study has at least 6 major limitations.First, we might have overestimated the neurologic diagnoses and symptoms prevalent in the continuity clinic as we enumerated ICD codes across multiple visits (NPV and RPV) for a patient.On the contrary, there could be an undercount of diagnoses and symptoms because of the onerous nature of the ICD system because residents may not have entered all the pertinent ICD codes relevant to an encounter.This may particularly be the case in the early part of residency.Because only 1 ICD code is required to close and bill a patient encounter, there is no incentive to enter other diagnosis or symptom codes.Furthermore, the role of supervising attending in choosing ICD codes for each encounter was not specifically addressed in our study, and it may have made an impact on the results.Second, this is retrospective study from 1 residency program, and it does not account for interinstitutional differences in disease prevalence, clinic location (urban vs rural), availability of subspecialists in the department, and the presence of other learners (fellows), all of which may affect resident experience.Third, this study evaluated 1 portion of the resident's outpatient experience because in most programs, including ours, residents gain significant skill and knowledge during their subspecialty clinic rotations.Fourth, we did not directly compare the inpatient and outpatient experience of the same set of residents.Fifth, administrative health data have not been previously used in neurology education research.3][14][15] Sixth, the sample size of 5 residents evaluated in this study was small.We studied just a single class of residents (from a program complement of 20) to obtain preliminary data as this topic because it has not been studied in great detail in the past.We intended to use these preliminary data to identify areas of improvement in our program and implement changes to our outpatient resident experience.We are currently collecting data prospectively from a larger cohort of residents in our program.Despite these limitations, this study provides first insights into the outpatient experience of neurology trainees during residency and adding to the limited literature on the topic.Unlike previous studies from the United States, which reported single trainee experience from urban academic centers, we report the experience of a cohort of 5 residents of the same class from a program, which primarily serves a rural population.
We found that continuity clinic experience for the residents may have been diverse, but the total number of patients evaluated by each resident over the course of training may not be adequate to prepare them for independent practice.Furthermore, we found that certain neurologic disorders tend to be referred to subspecialty clinics (neuro-oncology, sleep neurology) rather than to the pool of continuity clinic.In response to these findings, we made changes to our program's outpatient curriculum and structure.The major change was introduction of a mandatory outpatient rotation called the subspecialty clinic rotation.During this rotation, residents get an opportunity to evaluate new and return patients while working with various neurology subspecialists for 2 months every year (PGY-2-PGY-4).The schedule for this rotation is made by the program director to ensure relative equity and diversity of clinical exposure.We are currently gathering data from this rotation to determine objectively whether the changes implemented broadened resident's clinical exposure to neurologic disorders in the outpatient setting.
We also made other changes to our program's outpatient training structure based on the results of this preliminary study, which are ongoing.Currently, we are collecting data prospectively from a larger pool of residents in an attempt to improve the quality of data.At the same time, this would allow us to monitor residents' clinical experience as they advance through training.We are also obtaining data from community neurology practices affiliated with our department to compare the disease mix encountered in a general neurology practice with that of our trainees.Please see Table 4 for a summary of changes to our program based on the results of this study.
There are several knowledge gaps in the subject of resident education and training in the outpatient setting.First, the relative value of inpatient and outpatient training in preparing residents for independent neurology practice is unknown.Future studies comparing resident experience in the inpatient and outpatient settings in the same program during the course of residency may help answer this.Of interest, teaching medical students about neurology in the ambulatory setting compared with inpatient setting was found to be effective. 17econd, within the outpatient setting, the educational value of continuity clinics and subspecialty clinics has not been evaluated separately.Third, differences and similarities in clinical experience among residents at the same level of training within and across residency programs has not been evaluated.
Our data hint at interresident variability; however, these preliminary findings need to be explored further.We suggest that research studies that collect data prospectively across multiple residency programs can fill some of these knowledge gaps.
Beyond its positive impact on resident experience, educational research in the outpatient clinic offers opportunities for quality improvement.For instance, the ambulatory experience of residents considering fellowships that are primarily based in the outpatient setting (e.g., movement disorders, neuromuscular medicine) can be improvised; thus, these residents evaluate patients with such disorders early in their training (PGY-2), so they can explore their interests well before fellowship applications are due.Another potential quality improvement intervention could be longitudinally monitoring trainee progress in the outpatient setting, including diversity, and extracting the most educational value out of it.Residents who are lagging behind their peers in the evaluation and management of certain neurologic diseases can benefit from more patient encounters pertaining to these disorders.Furthermore, educational research in the outpatient setting can inform decisions about restructuring neurology training such that it is in sync with the requirements of modern neurology practice.
During the course of their residency, 5 neurology residents evaluated a wide range of neurologic disorders in their continuity clinics with headache, neuromuscular, and movement disorders making up half of their workload.This clinical experience appears to be different from previous reports of resident experience in the inpatient setting.We implemented changes to our program's outpatient curriculum based on the results of this study with the aim of improving the quality of residents' outpatient exposure during training and supplementing their continuity clinic experience.There is a need to further evaluate the clinical and educational experience of neurology residents in the outpatient setting.

GlossaryACGME=
Accreditation Council for Graduate Medical Education; ICD-9 = International Classification of Disease, Ninth Revision; ICD-10 = International Classification of Disease, Tenth Revision; NPV = new patient visit; PGY = postgraduate year; RPV = return patient visit.

Figure 1
Figure 1 Distribution of International Classification of Disease (ICD) Codes in 3 Major Categories

Figure 2
Figure 2 Neurologic Diagnoses Encountered in Continuity Clinic Over the Course of Training

Figure 3
Figure 3 Types of Patient Visits and Neurologic Diagnoses Evaluated During Neurology Residency When Divided Into 2 Halves Abbreviations: ICD = International Classification of Disease; PGY = postgraduate year.

Table 1
Number (Frequency) of International Classification of Disease Codes Pertaining to Neurologic Diagnoses Entered by Each Resident

Table 2
Number (Frequency) of International Classification of Disease Codes Pertaining to Neurologic Symptoms Entered by Each Resident

Table 3
Comparison of Neurologic Diagnoses Reported in Studies of Neurology Resident Experience in the United States a Data not reported.

Table 4
Systemic Analysis of Study Findings and Changes Instituted to Our Residency Program