All second-year medical students attending the University of Toledo College of Medicine and Life Sciences were eligible to participate in the study. The trial was explained in detail to the students, and they were assured of the anonymity and confidentiality of personal information for all responses. The University of Toledo Institutional Review Board gave approval for the study. A total of 176 medical students were eligible to participate. Ten students were unable to participate due to infection with COVID-19, leaving a total of 166 students who participated in the SPE's.

We undertook a mixed-methods study to determine the impact of a peri-simulation educational video on student performance and learning in two simulations. A cross-over type design for the video intervention was implemented, in which students either received the video before the first SPE (SPE-1) or the video between SPE-1 and the second SPE (SPE-2). A pre-test multiple-choice questionnaire (MCQ) was administered prior to SPE-1, and a post-test MCQ was administered after SPE-2, followed by a voluntary feedback survey and a debriefing of the experience with a faculty physician. The SPE scenarios encompassed a patient with asthma and a patient with chronic obstructive pulmonary disease (COPD).

A two-stage randomization strategy was implemented, as shown in Figure 1. First, students were randomized into 32 groups of approximately equal size (5–6 students each) by faculty independent of the research team. Assignment of students and the randomization process was concealed from the students and the research team. The 32 groups were then randomized into four subgroups (A, B, C, D). The four subgroups (A, B, C, and D) had a pre-randomization of the location of the simulation room and the SPE scenario.

Subgroups A and C both received video interventions before the first simulation scenarios and rotated thru the same environment: room, medical manikin, faculty, and simulation operator. Subgroup A was given the COPD simulation scenario first, followed by the asthma scenario and subgroup C was given the asthma simulation scenario first, followed by the COPD scenario. Subgroups B and D both received video interventions between the simulation scenarios and rotated thru the same environment: room, medical manikin, faculty, and simulation operator. Subgroup B was given the COPD simulation scenario first, followed by the asthma scenario and subgroup D was given the asthma simulation scenario first, followed by the COPD scenario.

There was a total of four operators running the SPE environments. The operators ran the same case in the same room and manakin for the duration of the study. The operators were given a standardized training session for their respective cases to keep the language and overall atmosphere the same for each group.

The two SPE scenarios and the information the students were expected to obtain were developed by faculty, independent of the research team, as part of the standard undergraduate medical education curriculum. The SPE's were a formative, non-graded group activity designed as an opportunity for medical student learning.

The students were given the following information on a display screen prior to starting the SPE: "You will have 15 minutes to conduct a focused history and physical exam. You will present the case afterwards as you will do on your clerkship rotations."

The COPD case represented a 61-year-old male, one pack/day smoker since he was 15 years old, with worsening dyspnea over the past two to three days, acutely worse the morning of the presentation. Relevant high-yield objective information given to the students when prompted included a temperature of 37.1 °C, heart rate of 124 beats per min, respiratory rate of 28 breaths per min, blood pressure 149/94, pulse oximetry oxygen saturation of 84%, forced expiratory volume in one sec to force vital capacity ratio 60% of expected, brain natriuretic peptide level of 88 pg/mL, initial arterial blood gas pH of 7.3, pCO2 of 55 mmHg, pO2 of 55 mmHg, bicarbonate of 26 mmHg, and imaging consistent with acute COPD exacerbation.

The asthma case represented a 10-year-old boy with dyspnea after playing outside. Students were notified that he was visibly distressed in the waiting room, where he was bent over with his hands on his knees. Relevant high-yield objective information given to the students when prompted included a temperature of 37 °C, blood pressure of 104/68 mmHg, heart rate of 110 beats per min, respiratory rate of 30 breaths per min, pulse oximetry oxygen saturation of 84%, forced expiratory volume in one sec to force vital capacity ratio 65% of expected, pulmonary function test after beta-agonist therapy 85% of expected, initial arterial blood gas pH of 7.37, pCO2 of 55 mmHg, pO2 of 65 mmHg, bicarbonate of 22 mmHg, and imaging consistent with acute asthma exacerbation.

Simulation performance scores were calculated using separate standardized checklists for the COPD and asthma scenarios, respectively, as shown in Table 1. The checklists were modified versions of those used for the graded objective structured clinical examinations performed as part of the standard undergraduate medical education curriculum.

Two faculty physicians performed checklist modifications for the SPEs. It encompassed key aspects of the history of presenting illness (HPI), physical exam, assessment/plan/interventions (A/P/I), and the number of times the facilitator had to prompt students to keep the sim moving forward. There was a total of 44 checklist items for SPEs, with each checklist item worth 1 point.

Individual baseline student knowledge was assessed by completing a multiple-choice question (MCQ) pre-test examination before SPE-1, as shown in Appendix 1.At the conclusion of SPE-2, students were re-assessed using a post-test MCQ, which consisted of the same ten questions and answers (ranging from two to five possible answers) as the pre-test. The MCQ points per question were weighted based on difficulty as pre-assigned by a faculty physician on the study personnel. There were three questions weighted to be worth 2 points, one question worth 3 points, and the remaining six questions each worth 1 point, totaling a maximum of 15 points available. The MCQ examination was intended to measure student learning separate from the dimension of clinical performance. Expert physicians verified the content validity of the MCQ. However, no statistical certification of MCQ reliability was performed.

Students had the option to voluntarily complete an anonymous 5-point Likert Scale feedback questionnaire at the conclusion of the MCQ post-test. The questionnaire was composed of three questions intended to gather students' perspectives on the impact of the intervention, debriefing, and overall simulation training experience on their learning.

Two blinded independent evaluators viewed video recordings of the SPEs. Both reviewers were 4th-year medical students who were trained independently by a faculty to ensure the accuracy of the assessment. The reviewers were blinded from the intervention arms of the groups were a part of, and they were blinded from whether the group was in SPE-1 or SPE-2. The evaluators recorded whether the students performed the checklist items for the respective Asthma and COPD scenarios. A third independent reviewer compiled the two reviewers' checklists and found no instances in which the independent reviewers differed in their scoring. The MCQ pre-and post-tests, along with the feedback questionnaire, were performed using an online platform which recorded each student's responses. Faculty placed the students' scores into their respective groups such that individual students' scores remained deidentified, and performance scores could not be traced back to the individual name of the student.

The primary objective was to compare the performance effect as measured by an adapted analysis of student performance in a group setting of simulated patient encounters. The hypothesis that the Intervention = 0 was tested by means of student paired t-tests and independent t-tests. The scores were expressed as means plus or minus standard deviations (SD). The mean and SD of the within-group difference captures the treatment effect and the paired nature of the design; these were used as the basis for constructing the difference between the means confidence intervals and hypothesis testing. This analysis approach made two main assumptions (in addition to the normality assumption for the student's paired t-test): no period effect and no intervention-period interaction. Secondary analysis was performed to compare the sub-categorization of the scoring checklist, compare the multiple-choice questionnaire pre-test and post-test, and compare the COPD versus asthma performance scores from SPE-1 and SPE-2. All statistical tests were two-sided, and a p-value ≤ .05 indicated statistical significance. All statistical tests were performed using Microsoft Excel.

The cohort that had the educational intervention between SPE-1 and SPE-2 had a significant improvement in their performance in SPE-2. On further analysis, this improvement was mostly attributable to higher accuracy in the HPI subcategory. The improvement among the students with the educational intervention was most likely due to the intervention itself, as the analysis of all groups showed no significant difference between the performance on SPE-1 and SPE-2 (Table 2). When comparing the performance scores across the two SPE's for those who received the video intervention prior to SPE-1, there was no significant difference. This suggests the intervention had made the impact already, and that impact was retained in SPE-2.  Furthermore, this may indicate that the score improvement from the video intervention between cohort was not due to repeated exposure and practice.

The performance of the students in the SPEs showed the educational intervention had a positive impact on student performance. The students with the educational intervention prior to SPE-1 scored higher than those without the intervention, and although the educational intervention between-group scored higher on SPE-2, there was not a statistically significant difference between the intervention arms. This suggests that the intervention prior to the SPE-1 had a positive impact on student performance across both SPE-1 and SPE-2. When assessing the group performance by comparing the scores from COPD vs asthma, there was no significant result, suggesting that the cases themselves did not impact the group performance scoring assessment.

Both cohorts had a significant reduction in the number of times they needed to be prompted to continue progressing through the simulated patient encounters, which is most likely attributable to the experience of the simulation itself rather than the educational intervention. As previously mentioned, patient simulation is an experience where students are encouraged to try and fail. It often takes students an initial period to familiarize themselves with the simulation environment and be comfortable enough to progress through various portions of the simulation assertively. Thus, it is not surprising that students needed to be prompted less in SPE-2 as they were likely less apprehensive about the simulation experience in general.

Overall, both cohorts had similar baseline results from the pre-test multiple-choice questionnaire, which suggests that baseline knowledge did not impact the results. Furthermore, both cohorts had statistically significant improvement from the pre-test to post-test MCQ results. This lends to the fact that the exercise had a positive impact on student learning.

The major benefit of the cross-over design in this cohort is that it allowed within-group comparison, which would not have been possible in a conventional parallel-group design. In addition, this allowed all students to receive the educational intervention as part of the simulation experience. While a control group that did not receive the educational intervention in any respect would have been optimal to understand the exact impact of the educational intervention, regardless of its temporal placement, this was not ethically possible as every medical student needed to have the same amount of instruction during the SPE day.

There are several limitations to the study that we felt were worth noting. Our study used a crossover-type design for the intervention, rather than one group receiving the intervention and one no-intervention group. Although a true control group may provide a stronger argument, this design was used in an attempt to provide all the students with the same overall learning experience. Additionally, the simulated patient scenarios may have differed depending on the order in which the students' gathered information and asked questions. Although the operators of the case scenarios underwent training to standardize the experience, the presentation of the cases may have differed slightly between the groups from intrapersonal and interpersonal differences between the two operators running each simulation case. Despite simulation rooms being standardized for patient positioning and equipment necessary for the experience, there was a slight variation in the overall atmosphere. This difference included fixed structural variations of the rooms as they were built as either an operating room, a labor and delivery room, an emergency department room, or an intensive care unit room, which may have impacted how students approached the simulated patient. Finally, the SPEs in our study were formative, non-graded activities. As such, this may have impacted students' engagement in the SPEs and therefore their performance.

We would like to give a special thanks to the following for their contributions to the study: Karla Dixon, the Lloyd A. Jacobs Interprofessional Immersive Simulation Center staff, and The University of Toledo College of Medicine and Life Sciences.

The authors declare that they have no conflict of interest.