This prospective comparative study was performed at the University of Würzburg, Germany, during the winter semester of 2018/2019. The whole cohort of third-year medical students (n=145) was enrolled and randomly sorted into two groups. While learning objectives and teaching content were widely congruent in both groups, different didactic methods were used for knowledge transfer: Students in group 1 attended a seminar with role-playing. Students in group 2 completed an online course with a video example.

The newly designed smoking-cessation training was implemented in the existing teaching module on "prevention and health promotion". This is a compulsory subject in medical students' education at the University of Würzburg. Students' knowledge of the course content was assessed in an obligatory written exam at the end of the sixth semester. When considering ethical issues, we became aware that one didactic format might be more successful in transferring knowledge, which could lead to an advantage for this group in the written exam. These considerations and the study plan were presented to but waived from the submission by the ethics committee of the faculty of medicine. The students council and the centre of general medicine at the University of Würzburg advocated the project.

Objective and subjective criteria of knowledge, skills, and attitude were measured with different instruments of data collection: Questionnaires, questions in a written exam, and an objective structured clinical examination.

The web-based module and the seminar both focused on transferring application knowledge and practical counselling skills, following the 5A's approach. National and international guidelines recommend using the 5A's (ask, advise, assess, assist, and arrange) for brief verbal interventions in clinical practice.^33,34 This counselling model is a valid and widely used guide that has been shown to increase rates of quit attempts and smoking cessations.³⁵

Both interventions were designed as a one-time 90-minute course and comprised modules on knowledge and skills. Structure and content were identical. The same graphics and explanations were used in both interventions. Because of the different teaching methods, students in group 1 actively performed in a role-play, and students in group 2 passively watched a video example of counselling a smoking patient. The setting, patient's motivational status, and doctor's task in counselling were identical in the two didactic approaches.

Knowledge module (groups). After conveying basic knowledge on tobacco smoking (e.g., epidemiology, health consequences, physical and psychological aspects of tobacco addiction) and emphasising the role of doctors in tobacco cessation, different options of drug- and non-drug therapy were presented. The Fagerström test was introduced as an instrument to estimate patients' nicotine dependence.¹ We focused on 5A's model, a motivational approach for short verbal interventions. Individual steps of counselling were explained, and phrases for patient-focused communication were suggested.³³

Skills module - seminar (group 1): The classroom course was divided into small role-play groups of three students each. Each student was randomly assigned one of the prescribed roles of "doctor", "patient", or "observer".  Following the 5A's model, the "doctor" conducted a short counselling interview with the "patient", who is willing to quit smoking but needs professional assistance to successfully implement their vague plan. After seven minutes of counselling, the "observer" gave structured feedback (2 minutes). Three instructors were available for questions and provided additional feedback if necessary. The roles were changed for the second and third round of role-playing.

Skills module–online-course (group2): Participants watched a 14-minute video of a patient-physician conversation following the 5A's model.³⁶ Single steps of this model were presented and directly linked to a realistic and concrete example for counselling a tobacco-smoking patient. The video could be halted or repeated individually. Comparable to the function of the "observer" in the role-play setting, a background voice in the video commented on the doctor's approach in the conversation.

Presentation tools: Graphics and central messages during the oral presentation in the seminar were shown in a standard PowerPoint presentation. In the online course, we used the presentation software Prezi for transferring knowledge and skills. As Prezi allows to produce mind maps, the overall structure of the course remains clear and comprehensible even if some topics are explained in detail. We proposed watching the slides in a linear sequence, but students were able to leave the suggested path if they preferred thinking and learning in a different individual way.

After the intervention, all students were given access to a pocket card. This pocket card summarises the 5A's model and the Fagerström test for nicotine dependence.

During the introductory lecture for the winter semester 2018/2019, students were informed about the newly implemented teaching module and the accompanying research. They were asked for consent to participate in this educational study.

Every student in the seminar was given a certain date for the course (weeks 9, 10, and 11), while students in the online course group had access to the course for three weeks (weeks 9-11). Because resources were limited, the number of OSCE participants was calculated based on results from Stolz and colleagues.¹⁵ To provide a significance level of .05 and a power of 80%, the sample size for the OSCE was set at 19 students per group. Paper- and web-based questionnaires as well as the OSCE evaluation form were designed and digitalised using EvaSys 7.1.

Pre-interventional questionnaires were filled at the beginning of the semester (T1). Post-interventional questionnaires were filled directly after completing the intervention (T2). Students in the seminar were given printed questionnaires, students in the online-course followed a link to an online questionnaire covering the same content. OSCE (T3) and written exam (T4) were performed one week after all students participated in the intervention. To match the questionnaires at different points of time, students indicated a personalised six-digit code on each questionnaire. Students were only included in the analysis when a linkable pair of questionnaires at T1 and T2 was available, and OSCE results could be matched to an existing pair of questionnaires.

The program SPSS 26.0 was used to conduct statistical analyses. Descriptive methods such as mean, median, and standard deviation were used to show data. Potential group differences in sociodemographic items for categorical data were assessed with the chi-square test (χ²), and continuous data such as age with Welch's t-test or one-way Welch's ANOVA, as appropriate.³⁷

OSCE: Each item at the OSCE was evaluated by two examiners (graduate and undergraduate). The mean rating for each item for every OSCE participant was calculated. The means of each item were summed section by section (e.g., general communication, ask, and advise). When a value in one checklist, was missing only the second examiner's evaluation was considered. Sectional and overall percentage scores were calculated. To test the primary hypothesis, group differences in students' OSCE performance were addressed with the Mann-Whitney U-test. This test was chosen as data in the online group were not normally distributed (as determined by visual inspection) but homoscedastic. The same test procedure was used in previous studies on tobacco cessation training.^14,15 Sociodemographic and OSCE score associations were addressed with Spearman's rho correlation. The inter-rater reliability between graduate and undergraduate raters regarding the overall OSCE score was measured using the intraclass correlation coefficient (ICC). ICC (1,1) was chosen because each subject was assessed by a different set of randomly selected raters.³⁸

Questionnaire: To test our secondary hypothesis, pre-and post-ratings of self-assessed knowledge and attitude for every item were analysed using a paired t-test. Although some knowledge items violated the assumption of normality, this test was chosen as it is robust to nonnormality in large sample sizes (number of participants: 130).³⁹

Learning-gain differences in knowledge and attitude between the seminar and web-based group were analysed with Welch's t-test.³⁷ Learning gain was calculated based on a formula in which ceiling effects in knowledge growth are considered.⁴⁰

Except for the primary directional hypothesis, all tests were two-tailed. Test results were assumed to be significant if p<.05. Effect sizes were calculated with Cohen's d.^41,42

The overall OSCE scores were higher in the seminar group (Mdn = 70.8 %; IQR = 60.9%–76.2 %) than in the web-based group (Mdn = 62.8%; IQR: 55.9%–71.7%), but this difference was not significant (U=119, p=.087). When we analysed the individual sections of the 5A counselling model, the seminar group performed slightly better in four of the 5 As (exception: “Assess”). As shown in Figure 1, a significant difference was only found in the section “Assist” (Mdn = 66.7% (IQR: 47.2% –77.8%) vs. Mdn = 51.4% (IQR: 38.2% – 68.8%); U=110, p=.049, d= 0.57).

Overall school grades given by the raters were not different between seminar group (Mdn=2) and web-based group (Mdn=2), U=145, p=.582. There was a statistically significant strong positive correlation between the mean subjective global rating and the mean overall OSCE scores r₍₃₆₎=-.707, p<.001. Graduate and undergraduate raters assessed the students' OSCE performance equally (r₍₂₂₎=.704, p=<.001). The reliability measured by the ICC was 0.844 (95% CI 0.652 - 0.935), which indicated a good agreement.

The SPs rated counselling and motivational skills significantly better in the attendance-course group (M=4.7) than in the web-based group (M=4.2), t_(27.836)=2.0, p=.028, d= 0.67.

Students’ age (r₍₃₆₎ =.183, p=.285) and sex (r₍₃₆₎=.117, p = .495) was not significantly correlated with the OSCE scores. Because only the web-based group included participants with smoking experience, OSCE scores within this group were analysed depending on smoking experience: Overall OSCE scores for students without smoking experience (Mdn= 62.8%, n=14) and with smoking experience (Mdn=66%, n= 4) were not statistically significantly different, U=21, p= .457.

Students in the seminar group felt more confident in addressing tobacco smoking among patients, taking patients' smoking history and counselling them following the 5A's approach. Nevertheless, this self-assessed counselling skill did not vary significantly between the two groups (t_(123.946) = 1.321, p=.189).

Knowledge: Both interventions increased self-assessed knowledge in all five items significantly (p<.005, see Table 2). The didactic method had hardly any influence on knowledge growth: learning rates were equally high in both groups (see Table 3). Students reported significant learning gain in knowledge about risks, nicotine dependence, the 5A's model (two items) and the variety of therapy options (one item; multiple choice). 78.5% (n=102) of the students stated that they had learnt theoretic and application knowledge for counselling smokers mainly in this course. Students reported the highest learning success in the item "order of counselling steps (5A's model)". The lowest knowledge gain was achieved in the item on "risk awareness". However, more than one-third of the participants (37.7%, n=49) already fully agreed before they participated in the course about the risks of smoking. In the written test (T4), the majority answered two out of three questions correctly (42.1%, n=59). Only 17.1% (n = 24) of all students were able to answer all three questions correctly. Results were similar in both groups, t_(137.917)=-0.688, p=.493.

Attitude: The overall attitude towards smoking (six items) was not affected by course participation, t₍₁₂₆₎=0.072, p= .943. Even before they participated in the intervention, 120 students (93%) considered smoking to be a chronic addiction disease and recognised the important role of doctors in smoking cessation treatment. Nevertheless, after the course, only half of the students (n=72) were convinced that doctors could effectively influence patients' smoking behaviour. Consistency in attitude towards smoking was found in both groups. The didactic form had no influence, t_(122.180)=- 0.321, p=.749.

In addition to testing the primary and secondary hypotheses, the newly implemented course was evaluated. Both interventions were rated equally as “good”, (t_(126.549) =- 0.305, p=.761). Prior to the intervention, students in both groups preferred the online module to the seminar. After the intervention, more than 80% in both groups were satisfied with the didactic form they were assigned. Few students reported technical problems with the online module, all of which could be solved via e-mail support. Students in group 1 spent 90 minutes of educational time on the seminar, while those in group 2 completed the web-based training in 74± 25 minutes, t_(53.000) =4.795, p.001. Afternoon (39%) and evening (30%) were the preferred times for working on the online course.

All third-year medical students were enrolled, and response rates were continuously high. The participating group, therefore, appears to be representative for the entire cohort of students this year. The two interventions were conducted simultaneously, and all participants were equally advanced in their studies. As the teaching module was newly implemented, students' opinion and exam preparation were not influenced by more advanced fellow students. Except for the post-interventional questionnaire (T2), all data in both groups were collected at the same time. Information-sharing between students therefore hardly affects the results.

The teaching times in both interventions in our study are comparable. We collected exact data on time spent during the online course. Some studies avoid making an exact statement on the time spent online because no data have been collected or because educational times vary widely between different didactic approaches.¹⁵ It often remains unclear whether possible differences in knowledge and skills are due to different teaching methods or to additional educational time.

Our study has some limitations. As the curriculum at the Würzburg medical school only allows offering events to a clearly defined semester, the maximum sample size was predetermined by the 145 students in the current semester. Out study was conducted with a relatively homogenous group of students at a single medical school. The enrolled group is characterised by a high portion of female and non-smoking students. A comparison to other sociocultural environments or healthcare systems might therefore be limited.

Students in the online course watched a video in English of a counselling talk, but the role-play was conducted in German language. In the OSCE all students were required to express themselves in German. Students in the online-course group had heard some key phrases only in English before. This might have influenced their OSCE performance negatively.

The voluntary OSCE was performed with 19 students per group, a relatively small sample size. Students participating in the OSCE were significantly younger. A selection bias of highly motivated students into the voluntary OSCE group cannot be ruled out. The sample size calculation was based on a study of Stolz and colleagues¹⁵ that used a similar design but had one crucial difference: Stolz and colleagues compared a group (n = 35) using a web-based module alone to a group (n = 31) using a web-based module plus additional role-playing for tobacco cessation training. This second group spent considerably more learning time, which might have contributed to a great difference between the groups and to the large effect size.

In our study participants completed the same tobacco cessation training, either as a web-based module or as a seminar. This might have reduced the differences between the two groups' OSCE performance compared to the study of Stolz and colleagues. It can thus be suggested that our sample size was too small, and our study was underpowered. Based on a calculated effect size of d=.44 and a sample size of 18 participants per group, the achieved power was indeed only 36%. To gain a power of 80%, all 130 students should have been included.

Finally, we did not address the problem with multiple comparisons. If the level of significance were adjusted to the number of p-values calculated, some group differences, for example of the item "Assist", would no longer be assumed significant.

Gaining knowledge and practical counselling skills is essential for treating patients who smoke and should become an integral part of undergraduate medical education. Because curricula are crowded, and the opportunities of web-based learning grow constantly, medical schools need to determine the best-suited didactic approach to teach smoking cessation successfully and efficiently. Results of this prospective study show that teaching online is a successful and time-efficient didactic method in smoking cessation training.

The inverted classroom model might include specific advantages of both didactic formats. Following this approach, factual knowledge is transferred online and can be trained and applied in a subsequent attendance phase.⁵³ Because our two teaching modules are well examined and are proven to be effective, they might be adjusted and combined into an inverted classroom course. The effectiveness of this teaching approach could be compared to results from our study.

Finally, the long-term effects of these two teaching interventions are still unknown. To this end, we are currently working on a follow-up study to assess long-term results of students' knowledge and skills in smoking cessation two years after completing the module.

We would like to thank all students and simulation patients for their participation in this study, especially in the OSCE. We would also like to thank Christoph Müller, who helped create the questionnaires.  This publication was supported by the Open Access Publication Fund of the University of Wuerzburg. Other expenses were covered by the Department of General Practice, University Medical Centre Würzburg.

The authors declare that they have no conflict of interest.