Academic achievement of students tutored by near-peers

Objectives: To compare the academic achievement of medical students tutored by near-peers and medical students tutored by faculty. Methods: A retrospective comparison study was conducted. In a total of 36 courses, 24 medical student groups were tutored by either faculty members or near-peers, from 2005 to 2010. To compare academic achievement students we used the test scores for individual courses and a combined overall standardized score for all courses together. Results: A total of 1201 and 8722 students were tutored by near-peers and faculty members, respectively. Of 36 courses, the mean test scores of five courses were higher for faculty members tutoring and the mean test scores of 29 courses were higher for near-peers tutoring. Additional analysis of standardized test scores showed that students who were tutored by peers outperformed students who were tutored by faculty members (t(9921) =5.345, P<0.05; Cohen’s d=0.17). Conclusions: Our data suggest that junior medical students are not put at a disadvantage when being tutored by senior medical students. It appears that near-peer tutoring has at least similar benefits as faculty tutoring.


Introduction
Peer teaching and peer assisted learning are methods of instruction that are increasingly being used in higher education in general as well as medical education. 1,2Reasons to consider incorporating peer teaching into medical education curricula are several 3 and include theory-based assumptions that students benefit from being taught by near-peers and that student-tutors themselves benefit from the act of teaching peers. 4The focus in this paper is to further substantiate the argument that being tutored by a near-peer does not negatively affect learning.
Theory-based notions that students would benefit from being taught by near-peers, i.e. students who are somewhat more advanced, can be roughly divided in two hypotheses.6][7][8] These hypotheses are that cognitive congruence and social con-gruence between teachers and students enhance their learning.The cognitive congruence hypothesis posits that students more readily understand explanations of teachers who have a semantic network that shows a similarity of thinking and understanding and of difficulties of comprehension. 6"Unconscious competence" has been widely used as a term to signify how experienced experts may have developed their expertise in a way that their knowledge has become tacit and even inexplicable. 9This may hamper their ability to explain content matter in a way that novices can comprehend.In contrast, teachers who have only recently gone through the phase of acquiring knowledge may more easily reach novice learners with their knowledge.The social congruence hypothesis refers to assumed similarities in social and motivational aspects of learning.Teachers close to learners may better understand their circumstances and Correspondence.Th.J. ten Cate, Center for Research and Development of Education , University Medical Center Utrecht HB 4.24, PO Box 85500, 3508 GA Utrecht, The Netherlands.Email: t.j.tencate@umcutrecht.nl their motives to put or not put effort in studying, and from that perspective could be more successful in stimulating them to learn. 6tudents being tutored by a "congruent" senior student may benefit, but the experience of an expert teacher adds a different value that is unquestionable.Historically, the expert teacher has the necessary fund of knowledge, both because she/he possesses the relevant information, but also because of the didactic choices he or she makes to flexibly provide this information in the right time to the right learners.In the past decades, two developments, visible in medical curricula, have changed this traditional teacher role.One is the arrival of problem-based, usually smallgroup teaching methods.Many medical curricula, predominantly in the western world, have adopted problem based learning (PBL) or variants of this method to a lesser or greater extent.One characteristic of PBL is the change from an information-providing lecturer to a coaching role for the teacher. 10The other, a more recent development, is the arrival of the internet as a quick and convenient source of expert information, making education less dependent on personal expertise of teachers.Medical educators have often debated whether a good teacher must be a knowledgeable content expert or primarily an expert in facilitating student learning. 11In the latter case, conceivable that a teacher with a less content expertise could be called an 'expert' teacher.Near-peers tutoring younger students in the same curriculum they went through may have a sufficient level of content expertise to facilitate student learning.
In our study, based on administrative information collected over a 5 year period in various curricular modules that were led by either faculty tutors or near-peer tutors (NPTs), we sought to establish whether students tutored by near-peers (i.e.medical students of a higher year) received, on average, similar or different scores on final module examinations.
The question whether tutoring by near-peers or by regular teachers differ academically has been addressed multiple times.3][14][15][16] De Volder and colleagues suggested a contradiction of the 'congruence' hypothesis, as in one of three PBL modules studied, students of teacherled groups performed better than those in student-led groups, while both other modules showed no significant difference. 17While other studies report no significant difference 18,19 , some authors found a significant benefit for students taught by peers. 12,20xcept for the De Volder study 17 , all of these experiments were specifically designed to test the hypothesis that students do not learn less if taught by peers and included random assignment of students to one of two conditions.We do not know whether studies have been carried out that have never been published because the results did not support their expectations, which would represent a publication bias. 21This is one source of uncertainty about the effect of near-peer teaching on academic achievement of learners.Another potential shortcoming is that even in randomized experiments, blinding students for conditions is difficult.Students usually know in which condition they are and may be conscious of the fact that they are part of an experimental study.This can affect their efforts before they take the exam, on which date outcome measures are based.At the University Medical Center Utrecht near-peer teaching has become an established practice.Since 2004, about 300 senior medical students have served as near-peer tutors, in problem oriented teaching modules.Students in this curriculum have most of their preclinical education in groups of 10 to 14 students in block-modules of 4 to 6 weeks, guided by tutors with a facilitating tutor task.Of all 24 groups per block period (over 300 students in total), 10 to 15% is tutored by senior medical students taking an elective teacher training rotation 21 .By the end of medical school virtually every student has encountered a sixth year medical student as a group tutor once or more during their preclinical curriculum.This study reports on naturalistic data, collected over a five year period in which near-peer teaching was utilized, that enable a comparison of achievement of junior medical students taught by senior medical students, compared to those taught by faculty.Naturalistic, quasi experimental data reflect actual educational circumstances, which can provide evidence to support the validity of conclusions.
A drawback of such non-randomized design is that teaching conditions are not fully controlled.We cannot know for sure that near-peer tutors were not allocated to systematically better or worse groups of learners.We therefore included a secondary data analysis.Given the fact that group composition remained the same over a number of adjacent block periods, we were able to compare groups who had been previously tutored by near-peers, with groups who had been previously tutored by faculty.This comparison was now done in a next module.We also checked test results of groups before they had a subsequent course with the two conditions of near-peer tutoring and faculty tutoring.This gave the opportunity to investigate whether groups of the NPT condition would also differ from non-NPT groups in other tests i.e. that were not prepared by being tutored in the near-peer condition.If peer-tutored students would outperform faculty-tutored students in previous or subsequent blocks, then the benefit should not be attributed to the peer tutoring but to the fact these student groups are probably just better than other student groups.In this way we partially compensated for the nonrandom allocation of students to conditions.

Methods
Ethical approval for this study was granted by the Ethical Review Board of the Netherlands Association for Medical Education in August 2010.Since 2004, final year students at ten Cate et al.  Academic achievement of students tutored by near-peers the University Medical Center Utrecht's medical school are offered the opportunity to participate in an elective six week teaching rotation.One of the obligations of this rotation is to participate in tutoring junior medical students in regular courses.During this elective, teaching students receive supervision, must read medical education literature, write a paper about medical education, contribute to educational development (e.g.improving a study guide, assist with constructing test items) and discuss a video recording of their own teaching.They are intermittently observed while teaching.A more detailed description of this process was published elsewhere. 22he near-peer teaching task comprises predominantly tutoring small problem-oriented group sessions with 10 to 14 students in multidisciplinary integrated modular blocks of four to six weeks.The teaching generally comprises the tutoring of a full series of 8 to 10 problem-oriented sessions for one or two of the (usually) 24 groups during a block.The tutoring includes guiding a student group through the content matter when necessary, explaining subject matter when necessary, leading discussions and providing instructional guidance.Generally, this task is much like any tutor role in problem-based learning.
In rare cases skills training is involved or lecturing to a small group; these courses were not included in this study.

Preparation of students for teaching
All sixth year students who start tutoring have had a oneweek teacher training course with an emphasis on small group teaching, including one teaching session with preparation and debriefing afterwards. 23During the teaching rotation NPTs receive guidance in a weekly session with one to six other NPTs who take the elective simultaneously.Individually they are coached by regular experienced teachers of the module in which they serve as a tutor.Typically, in 6 to 10 of the 30 hours this experienced teacher is silently present.During at least six hours (i.e. three sessions), student-tutors must be observed and evaluated with a written report including feedback.

Courses
Most of the near-peer tutoring is done in multidisciplinary integrated block courses with groups typically meeting twice a week.
All courses in the medical curriculum of the University Medical Center Utrecht, in which near-peer teaching occurred (January 2005 through April 2010), were screened for meeting inclusion criteria for this study.These course criteria were: (1) mandatory course; (2) end-of-course written examination that all students take simultaneously; (3) the exam is administered within one week after the series tutorials; (4) some groups are tutored by near-peer tutors while the other groups have faculty tutors.Exclusion criteria were (a) courses for which data were missing; (b) courses without a regular examination; (c) courses in which near-peers tutored occasional sessions of a group, and (d) preclinical courses immediately preceding a clerkship, with postponed examinations.Also, courses with a final grade composed of different parts, e.g. a paper and an assessment, were excluded.
The blocks involved all have a study guide and a guide for teachers, usually prepared by a multidisciplinary team."Circulation" and "Metabolism" are examples of modular blocks that were included.All tutors are attached to one of more groups, and faculty come from different preclinical of clinical disciplines, e.g.Circulation could include anatomists, cardiologists and physiologists.Consequently not all faculties would necessarily always be an expert in all content areas of each session.
In the same period 168 sixth year students took an elective student teaching rotation and served as NPTs.Faculty teachers tutoring student groups all have a university degree, most have a doctorate degree in medicine, some are in training for a specialty or a PhD degree and they come from a wide range of preclinical and clinical departments of UMC Utrecht.The exact number of faculty tutors cannot be given, as the registration is not complete.The total number of faculty teachers involved in the preclinical courses over the five years can be estimated between 400 and 500.

Student population
The population of learners were all regular students taking curricular courses in program year 1 to 3 between 2005 and 2010.

Reliability estimates
At the end of each included course students take a written examination consisting of multiple-choice questions, open answer questions or true-false questions or combinations of these.The reliability of most of these examinations is calculated and regularly reviewed by a test quality committee.Internal consistencies (Cronbach's alpha) could be identified for most but not all courses between 2005 and 2010.Most tests show alphas between 0.60 and 0.70.For example, the registration of psychometric data in the academic year of 2007-2008 reveals an average value for Cronbach's alpha of 0.67 for all registered 27 tests in program years 1 through 3. Most tests had additional openended questions that were not included in the psychometric documentations.The calculation of psychometric values is not fully complete, but the tests seem on average to show reasonable Cronbach's alpha reliabilities.
All scores for these examinations are calculated on a scale from 1 to 10 and administered in the school's database, which was used to retrieve information for this study.As outcome measure we used the mean scores of learners tutored by NPTs versus the mean scores of learners tutored by faculty.

Data analysis
First, mean scores were calculated across all students within each group.For each course, the mean score for NPT-led and faculty-led groups were compared.We did not test for significance per block because of the large number of blocks and comparisons.Effect size statistics, using Cohen's d, were assessed to show the magnitude of the difference between the means of each group.The value of Cohen's d can range from -1.0 to +1.0.An effect size (d) of 0.2 is considered small, 0.5 medium, and 0.8 or higher large. 24omparisons resulting in a negative effect size equate to a higher mean score for learners tutored by faculty; positive effect sizes imply a benefit for NPT-tutored learners.The average effect size for all courses was calculated.
Next, all scores for students in all courses were transformed to Z-scores (i.e., with mean score of 0, standard deviation set to 1).A student's t-test for independent groups was then performed to compare the standardized scores for all learners in NPT-tutored and faculty-tutored conditions, to draw a general conclusion.
To exclude bias because of accidental allocation of NPTs to 'good' groups, we conducted a secondary analysis comparing the same student groups' exam results in adjacent courses.If in these adjacent courses near-peer tutoring also happened for some groups, we then excluded these NPTtutored groups from the analyses.Student's t-tests for significance and effect sizes were calculated.Data were analysed is SPSS version 15.0.Effect sizes were analyzed by a web-based calculator. 25

Courses and students
From 2005 to April 2010, a total of 36 courses met the inclusion criteria.The teaching rotation was introduced in September 2004, but we were unable to retrieve information on group numbers, tutored by the eight NPTs between September 2004 and January 2005.Included were 12 different courses taught in program year 1 to 3 for different cohorts, and sometimes a course was taught more than once per year.The number of groups tutored by NPTs varied from one to seven per course; in total 46 NPTs participated in these courses.The remaining groups in each course were tutored by faculty.The mean group size in each tutorial was 14 students.
Examination results of 9923 learners were included.Of these, 1201 had been tutored by a near-peer tutor, 8722 were tutored by faculty tutor.

Per course analysis
For each course, the learners were divided into two groups, one tutored by NPTs and the other tutored by faculty.For each group a mean score was calculated.The means and standard deviations allowed us to calculate differences of means in terms of effect sizes (Table 1).Five courses showed a negative effect size (all small, d=-0.01 to -0.19); two courses showed no detectable difference; twenty-nine courses showed positive effect size (ranging from small to large: d=0.1 to d=0.8).The mean effect size was +0.23, which means that on average for all courses the groups tutored by near-peer tutors showed higher scores than faculty-tutored groups.

Per student analysis
To compare the scores of all students tutored by a NPT versus those tutored by a faculty tutor, a Z-score for each student for each exam was calculated by transforming mean scores per exam to zero and the standard deviation to 1.A mean positive Z-score of 0.151 (SD=0.96)was calculated for the group tutored by a near-peer tutor and a mean negative Z-score of -0.016 (SD=1.00)for the group tutored by a faculty tutor.There was significant difference between the two groups (t (9921) = 5.435; P<0.05), but the effect size was small (d=0.17).
We were unable to randomly allocate sixth year students to groups of learners, and therefore could not guarantee that NPTs had not been allocated to the 'better' groups.
We investigated this accidental possibility.In fourteen courses it was possible to compare the scores of learners, who had been taught by near-peer tutors, with the scores of learners who had been tutored by a faculty tutor in a directly adjacent course (the course directly preceding or the course directly following) without the same tutoring conditions.In these secondary analyses, we excluded the groups that were, at that time, tutored by a near-peer.These fourteen comparisons constituted seven pairs of courses.In all these adjacent courses, other sixth year students acted as tutors, but for different groups.For seven courses, seven subsequent courses could be used as controls.For the other seven courses the previous course could be used as a control.For example, students from different conditions in course A were compared on their scores in course B, and students from different conditions in course B were compared on their scores in course A.
In most of the fourteen secondary comparisons we indeed found that the groups who had recently been, or were about to be, tutored by an NPT in an adjacent course, scored slightly higher than their faculty-tutored counterparts (Table 2), but none of these differences reached significance.

Discussion
We found that near-peer teaching, on average, did not lead to a lower level of achievement than faculty teaching.Indeed, most differences found were in favor of those learners who were tutored by NPTs.
We draw this conclusion with some caution, as we cannot exclude that some allocation bias has occurred.The students who belonged to groups that were tutored by near-peers appear to do slightly better in other courses too, ten Cate et al.  Academic achievement of students tutored by near-peers but his benefit was nowhere significant.We do believe it is safe to say that near-peer tutoring does not lead to a decrease in academic achievement if compared to faculty tutoring.Many studies that compare teaching methods show no difference on outcome measures such as regular tests. 26,27One important reason is that students are motivated to pass a test.McLeash suggested that students adapt their study effort when preparing for tests if the teaching was felt inadequate 28 which means regular tests as outcome measures of teaching comparisons may be invalid. 29We did not measure study effort, and cannot compare groups on this variable, which leaves us with a somewhat black-box nature of the causes of the effect found.The least we can say that at the end of the day, being tutored by a near-peer does not negatively affect academic achievement.Even if nearpeer tutoring has led to a higher study effort, it is questionable whether this should be viewed as an adverse outcome, as education also has a function to stimulate students to spend effort in studying.To our knowledge this is the first long-term naturalistic study that establishes information on academic achievement of students taught by near-peers, compared to achievement of students taught by regular teachers.Its naturalistic nature excludes a possible "Hawthorne" bias, that may result from the fact that subjects are aware that they participate in an experiment. 30We cannot exclude publication bias with the literature reports of studies in peer teaching, and one strength of our study is that we have used all available data since this model of near- peer teaching was introduced in our school.In some courses faculty-tutored groups did better and in other groups student-tutored groups did better.We conclude that final year medical students, in a role of tutor for small group teaching, can at least provide as high quality education for more junior medical students as faculty.We also know that students like to be tutored by them. 16,22,31Is this surprising?In some way it is.Faculty may be considered more experienced in teaching and it may seem slightly disappointing that these faculty teachers, with their richer experience, their deeper knowledge and training in teaching on average do not outperform student teachers, at least not in terms of learners' outcomes.Conversely however, the near-peer tutors in a teaching rotation were closely monitored, had all of their time available to teach and prepare for it, had taken a short teacher training course 23 and had voluntarily chosen this elective.If faculty had this opportunity and would be willing and able to invest as much energy, the outcomes may have been different.This may be considered as a threat to the validity of our finding.Faculty in our University Medical Center Utrecht are motivated for teaching, but teaching for many of our faculty is not a primary focus, compared to patient care and research.Additionally, some faculty do not feel as comfortable tutoring a multidisciplinary small group as they would feel in didactic teaching about their own specialty.The findings may thus reflect the circumstances that stimulate and motivate teachers to do their teaching, rather than their status as faculty versus senior student.But it may be questionable whether faculty can ever receive such intensive coaching and be so fully available for teaching as our senior medical students in the elective teaching rotation.Engaging senior medical students in teaching may be an undervalued, but useful building block of a medical curriculum.Teaching others those things, that one has learnt not long before, consolidates ones knowledge and prepares for future cognitive development.Therefore, intermediate trainees may deliberately be entrusted with teaching responsibilities as a natural part of their training which may also accelerate their own learning. 3We believe this can be done with no negative effect on junior learners.
Our findings have not uncovered the reasons or mechanism why near-peer tutoring seems to 'work' satisfactorily, and hypotheses of cognitive and social congruence [4][5][6][7] are neither confirmed nor invalidated.It would be worthwhile to be able to design a future experiment with experienced teachers, receiving similar guidance as teaching rotation students and then determine whether the congruence hypotheses still hold.

Table 1 .
Mean scores of learners tutored by near-peer teachers versus faculty members (N=10723)

Table 2 .
Comparative scores of students in the NPT-or faculty-condition on other, adjacent exams (N=14) ten Cate et al.  Academic achievement of students tutored by near-peers