The Relationship between Student Perceptions of Tutor Performance and Tutors' Background in Problem-based Learning in South Korea

7 © 2011 Eun-Kyung Chung et al. This is an Open Access article distributed under the terms of the Creative Commons Attribution License which permits unrestricted use of work provided the original work is properly cited. Abstract Objectives: This study was conducted to investigate student perceptions of tutors' group facilitation skills and to evaluate the difference in student perceptions of tutor performance according to the tutors' background in problem based learning (PBL). Methods: This study used a cross sectional design. One hundred fifty third-year medical students at Chonnam were asked to assess their tutors' performance at the end of each PBL tutorial using a self-administered questionnaire. The information collected in the questionnaire addressed tutor performance with regard to constructive/active learning, self-directed learning, contextual learning, collab-orative learning, and intrapersonal behavior as a tutor. Tutor background information, such as gender, age, basic or clinical science qualifications, faculty or non-faculty appointment status, and PBL tutor training program attendance was collected from secondary data provided by the school administration. We performed multiple linear regression analysis using the total student perception score as the dependent variable to compare student perceptions of tutor performance according to the tutors' background. Results: The mean score for the 11 items on the questionnaire varied between 4.03 and 4.17 on a 5-point Likert scale. The assessment of student perspectives on tutor performance revealed that students have positive perceptions of tutors' performance in PBL, particularly tutors who were faculty (β = 0.255, p = 0.035) and those who had participated in the PBL tutor-training program (β = 0.224, p = 0.046). Conclusions: The study results indicate that exploring how tutors' characteristics influence their performance can be used in planning PBL tutor recruitment and designing PBL tutor-training programs.


Introduction
Tutors in problem-based learning (PBL) tutorials have a complex role to play in facilitating students' learning.This includes providing support for students' acquisition of content knowledge and skills in critical thinking, coaching of group processes and modeling of reflective practice. 1ecause the tutor plays a central role in PBL, the characteristics and skills of effective tutors have received relatively more attention in medical education research.The most common focus of inquiry regarding tutor performance in PBL has been whether the tutor should be an expert in the content matter related to the problem under study.3][4] Studies investigating process differences between content expert and non-content expert tutors have found that content expert tutors tend to use their subject-matter expertise more to direct the discussion in the tutorial group, whereas non-content expert tutors use their process-facilitation expertise more to direct the tutorial group. 5tudies investigating the related factors to tutor performance have found that a tutor's performance is also dependent on the quality of the cases, structure of PBL courses, link with students' level of prior knowledge, and the functioning of tutorial groups. 5Therefore, there may be other factors that contribute to a tutor's influence on student learning in a tutorial setting, and more research is needed about the relation between tutor characteristics and the tutorial group process.Furthermore, there has been a lack of studies exploring the relationship between tutor performance and tutor background in PBL. 6ble 1.Tutors' general characteristics (N=86) Although tutor performance can be evaluated with selfassessment or by other individuals, such as students, peers, or examiners, desirable tutor skills from a student point of view have received considerable attention.Dolmans et al. reported that students can provide tutors with useful information to improve their performance. 7By contrast, Papinczak suggested that, within the PBL tutorial environment at least, regularly evaluating tutors can create mistrust and confusion among the medical school, the tutor and the students on several levels; she recommended enhanced information provision and frank discussion to avoid such problems. 8PBL approaches for medical education have been in use in Western countries for more than 40 years, but its use in Asian countries commenced quite recent.Because of cultural differences, the perception of tutor performance from the perspectives of students may be different. 9he purpose of this study was to investigate student perceptions of tutors' group facilitation skills and to evaluate the differences in student perceptions of tutor performance according to tutors' background in PBL.

The PBL Curriculum and Study Participants
The PBL program consisted of three, 2-hour sessions per PBL tutorial designed for third-year medical students at Chonnam National University Medical School.Students attended two small-group discussions and a colloquium.Students attended tutorial groups with approximately 10 students, and the discussion focused on the problems.During the discussion, learning issues emerged that required further self study.Between the tutorial meetings, students individually searched for information about these learning issues.In the next tutorial meeting they reported to one another what they found, synthesized the acquired information, and applied it to the problem.In the colloquium, some students gave presentations, and the case writer provided additional information when necessary.One hundred fifty students, divided into 14 small groups, participated in nine PBL tutorials.Eighty-six tutors guided the individual tutorial groups.Each tutor taught an average of 1.5 groups.All tutors were given the opportunity to participate in the study and agreed to be subject to the student evaluations.The tutor characteristics are presented in Table 1.Of the tutors, 71 (82.6%) were male, and 46 (53.5%) were over 40 years of age.Furthermore, 66 (76.7%) were clinical physicians, and 72 (83.7%) held academic faculty appointments.Although each tutor was obligated to attend a faculty development seminar focusing on the principles behind problem-based learning and tutoring during the study period, 68 (79.1%) tutors had already attended these trainings.The Institutional Review Board determined the study exempt from human subjects' research regulations.Consent was obtained from all participants.

Data Collection
This study used a cross sectional design.The authors first provided opportunities for students to learn more about tutor evaluation objectives and strategies, and conducted the survey.One hundred fifty third-year medical students were invited to assess their tutors' performance at the end of each of nine PBL tutorials, they were asked to anonymously indicate the degree to which they agreed with each statement on a Likert scale ranging from 1 to 5 (1 = strongly disagree to 5 strongly agree) using a self-administered questionnaire.The number of students participating in each tutorial and their response rate are presented in Table 2.The average response rate was 85.1%.The student-centered learning approach of PBL means that for tutors, content knowledge is secondary to proficiency in group facilitation. 10Tutors are required to facilitate active learning by students, foster critical thinking skills, and increase a commitment to continued learning. 11Thus, this study focused on the evaluation of tutors' group facilitation skills, using the questionnaire developed and validated by Dolmans and Ginns. 12Two of the authors who are medical education expert and speak fluent English and Korean translated this questionnaire and verified the accuracy of translation.The questionnaire consists of 11 statements related to tutor performance in containing 4 learning factors and 1 behavior factor: constructive/active learning (3 items), self-directed learning (2 items), contextual learning (2 items), collaborative learning (2 items), and intrapersonal behavior as a tutor (2 items).The authors calculated the average scores for the five factors then summed the scores of 11 statements for total students' perception.The Cronbach's alpha reliability coefficient was 0.734, and the 95% confidence interval was 0.670, 0.796.Background information about the tutor was collected from secondary data provided by the school administration.These data included gender, age, basic or clinical science qualifications, faculty or non-faculty appointment status, and PBL tutor-training program attendance.This information was not revealed to the students, except for the tutor's gender.

Data Analysis
Mann-Whitney U tests and Student t-tests were used to compare student perceptions of tutor performance according to the tutors' background.We performed multiple linear regression analysis using the total student perception score as the dependent variable.All analyses were performed using SPSS software version 17.0 (SPSS, Inc., Chicago, IL, USA).

Student perceptions of tutor performance
Of the five factors measured, the students gave the highest scores to intrapersonal behavior as a tutor and the lowest to stimulating self-directed learning.The mean score for the 11 items varied between 4.03 and 4.17.The highest scoring item was for tutors' knowledge of their strengths and weaknesses.The lowest scoring items were a tutors' ability to stimulate students to understand the underlying mechanism/theories and a tutors' ability to stimulate students to search for various resources (Table 3).

Comparison of student perceptions of tutor performance and background
Table 4 summarizes the results of the univariate analysis according to the tutors' background.
Mann-Whitney U tests of relevance of academic appointment revealed the faculty tutors had significantly higher scores than did the non-faculty on the contextual and collaborative learning, intrapersonal behavior as a tutor and total students' perception scores.Those who participated in the PBL tutor-training program had significantly higher scores on self-directed learning scale and total students' perception scores.The tutor's gender, age and qualifications were not significantly associated with student perceptions of tutor performance.

The relationship between student perceptions of tutor performance and tutors' background in PBL
The multiple linear regression analysis indicated that the total students' perception score was positively associated with tutors who were faculty (β = 0.255, p = 0.035) and those who had participated in the PBL tutor-training program (β = 0.224, p = 0.046).This model was statistically significant (F = 2.5, R 2 = 0.137) (Table 5).

Discussion
Tutors have an important role in facilitating learning in PBL. 13 Our study was designed to investigate student perceptions of tutors' group facilitation.The mean score for the 11 facilitation skill items varied between 4.03 and 4.17, as compared to Dolman et al., who found the mean score varied between 3.37 and 4.20.In both studies, the lowest scoring items involved how to simulate students to search for resources.An important implication of this finding is that faculty development should place more emphasis on how to guide students in seeking information, prioritizing and planning their approach, making decisions, assessing, interpreting, evaluating, comparing and weighing evidence, and using resources. 14he second purpose of this study was to evaluate the relationship between student perceptions of tutor performance and tutors' background in PBL.Students rated tutors who were faculty and those who had participated in the PBL tutor-training program higher than non-faculty and those who had not participated in PBL tutor-training program.Faculty tutors scored significantly higher than non-faculty tutors in three of five factors including self-directed learning, contextual learning, and collaborative learning.Those who had participated in the PBL tutor-training program received significantly higher scores on contextual learning and intrapersonal behavior as a tutor.Baroffio et al. 15 found that participation in tutor workshops improved participants' understanding of the PBL principles, repertoire of teaching strategies, knowledge of problem content, and ability to guide student learning.Also, Jafri et al. 16 found that medical residents are an effective supplement to faculty members in facilitating PBL sessions, providing specific education in teaching methods such as facilitation of the PBL modules and participation in teaching workshops.Our study contributes to the body of research that highlights the importance of faculty development programs that provide training in effective tutoring techniques.
Furthermore, the most common area of inquiry in the domain of desirable PBL tutor characteristics concerns whether a tutor should be an expert in the content matter under study.9] In our study, tutors' qualifications were not significantly associated with students' perception of their performance.
The findings in this study should be interpreted considering the following limitations.First, one of the inherent limitations of educational research using a questionnaire is the role of subjectivity.Second, our study was limited by our focus on the characteristics of successful tutors.A successful tutor may not necessarily be an effective tutor.To evaluate effective teaching by a tutor, student achievement scores and other outcome indicators should be used as a basis for assessment.Third, the small sample size and single institution survey limited the statistical power and affected the statistical significance of the results.Fourth, further research to include other factors that can contribute to tutor performance should be undertaken.

Table 4 .
3Comparison of student perceptions of tutor performance and background (N=86)  Constructive/active learning,  Self-directed learning,  Contextual learning, ‡ Collaborative learning, ¶ Intrapersonal behavior as a tutor

Table 5 .
Multiple linear regression analysis using total students' perception score as dependent variable (N=86) Despite the aforementioned limitations, the study findings suggested that students had a positive perception of tutors' performance in PBL, particularly for tutors who were faculty and those who had participated in the PBL tutortraining program.This result can be used in planning PBL tutor recruitment and designing PBL tutor-training programs.Faculty who participated in PBL tutor-training programs must be recruited as a tutor and effective faculty development programs that are tailored to tutors' needs about their ability to guide students' learning (e.g., simulating students to search for resources) should be provided.