To compare the Problem-based learning (PBL) with the traditional lecture-based curricula.
The single best answer Multiple Choice Questions (MCQ) and the Objective Structured Clinical Examination (OSCE) were used to compare performance of the lecture-based curriculum with the PBL medical student groups. The reliability for the MCQs and OSCE was calculated with Kuder-Richardson formula and Cronbach’s alpha, respectively. The content validity of the MCQs and OSCE were tested by the Independent Subject Experts (ISE). The Student’s t-test for independent samples was used to compare the item difficulty of the MCQs and OSCE’s, and the Chi-square test was used to compare the grades between the two student groups.
The PBL students outperformed the old curriculum students in overall grades, theoretical knowledge base (tested with K2 type MCQs) and OSCE. The number of the PBL students with scores between 80-90% (grade B) was significantly (p=0.035) higher while their number with scores between 60 to 69% (grade C) was significantly p=0.001) lower than the old curriculum students. Similarly, the mean MCQ and the OSCE scores of the new curriculum students were significantly higher (p = 0.001 and p = 0.025, respectively) than the old curriculum students. Lastly, the old curriculum students found the K2-MCQs to be more (p = 0.001) difficult than the single correct answer (K1 type) MCQs while no such difference was found by the new curriculum students.
Suitably designed MCQs can be used to tap the higher cognitive knowledge base acquired in the PBL setting.
The Problem-based learning (PBL) model is now a well-established learning method, where students take center stage in case-based, self-directed learning.
Numerous systematic reviews and meta-analyses comparing PBL with the traditional curriculum have failed to provide unequivocal support in favor of the PBL.
The School of Medicine, Kuwait University, admits about 100 students every year and offers a 7-year undergraduate teaching program; 1st year is pre-medical; 2nd to 4th years are pre-clinical, and the 5th to 7th years are clinical. Some students drop out as years progress while some failed students take re-sit exam with the previous year’s class. The faculty curricula, for both preclinical and clinical programs, were reformed in 2005-6. The traditional teacher-centered, didactic lecture-based classroom teaching was replaced with the student-centered, small group, problem-based learning (PBL). The weekly learning objectives (WLOs) for the 8-week psychiatric rotation were identified, and the didactic lectures (N= 24) were replaced with 14 (2 for each case) PBL group meetings. Based on the WLOs, seven PBL cases, comprising psychoses, affective disorders, anxiety disorders, somatoform disorders, organic mental syndrome (delirium), substance abuse, and child psychiatry (attention deficit disorder), were identified. During the daily Case Conference meetings (N=28), the students, under supervision of their respective tutors, made case presentations and demonstrated the clinical findings with live patient interviews.
A cross-sectional descriptive study design was used. The data was accessed from the on-site evaluations of both the old and the new curricula students.
The participants included 91 students (48 males; 43 females) of the 2009/2010- year and 69 students (48 males; 43 females) of the 2011/2012- academic year. The old curriculum students included 16 re-sits while 25 students had dropped out from the new curriculum class. All the old curriculum students were examined only once at the end of the academic year while the new students were examined twice; once at the end of the 8-week psychiatry rotation, and then during the annual examination.
The institutional scientific and research committee was approached for ethical approval which allowed the study to proceed without any further review as it did not involve human subjects or animals and the study data consisted of students’ performance in examinations.
In accordance with the weekly learning objectives, a total number of seven PBL cases were identified. Each PBL case was discussed in two sessions. A typical PBL group, comprising 5 to 6 students, began the session (one) with analysis of the unfamiliar terms and defining the stimulus (case trigger) in a concise statement (describing the nature of the disorder) followed by the brainstorming session. Drawing upon their pre-existing knowledge and engaging in the process of elaboration, the students brought forth different hypotheses to explain the possible underlying (psychopathological) mechanisms for the given clinical case manifestations. This exercise helped students identify gaps in their knowledge in ascertaining clinical manifestations, differential diagnoses, and management of the given case. The students then dispersed to engage in self-directed learning. Using a variety of resources, they sought out and learned additional information which helped to bridge the gaps in their pre-existing knowledge. Additional information containing details of history, clinical manifestations, and investigations was sent to them during this time. In addition, the students clerked and presented patients with similar manifestations during the once daily Tutor-supervised Clerkship sessions and the Case Conferences. This helped them to narrow down the hypotheses (generated during the previous group meeting) to the main diagnosis and formulate the individual management plans for the case. The second PBL group meeting, called the Reporting back session, was conducted six days later. It began with 5-7 minute learning topic presentations by the students. Then, the case review was carried out in which the underlying mechanisms (psychopathology) for the given clinical manifestations were discussed and the management plan formulated. The session ended with the process review when the group members reflected on the productivity of the PBL sessions. The role of the PBL facilitator, a subject specialist, was limited to maintenance of cooperative, productive, and positive group environment. Using his content knowledge sparingly, he helped foster receptive and non-judgmental group culture, encouraged free and frank expression by all group students and ensured that the group remained focused.
Grades* | 2012 (new) N=69 | 2010 (old) N=91 | **p-value | |||
---|---|---|---|---|---|---|
Number | % | Number | % | |||
A | 90%-100% | 3 | 4.35 | 3 | 3.30 | 0.941 |
B | 80-89% | 23 | 33.3 | 16 | 18.4 | 0.035 |
C+ | 70-79% | 38 | 55 | 42 | 46 | 0.338 |
C | 60-69% | 5 | 7.2 | 26 | 29 | 0.001 |
*Four students failed in old curriculum; **Chi-square test
The 2009-10 assessment included 220 MCQs and eight OSCE stations. The 2011-12 examination consisted of 225 MCQs and ten OSCE stations.
It has been argued that MCQs most often test basic cognitive level of the knowledge-base, namely the factual information recall and comprehension with little consideration for the degree or depth of cognitive level of the learning involved.
The clinical skills and professional competency were assessed with the 7-minute couplet-OSCE stations. The clinical competency level of the OSCE stations conformed to the course objectives and covered a variety of clinical problems including insomnia, suicidal risk, cognitive impairment, delirium, depressed mood, hallucinating patient, panic disorder, somatization disorder, psychoeducation and counselling for patients and their families. Each station was followed by a couplet, pen-and-pencil station in which students were required to answer questions concerning diagnosis/management of the clinical problem, assessed in the previous station.
Items | Mean ± SD | *p-value | |
---|---|---|---|
2012 (new) | 2010 (old) | ||
MCQ | 67.92±9.7 | 61.11±6.8 | 0.001 |
OSCE | 79.91±5.1 | 76.91±10.1 | 0.025 |
*Student’s t-test for independent samples
Independent subject experts (renowned senior professors visiting as external examiners) cross-checked the MCQs and categorized them into K1 or K2 type. Most (104/225, 79.6%) of the new curriculum questions were classified as K2 type while most (144/220, 80.0%) of the old curriculum questions were categorized as K1 type. The performance of the students on both K1 and K2 type MCQs was compared to see if performance differed in the two groups.
The MCQs internal reliability coefficient, indicator of the homogeneity of test items,
The data were analyzed on SPSS, version 20. The Student’s t-test for independent samples was used to compare the post-hoc mean item difficulties between the old and new curricula MCQ scores The Chi-square tests were used to compare the grades for the two groups of students.
Curriculum | n* | Measured Item Difficulty | **p-value | |||
---|---|---|---|---|---|---|
K1-Items | K2-Items | |||||
n* | Mean ± SD | n* | Mean ± SD | |||
Old | 220 | 144 | 0.68 ± 0.24 | 76 | 0.56 ± 0.26 | 0.001 |
New | 225 | 121 | 0.67 ± 0.26 | 104 | 0.64 ± 0.25 | 0.545 |
p-value | 0.693 | 0.036 |
*Number of MCQs; **Student’s t-test for independent samples
The reliability coefficients of the new and the old curricula MCQs were 0.8 and 0.72 respectively, which lie well within the acceptable range. The Cronbach’s alpha values for the OSCE were 0.75 and 0.82, for the new and the old curriculum, respectively, which again were well within acceptable limits.
The number of the PBL students with scores between 80-90% (grade B) was significantly (p = 0.035) higher while their number with scores between 60 to 69% (grade C) was significantly (p = 0.001) lower than the old curriculum students (
The post-test item analysis showed that the old curriculum students found K2-questions to be significantly (p = 0.001) more difficult than K1-questions while no such difference was found by the new curriculum students (
Our findings suggest that the PBL-based curriculum students performed significantly better than the didactic lecture-based curriculum students both in theoretical knowledge base (K2 type MCQs) and clinical examination (OSCE). The proportion of the new curriculum students with top grades was significantly higher while their number with lower grades was significantly less than the lecture-based curriculum students. Moreover, the old curriculum students found K2 questions to be more difficult while the new curriculum students found no such difference between the K2 and K1 type questions. Similarly, the mean OSCE scores of the new curriculum students were significantly higher than the old curriculum students.
The PBL-based curriculum has been shown to be more effective in promoting clinical competency with little or no impact on the theoretical knowledge base.
Since the K2 type MCQs measure comprehension, analysis, integration, and application of knowledge,
Our findings are consistent with the previous studies reporting significant improvement in the PBL-driven curriculum students’ deeper understanding of the knowledge base; greater student satisfaction, and more positive student attitudes and perceptions of education.
It is important to mention some shortcomings of the study here. First, although mainly driven by PBL, the delivery of our curriculum involved complementary teaching activities including the tutor-supervised, small group clerkship sessions; teaching OSCE sessions; and the Case Conferences, which served as additional learning forums for the students. The PBL has generally been construed as a general construct with little consideration for the complexity of its implementation and the multiple factors likely to affect the outcome of this approach.
In summary our experience of replacing the traditional didactic lecture-based with the PBL-driven, curriculum has shown promising results. The synchronization of the different teaching methods, namely aligning the tutor-supervised clerkship sessions and the Case Conferences with the PBL case for the given week, in addition to promoting the PBL processes, resulted in deeper understanding, and superior performance, of our students in the broad knowledge base and clinical competency. This study contributes to the understanding of different educational approaches and describes the usefulness of K2 type MCQs as a reliable and valid tool to evaluate efficacy of the PBL. In particular, it suggests a novel approach to compare the outcome of the PBL with the traditional lecture-based teaching in undergraduate medical students. Further methodologically sound studies are needed to establish the usefulness of the K2 type MCQs to measure the higher cognitive level and deeper understanding of the broad knowledge base, attributed to the PBL curricula.
The authors declare that they have no conflict of interest.
K1/K2 type multiple choice questions