Students ’ evaluation indicators of the curriculum

Objectives: The purpose of this study was to examine indicators and methods that the allied health students use for evaluating the curriculum quality and teaching. Methods: A cross sectional study was conducted. Questionnaires were developed and administered. A total of 383 students at the Nanjing Medical University participated in the first round to identify initial indicators. A total of 41 medical educators ranked the identified factors in order to construct the questionnaire for evaluating the curriculum. A total of 2148 students completed the questionnaire in the second round. To evaluate the psychometric properties of the questionnaire, factor analysis and the reliability coefficient were used. Results: The initial indicators yielded in factor analysis constituted 16 factors in the first round and were reduced to 10 by medical educators with an internal consistency reliability of 0.79. The correlation coefficients of 10 indicators were found to be less than 0.45 with an average of 0.27. Students valued the autonomous learning ability as a key indicator for teaching and learning (r = 0.65; p < 0.0005). Conclusions: The result shows that the identified indicators are valid and reliable to measure the quality of the curriculum. The importance Chinese students placed on autonomous learning may suggest new initiatives to new curriculum.


Introduction
Curriculum plays a key role in education.Since the late 20 th century, along with the introduction of western curriculum theories and the reform of Chinese higher education curriculum, the importance of curriculum evaluation in improving the quality of education and training has been increasing.Medical schools across the world used different approaches in curriculum evaluation to make informed decisions regarding the effectiveness of the components of the curriculum and outcomes and any future actions related to revising it. 1,2Curriculum evaluators collect reliable data in order to examine how well the curriculum is operating. 3lthough medical education literature described the process of curriculum evaluation using quantitative and qualitative methods, most of them reported information obtained from educators' point of view and less attention paid to students' perspective.In addition, students normally evaluate the effectiveness of their own teachers and were given little opportunity to comment on the curriculum.
Curriculum planners have a key role in the evaluation process and students input in evaluation of the curriculum have been overlooked. 4,5In fact, students, as the direct learning agents and beneficiaries, are supposed to be more directly involved in the evaluation of curriculums.Ideally, monitoring and evaluating the curriculum should be based on students, teachers, and administrators input. 6On the other hand, most of the available curriculum evaluation reports have focused on one subject area of interest and a comprehensive evaluations of all modules on the curriculum were rarely carried out. 7,8 erefore, the purpose of this study was to examine indicators and methods that the allied health students use for evaluating the curriculum quality and teaching.This study was based on a combination of 27 courses which can reflect the curriculum quality in a macro-level rather than a specific perspective.Considering students' perceptive in the evaluation process of the curriculum may improve the Correspondence: Wen Tang, Centre for Educational Evaluation and Assessment, Nanjing Medical University, China.Email: wtang@njmu.edu.cnquality of teaching and learning in our universities.Information provided by students can show how well they have achieved, the learning outcomes and their attitudes toward the curriculum and teaching.

Indicator generation and Participants
A cross sectional design was used for the study.The study was conducted in Nanjing Medical University, China.We produced a preliminary 34-item questionnaire using the literature review and consultation with experts and students to evaluate the curriculum quality and teaching.In the selected items for the questionnaire, the preliminary questionnaire was distributed to students and they were asked to rate the curriculum quality on 9 courses (Physiology, Biochemistry, Anatomy, Pathophysiology, Microbiology and Immunology, Diagnostics, General surgery, Internal Medicine, and Surgery).The questionnaire was returned by 383 students in the fourth and fifth year (284 fourth and 135 fifth years).Of these, 108 students were Clinical Medicine (seven-year program), 190 Clinical Medicine (five-year program), 28 Stomatological program and 57 Nursing program.Using factor analysis, the number of items was reduced from 34 to 16.These items are considered as performance indicators of the curriculum evaluation.
In order to establish the content validity of the questionnaire for evaluating the curriculum, we asked 41 experts to rank the 16-item questionnaire, graded as very necessary, quite necessary and not necessary.The questionnaire items were reduced from 16 to 10 based on medical educators input.Among experts, 15 were specialists either in clinical medicine or nursing, 14 in basic science, preventive medicine and pharmaceutics, 3 in humanistic medicine and 9 in medical education management.Sixteen experts were recruited from other universities.

Procedures
To collect data, the 10-item questionnaire was embedded in the university website.Medical students rated each item based on 27 courses on a 3-point scale ranging from 1 (disagree) to 3 (agree).These courses were Basic Chemistry, Physics, Cell Biology, Physiology, Systematic Anatomy, Histoembryology, Organic Chemistry, Pathology, Pathophysiology, Microbiology and Immunology, Morphological comprehensive experiment course, Functional Experimental Course, Pharmacology, Diagnostics, General Surgery, Surgery, Paediatrics, Gynemetrics, Bioethics, Medical Psychology, Hygiene Toxicology, Statistical Epidemiology, Medical Statistics, Epidemiology, and Occupational Hygiene.A total of 2270 students filled in the web-based survey (no student claimed about ethical issues or conflicts) and 2148 valid questionnaires were collected, however, 122 incompletely papers were considered invalid (effective returns-ratio is 94.63%).Of these, 187 students were Clinical Medicine (seven-year program), 1747 Clinical Medicine (five-year program) and 214 preventive program.Further demographic analysis showed that 305 of these were freshman, 232 sophomores, 689 juniors, 593 seniors and 329 students in their fifth year.On average each course was assessed by approximately 79 students.

Statistical analysis
Data were analyzed using the statistical software SAS, version 9.1.3and STATA, version 9.2.The principle component analysis (PCA) was run on items to extract the number of factors.For factor rotation, the oblique Promax rotaton was employed.The correlation coefficient was run to investigate the relationships between the items.The reliability of the questionnaire was assessed using the Cronbach's alpha coefficient.

Validity
Using factor analysis, 16 common factors were selected from the 34-item questionnaire that account for 88.64% of the variance.The common factors identified were in good agreement with our expectations. 10Experts selected 10 factors out of 16 as key indicators for evaluating the quality of curriculum (Table 1).The PCA produced 9 factors that accounted for 95.29% of the variance (Table 2).From Table 2 we can see that each factor contains one item (indicator) with factor loadings greater than 0.9786.Table 2 also shows that there is no overlap of the significant factor loadings for the items, providing each factor measures one construct which is discriminant from other factors.Table 3 shows the correlation coefficients of 10-item questionnaire.Table 4 shows the item means, standard deviations and item-total correlations for students.Item-total correlations ranged from 0.52 to 0.73.Although the mean score for item 9 (training students' autonomous learning ability) was ranked second from the last (Mean = 7.20), its item-total correlation (the correlation between the item score and the overall assessment score) is quite high (p < 0.0005).Item 9 was also ranked second.It is of note that there was a statistically significance between item 9 and other items (p < 0.0005).

Reliability
The Cronbach coefficient alpha of the preliminary 34-item questionnaire and the 10-item questionnaire was 0.92 and 0.79 respectively, indicating a satisfactory reliability.

Discussion
The quality of curriculum teaching is essential in higher school education.It is of great theoretical and practical importance to improve the teaching outcome through the construction and implementation of scientific and specific curriculum evaluating system.The purpose of this study was to examine indicators and methods that students use for evaluating the curriculum quality and teaching.Since every aspect of teaching is involved in the design of Stu- Our results show that the students have a very good recognition and expectation of their teachers' teaching methods with regards to self-directed learning.Therefore, it might be argued, that the training of autonomous learning ability as an indicator for evaluating can be enhanced if we increase the weight of the indicator and urge its importance in modern education.In addition, we need to encourage medical educators to motivate students to work and learn by themselves. 12At present, among all course evaluation related literature we haven't found any quantitative study of evaluation indicators aiming at fostering medical students' self-study capacity.For example, the authors searched MEDLINE (1970-December 2011) and PubMed to identify all studies of curriculum evaluation and self-study or autonomous study toward undergraduate medical students.Of the 163 studies identified initially about curriculum evaluation, only one concerned the student's attendance. 13herefore this study is a meaningful effort made in the research and practice of course evaluation.
Another survey of this study showed that students' evaluation of the curriculum quality varied with the experts', and that the national top-level courses, evaluated by experts, and the general curriculum showed no statistically significant difference in students' evaluation.(Mean score for national top-level courses = 75.53;for general curriculum = 76.41;p > 0.05).It is indicated that in the course evaluation, the evaluation indicators should be analyzed from the unique perspective of physical and psychological characteristics of college students, which is of important theoretical and practical significance to help us to obtain more comprehensive and valuable feedback from the teaching.
In conclusion, experimental results show the 10-item questionnaire is brief, easy to administer, valid and reliable and it can serve as students' evaluation indicators of curriculum.This selection of students' curriculum evaluation indicators illustrated essential factors related to the curriculum teaching process such as has been added curriculum planning, teaching effectiveness, and the quality of teaching material, teaching methods and etc.The most important feature of the above evaluation indicators is its facilitation on students' autonomous leaning ability.However, this study was conducted only in one medical college at present, and its widespread usability needs further validation.

Table 1 .
The 10 -item questionnaire as indicators of students' evaluation indicators of the curriculum by experts

Table 2 .
Loading factors of the Principle Component Analysis after oblique rotation

Table 3 .
Correlation coefficient of 10 items/indicators

Table 4 .
Mean, standard deviation, item-total correlations and weights on the 10-item questionnaire