A qualitative study of foundation doctors ’ experiences with mini-CEX in the UK

Objectives: To explore foundation trainees’ experiences with the mini clinical evaluation exercise (mini-CEX). Methods: Data were gathered from interviews with thirty foundation year one trainees who had completed a minimum of six assessments with the mini-CEX tool. Interview transcripts were analysed using a template thematic procedure. Results: Trainees reported divergent views regarding the goals and the standard of the assessment. Surface and strategic approaches were the commonly reported approaches to the mini clinical assessment. Assessment standards were frequently subverted in favour of retrospective, ad hoc and absentia approaches. Non-specific feedback was common and explicit action plans were infrequent. All trainees applauded the formative potential of the mini-CEX but there were mixed perceptions of its educational value. Some considered it useful and spoke of their positive experiences in terms of theoretical knowledge, clinical competencies and daily clinical practice. Others considered their experiences unremarkable; and some other respondents felt mini-CEX was valueless, and described it as an administrative burden. Respondents felt that, trainee’s personality, maturity and learning style, ambiguity in the use of assessment result, assessor’s skills and attitude, and the clinical environment, were important to their experiences with mini-CEX. Conclusions: The trainees recognised the potential of the mini-CEX. Adequate knowledge about the assessment standard and guidelines were generally lacking. There was perceived tension between mini-CEX formative and summative purposes. This and other contextual factors probably shaped the trainees’ experiences with mini-CEX.


Introduction
In the UK government initiated Modernising Medical Careers programme, 1 newly graduated doctor trainees spend their initial two years in the foundation programme. 2uring this phase a number of workplace-based assessments (WBAs) are used to assess trainees' clinical competences. 3The mini-CEX is a workplace-based assessment to determine the clinical performance of the trainees, 2 provide concrete evidence of competence and drive doctors' learning. 4,5he mini-CEX was developed and piloted in the USA 5,6 and is considered well suited for formative assessment. 7An evaluator observes a trainee's performance in a normal clinical encounter with a patient for 15 minutes, then the trainee receives immediate feedback to help bridge the gap between actual and expected performance. 8Trainees must complete six assessments over a year in various clinical settings, with cases of diverse complexities, and with different assessors. 8The aim of formative assessment is to promote students' learning but achievement of this aim is not automatic.Research on the mini-CEX has been carried out over the last one and a half decades, and recent systematic reviews have confirmed its strong validity, reliability, and feasibility. 9,10lthough the educational impact of formative assessment is well documented outside the sphere of medical education, 11,12 concrete evidence of its educational impact is lacking in medical education. 9,10,13The purpose of the focus group study was to explore the foundation year one doctors' experiences with mini-CEX at Royal Derby hospitals in the UK.In particular four questions were asked: (1) how did foundation year one trainees at Royal Derby Hospitals react to mini-CEX as a formative assessment instrument?; (2)   Correspondence: Olukayode M. Tokode, Medical Education Unit, The Medical School, University of Nottingham, NG7 2UH, UK.Email: mcxot2@nottingham.ac.uk how did the doctors describe the process of mini-CEX assessment?(3) how did the foundation trainees perceive the contribution of mini-CEX to their education?and (4) how could the process of mini-CEX and other emerging themes be used to explain the experiences of the foundation year one doctors?It is anticipated that the result of the study could be fed back to the mini-CEX assessment process to improve instructional effort and experience with mini-CEX.

Methods
The research was conducted in July 2008 at two Universityaffiliated district general hospitals.All the 60 foundation year-one trainees from the 2007/2008 cohorts of house officers in the two hospitals were invited for the study.Fifty (83.3%) trainees responded to the invitation to participate in the study following a reminder email.Twelve (24.0%) of this number dropped out because of annual (3 trainees) and study (2 trainees) leave and difficulty in arranging mutually convenient time for the focus groups (7 trainees).Eight (16.0%) trainees were excluded from the study because of unwillingness (3 trainees) and previous mini-CEX assessment with (4 trainees), and a close professional relationship with the researcher (1 trainee).Thirty (50.0%) foundation year-one trainees were interviewed.Participation in the study was voluntary.
Participants were recruited via the Trust Medical Education Department through emails.A second reminder email was sent to those who did not respond.The focus group moderator arranged a mutually agreeable date and time for the group interviews.The participants were informed about the study, and consent was obtained for their participation and for audio recording the interviews.The study was approved by the Trent Research Ethics Committee (REC.REF/NO.08/H0405/32).

Induction of junior doctors
In the UK, all medical trainees undergo a comprehensive all-day, mandatory induction programme that includes lectures on workplace-based assessments, at the beginning of the year.The trainees are given an information pack that contains their curriculum and the competencies required to complete each level of training.Details regarding the various types of assessments, the number of each to be completed, the appropriate assessors, when the assessments need to be done and how they are to be arranged are discussed.In the UK a satisfactory performance in the Workplace-based assessments is required for academic and professional progress.A struggling trainee requires a remediation programme to progress.

Data collection
Data were collected via semi-structured focus group interviews, which were held in the private rooms of the hospitals, and the interview periods were bleep-free.Thirty first-year foundation trainees (17 women, 13 men) participated in five focus groups.Twenty two participants had entered medical school via the undergraduate route while 8 participants entered via the graduate-entry route.The mean duration of the focus group sessions was 75 minutes.All the trainees had completed a minimum of six mini-CEXs at the time of the study.A total of 196 assessments were completed: with consultant, 109 (55.6%); with specialist registrar, 61 (31.1%); with senior house officer, 11 (5.6%); with research fellow 9 (4.6%); and with specialty doctor, 6 (3.1%).
After a welcome address by the moderator and participant self-introduction, the participants stated the number of mini-CEXs they had performed and the grade of the evaluators.Further interview questions included how interaction with seniors and feedback received during mini-CEX had influenced trainees' learning approaches for the assessment and how their learning has influenced their daily clinical practice.The interview guide (Appendix I) provided organised questions, but flexibility was exercised in framing questions to probe emergent topics.The moderator confirmed points of views by paraphrasing and formulating further questions on the emergent topics.The focus group interviews were digitally recorded with a DS-50, and an external professional transcriber provided verbatim transcription.

Data analysis
Data were analysed using the template thematic procedure.Template thematic analysis involves the production of a template (a list of a priori codes) which is organized in a hierarchical structure 14 , and following the protocol adopted by Meade et al. (2011). 15Template thematic analysis is considered appropriate when the researchers wish to focus on specific aspects of the study outcome as defined a priori. 14The preliminary coding template was developed on the basis of the focus of the study and emerging themes were added as the analysis progressed; the codes were organised hierarchically such that the template contained a number of higher-order themes along with various lowerorder themes.Coding consists of a collection of statements which are transformed into clusters of meanings expressed in psychological concepts.Through careful re-reading of the transcripts and application of the amended drafts to the data, the iterative process of recoding and amending the template progressed until the authors were satisfied that the template had reasonably described the data.

Results
The themes derived from the data analysis fell into five categories: (1) reactions to the mini-CEX, (2) approaches to mini-CEX, (3) educational experience with mini-CEX, (4) possible contribution of contextual factors and (5) suggestions for improvement.

Reactions to mini-CEX
The reactions of the trainees to mini-CEX fell under two main headings: its purpose and value.

Purpose of the mini-CEX
Some trainees considered it formative: "to assess you and then obviously give you feedback to help you to improve"; Some judged it summative: "does it really need to come down to this CEX examination just to be formally assessed?"or "I think mini-CEX is like an exam".
Others were unsure of its goal: 'But I still don't quite understand what they're meant to be about'.
The participants also mentioned that many of the assessors did not possess an adequate understanding of the mini-CEX: "A lot of them just don't know that much about the process, so you have to educate them about what it is before you undertake it"; "so they had no idea".

The value of the mini-CEX
Experiences with the value of the mini-CEX were mixed.In the first group, the participants considered it worthwhile and useful, and spoke of its several positive aspects.Some liked the opportunity the mini-CEX afforded them for interaction with seniors; others indicated that the program gave useful feedback: "It's concrete, it's evidence to say I can do this, or to prove that you are improving as you should be, that you're hitting the right marks in your progression".Some liked the formative potential, citing the 'one-to-one contact' as 'very, very positive'.The second group was much less enthusiastic, with some participants indicating that lack of value may arise from poor execution: "I think, you know, the idea behind it is relatively good, but in practice it's not".
Others perceived the exercise as 'busy work', a 'paper-filling exercise', and a 'false scenario', with another person summing it up as 'very flawed'.
Finally, some participants believed that the mini-CEX may be either an invalid assessment that does not fairly represent the trainee or one of a series of administratively imposed steps to check off.One person summed up the former idea thus: Those who regarded it as an administrative control characterized the programme as part of the 'hoops' one must jump through, or even an 'obstacle', and one person characterized the educational experience as 'minimal, minimal, I would say'.

Approaches to the mini-CEX
The approaches of the trainees to the mini-CEX will be discussed under three broad sub-themes: learning behaviours; assessment organisation; and feedback.

Learning behaviours
The trainees described five approaches to their learning: repetition; pre-assessment reading; post-assessment reading; self-directed learning; discussion and questioning; and risk avoidance.Firstly the trainees indicated that they used familiar cases for their assessments which their seniors had examined and on which they had seen clinical signs elicited: "you already know why they're in hospital because you've been part of the team that's looking after them".
They mentioned that performance in this setting is by rote and represents a false self-presentation: "so you're presenting a bit of a false picture of yourself".

Others viewed the mini-CEX as an examination that one needs to prepare for to get high marks:
"what you'll do is you'd read up on it, and you'll do it, and it'll be all right, and someone says you've done well, tick the box, there you go".Some other participants reported that they followed up the action plans reached during the mini-CEX: "I read up the clinically relevant bit"; "it's something that I did, go into the books and try and understand".

While others ignored them:
"… the anatomy of the epidural and how it works; that's something that needs to be read up on, but so far it's not something that I've done".
However, some trainees utilised the mini-CEX for meaningful self-directed learning by leading exploration of improvement opportunities with the assessors.This they do by requesting ideas for improvement: "I would ask him is there an appropriate text that he would advise me to read.I felt that was a good way of improving myself"; and engaging in meaningful discussions and questioning that deepened their knowledge and understanding of the case chosen for assessment:

"We discussed the score the patient got, what the diagnosis was and what you might do if the score was higher or lower"; "I mentioned the saturation of oxygen, and then I got questioned upon how saturation of oxygen's measured and the accuracy of it".
Finally, some of our respondents avoided the risk of low grades by having the assessment conducted by their friends: "It's the same for everyone else because I'm sure they'll choose people that are close to them, who will give them good marks, etc.,"; or pick easy cases: "you usually pick the ones that you know you're going to be good at"; because they do not want low grades: "you don't want a bad score"; "nobody wants to have a bad result"; instead of selecting more challenging cases from which they could learn: "in fact you should pick things that you know you can't do, but maybe you want to get a bit more out of it when you ask someone to assess you".

Assessment organisation
Responses to mini-CEX assessment organization produced four categories: Formal; retrospective; ad hoc; absentia.

Formal organisation
Some trainees mentioned that their mini-CEX involved observation by the assessors: "I went and did the mini-CEX, and they observed me and gave me feedback there and then".

Retrospective
Many trainees mentioned that prior clinical performance without assessment intention had been adopted as the mini-CEX for them:

Ad hoc
Other trainees reported that mini-CEX assessments are frequently rushed and lack detailed evaluations and feedback: "There were one or two where it was literally a spur of the moment let's do this as a CEX.So obviously they didn't prepare for that".

Absentia
Other trainees had never had their performance observed by an assessor:

Feedback
The opinions of the trainees about the received feedback fell under the following categories: self-reflection; strengths; weaknesses; compliments; action plans.

Self-reflection
Almost all the trainees mentioned that they had an opportunity for self-reflection.

Strengths
Most trainees indicated that the strength of their performance was frequently highlighted: "You've done this well; that's very good; I like the way you did this; I like the way you did that.";

Weaknesses
Others said that weak areas were identified during feedback: "I was told I need to grow in confidence and speak louder; I was told…that I wasn't as precise as I should have been";

Compliments
Some participants mentioned that their feedback consisted of mere compliments: 'Fine, well done, no problem'; without comments on their strengths and weaknesses: "They didn't say, 'Oh you should do this differently, or you should do this better or this was good'".

Action plans
The trainees held three views on action plans: some were enthusiastic to mention that they received specific and usable action plans: "Teach medical students; read up on specific chapters on clinical examination; read up on the care of acutely ill patients; spend more time in SAU; draw a diagram to aid explanation to patient"; Others stated that they received general action plans that they considered unusable: "sort of give you a generic 'yeah you're doing fine, carry on as you are'".
A number of trainees were not pleased because their mini-CEX assessments lacked action plans: "So things like improving confidence.That's not an action plan.And so, you know, there's nothing that I need to go away and do to achieve that.I can't kind of, wake up one morning 'right, today I'll be more confident"; "There's one thing that was probably lacking from my assessments; it was feedback regarding how to improve for the future.You know everybody seemed quite happy to just take a few things, write something positive, and then leave it at that".

Educational experiences with mini-CEX
The educational experiences with mini-CEX described by the participants can be outlined under the following subthemes: experience on generic competence; experience on clinical skills; experience on theoretical knowledge; and no educational experience.

Experience on generic competence
In this category, participants indicated several areas including improved confidence; improved communication skills; improved organizational skills; enhanced interpersonal skills; improvements in reflective practice.

Improved confidence
Some participants mentioned that good feedback contributed to improved confidence in interacting with patients and performing their clinical work: "When you get positive feedback, it gives you confidence; and in an acute situation, you are likely to be much more relaxed and confident in performing and seeing patients".

Improved communication skills
Some other trainees linked improvement in their presentation and communication skills to mini-CEX and indicated that they had learnt to communicate appropriately with patients and their relatives: "One of the aspects is the breaking of bad news; I wasn't competent enough before, but because of mini-CEXs, I am quite confident now in terms of doing it".

Improved organizational skills
Others perceived that mini-CEX had increased their organizational skills: "Having to complete X number of mini-CEXs in each job has made me more organized".

Enhanced interpersonal skills
Some trainees mentioned that mini-CEX developed their interpersonal skills, asserting that doing mini-CEX means interacting with the seniors: "I have got to do mini-CEX; please come and watch me; in a way it was persuading them to come and interact, so it was useful".

Reflective practice
Others reported that mini-CEX fostered reflection on their practice: "Although we get experience, we don't reflect on it; and I think one of the good experiences with mini-CEX is that it makes me think about the situation and learn from the cases".

Experience on clinical skills and practice
In the second category, trainees perceived that they had learnt a new set of clinical skills from the seniors' feedback after undertaking mini-CEX: "So he gave me feedback, saying such was the way it had to be done, and I didn't really do it that way despite spending quite a few weeks on the neuro-ward.So being shown by a consultant how to perform a neurological examination on patients that are not going to listen to your instructions and won't lift their legs [or relax] when you tell them to … was very beneficial."Some others confidently mentioned that some of their inappropriate clinical skills were corrected when seniors observed them and gave feedback.They identified a number of behaviours needing correction, such as cutting corners, illogical order of examination, and skipping things while rushing through examinations "…skipping over things for example, and then just missing things out….Sometimes you forget if you're just rushing through".
Here again, the perceptions of the trainees were mixed.Some participants described positive experience with mini-CEX in areas of communication practices: Others were not confident that they have had major positive experiences with mini-CEX in terms of their clinical practice, describing whatever positive experience they had with the assessment as 'small things', 'minimal', and 'very little, very little'.Still some other participants did not see any connection between mini-CEX assessment and their practice.Their experience is well captured in one comment by a participant: "It's not modified my practice or made me a better doctor, I don't think, from doing it at all."

Experience on theoretical knowledge
The trainees were divided into those who felt that mini-CEX had increased their knowledge and those who perceived that it had no educational value.In the first group, participants were confident that mini-CEX had increased their background knowledge of clinical medicine either directly in discussions with the seniors:

"... they will say, where else would you hear that kind of chest sound? And you just go into a bit more detail. So you learn quite a lot if someone sits down and you do quite a bit of discussion";
Or indirectly as they followed the action plans with further reading: "I didn't know you can do a herniogram; it's something that's new to me.So again, that was suggested to be read up".
In the second group, some trainees did not report any educational experience with mini-CEX and they discussed this from several perspectives.Some felt that their improvement was due to normal work experience while others linked their improvement to feedback from seniors outside mini-CEX assessment: "I think you tend to get feedback from your consultants or your registrars anyway".Some other trainees attributed the absence of educational experience with mini-CEX to poor implementation, citing the lack of interest of some senior doctors in education and training as the reasons: "I think it depends on who is the assessor really.The assessors who are very keen on education … really participate and go with you through the assessment.But assessors who are not really keen just want to sign you off really quickly".

Possible contribution of contextual factors
The trainees felt that positive experience with mini-CEX may be hindered by trainee-related factors; assessor-related factors; assessment tool factors; assessment environmental factors.

Trainee-related factors
The trainee factors relate to learning style, maturity and personality.Some trainees felt that mini-CEX assessment was at odds with their learning style:

"The way I do things is through watching other people do things and being formally taught on how to assess a patient; it's not by doing a clinical examination and then being fed back"; (male, undergraduate) "I'm kind of worried about asking people to watch me do a procedure; I would rather kind of just learn it on my own"; (female undergraduate)
Other participants perceived that as mature trainees, they felt more comfortable with the mini-CEX: "I see some of the people who are straight out of medical school who really are very passive when it comes to interacting with their consultants; they're right and I'm wrong and all the rest of it.But I think I'm a bit older and I'm willing to actually have a conversation with the boss"; (male, graduate entry)

"if you build up a rapport with your consultant, you do get little bits of feedback from them and it's a natural process"; (male, graduate entry)
Still some respondents reported some personality issues and asserted that they deliberately deviated from the assessment standard and conducted it in ways that suited them: "I suppose, like, I may have read the guidelines on what I'm supposed to get them to do.But like I said, all I care about is for them to fill out the form and once that's done I don't really ponder upon it so much"; (male, undergraduate)

Assessor-related factors
The concerns expressed by the trainees about the assessment tool relate to perceived rating insufficiency: Some of the participants described several assessor related factors such as:

Lack of interest "Assessors [are] not really keen; they just want to sign you off really quickly" (male, graduate entry)
Inadequate knowledge "There's not enough education amongst the people who are assessing us"; (male, undergrad)

Lack of feedback skills
"A lot of people are reluctant to give negative feedback direct face-to-face"; (female, undergrad) "I think people tend to shy away from direct criticism"; (female, grad) Others mentioned that some assessors consider mini-CEX valueless: "They do them for a paper-filling exercise rather than teaching particularly"; (male, undergrad) But some trainees rated mini-CEX by consultants more useful: "A lot of the time you tend to get more useful tips from the consultants, maybe because of their experience; I don't know"; (female, undergrad)

Assessment tool-related factors
"The current scoring system is too vague"; (female, graduate entry) "I think there should be just somebody who's not satisfactory, somebody who's satisfactory, and someone who is excellent"; (female, graduate entry) or its subjective rating: 'So certain consultants will mark very low, certain will mark very high for the same trainee's performance"; (male, undergrad) or its overall rating accuracy:

Assessment environmental factors
Many trainees considered mini-CEX stressful: "They're not enjoyable experiences"; (female undergrad) "It's probably the most difficult one to do"; (female undergrad) "I think that was the most frustrating"; (female undergrad) "I think it's a bit of a chore for everybody"; (female undergrad) and hard to undertake because of competing clinical duties (male undergrad): "I was so busy that it was not something that I was able to do"; (male undergrad) "It was a lot easier to do in medicine than it was in surgery"; (female, undergrad) and lack of assessor's time: (male undergrad) "Just difficulty in pinning people down"; (male undergrad) "so you're sort of relying on people's goodwill to do it"; (female undergrad).

Suggestions to improve mini-CEX
The focus group participants made several suggestions that they perceived could improve mini-CEX formative assessment.They preferred spending more time with the consultants: "spending time with a senior will be beneficial, but not necessarily in the form of a mini-CEX".

more constructive feedback:
"but you know, you want to have supervision with a consultant on a regular basis and have feedback from them"; "the thing that I do like is getting feedback from seniors, but I don't think it should be by rating"; clear assessment purpose and standard

"I think that whatever assessment, you need to have clear guidelines that you need to have a clear objective in terms of what you're expecting the tool to tell you"
and improvement of the assessment tool "I think it's hard to give everyone the same marks for such a varied aspect of things".
Other propositions include mandatory training for assessors; more assessment responsibility for consultants with an interest in medical education; more consultant involvement in assessments: "I mean the consultants should be assessing us because they are much more experienced".

Dedicated time for the mini-CEX "I think maybe there should be a set time every week that you get to sit with your supervisor and do a mini-CEX";
They also felt that the training for the mini-CEX should not be limited to lectures; they mentioned that they would like a model of meaningful mini-CEX to be demonstrated to them: "But I think perhaps if we had more examples of mini-CEXs being done in front of us and seeing how the mini-CEX really should be done, I think we would get more ideas out of it and know what to do".Some trainees also felt that mini-CEX might be more meaningful and useful if the cases were selected by the assessors: "If a consultant says "you do a mini-CEX on this patient", now that would be more helpful because you've not got all this time to prepare for it".

Discussion
This study employed a qualitative approach to explore the experiences of the foundation trainees with mini-CEX in the areas of their learning approaches and clinical practice.Unlike previous research, 4 this study shows that the experience of the trainees with mini-CEX is possibly related to the trainees' level of maturity, personality and learning style; and the knowledge of the assessment shared by the assessment participants.
The trainee and the assessor are the key actors that determine the value of formative assessment 16 and collaboration between them is very vital 17 to achieve this end.The differing views of the trainees regarding the purpose of the mini-CEX suggest that the main players in the assessment may not have a shared knowledge of assessment goals.In the absence of this, the trainees may draw on their internal conceptions, formulate their own task goals, and adopt tactics and strategies that often differ from the curriculum goals. 18Although a number of trainees described mini-CEX as worthwhile and useful, and valued the feedback they received, some trainees were quite dissatisfied with their experiences.
They mentioned that positive experience with mini-CEX is limited.This finding agrees with a previous study which recorded poor foundation trainee satisfaction with mini-CEX 4 , but contradicts, on the other hand, the findings of other studies 6,19,20 that recorded high participant satisfaction.It is noteworthy that the participants in these studies were mature graduates; thus, it is possible that the foundation trainees were too inexperienced in the clinical environment to have developed a robust capacity for combining learning and clinical practice in busy hospitals.
Workplace-based assessments (WBA) play an important role in aligning learning with assessment, based on the notion that assessment steers learning 21 .However, learning approaches are very fluid and the approach that a learner adopts depends on the context and nature of the assessment. 22This study suggests that many foundation trainees may be orientated towards superficial and strategic learning for the mini-CEX.They may have accomplished these by replicating observed performances, choosing easy and familiar cases and conducting the assessment with senior friends in retrospective, ad hoc and absentia fashions.Some trainees even ignored the action plans when they were given suggesting that the pre-assessment reading may have been carried out to obtain high grades rather than to learn.On the other hand, the opinions of some trainees suggest deep learning behaviours characterised by self-direction, questioning, discussions, and requests for resources to enhance learning.Surface learning is seen in the presence of anxiety about failure 23 strategic learning is concerned with making a good personal impression and achieving high grades; 24 and deep learning behaviour leads to comprehension of the subject matter. 22Formative assessment is known to promote deep learning 17 but the summative use of mini-CEX in this instance may have encouraged the participants to pursue superficial and strategic approaches to learning.The use of WBAs improves public accountability of medical practice and transparency of standards, and their formative use identifies deficiencies, problems, and gaps in training and can be used to maximise training opportunities. 25The trainees' perception of, and approach to the mini-CEX may have reflected a perceived tension between the dual role of mini-CEX for formative and summative intentions as noted by previous researchers. 26Our respondents believe that mini-CEX's results may be used to validate their competency, determine their progress and suitability for full professional registration, and may help with interviews and employment purposes.Indeed other trainees who share this anxiety felt that formative assessments provide quantitative evaluations that can be used by interviewers and employers. 27This may create a mindset in which survival takes precedence over development.
Good feedback should help clarify the expected standard and provide opportunities to close the gap between actual and desired performance. 18Identification of the performance gap is possible only if standard performance is known, actual performance is observed, and its significance is recognised.Our findings suggest that only a few trainees reported being frequently observed during their mini-CEX assessments.This means that opportunities to capture trainees' actual performance were probably lost.Furthermore, the ad hoc nature of some of the assessments may suggest that time was infrequently given to meaningful interactions between trainees and assessors.These may partly explain the reason why trainees were frequently not given usable feedback.The conception of feedback as a dialogue rather than information transmission from the assessor to the learner is known to improve formative assessment. 28Discussions with the assessor should assist the learner in developing his or her understanding of work standards, judging in what way his or her work relates to these standards, correcting misunderstandings, and obtaining solutions to difficulties. 18We noted that trainees were frequently asked to reflect on their performance before the evaluator gave them feedback.This procedure is in line with the principles of good feedback practice. 5It also helps equip the trainees with the evaluative and self-reflective skills that the evaluators possess. 18Research shows that frequent opportunities for learners to reflect on their goals, strategies, and outcomes are very effective in enhancing learning and development. 29The development of self-reflective skills aligns with the model of higher education, where learners are prepared for lifelong learning. 30Some trainees did not receive comments on their strengths and weaknesses.Without a clear understanding of their strengths and weaknesses, it becomes very difficult for the trainees to meaningfully utilise action plans for change and further development.
The usefulness of questioning during formative assessment has been noted outside medical education. 11The constructivist model of assessment focuses on the mental processes involved when trainees perform a task. 31These mental processes are uncovered when the dialogue between assessor and trainee is enriched with constructive questioning.Only a few trainees reported questioning during their feedback.This may suggest that feedback, in most cases, was probably directed at the products of performance and not its process.Learners are less likely to implement action plans unless the flaws in the mental process involved in a task are uncovered, the mental process is corrected, and the ownership of action plans is internalized. 32hile some trainees felt that they received specific action plans, many of our respondents were disappointed that the feedback was non-specific and unusable.Similar findings were reported by other researchers. 4,16Likewise, research has shown that feedback often compares learners' performance to a standard but frequently lacks information that could help learners close the performance gap. 32The paucity of specific action plans may be due to a number of factors, including lack of data on actual performance, since the trainee may not have been observed; inability to recognise significant weaknesses and address them with concrete action plans; lack of evaluators' tactical feedback delivery skills; and unclear curriculum standards against which to compare trainee performance.Additionally, the trainees may have deliberately blurred the performance gap through strategies and tactics designed to achieve their set goals, or they may have been up to the standard, requiring no further development actions.Even when the trainees seemed to have attained the required standard, useful lessons could have been learned if the process of performance had been scrutinised with appropriate questioning.
Regarding the educational experience with mini-CEX, Jackson and Wall (2010) 4 reported improvement in the critical judgment and organization of foundation trainees.The result of our study is in line with this finding as the trainees perceived improvement in their generic skills.These skills are vital for modern medical professionals.Unlike the previous study 4 however, a number of our participants reported acquisition of new clinical skills, correction of inappropriate clinical skills, and knowledge increase during mini-CEX assessment.The respondents were able to cite concrete examples to demonstrate their learning.Similarly a number of the trainees mentioned that the skills they learnt from mini-CEX assessment have been incorporated into their clinical practice.On the other hand, some trainees did not report positive educational experience with mini-CEX asserting that their experiences with the assessment are not superior to that of general learning on the job.The varying educational experiences with the mini-CEX as reported by the trainees may not be surprising.It may represent the reality in that the trainees would have had different performance gaps that could be bridged by the assessment and thus different educational experiences would be expected.Or the reported educational experiences may be related to heterogeneous assessment process, the skills of the assessors, the motivation of the trainees and other contextual factors.
Rogers's model posits that innovation adoption is determined by perceived benefit, consistency with existing beliefs and ease of use. 33Our study suggests that the mini-CEX may not align with learning style, personality, maturity, and the interpersonal skills of some trainees.The lack of alignment may partly explain the resistance of some trainees to the assessment.The high-pressure hospital environment with competing clinical duties, the difficulty in organizing the assessment and the varying availability of time and energy of consultants to give quality feedback may have contributed to the low acceptability of mini-CEX by some trainees.
The quality of feedback is known to correlate with the feedback delivery skills of the assessors. 16Our finding is in line with this notion.However, equally important to the poor engagement of the seniors is the possibility of a pedagogical conflict with the innovation.Many consultants were educated through behavioural educational philosophy, which they are familiar with.Research has shown that teachers live their beliefs 34 hence, the current constructivist educational philosophy embodied by the mini-CEX may run counter to their beliefs, norms, and mind set with regard to what counts as authentic education, and this may have contributed to their apathy about the assessment.This may partly explain lack of enthusiasm on the part of some assessors.
The trainees proposed a number of strategies to improve mini-CEX assessment; it seems reasonable to begin the assessment improvement by giving serious consideration to these suggestions.Attention should be paid to clear assessment standards and demonstration of meaningful mini-CEX assessment to the trainees during their training.Senior doctors may need encouragement to embrace the training of junior doctors as part of their responsibilities as leaders of the profession.Seniors that are especially interested in education may need to be identified and given more responsibility in the workplace-based assessment system, and extra time may have to be dedicated for the assessments.Robust integration of clinical duties with assessment commitments is desirable to improve assessment experience and reduce assessment related stress.More opportunities to improve consultant-trainee contacts for experiential learning may need to be explored.
This study has a number of limitations that need to be considered when interpreting the results.There is a potential for recall bias as the trainees were recounting their experience of mini-CEX over a period of one year.This long period indicates that the trainees had sufficient experience with mini-CEX and that the experiences the trainees were able to recollect must have been very significant to them.Another limitation of this study is that it relies on selfreported data, possibly weakening the strength of the evidence.The internal validity of the study may be limited because of the voluntary nature of the recruitment.This may mean that those who participated in the focus groups had stronger positive or negative views about mini-CEX than those who did not participate.The generalisability of the results may be limited because the study involved trainees from two related hospitals in the same location.However, a similar recent study 4 conducted in another centre corroborated our findings.

Conclusions
The trainees unanimously perceived the theoretical formative potential of mini-CEX, but in reality educational experiences with it are mixed.There seems to be a perceived tension between its formative and summative intentions which may have led to subversion of assessment standard by the trainees.This together with unclear assessment guidelines, assessors' apathy and inadequate skills may have contributed to the experiences of the trainees with the assessment.
This exploratory study raised a number of questions that could be pursued by further research.Further studies are required to understand the role of trainees' learning style, maturity, and personality in formative assessment.Studies are also needed to clarify how evaluator characteristics such as pedagogic beliefs and conceptions influence their performance in the context of formative assessments.

"
… for example, drawing diagrams to help patients understand what you are talking about";new methods of eliciting clinical signs:"… just get the child to hop up and down on one leg, because if they're peritonitic, they won't be able to do it"; and they indicated that the newly learned skills have been integrated into their routine clinical practice:"Oh, [I use my new skills] frequently; I actually used it on Friday"; "Ever since, I've used that trick to jump up and down on one leg".
like, 20 minutes to see me do something, and you know, everyone's busy.So that's the way I do it".
"None of my assessors have been with me.I suppose it defeats the purpose because they don't see me do a chest exam and they don't see me take a history, but I can't expect the registrar to sit for,