Change in attitudes and performance of critical care teams after a multi-disciplinary simulation-based intervention

Objectives: To conduct an in-depth exploration of the selfreported long-term change in attitudes and performance after a full-day multidisciplinary simulation-based course focussed on team management of emergency events in the Critical Care Unit. To address the current lack of knowledge of factors which can lead to improved teamwork performance and their measurement through identification of measurable markers of behaviour and attitude change. Methods: A purposive sample of course participants underwent semi-structured interviews one to five months after course completion. Responses were coded using grounded theory to identify instances of learning, changes in attitudes or clinical performance, and measurable behavioural and attitudinal markers for such change. Interviews continued until data saturation was achieved. Results: Twenty nine participants (15 doctors and 14 nurses) were interviewed. Doctors became more confident in delegating and including nurses in decision making, and nurses became more confident and aware of the need for effective communication. Doctors reported that their ability to assign team roles improved over the day and that they made more frequent use of closed-loop communication. Both professional groups reported improvement in communication in the clinical setting after participation, including better vocalising of thoughts and use of colleagues’ names. Attitudes to communication also improved and persisted in the clinical setting. Conclusions: Addressing gaps in current medical education knowledge, self-reported improvements in behaviour and attitudes translated to clinical performance after a simulation course. Measurable behavioural and attitudinal markers were identified that may assist with the development of evidence-based measurement tools in future team training work.


Introduction
2][3][4][5][6][7][8] Teams generally make fewer mistakes than individuals, and well-functioning teams make fewer mistakes than poorly functioning teams. 6,9,102][13] Despite the pressing need for the more widespread development of better teamwork skills, a need which can be expected only to become more urgent with the ever increasing complexity of clinical care, few health care providers receive specific training in teamwork. 1,14Furthermore, although modern clinical care is by necessity provided by multidisciplinary teams, the training of health professional groups at both undergraduate and postgraduate levels occurs separately, and this separation of training continues into the use of health care simulation.Genuine multidisciplinary team training has considerable appeal as a remedy for these Correspondence: Jennifer Weller, Centre for Medical and Health Sciences Education, Faculty of Medical and Health Sciences, University of Auckland, New Zealand.Email: j.weller@auckland.ac.nz shortcomings in training, but current evidence on the advantages to participants, and ultimately patients, remains unclear. 15,16Two key gaps in our knowledge in this area of medical education currently exist: first, evidence for the transfer of team learning from the simulated environment to clinical practice is very limited; and second, tools to reliably measure the degree of such learning do not exist.The research designs necessary to detect efficacy of medical education initiatives using simulation are currently difficult and expensive. 17New approaches to the problem of multidisciplinary team training through the synthesis of all available data may provide useful guidance 2,[17][18][19][20] -in the presence of equivocal quantitative data, a qualitative investigation can be illuminating and provide an excellent methodological starting point for the design of a more robust quantitative study. 21he current study therefore has two aims.The first is to conduct an in-depth analysis of the self-reported long-term change in attitudes and behaviours of participants after attendance at a full-day course incorporating high-fidelity simulation.The second is to draw from this analysis observable or measurable markers of behaviours and attitudes that could be used to quantify the transfer of new learning to clinical practice in future work.In the first part we aimed to demonstrate the self-reported efficacy of our full-day simulation course in the transfer of learning to clinical care.][24] The current work comprised a part of a larger project validating an existing teamwork measurement tool 22 and evaluating the impact of the intervention of the full-day course on performance in the simulator. 25

Methods
This study was conducted in 2008 as part of a larger project which is described in detail elsewhere. 22,25 he larger project was approved by the Northern Regional and relevant hospital Ethics Committees, and was conducted over 20 separate study days (10 hours each day) at a university simulation centre.Members of the study team and participants signed confidentiality agreements to ensure that details of the simulations and individual performances were not shared with others.

Participants
For the purposes of the larger project, all nine Critical Care Units in eight hospitals within the region were invited to participate in the full-day simulation course, and we recruited sufficient numbers of doctors and nurses to form 40 teams of four clinicians each -hence yielding a purposive sample of 160 participants in all. 22,25 h team comprised three nurses and one doctor from the same Critical Care Unit.To ensure the highest levels of fidelity during simulations, we took steps to ensure that the composition of teams was typical of those working in clinical care: hence we recruited a mixture of junior and senior medical and nursing staff, and staff who regularly worked with one another.

Description of the course
Each four-person team attended a full-day course at our university simulation centre.The learning goals were improved teamwork, and improved management of typical Critical Care emergencies: these being, respiratory failure requiring securing of the airway, and cardiovascular failure due to dysrhythmia.The outline of the day is described in Table 1, and included seven simulated emergency scenarios with debriefs, three case-based learning discussions, two skill stations and an interactive tutorial.All components of the course apart from the skill-stations addressed the principles of effective teamwork -between the second and third scenarios there was specific teaching on teamwork and crisis management, and on cardiac and respiratory emergencies.Skill stations on managing life-threatening cardiac arrhythmias and emergency airway control addressed use of equipment, treatment protocols, and practice of emergency drills -this curriculum is described in more detail elsewhere. 22,25 e simulation scenarios achieved a high degree of realism using a recreated Critical Care ward, a METI HPS full-body simulator and real drugs, fluids, equipment and disposable items.The teams were assisted in the simulations by a single study confederate, playing her actual role of a health care assistant, who would follow orders within her scope of practice.Each scenario was followed by a debriefing session with an experienced instructor, which allowed for both behavioural and clinical aspects of the team's performance to be explored.

Interview process
For the current study we took a systematic approach to the selection of interview participants and to the detection of data saturation. 26Semi-structured interviews were conducted with doctors and nurses drawn alternately, and one at a time, from the purposive sample of 160 participants who attended the simulation course until data saturation was achieved -thus ensuring equal representation of the views of each professional group.In order to ensure an unbiased sub-set of interviewees, each selection was performed randomly.Data saturation was deemed to have occurred when no new concepts were being raised by interviewees, at which time interviewee selection ceased.Questions for the semi-structured interviews were chosen in order to elicit from participants their reactions to the course, what they had learnt, and importantly, changes in attitudes or clinical practice subsequent to the course (example questions included: On reflection, what were the most useful things you learnt?Have you actually changed some aspect of your practice or behaviour subsequent to your participation in the study?).During interviews there were opportunities to explore each answer further and for additional comments.The interviews began one month after attendance at the simulation-based teaching intervention, and were conducted via telephone by one investigator (ST), recorded and transcribed.

Analysis
A grounded theory approach was chosen to analyse interviews in order to avoid any a priori conceptions of the views expressed. 27,28 his was undertaken by an investigator and a research assistant (JW, MS), using a constant comparison technique.Open coding was conducted on interview transcripts, followed by the clustering of coded items into categories and sub-categories.The two coders compared and agreed on the coding of data, and emergent categories, with progressive refinement of the coding framework until all data were consistently subsumed.Transcripts were coded with the use of qualitative data analysis software NVivo8 (QSR International, Melbourne, Australia) and reports then produced on each category.Categories were tabulated and illustrative quotes selected for the reporting of results.
The changes described by participants in how they interacted differently following the course should be amenable to measurement by direct observation.Therefore a set of specific behavioural markers for such events was extracted from categories for possible future inclusion in an observational measurement tool.Attitudinal changes cannot be directly observed, but are important in the success of safety interventions, and can be systematically studied. 23,29Therefore, aspects of attitudinal change evident in categories were also extracted and summarised.

Results
Over a period of one to five months after course completion, interviews were conducted with 29 participants (15 doctors and 14 nurses), taking a median (range) of 32 (9-62) minutes each, at which time data saturation was achieved.The final set of categories developed from the data is shown in Table 2.

Interview analysis
In the following the results of the analysis of interviews are presented as categories with illustrative quotes for each point and in an order following the results in Table 2.

Learning how to do something new/better/with more confidence
Doctors became more confident in delegating and stepping back from the scene, and including nurses in the decision making.For most, the course provided a useful opportunity to practice: "It was really good to have it all reinforced and to actually get to put it into practice" (Doctor, no.17).Doctors considered that simulation courses needed to occur on a regular basis to maintain confidence in these skills.
Nurses appreciated the practical side to the course, in particular having the opportunity to gain more confidence in managing clinical crises, which for many were not a regular occurrence in their day-to-day practice.The nurses learnt how to be more involved and vocal members of the team.Specifically, they reported an awareness of both the need to communicate and how to communicate more effectively and felt more confident in speaking up and providing input: "I would definitely say that the thing [the course] has given me a lot of confidence to speak up and if you feel like that, you know you have something to say, I wouldn't have a problem saying to the doctor, look what do you think about this?" (Nurse, no.37).
In addition by learning new practical skills during the course nurses felt they had become a more helpful assistant.

Change of attitude/new understanding/new appreciation of the roles of others
Doctors gained a greater realisation about how focusing on one task can narrow their overall view of the situation, and the importance of standing back and looking at the wider picture: "I suppose naively [I] thought that you could actually keep an eye on a number of things" (Doctor, no.11).
Related to this, there was a greater awareness of the importance of identifying a leader to direct decisions: "It brought the focus for me on to the urgency of making the decision and facilitating the intubation ... rather than the clinical skill itself" (Doctor, no.23).
With this awareness, there was also a new appreciation of the expert assistant role of the nurse.
Doctors also became more aware of the importance of sharing information and goals in preventing adverse events, the need to verbalise their thought processes and to state what they may consider to be the obvious in order to keep everyone in the loop.
Nurses became more aware of the burden of care that doctors carry and how much information they are required to manage, and in turn the importance of the nurse's role in pointing out concerns and coming in with a fresh set of eyes: "It has highlighted ... that they [doctors] need to be able to take on information, process it and spit it out at a great rate of knots, but also it has highlighted that just because they are the doctor doesn't mean that they can absolutely see everything, that we have to work together as a team to get a desired outcome" (Nurse, no.82).
In a related vein, nurses gained a new understanding of the importance of the "team", and not necessarily just the individual's "skills", for good patient outcomes.

Skills/attributes of team or leader
Doctors appreciated nurses being willing to step in and do whatever role they were assigned, giving the opportunity for the doctor to step back and make the decisions.Doctors also valued a nurse's honesty and willingness to speak up and say what they thought and/or make suggestions: "and they [doctors] were very quick to say when they thought something was done well ... and sort of make something better when they thought it wasn't being done quite so well ...And I really valued that, because I think it was much more constructive"(Doctor, no.28).
Doctors appreciated working with experienced nurses with whom they were familiar.Knowing that a nurse was competent and could be trusted to successfully complete a task helped the doctor to feel more confident in delegating tasks.
Nurses recognised a number of attributes in doctors that hindered the way the team worked together: not having confidence in their own ability to lead the team; not instilling confidence in the team; not making the nurse feel acknowledged and valued; and not verbalising their thoughts well.Nurses appreciated it when doctors communicated their treatment plan to the team: "[He is] clear in his way of communicating where he thinks we're at and what we all need to do to achieve that, which makes it very easy as a team" (Nurse, no.46).
In addition nurses appreciated it when doctors were open to their suggestions letting them put their "penny's worth in": "he encourages input from all his team, which makes it much better" (Nurse, no 84).

Change in perceived behaviour over the course of the day
Doctors felt that as the day progressed they improved at providing directions and assigning roles to nurses.Over the day, doctors more often reported that they used closed-loop communication (i.e.where a response is required to ensure the communication has been received and acted upon).Communication practices also became more streamlined throughout the day.For example, a nurse described how they started with a separate nursing and medical handover but combined this into one as the day progressed, which reduced the chance of overlooking important information or actions.In terms of team dynamics, doctors recognised that as the day progressed the team became more focused and unified, with improved role clarity.

Change in practice/communication
Doctors felt that their communication in the clinical setting had changed since the training day.Doctors were now using/saying colleagues names, allocating tasks, using closed-loop communication, vocalising thoughts, and requesting that other team members speak up: "what I did there [giving an example of a clinical crisis], which I think is probably different from how I would have done it before the training day, was very specifically allocate roles for specific people and tell them what I expected of them ... Giving them instructions to feed back to me and allocating somebody purely to give drugs" (Doctor, no.23).Doctors reported they were also stepping back and taking more of an overview of the clinical situation, which in turn helped with coordination of the team response.Many doctors described how they had a calmer approach to managing an event now, and how their confidence to handle a crisis had improved.
Nurses described changes in their communication, in particular that they were vocalising and asserting themselves more than before the course.The nurses reported speaking up to take the load off the team leader and to clarify that a task was being carried out.One nurse even described verbalising on behalf of other members of the team: "He would say 'can someone get a blah blah [sic] drug' and you would know that somebody has left the room and you could say 'so and so is getting that drug now'" (Nurse, no.53).

Change in attitude
Both doctors and nurses mentioned a change in attitude in terms of a greater appreciation of the importance of communication for underpinning effective team dynamics and patient outcome.Even "silly, incidental communication" (Nurse, no.62) was viewed as having a positive influence on the promotion of more effective team communication.

Change in processes in the workplace
One doctor described how his/her unit was now considering introducing debriefs after each major incident.One nurse educator spoke about plans to run monthly scenariobased teaching for clinical staff in the unit.

Opportunity to work in a multidisciplinary team
Doctors and nurses appreciated the realism of the training day and that it reflected the team-oriented approach to patient care in the critical care unit:

Opportunity to reflect on their behaviour in debriefs
Participants valued the debriefing discussions and thought they were a good model for clinical practice.Debriefing provided an opportunity to hear everyone's concerns and perspectives, and to learn new strategies for the future.

Opportunity to practice for rare emergencies
Doctors and nurses found it beneficial to practice critical situations a number of times over the course of the day, which in turn would help them to better deal with major incidents in real life: "the more practice that you have with [clinical crises] the more aware and the better you are to deal with them in real life" (Doctor, no.21).

Development of measurement tools -behavioural and attitudinal change
From the analysis of interviews we identified behavioural events reported by participants reflecting their improved teamwork as transferred to their clinical workplace.These constitute a set of identifiable behavioural markers that would be amenable to measurement through a direct observation tool (Table 3).1][32][33] For the purposes of triangulation with our quantitative work from the larger project, 22 we have also indicated the factor numbers and categories of a validated teamwork performance instrument which correspond to each behavioural marker generated in this study (Table 3).Qualitatively derived behavioural markers correspond to 14 of the 23 quantitative factors of our validated teamwork instrument. 22imilarly, a number of reported attitudinal changes emerged from the data after analysis, which could inform the development of items in an attitudinal questionnaire or other measurement tool (Table 4).For example, detecting doctor's increased appreciation of nursing staff speaking up to alert him or her to relevant aspects of a patient's care may be an important attitudinal change in improving and sustaining team communication.

Discussion
The use of simulation in clinical curricula is becoming increasingly common, including procedural skills training, application of theoretical knowledge to practice, and teamwork training.However, even simulation-based initiatives in team training in health care are often unidisciplinary, with, for example, an anaesthetist confederate playing the role of the surgeon during an anaesthetically focussed simulation run for the training benefit of the anaesthetist participant.Surgeons and nurses also tend to have their own simulation courses.We propose that this separate approach to training reinforces professional boundaries, helps create what have been called professional silos, and may in fact reinforce stereotypes. 8,29 n a multidisciplinary clinical team, people within one silo often manifest an unwillingness to speak up to challenge those in other silos, and this can lead to loss of situational awareness, lack of engagement in team decision making and poor agreement on shared goals.Without a holistic team approach to patient care and safety, aspects of patient management may be seen as the responsibility of another discipline. 34he primary strength of our study is that it was conducted using genuine multidisciplinary clinical teams and high-fidelity simulation, thus allowing the systematic study of the self-reported benefits of multidisciplinary simulationenhanced team training.However, our study also has a number of limitations, perhaps the primary one being that self-reported data is subjective and does not provide evidence that change has actually occurred in workplace practices.Other limitations include the lack of any control group and the fact that our qualitative results are not generalizable beyond our study sample.However, in the absence of definitive quantitative data on the value of team training, it was our intention to conduct a systematic qualitative study.The behavioural and attitudinal markers generated by our study should inform further, more definitive, quantitative work in this area, thus addressing some of the weaknesses of the current study.
The results of this study support the value of multidisciplinary, simulation-enhanced team training in health care teams. 25Participants reported improving their teamwork skills, a positive change in attitudes towards, and understanding of, the value of teamwork in health care, and transfer of this learning to practice in their clinical workplace.Much of the learning and attitude changes described in this paper depended on the interaction between doctors and nurses during the course, supporting the recommendation of the United States' Institute of Medicine 'to train in teams those who are expected to work in teams'. 1 We have previously reported on the improved performance of these same Critical Care teams over the course of the day in both team behaviours and the more technical aspects of clinical management. 22,25 hese were measured by pre-and post-course team assessments using simulated cases with an end-of-course questionnaire.The data reported here builds on that work by providing self-reported evidence from course attendees that what they learnt from the course was retained over a period of months, and in fact transferred to use in clinical practice.In a previous simulation-based study of anaesthesia teams, we also found that much of the reported learning was very much dependent on the multidisciplinary nature of the intervention. 35While our current evidence is limited to self-reported qualitative data, a consistent picture is emerging from all sources of evidence supporting the value of multidisciplinary, simulation-based team training for health professionals.
While there has been increasing acceptance of the concept of multidisciplinary team training of health professionals, 7,8,36 attempts to demonstrate improved outcomes stemming from these interventions have often shown weak evidence of clinical or technical benefit. 3,15,37On the other hand large-scale interventions such as the WHO Safe Surgery Checklist have shown substantial outcome benefits in terms of reduced patient adverse events -some significant benefits of which were in categories not specifically targeted by checklist items. 6Authors of the checklist have speculated that this additional benefit may be due to the better team communication engendered by the act of carrying out the steps of the checklist itself, including individual team members introducing themselves by name. 33The value of knowing and using other team members' names and of "incidental" communication are in fact two factors identified in the behavioural and attitudinal markers generated in our study.
6][17] The changes described by our study participants in how they interacted differently following this multidisciplinary team training course would be amenable to measurement by direct observation or through analysis of video recordings.
The set of behavioural markers generated by this study provides useful data to inform the development of tools for use in an observational study to measure such changes (Table 3).For example, such a tool could be used for the more explicit measurement of the behavioural improvements of interventions such as the WHO Surgical Safety Checklist on team interactions. 6If repeated measures were taken with such a validated tool before and after an intervention, effects on team performance may be reliably detected.
9][40] The first step on the way to remedying this situation, in terms of behavioural or practice change, relies on prior attitude change. 29Thus, detecting changes in attitudes is an important goal when pursuing improved teamwork performance, and an ideal target for measurement.The attitudes of healthcare teams after multidisciplinary training and practice could be more specifically investigated using information arising from this study (Table 4).Repeated measures of a tool with such items, used in conjunction with a behavioural measurement tool, could allow the identification of the changes in attitudes associated with, and underpinning, objectively identified behavioural changes.
Self-reported data, while valuable, does not provide evidence that change has actually occurred in workplace practices, teamwork behaviours or attitudes towards teamwork, underscoring the need for more robust data obtained from validated observational and attitudinal studies to consolidate the present findings.[44] In particular, it is of interest that the markers of behavioural change reported to have transferred to the clinical workplace derived by qualitative means in the present study correspond well to the quantitatively derived factors of a validated teamwork performance instrument developed by our group in previous work. 22This may allow us some confidence in our present results and in their ability to help inform the on-going development of more rigorous, evidence-based tools.
Our participants attended in groups from a limited number of workplaces and may have continued to reinforce and further develop the changed attitudes and behaviours induced by the course.The extent to which our positive results would be seen if individuals from different departments attended is unknown.It is also unknown whether it is necessary to train a critical mass of staff in any particular workplace in order to sustain teamwork improvements within that environment.

Conclusions
In this project, 40 established teams from eight Critical Care units attended a day-long course and then returned to their workplaces.Participants felt that as a result of the multidisciplinary simulation-based course, communication and team orientation were improved, and they had a better appreciation of the teamwork roles of others.A major factor in the perceived value of the course was learning from one another.Attitudes to communication also improved and persisted in the participant's clinical setting.
Key gaps in our knowledge in medical education currently include the specific factors that reliably lead to improved teamwork performance in the clinical setting, and how to best measure such improvements.Our results address these key gaps by providing self-reported qualitative evidence of improved team performance and communication during a simulation-based course, improvements which were also reported to translate to participant's clinical workplace.Furthermore, our results identify the behavioural and attitudinal markers for such improvements and therefore may inform the development of new instruments to measure transfer of learning to the clinical workplace, and guide future training initiatives.

Table 2 .
Results of the analysis of interviews (N=29)  Skills/attributes of team or leader -what participants noticed during the day about the team processes that either helped or hindered the way the team worked together to achieve their goal  Change in perceived behaviour over the course of the day Evidence of change in behaviour subsequent to participation in the SBL intervention  Change in practice/communication  Change in attitude  Change in processes in the workplace Reflections on the SBL intervention  Opportunity to work in a multidisciplinary team  Opportunity to reflect on behaviour during debriefs  Opportunity to practice rare emergencies * Simulation-based learning "It's not the kind of hierarchical doctor/nurse scenario, which you find out in the [general] ward ... I think to replicate that in the training makes it more valuable ... the bottom line is that they are working together, hopefully as a team, for the good of the patient aren't they, and to integrate as much of the training as possible so they can both see from the other's eyes what they are responsible for, what they're going through, I think that would be invaluable" (Nurse, no.46).

Table 3 .
Components of teamwork and behavioural markers expressed by interviewees that transferred to the workplace(N= 29) * In brackets, following each behavioural marker are the corresponding factor numbers and categories of a validated teamwork performance instrument.22 † LTC, Leadership and Team Communication; ‡ MPM, Mutual Performance Monitoring; ¶ VSI, Verbalising Situation Information (see results)