A pilot study using Cultural Consensus Analysis to measure Systems-Based Practice performance

C. Scott Smith1, Magdalena Morris2, Francine Langois-Winkle3, William Hill4 and Chris Francovich5

1University of Washington, Seattle, USA

2Apollo College, Boise, USA

3Saint Luke's Regional Medical Center, Boise, USA

4Northwest Regional Faculty Development Center, Boise, USA

5Gonzaga University, Spokane, USA

Submitted: 25/03/2010; Accepted: 26/04/2010; Published: 04/05/2010

Int J Med Educ. 2010; 1:15-18; doi: 10.5116/ijme.4bd5.6893

© 2010 C. Scott Smith et al. This is an Open Access article distributed under the terms of the Creative Commons Attribution License which permits unrestricted use of work provided the original work is properly cited. http://creativecommons.org/licenses/by/3.0

Objectives: This pilot study was designed to compare the performance of 'relationship-based' statements from a cultural consensus analysis, a standard anthropological technique for measuring value differences, with 'gold standard' patient and nursing satisfaction surveys often used in 360° evaluation of the systems-based practice competency.

Methods: We performed a cross-sectional correlation study in a teaching clinic in the United States. A single research assistant approached a convenience sample of ten residents, ten clinic staff, and ten patients per resident (120 participants). The cumulative scores for each resident on patient and nursing satisfaction surveys were compared to the average difference in cultural consensus analysis ranking between patient or nurse and the resident for two statements using Spearman rank correlation coefficients. These statements were selected because they represent the 'relationship-based care' pole in a previously validated conceptual model of clinic.

Results: The correlation between patient satisfaction cumulative scores and the difference in patient and resident cultural consensus analysis rankings on 'goals' was -0.527 (less difference between residents' and patients' value ranking correlates with higher satisfaction). The correlation with 'changes' was -0.351. The correlation between nursing satisfaction cumulative scores and the difference in nursing staff and resident cultural consensus analysis rankings on 'goals' was -0.086. The correlation with 'changes' was -0.415.

Conclusions: Systems-based practice is a notoriously difficult competency to evaluate. These moderate correlations in the expected direction between commonly used 360° evaluation instruments and this cultural consensus analysis tool suggest that it can provide an alternative measure of this competency.