Measuring medical students ’ sense of community and satisfaction with a structured advising program

Objectives: The purpose of this study was to assess changes in students’ perceptions of the medical school sense of community and their satisfaction with advising following the initiation of a structured advising program at the Johns Hopkins University School of Medicine. Methods: Participants included School of Medicine graduates from 2006 to 2009. Total sample size was 471. The response rate was 367/471 (81%). A cross-sectional survey design was utilized to assess students’ perceived differences in advising and sense of community. Scales were developed to assess these constructs. Scale scores and responses to individual items were compared by graduation year. Results: Statistically significant increases over time were noted on the Advising Satisfaction (F (3,363) = 6.54; p≤0.001) and the Relational Community (F(3,363) = 2.76; p=0.042) scales. Using simple logistic regression, unadjusted Odds Ratios show that students graduating in 2009 report higher quality of advising during their clinical years (OR 2.87, 95% CI=1.56-5.28), around academics (OR 2.54, 95% CI=1.414.60), career planning (OR 1.87, 95% CI=1.03-3.38), stress management (OR 2.88, 95% CI=1.56-5.30), and an enhanced sense of community (OR 2.55, 95% CI=1.40-4.64), and social integration (OR 2.07, 95% CI=1.15-3.71) compared to 2006 graduates. Conclusions: We found, that students were more satisfied with advising and the medical school environment over the four years of maturation of the Program. A structured advising program that fosters meaningful, longitudinal relationships between students and advisors may enhance students’ experiences during medical school.


Introduction
Medical schools are increasingly offering expanded networks for student advising to supplement the traditional role of the Office of Student Affairs and the more informal relationships that may develop between students and faculty.Such efforts recognize the importance of guidance in career development and success as well as the fact that some students may not identify faculty advisors or mentors on their own.Advising programs (also described as learning communities, mentoring or docent programs) at different institutions encourage faculty to provide support for students in various ways and utilize different models such as one-on-one or group advising relationships between students and faculty.Specific content areas of focus include academics, career planning, professional values, communication and clinical skills training, medical economics, time management and research.][3][4][5][6] The traditional medical training environment is characterized by an intense workload, a competitive and individualis-tic culture, and high learner stress.Some learners may struggle to establish a network of support and resources on their own.Structured advising programs may have a substantive impact on the personal and professional growth of students by fostering meaningful longitudinal relationships between students and faculty and influencing the social and learning environments of the medical school.Advising programs may contribute to the development of a "Relational Community" in which interactions between individuals are based on group consensus, shared values, common goals, and a sense of identification. 7The relational culture of an organization is enhanced by positive interpersonal interactions, trust, and collaboration.Recent literature suggests that a pervasive negative relational culture in schools of medicine may undermine the expressed mission of those institutions to educate students to be compassionate, caring, and ethical physicians. 8,9t the Johns Hopkins University School of Medicine (JHUSOM), a structured system for advising was initiated in 2005.The Colleges Advisory Program (CAP), similar to the experience at other medical schools 2,10,11 was created partly in response to students' requests for greater interaction with faculty.The CAP provides one-on-one longitudinal advising for each student with a designated faculty advisor throughout his or her four years of medical school.We hypothesized that students' satisfaction with advising and the medical school's sense of community and relationships would increase over time as as the CAP evolved, expanded its roles, and as student exposure and engagement increased.The goal of this study was to assess changes in student satisfaction with advising and student perceptions of the relational community during the first four years of the CAP.

Description of the JHUSOM colleges advising program
The initial structure of the program was loosely patterned after other institutions that had developed advising or mentoring programs for students and through discussions of the AAMC Learning Communities Institute (a collaboration of medical schools with similar programs).In the fall of 2005, extensive collaboration between CAP faculty and students occurred to develop a vision and goals for the Program which fit the needs of students and the institution.The vision of the Program is to develop a welcoming community of students and faculty, dedicated to enhancing personal and professional growth and career development and to promote collaborative learning, clinical skillbuilding, longitudinal advising, and interpersonal connections between students and all (interested) members of the school of medicine.One-on-one, longitudinal facultystudent relationships are emphasized as core to the program along with a strong emphasis on the entire CAP as an important community for social support and learning for students.Within the CAP, twenty-four core College faculty are paired with five students from each medical school year.Advisors and students are expected to meet at least quarterly, as well as on an as needed basis.Advisors also teach the Introduction to Clinical Skills course to their group of 2nd year advisees.CAP faculty receive financial support for their work with students.CAP faculty meet monthly for regular faculty development sessions to enhance their advising and teaching skills and have developed a comprehensive advising resource guide that specifically covers content area related to academics, research and scholarly activities, career planning, extracurricular, leadership and service opportunities, and student assistance resources.
Led by a cadre of student leaders mentored by program faculty, the CAP now coordinates and supports many traditional medical school events, such as the White Coat Ceremony, a peer-advising program, revisit weekend for prospective students, forums to share information such as about transitioning to the wards and applying to residency, as well as new social events including a competitive yearly sporting event "The Colleges Bowl." Students graduating in 2006 and 2007 had limited contact with their advisors for several reasons.First, many students had already made career plans prior to the start of the advising program.Second, these students had already navigated through several years of medical school and may have already established support networks to meet their advising needs.In contrast, students graduating in 2008 and 2009 had more comprehensive exposure to the program including interactions with their advisors through regular advising meetings, social activities, and during the Introduction to Clinical Skills course.For these reasons, as well as the growth of the program, we hypothesized that student satisfaction with advising and sense of community would increase over the study period.

Study design
This is a cross-sectional survey of all graduating medical students at the Johns Hopkins University School of Medicine over a four year period.

Participants and data collection
We surveyed all graduating medical students (identified through the registrar's office) in the spring of 2006, 2007, 2008 and 2009 using a confidential, web-based survey.The total sample size of graduating students from these years was 471.The response rate was 368/471 (81%).Response rates for the years 2006, 2007, 2008, 2009 were 96/118 (81%), 99/128 (77%), 80/101 (79%), 93/124 (75%) respectively.Students were sent an email inviting them to complete the survey.Participation was elective.Each year of the survey administration, students who completed the survey were entered into a drawing to win one of five $100 restau rant gift cards.Multiple email reminders were sent to encourage participation.The study was approved by the Johns Hopkins Medical Institution's Review Board.

Survey development and content
The survey was developed through review of the relevant literature on advising and mentoring in medicine [1][2][3]5,[12][13][14] and input from the CAP faculty advisors. The surey was revised after piloting the questions on recently graduated medical students and faculty who work closely with medical students.Survey content included: student demographics, sense of medical school community and social environment, relationships with faculty and other students, satisfaction with advising around academics, research, career planning, work/life balance, stress management, what it is like to be a physician after training is completed, comfort with career choice, and overall satisfaction with advising.To assess satisfaction with advising, students were asked to rate their level of agreement with the statement, "Overall, I am satisfied with the guidance and advising that I have received during medical school."Questions were formatted as multiple choice, yes or no, short answer and 5 or 6 point Likert scales.

Scale development
We initially used exploratory factor analysis to determine how well specific survey questions measured students' satisfaction with advising and their perception of the medical school environment.We conducted separate factor analyses to develop a scale for each domain (Advising Satisfaction and Relational Community) using variables related to each construct.We first analyzed all variables with Likert response options for evidence of sufficient variability and then for the mean sampling adequacy (MSA).We considered a MSA of 0.65 as a minimum requirement for inclusion of the variable in the factor analysis.An Eigen value of 1.0 was set as the minimum to extract a factor.For the factor analysis, values >0.40 represented a clear loading on a factor, and values of 0.35 -0.39 represented borderline loading.We considered loadings of >0.40 on two or more factors to represent straddling.When multiple factors emerged from principal components analysis, we used two rotations, Varimax (orthogonal) and Promax (oblique) to identify factors.
Total scale scores for individual survey participants were created by summing the responses to each Likert item (1-5 or 1-6 item response options).The minimum and maximum scores for the Advising Satisfaction Scale and the Relational Community Scale were 8-40 and 4-20, respectively.
We substituted the group mean response for respondents who had missing data.Mean substitution was used only when more than 75% of all items were answered.It was needed for only 2 subjects.

Scale reliability and validity
To evaluate reliability, we computed Cronbach's alpha and examined whether the scale alpha increased with the omission of any single item.Decisions to omit or include specific items were based on the change in the alpha and weak item-to-total correlations along with theoretical considerations.To establish the internal validity of the scales, we ran Pearson and Spearman correlations of the Advising Satisfaction and Relational Community scales with the single Likert item that was designed to evaluate overall student satisfaction with advising during medical school.In theory, the Advising Satisfaction scale should correlate with overall advising because the variables in the scale focus on important individual advising topics.The Relational Community scale addresses students' sense of closeness to and accessibility of faculty and sense of community (feeling of belonging, the sense that the individual and the group matter to each other, expectations that members' needs will be met through group resources). 15To the extent that a student feels strongly that their advisor is accessible, supportive and provides resources, this should also correlate with overall satisfaction with advising.Spearman and Pearson correlations agreed very closely.We also ran an analysis of variance (ANOVA) with a linear contrast to demonstrate whether increasing levels of overall satisfaction (a 5-point Likert item) were related to increasing scores on the Advising Satisfaction and Relational Community scales.

Measuring student perceptions of the CAP by year of graduation
To compare the scale scores by year of graduation, we used an ANOVA and conducted multiple comparisons of groups using Tukey's test only when the overall F test was significant at 0.05.Prior to conducting the ANOVA, we examined the distribution of the scales for evidence of nonnormality, skewness, and outliers.No outliers were observed and both scales closely approximated a normal distribution with skewness and kurtosis close to 0.
When exploring the changes over time for individual survey items, we dichotomized relevant Likert items and compared variables using crosstabs and chi-squares to develop a better understanding of student perceptions.Decisions to dichotomize the variables were based on the distribution of the data and for ease of presentation.Reponses were dichotomized at the top two response categories.To characterize the magnitude of the differences in cohorts, simple logistic regressions were run, using the 2006 cohort as a reference group.Effects are shown as unadjusted odds ratios (OR) with 95% confidence intervals (CI).
To ensure that graduation year cohorts did not differ by characteristics that might explain potential differences in satisfaction with advising and perception of the relational community we compared the following variables: gender, age, race, marital status and amount of educational debt by year of graduation.Chi-Square tests were used to compare proportions and t-tests to compare means.There were no significant differences in any of these characteristics between cohorts.Therefore, we did not control for these variables when comparing cohorts.SAS Version 9.0 was used for all analyses.

Results
Table 1 demonstrates the demographic characteristics of the participating students.There were no differences in demographic variables by year therefore all study years were combined for presentation.There were no significant differences between responders and non-responders in terms of gender (53% vs 43% female; p=0.391) or race (48% vs 54% non-white; p=0.556).* There were no differences in variables by year therefore data was combined for all years (2006, 2007, 2008 and 2009).Not all calculations are based on an N of 368 due to missing data which represents less than 1% of responses.† Other includes African American, Native American, Hispanic, mixed race and non-Caucasian other.

Advising satisfaction scale
Factor analysis was used to create an Advising Satisfaction scale using ten questions that focused on various aspects of advising including, medical students' experiences during preclinical and clinical years and around specific topics such as academics, research, career planning, work/life balance, insight into what medical practice is like following training, and stress management.Two items which asked about confidence and comfort with career choice were excluded due to poor factor loading values.For the remaining eight variables, the mean sampling adequacy was sufficient (overall 0.90, range 0.78-0.94)and a single factor solution was obtained.The factor loadings for the eight question scale are shown in Table 2. Cronbach's alpha was 0.89.Question to total correlations indicated that all questions correlated well with the total scale (ranging from 0.57-0.75).
Examination of the change in alpha while deleting one item at a time suggested that all items contributed to the alpha and should be retained.

Relational community scale
Eight questions addressed medical students' sense of the relational community.These questions asked about students' perception of accessibility of faculty, relationships with faculty and other students, sense of community and social integration.Four items related to student-student relationships were excluded due to poor factor loadings.
For the remaining four questions, mean sampling adequacy was sufficient (overall 0.79, range of 0.71 -0.81.)We obtained a factor solution with a Cronbach's alpha of 0.81 (Table 3).Item to total correlations (ranging from 0.33-0.70)indicated that all items correlated sufficiently with the total scale.Consecutive deletion of single items did not alter the alpha sufficiently to warrant leaving any items out of the final scale.
Pearson correlations were high between the Advising Satisfaction and Relational Community scales and a single Likert item designed to assess overall satisfaction with advising during medical school (0.79 and 0.64 respectively).ANOVA with linear contrast demonstrated a strong linear relationship between the five Likert (strongly disagree to strongly agree) categories of overall satisfaction with increasing scores on the Advising Satisfaction (F(1,359) = 418.52;p<0.001) and Relational Community scales (F (1,359) =188.14;p <0.001).

Comparisons of scales by year of medical school graduation
Based on multiple comparisons, significant increases in scores were noted for the Advising Satisfaction Scale between 2009 and 2008, 2007, 2006, and

Comparisons of individual variables by graduation year
Changes in responses to individual survey items were compared by graduation year using simple logistic regres-sion and are presented as unadjusted ORs with 2006 as the reference year.Table 4 demonstrates a steady improvement for many survey items for each year compared to 2006, the first graduation year after initiation of the CAP.Progressively higher scores are noted for each year in the perceived quality of advising during the final two years of medical school as reported by students compared to the reference year, 2006.Compared to the 2006 graduating class, students in each subsequent graduation year reported an increase in their sense of community.Students in the class of 2009 were more likely to report being satisfied or very satisfied with seven of twelve key survey items.

Discussion
To our knowledge, our study is the first to demonstrate a positive impact of a student advising program using scales developed specifically to measure student satisfaction with advising and perception of the relational community during medical school.Specifically our study demonstrates that with the maturation of the CAP and increased student contact with their advisors, students are now more satisfied with advising and have an increased sense of relational community.Our study employed a large sample, with a robust response rate and multiple cohorts to develop the scales.Other researchers have focused on the impact of the learning environment on medical student well-being 16 or demonstrated a decline in the learning environment as students progressed through medical school. 179][20] Rosenbaum's study demonstrates that a "learning community" based on increased relationships among students and between students and faculty enhanced students' sense of the learning environment. 6Our study adds to the literature by presenting an expanded model for viewing the medical school environment, as a "Relational Community" and developing a scale to measure this construct.
Our study found an improvement in student perceptions of their relational community during medical school.This view held true when this domain was measured using a multi-item scale and when students were asked about this using a single item assessing sense of community.The Relational Community scale was highly correlated with a single item measure of overall satisfaction with advising during medical school.Student perceptions of close relationships with accessible faculty contributed to the high alpha obtained for this scale.
The strength of a relational community relies on the sense of community that exists among members."Sense of community" develops when: (i) individuals have a feeling of belonging to a group, (ii) a high value is placed on relationships, (iii) individual needs are met through group  resources, and (iv) group members have a sense of shared history and experience. 15Relationships and social networks are critically important for the education of adults in that they have been linked to trainees' desire to learn and to the behaviors they choose to emulate. 21The relational community experienced by learners may represent an important part of the learning environment and have a profound impact on the development of professional identity among students. 92][23] The structure of the CAP, with its opportunities for regular and meaningful interactions among students and between students and faculty, may provide the scaffolding for the development of a true relational community. 24Furthermore, as such advising programs expand their scope to include more social and educational activities, the values and goals that they embody (generally those of the formal curriculum) may diffuse into space that was formerly encompassed by negative features of the hidden curricula. 24,25Finally, as medical schools and graduate medical training programs are increasingly required to provide evidence of an environment that promotes the professional development of learners, our scales may serve as tools to measure key aspects of the learning environment (Liaison Committee on Medical Education Accreditation Standards).
With the maturation of the CAP, students perceived higher quality advising during the 3rd and 4th years.The second half of medical school is often the most challenging.During the "clinical years", students experience increasing responsibility for patient care and must begin to make important career decisions.The guidance provided by a designated advisor in our program may have helped students to navigate this often difficult period in their training.Supportive relationships have been shown to be critical in the personal and professional development of practicing physicians and medical trainees, especially in high stress situations. 26,27mong undergraduates, students' perceptions of being part of a community have been shown to have a positive effect on preventing burnout. 28e found the most striking changes in responses from the class of 2009.Students reported significant improvements in over half of the study questions.These students had the greatest exposure to their advisors and experienced the evolution and maturation of the program from its inception.Thus, re-emphasizing the importance of longstanding relationships and educational continuity between advisors and advisees. 29

Limitations of the study
Limitations of this study should be considered.First, this is a cross-sectional study and, therefore, cannot address causality.Second, factors outside of the Colleges Advising Program, such as changes in the curriculum or admission process may have contributed to students' satisfaction with advising and sense of community.However the authors are not aware of any major changes that occurred within the school of medicine during this time which would have significantly impacted the majority of students besides the implementation of the Colleges Advising Program.Similarly, overall student satisfaction with advising was higher when comparing the final study year to each prior year, despite that fact that some individual items did not increase significantly.There may be specific aspects of advising within the CAP that our survey items did not capture and that requires further study.Third, although our response rates were high, there is the potential for a non-response bias.Students who chose not to respond may have had less favorable experiences with the CAP than those who did respond.Finally, our outcome measures have not been externally validated.However, we did demonstrate high internal validity of our scales by finding a positive association between scale scores and a single item question assessing overall satisfaction with advising during medical school.There is also evidence for content validity for the scales in that they were created from survey items developed from the expert opinions of faculty who have considerable experience advising students and the published literature on the topic.We have also demonstrated internal construct validity evidence through factor analysis.
In conclusion, we found a significant increase in student satisfaction with advising and perceptions of the relational community during medical school after initiation of a structured advising program, in particular among students who had experienced the program throughout the entire four years of their training.Enhanced satisfaction in these areas may lead to other positive outcomes for students such as academic, clinical and career outcomes, however this requires further investigation.

Table 1 .
Demographic characteristics of the 368 responding graduating medical students*

Table 2 .
Factor analysis for variables measuring student's satisfaction with advising (N=368)* * Medical school graduation years combined 2006, 2007, 2008, and 2009.†Mean score and standard deviation based on responses using a 5-point Likert scale.

Table 3 .
Factor analysis for variables measuring student's satisfaction with advising (N=368)*

Table 4 .
Comparisons of responses to survey items included in advising satisfaction and relational community scale by medical school graduation year using simple logistic regression and presented as unadjusted odds ratios (OR) (N=368)* ¶ (Very satisfied and satisfied vs neutral, dissatisfied, very dissatisfied).