Attitudes toward osteopathic medicine scale: development and psychometrics

Objective: To develop a valid and reliable instrument for measuring attitudes toward osteopathic medicine. Methods: Participants included 5,669 first-year students from 33 U.S. colleges of osteopathic medicine, who completed an online survey at the beginning of the 2019-2020 academic year. Using data from the nationwide Project in Osteopathic Medical Education and Empathy, we developed a 13-item instrument: Attitudes Toward Osteopathic Medicine Scale (ATOMS) and demonstrated the validity and reliability of its scores. The social desirability response bias was controlled in statistical analyses. Results: The corrected item-total score correlations were all positive and statistically significant, and the effect sizes of item discrimination indices were large. Cronbach’s coefficient alpha reliability was 0.83. Construct validity, corroborating face and content validity of the ATOMS, was supported by three components, emerged from factor analysis: “Perspectives on Osteopathic Medicine,” “Osteopathic Diagnosis and Treatment,” and “Holistic-Integrative Care.” Correlations between ATOMS scores and scores of cognitive empathy, emotional empathy; orientation toward interprofessional collaboration; lifelong learning; and burnout were statistically significant in the expected direction, providing validity evidence for the ATOMS. Using the method of contrasted groups, significant differences in the ATOMS scores were found by gender, ethnicity, academic background, and career interest in the expected direction, supporting the validity of the ATOMS scores. National norms were developed to assess individual scores alongside national percentile ranks. Conclusions: The ATOMS, developed in a nationwide study, supported by strong psychometric evidence for measuring orientation toward osteopathic medicine, has implications for the assessment of osteopathic medical education, patient outcomes, and admission decisions.


Introduction
Diagnosis and treatment of illness in the context of holistic care was recognized in 1874 by Andrew Taylor Still, MD, who called his view of medical care "osteopathy" and founded the first osteopathic medical school in 1892 in Kirksville, Missouri (currently A.T. Still University-Kirksville College of Osteopathic Medicine). The core tenets of osteopathic medicine specify that a human is a unit of the physical, mental, and social/spiritual; that the body is capable of self-regulation; and that holistic treatment should be based upon an understanding of body unity, self-regulation, and interrelationships of structure and function. 1,2 Fundamental osteopathic medical competencies include the application of osteopathic manual diagnosis and treatment; the ability to work effectively with other health care professionals as members or leaders of an interprofessional collaborative team; and demonstration of humanistic behavior such as empathy, altruism, compassion, respect, integrity, honesty, and trustworthiness. 3 While osteopathic and allopathic medical education systems currently share most of the aforementioned features in educating physicians-intraining, osteopathic medicine emphasizes manipulative diagnosis and treatment and holistic care.
Attitudes toward specific features and tenets of osteopathic medicine contribute to the career decisions of applicants and to the practice of medicine by graduates of osteopathic medical schools. Also, the measurement of such attitudes with a psychometrically sound instrument would be crucial for the assessment of osteopathic medical, educational outcomes. However, while reviewing the relevant literature, we noticed a limitation in empirical research in identifying core components of the attitudes toward osteopathic medicine. There are a few instruments intended to measure attitudes, orientation, and beliefs toward aspects of osteopathic medicine, such as osteopathic principles, 4 osteopathic manipulative treatment, osteopathic philosophy, [5][6][7][8] and osteopathic education. 9 However, these instruments have not been supported by strong psychometric, especially validity evidence. Study participants were often accessible samples from a single institution and insufficient in size.
There was a need for a valid and reliable instrument, developed without suffering from the aforementioned limitations, for measuring attitudes toward osteopathic medicine. In response to this need, we designed this study to develop a psychometrically sound instrument for measuring empirically derived aspects of osteopathic medicine, with potential implications for the assessments of osteopathic medical education outcomes and clinical outcomes of osteopathic care, and to monitor changes as physicians-in-training progress through medical school and postgraduate medical education training. Also, our intention from the onset of the study was to use nationwide data to provide a national norm table for osteopathic medical students to assess their scores on the instrument against the national norm, and possibly use each osteopathic medical school applicant's converted national percentile rank as a supplementary measure for admission decisions.

The nationwide project
This study is part of the landmark nationwide Project in Osteopathic Medical Education and Empathy (POMEE), a twophased project sponsored by the American Association of Colleges of Osteopathic Medicine and cosponsored by the American Osteopathic Association, in collaboration with the Cleveland Clinic and Sidney Kimmel Medical College at Thomas Jefferson University. Phase I, a 2-year cross-sectional study completed in 2018, laid the foundation for Phase II, a 5-year longitudinal study of changes in empathy and other personal qualities, including attitudes toward osteopathic medicine as students progress through medical school. Data for this article were retrieved from the database of POMEE-Phase II.

Research design and study cohort
Participants in this survey research included 5,669 first-year matriculants to 33 of 36 (92%) of U.S. colleges of osteopathic medicine in the 2019-2020 academic year who voluntarily participated in the study.
The research team at Jefferson obtained exempt status approval for the project from Thomas Jefferson University Institutional Review Board (IRB); all other participating colleges also received IRB approval from their college.

Study survey
The study survey included questions regarding participants' demographics, undergraduate major, and career interest, plus the following instruments:

Attitudes Toward Osteopathic Medicine Scale
We developed a new instrument: Attitudes Toward Osteopathic Medicine Scale (ATOMS); seven items of this instrument were adapted (with author permission) from a 29-item Integrative Medicine Attitudes Questionnaire; 10 six items were developed by two of this study's authors (MH and LHC) for another study with osteopathic medical students. 11 Permission to use selected items from the Integrative Medicine Attitudes Questionnaire was obtained from the author of the questionnaire. Items were answered on a 7-point Likert scale (1=Absolutely Disagree, 7=Absolutely Agree).

Cognitive (Clinical) Empathy
We measured clinical empathy using the Jefferson Scale of Empathy (JSE, 20-item, medical student version), a broadly used and validated instrument for measuring clinical empathy in the context of patient care, developed based on the conceptualization of empathy as a predominantly cognitive attribute. Evidence from medical school student samples in the U.S. and abroad supports psychometrics of the JSE 12 (pp. 84-128, 276-286) and specifically in first-year matriculants of osteopathic medical schools (POMEE Phase I). 13 Moreover, the JSE has been recognized as the most studied instrument in medical education research, 14 and the most frequently used instrument for measuring clinical empathy in medical education. 15

Emotional Empathy
Cognitive and emotional empathy could have different consequences in the context of patient care. 12 Because of its affective nature, excess emotional empathy (synonymous with sympathy) can be detrimental to patient care. 12 We included emotional empathy to differentiate the effects of cognitive empathy (understanding patient's suffering) from emotional empathy (e.g., feeling patient's pain) on outcome measures. We selected the following two scales from the Interpersonal Reactivity Index (IRI): "empathic concern" and "personal distress." [16][17] Each scale contains seven items. The total score of these two scales was considered as an indicator of emotional empathy. [16][17][18] Moderate correlations between scores of the IRI scales and the JSE have been reported in medical students. 19 Permission to use this instrument in this study was obtained from the author of the IRI.

Attitudes Toward Interprofessional Collaboration
We used the Jefferson Scale of Attitudes Toward Interprofessional Collaboration (JeffSATIC), a 20-item validated instrument measuring orientation toward interprofessional collaboration and teamwork in health professions students and practitioners. Evidence supporting measurement properties of this instrument has been reported in a multi-institutional and multi-national study of health professions students. 20

Attitudes Toward Lifelong Learning
We used the Jefferson Scale of Attitudes Toward Physician Lifelong Learning (JSPLL), a 14-item instrument adapted from the Jefferson Scale of Physician Lifelong Learning, 21 for administration to medical students. 22 Evidence has been reported supporting measurement properties of the JSPLL in physicians 21 and medical students. 22

Burnout Measure
We used the Shirom-Melamed Burnout Measure (BM), 23-25 a 14-item instrument to measure overall burnout experiences. The instrument has been used in a multi-institutional study with medical students. 26 Permission to use this instrument in this study was obtained from its author.

"Good Impression" Response Bias
Respondents to self-reported personality tests can manipulate their answers to produce disingenuous responses, known as the "social desirability response set." We used the "Infrequency" Scale of the Zuckerman-Kuhlman Personality Questionnaire (ZKPQ) 27 to control for the effect of the social desirability response bias. This 10-item scale identifies subjects with invalid records due to an exaggerated "good impression" response bias. Scores higher than three on this scale indicates questionable validity of the respondent record. 27 This scale was previously used with medical students to detect and control for the tendency to make "good impression" responses, 12,28 and in POMEE-Phase I. 13,[29][30][31] Procedures Two pilot studies were undertaken with volunteer osteopathic medical students and medical education researchers to improve the clarity and comprehensiveness of the study survey, to detect any possible technical issues in its online administration (pilot study 1), and to test it when using desktop, laptop, and mobile devices (pilot study 2).
One or two research coordinators from each participating college or campus were selected to serve as liaisons between students, the colleges, AACOM, and the research team at Jefferson. Research coordinators and the AACOM research team arranged with college administrators to schedule an appropriate time for online group administration of the study survey at local campuses and helped to maximize response rates.
Participants were informed that their email addresses would be used as a unique identifier to track, match, and merge data from multiple survey administrations. Before the administration of the survey, students received an email signed by the dean of their medical school that included a brief message about the importance of the project and its goals. Subsequently, students received another email message, encouraging them to participate as an "indispensable" stakeholder of this landmark project, signed by Robert Cain, DO, president and CEO of the AACOM, and Leonard Calabrese, DO, of the Cleveland Clinic (principal co-investigators of the project).
We administered the initial web-based study survey at the beginning of the 2019-2020 academic year prior to the start of medical school classes. Our study survey accompanied the AACOM matriculating student survey. Respondents were given the option to voluntarily complete the accompanying study survey. They could also voluntarily enter their email addresses to receive feedback on their empathy scores. Online administration of the study survey was managed by the AACOM research team.

Statistical analyses
We calculated Cronbach's coefficient alpha, and examined corrected item-total score correlations, item discrimination effect sizes, underlying factor structure, and used bivariate correlations (Pearson), multivariate regression analysis, and the method of contrasted-groups to confirm the validity and reliability of the ATOMS, developed in this study. The Statistical Analysis System (SAS for Windows, version 9.4) was used for statistical analyses.

Results
A total of 5,979 students of 7,781 total first-year matriculants in all U.S. colleges of osteopathic medicine (77%) submitted their online survey. Excluded were incomplete surveys and respondents' records with questionable validity (scored>3 on the Infrequency Scale of the ZKPQ). Therefore, the final sample for statistical analyses included 5,669 students; 2,653 selfidentified as male (47%); 2,964 (52%) as female; and 52 (< 1%) did not identify as either male or female.

Preliminary Study of the ATOMS
We performed a preliminary study to examine corrected item-total score correlations and explore underlying factors of the initial instrument (14-item). The corrected item-total score correlations were all positive and statistically significant with the exception of one item that read: "A patient is healed when the underlying pathological processes are corrected or controlled", for which the item-total score correlation was negative and negligible (r=-0.10), with a non-substantial factor loading. We deleted this item; thus, the final ATOMS contained 13 items used for further statistical analyses (see Appendix A).

Item-Total Score Correlations
The corrected item-total score correlations of the final 13item ATOMS instrument (calculated based on the correlation between each item score and total score, excluding the corresponding item from the total score) were statistically significant and moderately high, ranging from a low of 0.29 (p< 0.01) for this item: "Therapeutic touch has been discredited as a healing modality" (a reverse-scored item) to a high of 0.61, p<0.01) for this item: "Osteopathic manipulative therapy is a valuable method for resolving a wide variety of musculoskeletal problems". The median correlation was 0.49 (Table 1).

Effect Sizes of Item Discrimination Indices
The effect sizes of item discrimination indices were calculated by subtracting the item mean score for the top 33% AT-OMS scorers from the mean score of the same item obtained by the bottom 33% ATOMS scorers, divided by the pooled standard deviation of the corresponding item (Table 1). These effect sizes were analogous to Cohen's d statistics. 32 All of the effect sizes were substantially large (> 1.01).

Exploratory Factor Analysis
We examined the underlying construct of the 13-item AT-OMS by conducting exploratory factor analysis, using principal components with oblique (promax) rotation to allow correlations among the factors (Table 1). Three factors emerged, each with an eigenvalue greater than 1 (Kaiser Criterion). The eigenvalues before rotation were 4.41, 1.59, and 1.04, and accounted for 34%, 12%, and 8% of the total variance, respectively. The scree test showed that the plot of eigenvalues leveled off after the third extracted factor, supporting the retention of the three factors. The Kaiser-Meyer-Olkin measure of sampling adequacy (MSA) showed an overall index of 0.88, indicating that data were adequate for factor analysis. Bartlett's test for sphericity indicated that the intercorrelation matrix was factorable (χ 2 (42)=463.82, p<0.0001).
The first factor was entitled "Perspectives on Osteopathic Medicine" (rotated factor loadings ≥ 0.42 in its five items). A typical item representing this factor is: "A strong relationship between patient and physician is an extremely valuable therapeutic intervention that leads to improved outcomes." The second factor, "Osteopathic Diagnosis and Treatment", included five items with factor loadings ≥ 0.46. A typical item representing this factor is: "Touch and tactile approaches may not serve a significant purpose in patient care" (a reverse-scored item). The third factor, "Holistic-Integrative Care", included three items with factor loadings ≥ 0.54. A typical item representing this factor is: "The osteopathic philosophy of holistic care greatly influenced my decision to attend an osteopathic school." The Cronbach's coefficient alphas for the three extracted factors were 0.77, 0.71, and 0.73, respectively.

Descriptive Statistics
The obtained mean and standard deviation of ATOMS scores were 73.9 and 9.5, respectively; the possible and actual score ranges were 13-91 and 29-91, respectively.

Criterion-Related Validity
We examined bivariate Pearson correlations between scores on the ATOMS and those of other personal quality measures used in the study ( Table 2). All obtained correlations were statistically significant, ranging from highs of 0.60 (p<0.01) for interprofessional collaboration, and 0.58 (p<0.01) for clinical empathy, to a low of 0.17 (p< 0.01) for emotional empathy. Correlation between scores of the ATOMS and burnout measure was statistically significant and negative (r= -0.29, p< 0.01).
We performed multiple regression analysis to examine the unique contribution of each of the personal quality measures in predicting scores on the ATOMS. Table 2 shows standardized regression coefficients (β), unstandardized regression coefficients, standard errors, t-values, and statistical significance for the unique contributions of the regressors in predicting ATOMS scores.
Measures of interprofessional collaboration (β=0.33), and clinical empathy (β=0.30) provided the most unique and positive contributions to predicting ATOMS scores in the multivariate model, and orientation toward lifelong learning (β=0.13) and emotional empathy (β=0.08) provided the least. The burnout measure showed a statistically significant negative contribution. The adjusted multiple correlation was R=0.68, meaning that 46% (R 2 =0.68 2 =46%) of the variation in the ATOMS scores could be accounted for by the five regressors ( Table 2).
In the additional analysis, we found a significant inverse association between clinical (cognitive) empathy and burnout (r=-0.21, p <0.01), whereas the correlation between emotional empathy and burnout was positive (r=0.14, p<0.01). This pattern of finding was expected as described in the discussion of findings.

Validity Evidence by the Method of Contrasted Groups
Significant differences have been found on scores on the JSE and gender (in favor of women), 12,31 and on ethnicity (in favor of African-American and Latinx vs Asian-American and White medical students), 31  (a) Based on the content of items with high factor loadings, Factor 1: was entitled "Perspectives on Osteopathic Medicine", Factor 2: "Osteopathic Diagnosis and Treatment", and Factor 3: "Holistic-Integrative Care". Items are sorted by descending order of factor loadings within each factor. Number in parentheses refer to the appearance of the items in the ATOMS. (b) Correlations between scores on each item and the ATOMS total score by excluding the corresponding item from the total score, all were statistically significant (p< 0.01). (c) Effect size estimate (Cohen's d statistic) of the discrimination index was calculated by subtracting the item mean score of the ATOMS high scorers (top 33%) from the item mean score of the ATOMS low scorers (bottom 33%), divided by the pooled standard deviation of the corresponding item.

academic background (in favor
of those with undergraduate college majors (in favor of those with college majors in social and behavioral sciences, and arts and humanities) 31 and career interest (in favor of medical students who planned to pursue "People-Oriented" specialties such as general internal medicine, family medicine, pediatrics, and psychiatry versus others interested in "Technology/Procedure-Oriented" specialties such as pathology, anesthesiology, radiology, and surgery. 12,31 Because of significant and relatively large correlations we observed in this study between the ATOMS and JSE scores, we expected to similarly find significant differences on scores of the ATOMS by gender (in favor of women), ethnicity (in favor of African-American and Latinx), academic background (in favor of those with college majors in social and behavioral sciences, arts and humanities), and career interest (in favor of those planning to pursue "People-Oriented" specialties. Using analysis of variance, we examined group differences on the ATOMS scores by gender, race/ethnicity, academic background, and career interest to find out if group differences were in the expected direction. Means, standard deviations, and summary results of statistical analyses are reported in Table 3.

Gender Difference
The ATOMS mean score for men was 71.5 (SD=10.0), and for women was 76.1 (SD=8.3). Gender difference in favor of women was statistically significant (F(1,5615)=362.31, p< 0.0001). The difference was also practically important, as indicated by the effect size of 0.51.

Race/Ethnicity Differences
The highest mean score on the ATOMS was obtained by Black/African American students (M=77.27, SD=9.2), and the lowest by Asian students (M=72.88, SD=9.5). The mean scores of the White and Hispanic/Latinx/Spanish origin groups were in between the other two groups. The differences in favor of the African/American group versus Asian, White, and Hispanic/Latinx/Spanish origin groups were statistically significant (F(3,5099)=15.80, p<.0001). Also, Hispanic/Latinx/Spanish origin groups obtained mean scores that were significantly higher than those obtained by White and Asian groups. The race/ethnic differences were practically important (effect size between the highest and lowest scoring groups=0.47).

Academic Background
Respondents were asked to report their undergraduate major by choosing from a list of 56 undergraduate majors (sorted alphabetically). For statistical analysis, we grouped the undergraduate majors into the following four broad categories: "Biological Sciences," "Chemical/Physical Sciences," "Social/Behavioral Sciences," and "Arts and Humanities." We compared respondents with different undergraduate majors on ATOMS scores. The majority reported their undergraduate degree in "Biological Sciences" (n=2,833), followed by those who majored in "Chemical/Physical Sciences (n=755), "Social/Behavioral Sciences" (n=264), and "Arts and Humanities" (n=91). The lowest ATOMS mean score was obtained for those who majored in "Chemical/Physical Sciences" (M=71.8, SD=9.7), which was significantly lower than the scores in the other three academic background groups (F(3,3939)=13.04, p< .0001).

Career Interest
Respondents were asked to choose the specialty they planned to pursue after graduation from medical school from a list of 33 specialties, most frequently pursued by graduates of colleges of osteopathic medicine. Based on other studies with allopathic 33 and osteopathic medical students, 31 we divided the specialties into three broad categories: "People-Oriented" (e.g., family medicine, internal medicine, obstetrics and gynecology, and pediatrics); "Technology-/Procedure-Oriented" (e.g., anesthesiology, dermatology, ophthalmology, orthopedic surgery, radiology, and surgery); and "Other" (including specialties chosen by fewer than 20 matriculants).

National norms
Using a national sample in this study provided a unique opportunity to develop national norms for the ATOMS scores that will enable medical colleges to determine the percentile rank of any new matriculant to osteopathic medical schools. Because of the gender difference in ATOMS scores observed in this study, we calculated percentile ranks for men and women separately (Table 4). For example, if the ATOMS score of a male matriculant is 80, first find the score interval that includes a score of 80 (79-80 score interval in Table 4), then find the corresponding national percentile rank displayed in the row for that score interval in the table. The corresponding national percentile rank in the table for a male matriculant with an ATOMS score of 80 is 78%, meaning that a score of 80 places a male matriculant in the 78th percentile rank of all first-year male matriculants. However, a female matriculant with an ATOMS score of 80 would be at the 61st percentile rank. If the gender is unknown, then the percentile rank on the norm table for men and women combined can be used to estimate.

Discussion
Using a nationwide sample, we developed and validated an instrument to measure students' orientations toward osteopathic medicine. Moreover, the study allowed us to prepare national norms that will enable medical colleges to determine the percentile rank of first-year matriculants in U.S. osteopathic medical colleges. More importantly, in all statistical analyses, we controlled for the effect of social desirability bias by excluding those who attempted to give a "good impression" response and scored above the cutoff of the Infrequency scale of the Zuckerman-Kuhlman Personality Questionnaire. 27 This study is unique, because to our knowledge, with the exception of studies in which we retrieved data from the Project in Osteopathic Medical Education and Empathy (POMEE), no other published study in medical education has been undertaken in which a large nationwide sample of medical students participated, and in which the social desirability response bias, which is a shortcoming of self-reported personality tests, was controlled.
Our findings on the magnitude and direction of item-total score correlations indicate that items of the ATOMS contribute significantly and positively to the total score. Cronbach's alpha coefficient of 0.83 for the ATOMS scores is in the acceptable range for psychological and educational tests. The large magnitude of effect sizes of item discrimination indices confirms the ability of ATOMS items to discriminate between students with the most favorable and the least favorable attitudes toward osteopathic medicine. The large magnitude of effect sizes indicate that the difference in mean item scores between high and low scorers in favor of high ATOMS scorers were not only statistically significant but also practically (clinically) important. 32 The three underlying factors of the ATOMS that emerged from factor analysis not only corroborate the face and content validity of the instrument but also made it possible to recognize and quantify core components of orientation toward osteopathic medicine. The criterion-related validity of the ATOMS scores was supported by statistically significant and positive correlations with scores of conceptually relevant measures. In particular, higher correlations with scores from the orientation toward interprofessional collaboration and clinical empathy (conceptually more relevant to competencies of osteopathic medicine) support the "convergent" validity of ATOMS scores. Conversely, lower correlations with measures of attitudes toward lifelong learning and affective empathy (conceptually less relevant to core tenets of osteopathic medicine) support the "discriminant" validity of ATOMS scores.
Patterns of findings in the expected direction obtained by using the method of contrasted groups provided additional evidence in support of the validity of the ATOMS scores. Because of the significant correlation found between ATOMS and clinical empathy (JSE) scores, we expected to find group differences similar to those in our previous research on empathy. For example, the ATOMS mean score was significantly higher for women than for men, a pattern of difference observed for clinical empathy in allopathic 33,34 and osteopathic medical students. 31 Also, group differences in ATOMS scores by race/ethnicity was consistent with previous findings regarding JSE scores in osteopathic medical students. 31 Similarly, differences in ATOMS scores by academic background were consistent with previous findings regarding JSE scores among osteopathic medical students. 31 Group differences in ATOMS scores by career interest were consistent with previous findings regarding JSE scores among allopathic 33 and osteopathic medical students. 31 The inverse relationship between ATOMS scores and burnout scores also supports the validity of ATOMS scores, consistent with other studies. 35 Additional research is needed to explain the difference in the direction of correlation between ATOMS scores and clinical empathy scores as opposed to emotional empathy scores. Perhaps the cognitive nature of clinical empathy (measured by the JSE) as opposed to affective nature of emotional empathy (measured by the subscales of the IRI) could explain their corresponding positive and negative correlations with the ATOMS scores.
A limitation of the findings is that national norms developed for new matriculants cannot be used for students in different years of medical school unless further empirical evidence verifies that ATOMS scores do not significantly change as students progress through medical school, which seems unlikely, based on previous findings. 6,30 administrative assistant at Jefferson, for managing administrative aspects of the project in addition to editorial and stylistic modifications of the manuscript; and Pamela Walter for her editorial polishing help. We thank Dr. Craig D. Schneider for granting us permission to adapt and use 7 items from the Integrative Medicine Attitude Questionnaire, Dr. Mark Davis for granting us permission to use two scales of the IRI, and Dr. Samuel Melamed for his permission to use his burnout scale. We are also thankful to the deans of participating colleges of osteopathic medicine who notified and encouraged the study cohort in their schools to participate and continue their cooperation in the project, and finally we thank thousands of osteopathic medical students who participated in this study and completed the study survey.

Funding support
This study was funded by the American Association of Colleges of Osteopathic Medicine (AACOM) and cosponsored by the American Osteopathic Association (AOA). The funding was terminated at the end of the 2019-2020 academic year, due to changes of leadership in the AOA and shift of research priority.