Recognizing diverse forms of scholarship in the modern medical college

Objectives: To determine which metrics of scholarly output have the most impact on decisions for faculty promotion, as new forms of scholarship are evolving at medical schools to answer an emerging need for increased skills specialization and interdisciplinary collaboration. Methods: University of Iowa Carver College of Medicine (UICCOM) data was used to test associations between important scholarship parameters and promotions outcomes. Two analyses were carried out. One evaluated summary data for tenure track faculty seeking promotion from Assistant to Associate professor over a 5-year period, and correlated success with the number of publications over different periods of the research career. The second sought to identify predictors of a positive tenure decision for faculty with an MD and/or a PhD, examining factors such as research publications, grant awards, timing of career transitions and departmental affiliations. Results: Promotion correlated significantly with the number of first (or last) authored publications since initial appointment (p<0.001), but not with the number of lifetime peer-reviewed publications. The best predictor of tenure was the receipt of at least 1 grant award; this held true for both MDs and PhDs. Conclusions: Despite the evolving need for faculty member involvement in interdisciplinary and collaborative projects that may not provide them with authorship opportunities or independent funding, our data suggest that they continue to be assessed based on these criteria. Medical colleges are encouraged to adopt more inclusive scholarship definitions that better fit with the reality of todays’ academic mission.


Introduction
Scholarship may be the heart of academic medicine, but scholarship assessment is its Achilles' heel.At a time when most institutions face funding challenges, there is mounting pressure to evaluate and promote the scholarly productivity of their faculty.Yet, there are few objective measures of scholarly productivity that can universally be applied for evaluating success across a range of academic disciplines and missions that constitute a typical medical college.
Scholarship assessment is also a challenge at the University of Iowa Carver College of Medicine (UICCOM) where the faculty largely consists of clinicians providing care, scientists performing research, and clinician-scientists having a hybrid of responsibilities.While academic and professional interests among these groups vary, scholarly achievement in the tenure track has traditionally been defined as peerreviewed publications and the attainment of external Correspondence: Donald W. Black , Department of Psychiatry, University of Iowa Carver College of Medicine, 2-126b Medical Education Building, Iowa City, IA 52242, USA.Email: donald-black@uiowa.edufunding, preferably through the National Institutes of Health (NIH).
These simple metrics are no longer sufficient in the evolving academic mission.At UICCOM, we have an increasing number of faculty members with specialized skills whose scholarly output is less conventional.This is in part due to the introduction of a "clinical track" at UIC-COM in 1995 for those whose main job is to provide patient care and to teach, rather than to publish original research or obtain grant funding.Commonly referred to as "clinicianeducators" or "clinical-scholars" at other institutions, nearly 45% of faculty members at UICCOM now fall within this track, a percentage that is likely to grow.While evidence of scholarly productivity is necessary for promotion, for clinical track faculty this could take the form of educational excellence, programmatic development, or administrative achievement.][3][4][5][6] Even among those on the more traditional "tenure track", there are faculty members, often holding PhDs, who pursue highly interdisciplinary forms of research that do not easily lend themselves to independent publications and/or funding.Examples include biostatisticians and those involved in neuroimaging science.In each case, these individuals may not have their own external funding or peer-reviewed publications, yet they play an essential role as part of a well-oiled and productive interdisciplinary team.In developing institutional programs to promote scholarly productivity, we are challenged to ask: "Will our traditions and policies of scholarship assessment promote these growing areas of uniqueness or antagonize them?"Clearly, medical colleges should recognize and encourage increasingly diverse forms of scholarship that drive interdisciplinary and forward-thinking enterprises.
In light of these considerations, we thought it appropriate to examine scholarship from differing perspectives.Looking to our past, we used UICCOM data to examine associations between scholarship parameters and promotion outcomes.Looking to our future, we address the extent to which the Boyer's recommendations have been embraced by peer-institutions.Last, we synthesize this material and make recommendations for the consideration of promotion and tenure in the modern medical college.

Methods
The UICCOM has the tripartite mission of teaching, clinical service, and research.The near 1000 faculty members are divided among three tracks: tenure (45%), which emphasizes traditional forms of scholarship, for example hypothesisdriven research; clinical (45%), which emphasizes clinical, educational, or administrative excellence; and research (10%), a relatively new track focused on traditional scholarship without the protection of tenure, but too new to significantly contribute to our analyses.To assess potential associations between scholarship metrics and promotion, we examined anonymous summary data of all tenure track faculty seeking promotion from Assistant to Associate Professor from 2006 to 2011 (n=78).Three measures of scholarly productivity were compared, including total number of lifetime peer-reviewed publications, total number of peer-reviewed publications since appointment, and total number of first (or last author) publications since appointment.Means between those promoted (n=68) and those not (n=10) were compared using a two sample t-test.
In another analysis, data from 195 persons hired from 1992 to 2002 were used to assess predictors that impacted a faculty member's likelihood of attaining tenure.Predictors included number of publications at the time of appointment, grants received (0=none, 1=one or greater) by type (e.g., RO1/PPG, VA Merit, K, foundation, other NIH grants, other non-NIH grant); length of time (years) since degree to entering tenure track; primary appointment department (1 = clinical 2= basic science); mentor's primary appointment department (1=clinical 2=basic science); MD degree granting institution (NIH Top 30 Ranked Institutions, 0=No, 1=Yes), and fellowship institution (NIH Top Ranked Institutions 30, 0=No, 1=Yes)."Success" was defined as attainment of tenure (0=No, 1=Yes).Logistic regression analyses were initially run separately by faculty degree (MD, PhD) to identify relevant predictors of tenure and subsequently run together to increase statistical power.Faculty with both MD and PhD degrees (n=39) were too few for a separate regression analysis and were included in the analysis of the entire sample.The number of publications at tenure was not available for faculty members who left the institution prior to a tenure decision.For that reason, there were too many missing values for this variable to be used as a predictor in the regression analysis.
Because the data were collected for administrative purposes and were anonymous, there was no need to seek Institutional Review Board approval for their use.

Results
For the first analysis, peer-reviewed publications were a significant factor contributing to overall assessment of scholarly productivity (Table 1).The group had a mean (SD) age at appointment of 36.7(4.8)years.Their total number of publications since appointment and total number of first (or last author) publications since appointment were both statistically significant (p<0.001) in distinguishing those promoted from those not promoted.The total number of lifetime peer-reviewed publications did not distinguish the groups.
In the second analysis, we looked at whether there were predictors that significantly impacted a faculty member's likelihood of promotion in the tenure track.In this group, 46 faculty members (23.6%) were women and 83.6% were non-Hispanic Caucasians.Their mean (SD) age at the time of hire was 35.7(4.1)years.It was clear that success in obtaining external funding also influenced assessment of scholarly productivity.When data from MD and PhD faculty were analyzed independently, having received ≥1 grant was found in both cases to be the single best predictor of tenure.The type of grant related to tenure did not vary appreciably by faculty degree (MD/PhD).RO1/PPG, K, foundation, and "other grants" were all significantly associated with tenure for MD and PhD faculty.To obtain more precise estimates of predictor impact, the data were pooled and regression analyses run on the entire sample.The resulting logistic regression equation correctly predicted 87% of the tenure outcomes, with the predicted probability of attaining tenure by the type of grant awarded ranging from 0.23 (None) to 0.96 (R01/PPG).

Discussion
Promotions outcomes among tenure-track faculty were strongly associated with number of publications and presence of external funding.While the UICCOM has no policy regarding numerical requirements for publications or grants, these findings indicate that these factors clearly contribute to scholarship assessment.For many faculty members, grants and publications are de facto measures of productivity.We believe an analysis among more recent promotion decisions at UICCOM would yield similar results, although perhaps showing that the type of grant was of growing importance (with large potentially renewable grants such as NIH R01 grants favored).Considering these data, do our traditions and policies regarding scholarship assessment impede growing areas of unique contributions to scholarship?There are now tenure track faculty members at Iowa and elsewhere whose work does not result in first or last authored publications or external funding who may be at a disadvantage for promotion.For example, people who are part of an interdisciplinary team, those who pursue less common fields of scholarship (e.g., educational research), or those whose research may be described as involving qualitative (rather than quantitative) methods, could have difficulty with promotion.Likewise, with research funding becoming harder to obtain as the NIH budget remains flat (or declining), traditional promotions decisions may need to take this discouraging fact into account.
These analyses focus on persons in the tenure track and do not address the thorny issues that remain with clinical track promotions.First, because there is less of a tradition with this track at UICCOM, and current guidelines provide only a modest description of the types of scholarly productivity necessary, clinical track faculty may have a poor understanding of what promotion requires.Further, department heads, and others charged with advising and assessing a faculty member's appropriateness for promotion may not understand the requirements, or might inadvertently apply stricter tenure-track guidelines.Thus, any solution involves providing more detailed guidelines, educating department heads and faculty about them, and providing appropriate mentorship to help faculty understand the requirements of their track and what the promotions process entails.
The UICCOM is not alone in grappling with the assessment of diverse forms of scholarship.In considering scholarship assessment at UICCOM, we examined promotions guidelines posted on the Internet for many peer institutions.While there is substantial heterogeneity with scholarship expectations, particularly with regard to rank and track, most institutions emphasize the importance of assessing the independence and impact of a faculty member's scholarship.Few offer "formulas" suggesting how productivity can be quantitatively assessed.Some institutions offer a college-wide definition, while others defer to departments to establish standards appropriate to their needs and traditions.Given the influence and widely cited nature of Scholarship Reconsidered, we were not surprised that some institutions appear to have fully embraced Boyer's definitions (e.g., University of Washington, University of Michigan). 7,8 useful framework for defining and assessing diverse forms of scholarship was proposed in the 1990's by Boyer 7,8 and Glassick 9 who spearheaded projects for the Carnegie Foundation for the Advancement of Teaching.Boyer 7 argued that colleges and universities needed to adopt broader definitions to recognize that scholarship is not a uniform activity.He described scholarship as having four domains: discovery, integration, application, and teaching.Glassick 9 later proposed a method for assessing scholarship on the basis of clear goals, adequate preparation, appropriate methods, outstanding results, effective communication, and a reflective critique.Appreciating its advantages, this body of work has shaped the values of many institutions and some have chosen to incorporate many of these principles into their expectations of and definitions for scholarship.10 Arguably revolutionary at the time, Boyer's text is more than 20 years old and predates today's focus on terms such as "translation", "integration" and "interdisciplinary".We believe his definitions hold up remarkably well and are germane to the mission of modern medical colleges.
His four domains include: 1) Discovery, or the generation of new knowledge; 2) Integration, or placing new knowledge "into a larger pattern" 7 3) Application, or putting new knowledge into practice; and 4) Teaching (i.e., "educational scholarship") involves the transmission of knowledge.These domains are not discontinuous and independent; rather they intertwine and overlap.For example, a basic scientist may be involved in the scholarship of discovery, but if the work is funded by the NIH it may have elements of integration or application.A clinician's work might emphasize application, yet include both discovery and teaching.
In terms of teaching and educational scholarship, Boyer says: "If there is any reality about scholarship, it is that it is a communal act.It only takes on life when it is shared". 8haring of knowledge can occur through traditional didactic lectures, one-on-one mentoring, or through publication or other forms of public dissemination.This form of educational scholarship goes beyond excellent teaching.The true scholar not only brings a masterful approach to his or her content, methods and programs, and publishes or presents his or her work, but also builds a platform for other educators and researchers to use. 11A version of Boyer's recommendations for scholarship assessment could be a helpful tool in promoting diverse forms of scholarship.
Dissemination is a vital determinant of whether a scholarly activity has taken place: has the faculty member published a paper, book chapter, or abstract?Given a presentation at a professional meeting or conference?While creative work not leading to publications or presentations can be valuable in its own right, without the element of appropriate critique including traditional peer-review, it most likely cannot be described as scholarship (e.g., giving media interviews, providing expert testimony).The challenges of digital scholarship are relatively new, and its standards evolving, but it highlights the question of what constitutes appropriate critique for this medium. 12ommunity-engaged scholarship constitutes another challenge to how dissemination is considered.For example, while some products of community-engaged scholarship inherently involve peer-review (e.g., traditional peerreviewed articles, or policies at the community, state or federal levels), other types of important work may not (e.g., implementation of innovative intervention programs, resource guides, newspaper articles, presentations to policy makers).Boyer's scholarship domain of integration is probably the most vexing.Practical examples include research publications, review articles, collaborative grants, and translational investigations.True integration requires that connections be made across disciplines to bring new insights from specialized fields into a larger context.Dauphinee and Martin suggest that scholarship of integration has been slow to gain acceptance in the biomedical sciences because it is often seen by many faculty as potentially unrewarding or even risky to their careers. 14Promotion based on objective measures of scholarship such as publications or grants have been traditionally thought to encourage safer, less developed, non-collaborative research projects. 15Smaller institutions were found to be more amenable to integrative scholarship because there were fewer physical barriers to interactions and cross-disciplinary work than larger ones.Dauphinee and Martin recommended that leaders in academic medicine, particularly at places where departments are isolated from one another, needed new strategies to promote and support interdisciplinary research. 14s biomedical science has become more interdisciplinary and collaborative, some medical colleges have placed greater emphasis on scholarly integration.By incorporating recognition of collaboration in their definition or in tenure and promotions policies, 10 these medical colleges have made the tacit acknowledgment that biomedical research has become more specialized and complex.The complexity of modern biomedical research has created a need for investigators to connect with peers in other fields to bring relevant new insights and directions and skill sets to their work.To some extent, the recent emphasis at NIH on translational science has prompted institutions to actively promote collaborations between basic scientists and clinicians.In many respects translational medicine epitomizes the scholarship of integration in the biomedical sciences. 15,16It transforms knowledge gained from studies of genes, proteins and cells into innovative treatments for complex human diseases, thereby translating research discoveries into something useful to the public ("from bench to bedside").
Impediments to interdisciplinary collaboration remain.For instance, collaboration might take a faculty member out of his or her "comfort zone" because it requires learning new concepts and terminology from another discipline.It could temporarily compromise a person's productivity as he or she devotes time to starting up new ventures.Interdisciplinary efforts may initially need to be encouraged and fostered by medical college administrators.Administrators and faculty members need to think beyond the short term and consider what it takes to achieve long-term gains that may ultimately contribute to the greater good.To achieve this, medical colleges must provide scientific leadership and administrative support.Further, it is important to clarify how to assign credit for interdisciplinary and collaborative research, including determining authorship assignments (middle author vs. first/last authorship) and investigator status on grants (e.g., PI vs. co-PI,).This could raise questions about independence during the promotion process.Clearly, the timing, nature of the work, amount of funding available and commitment of the collaborator(s) will influence the success and impact of the integrative research on a person's promotion and tenure decision.While such activity may have been thought too risky in the past, we believe the benefits of collaboration outweigh the risks.
Many organizations have moved toward implementing policies and promotion rubrics reflecting Boyer's call to adopt broader definitions of scholarship. 4,10,17,18Despite the popularity of Boyer's model, its adoption has not been met with universal success.Schweitzer describes how the model was adopted at the University of Louisville in the early 1990's as one of several ambitious initiatives launched by the Board of Trustees. 17As discussed, faculty members were neither adequately prepared for the recommendations, nor appropriately educated about them.There was little buy-in or explanation about how Boyer's model could apply to "hard science".This experience shows that any redefinition of scholarship requirements must be accompanied by appropriate education about the standards and with input from stakeholders.

Conclusions
We believe Boyer's scholarship definitions are more relevant today than when first published.Traditional definitions of scholarship that emphasize publications and grants fail to appropriately recognize the many diverse forms of scholarship encountered in a modern medical college.While writing papers and obtaining external funding will continue to be core values for many, medical colleges must be able to appropriately support faculty members engaged in interdisciplinary and collaborative projects that encourage forward thinking enterprises.We encourage medical colleges to adopt and apply more inclusive scholarship definitions that better fit with the reality of today's academic mission.We recommend the following: 1. Defining scholarship: colleges should adopt more inclusive definitions that encourage creative and interdisciplinary enterprises.Each person has a contribution to make to the medical enterprise and scholarship definitions must take the faculty member's unique talents and areas of responsibility into account.Scholarship definitions should take the various tracks into account.See the Appendix for examples of work drawn from the literature that fall within Boyer's definitions. 4,10,172. Sharing responsibility: promoting diverse forms of scholarship is a shared responsibility.Department heads should establish clear expectations for scholarly activity with faculty members.Faculty members will be responsible for crafting personal statements, creating annotated curriculum vitae, or developing educator portfolios that explain and clarify the value of their work, including evolving standards of digital scholarship if applicable.3. Continue assessing: while greater emphasis should be given to guiding and mentoring junior faculty members (i.e., Assistant Professors) than those at other ranks in order to foster emerging careers, those in the upper ranks must not be forgotten.They should be provided on-going support and encouragement, particularly those who may have hit a "snag" in their careers.Faculty scientists and clinicians at all levels must be valued and their careers nurtured.Medical colleges cannot afford to waste talent.

Limitations
This paper evaluated data from a single medical college and presented data from the tenure-track only.Analysis of data from other medical colleges and other tracks may produce different results.

Table 1 .
Mean number (±SD) of peer-reviewed publications for tenure track faculty at the UICCOM

Appendix: Examples of diverse forms of scholarship
Publication of peer reviewed translational research.At a minimum, there should be dissemination through public venues that allow others to build on the work and critique it.While first/senior author authorships in professional journals or books is not required, a means of demonstrating a community valuing the specific contribution of the individual needs to be established. Publication of books, monographs, manuals, on paper or in electronic media  Invited or submitted presentations of translational data at major national or international meetings, major institutions, or research organizations  Demonstration of a sustained, externally funded, translational research program.At a minimum, there should be contributions to external funding agencies supporting the work, including salary support and, if appropriate, instrumentation. Invention or expert use of an instrument, technique, or process bringing advances from other fields into medical research, clinical practice, education, or outreach with the broader community  Demonstration of a tangible and medically significant "deliverable" resulting from the work which may include items such as new patents; instruments; software; policies at the community, state, or federal levels; publications; or educational programs that would otherwise not exist.A case for impact of the deliverable should be clear. Multidisciplinary work including peer-reviewed publications of research, policy analysis, case studies, meta-analyses, book chapters. Interpretation of scholarly work for the wider public.Publication of books, monographs, manuals, on paper or in electronic media. Invited or submitted presentations of original clinical data at major national or international meetings, major institutions, or research organizations. Demonstration of a sustained, externally funded, clinical research program. Development and evaluation of new forms of treatment, new surgical procedures, innovative intervention programs, or innovative diagnostic techniques. Organization of a new or reorganization of an existing, clinical service. Innovation or improvement of an existing clinical service, as evidenced by addition of new services, significant increase in the volume of patients, better patient outcomes, increased revenue production. Interpretation of scholarly work for the wider public.Teaching  Publication of peer reviewed educational research.Publication of books, monographs, manuals on paper or in electronic media. Invited or submitted presentations relevant to medical education at major national or international meetings, major institutions, or educational organizations. Published reviews of educational materials developed by the faculty member. Curriculum development: development of objectives, materials and methods, methods of evaluation, etc.  Organization of a new teaching program, or integration of teaching effort within or between departments. Development of teaching techniques. Development of short courses or "workshops" for students, residents and fellows, postgraduate professionals, and lay public. Development of teaching materials, such as the preparation of a syllabus, book of procedures, course of study, laboratory manual, development of testing procedures or other modes of evaluation.This would include educational efforts directed at students, residents and fellows, postgraduate professionals, and the lay public. Interpretation of scholarly work for the wider public.