Virtual global health in graduate medical education: a systematic review

Objectives To synthesize recent virtual global health education activities for graduate medical trainees, document gaps in the literature, suggest future study, and inform best practice recommendations for global health educators. Methods We systematically reviewed articles published on virtual global health education activities from 2012-2021 by searching MEDLINE, EMBASE, Cochrane Library, ERIC, Scopus, Web of Science, and ProQuest Dissertations & Theses A&I. We performed bibliography review and search of conference and organization websites. We included articles about primarily virtual activities targeting for health professional trainees. We collected and qualitatively analyzed descriptive data about activity type, evaluation, audience, and drivers or barriers. Heterogeneity of included articles did not lend to formal quality evaluation. Results Forty articles describing 69 virtual activities met inclusion criteria. 55% of countries hosting activities were high-income countries. Most activities targeted students (57%), with the majority (53%) targeting trainees in both low- to middle- and high-income settings. Common activity drivers were course content, organization, peer interactions, and online flexibility. Common challenges included student engagement, technology, the internet, time zones, and scheduling. Articles reported unanticipated benefits of activities, including wide reach; real-world impact; improved partnerships; and identification of global health practice gaps. Conclusions This is the first review to synthesize virtual global health education activities for graduate medical trainees. Our review identified important drivers and challenges to these activities, the need for future study on activity preferences, and considerations for learners and educators in low- to middle-income countries. These findings may guide global health educators in their planning and implementation of virtual activities.


Introduction
Global health (GH), a rapidly growing field focused on advancing international and interdisciplinary healthcare [1][2][3][4][5][6][7][8][9][10][11][12] while addressing health inequities, 13 is an increasingly common component of graduate medical education and international partnerships. 1,14 The COVID-19 pandemic disrupted in-person GH education (GHE) activities such as international clerkships and rotations 15,16 and worsened inherent inequities in GH. 17,18 Typical challenges encountered in GHE work, including distance, communication, and barriers to bidirectional exchange of staff and learners 6 worsened throughout the pandemic, 19,20 highlighting the need for thoughtful development of virtual GH curricula and practice. Since the start of the pandemic, much has been published on shifting graduate medical education activities into the virtual realm, but little research focuses on virtual approaches to GHE, particularly within GH partnerships where barriers such as poor internet access persist. 14,[21][22][23] While several papers discuss the use of virtual education for GH preparation, simulation, and education, 7, 24-29 ethical considerations in GH engagement, [30][31][32] and clear learner competencies for GHE within GH partnerships, 1,24,25,27,28 limited guidance exists regarding methods to virtually sustain or improve formerly inperson GHE activities during the pandemic or similar disruptive global challenges. Few previous papers focus on supporting partners in low-to middle-income countries (LMIC) during times of crisis, 26,33 and it is unclear how GH competencies can be reinforced virtually for learners in high income countries (HIC) while prioritizing the needs of partners in LMIC. 20,21,34,35 Last, to our knowledge no current studies examine faculty or learner preferences for virtual GHE activities (VGHEAs).
Virtual GH content is necessary and relevant now due to current travel restrictions, but this mode of engagement will undoubtedly be a key component of GHE moving forward. 15 Hindrances from financial constraints, ongoing travel restrictions, threats of future COVID-19 variants, and equitable access to vaccination may continue to limit in-person GHE activities. 19,20 VGHEAs may provide the GH community with lower cost, more attainable engagement strategies, and may facilitate mutual learning, goal setting, and problem solving.
There is a crucial need for evidence about VGHEA planning, implementation, and continuation, particularly regarding the specific needs of learners in LMICs, to guide GH educators and the creation of GH programming. This systematic review, therefore, aimed to identify and synthesize recent VGHEAs (including their enablers and barriers) targeting health professional trainees of any level, to document gaps in the existing literature, to identify areas of future study, and to contribute to preliminary foundational data to inform future best practice recommendations for GH educators.

Methods
We used the Preferred Reporting Items for Systematic review and Meta-Analysis (PRISMA) Protocols 2015 Checklist 36 to perform our systematic review, which we chose as the most appropriate methodology to summarize recent VGHEAs over our review period. We registered the general systematic review protocol with PROSPERO on February 14th, 2021. 37 Ethical approval was not required for our review.

Eligibility criteria
Inclusion in this review required that articles from the primary literature between 2012-2021 focus on existing and sustained GH curricula, programs, activities, or online content. Our definition of "GH content" included any activity highlighting health disparities due to resource level, geography, or access to care. The administration of the GH content had to be primarily virtual, not supplementary to an in-person activity. The target users of the content had to be health professional trainees of any level or specialty. We chose to include articles between 2012-2021 to focus our evaluation on more recent technology and on articles with more robust descriptions of virtual activities.
Our review excluded online content not otherwise described in the primary literature; general open access resources without a stated objective to reach trainees in underresourced or LMIC settings; descriptions of telemedicine services; and non-human GH topics. If multiple papers described the VGHEA, our review included only the most recent article. Our review also excluded Project ECHO (Extension for Community Healthcare Outcomes) 38 discussions, as they are not trainee-focused and were outside the scope of this manuscript. Please see Appendix 1 for full inclusion and exclusion criteria.

Search strategy
A medical librarian (M.S.) constructed a comprehensive search strategy to capture the concept of VGHEAs (Appendix 2). We used the strategy to search the following databases on November 4, 2021: Ovid MEDLINE®, Ovid Embase, Cochrane Library from Wiley, Education Resources Information Center (ERIC, via EBSCO interface), Scopus via Elsevier, Web of Science from Clarivate Analytics, and ProQuest Dissertations & Theses A&I. One co-author (N.E.H.) searched the grey literature sites per the strategy in Appendix 2. Two authors (N.L. and L.U.) also reviewed the references for pertinent articles.

Article selection
We used Covidence software 39 to manage the systematic review process. Two reviewers (L.U. and N.L.) performed the initial article screening by assessing titles and abstracts from the search. Article exclusion occurred if they lacked a GHE or virtual focus. After the initial exclusion process, L.U. and N.L. independently reviewed the full text of the remaining articles to determine whether articles met the predetermined eligibility criteria. Because the heterogeneity of articles included did not lend to formal quality evaluation, we jointly determined our parameters for making judgements and used three general ratings. "Good" and "fair" articles met inclusion criteria and included information on at least >75% or 50-75%, respectively, of planned data extraction points. "Poor" articles did not adequately meet inclusion criteria and/or did not contain sufficient information for data extraction. We included "good" articles, excluded "poor" articles, and further discussed "fair" articles to reach consensus. A third reviewer (S.K.L.) settled disagreements on inclusion or exclusion via collaborative consultation.

Data extraction
Members of the study team independently extracted data from the articles in an Excel spreadsheet. Three reviewers (L.U., N.L. and N.E.H.) then cross-checked extracted data. Extracted data included: activity type, synopsis, ownership, length, frequency, content delivery, cost, evaluation, outcomes; targeted participant type, numbers, and location; drivers/enablers, barriers/challenges, and impact. We organized the VGHEAs into 8 activity types: synchronous activities (e.g., discussions, conferences, chats, skills sessions, simulations, or lectures); asynchronous activities (e.g., modules, videos, or pre-recorded lectures); group learning or projects; shared cloud resources; complete online GH courses; virtual mentorship; paired learning ("twinning") experiences; and online discussion forums.

Data synthesis and analysis
We performed a qualitative summary of the data given the nature of the systematic review and the preponderance of descriptive statistics in included papers. We summarized descriptive data, identified common collective themes, and noted gaps in available information.

Topic/focus of VGHEA
The complete list of topics covered in the described VGHEAs are listed in Table 1. Most articles (68%, 27/40) focused on general GH topics (e.g., global health education, community health, or field experiences) while 32% (13/40) focused on GH topics linked to a medical specialty (e.g., anesthesia or surgical training in LMICs). While the vast majority (95%, 38/40) of articles focused on international GH, two articles (5%, 2/40) focused on local GH. One paper (3%, 1/40) had health equity and equitable partnerships as a key focus.
Overall, few articles (10%, 4/40) reported details about trainee characteristics and rates of activity completion. One article (3%, 1/40) documented dropout rate of trainees through duration of the program, another (3%, 1/40) reported a documented increased participation rate over a twoyear period during the activity, and two papers (5%, 2/40) provided a comparison of participation rates between trainees from HIC versus LMIC.

Evaluation and outcomes of VGHEAs
Most articles (90%, 36/40) discussed VGHEA evaluations. The most reported evaluation method was participant surveys (57%, 23/40 of articles). Different outcome measures discussed are available in Table 1, the most common being satisfaction with course, content, or teaching (60%, 24/40 of articles) and self-reported improvement in knowledge or skills (40%, 16/40 of articles). Detailed evaluation methods, however, were not a common feature of included articles.

Participation in VGHEAs
The 40 included articles described 66 countries (73%, 48/66 LMIC and 27%, 18/66 HIC) as having participated in the VGHEAs. A HIC (USA) was the most frequent consumer of VGHEAs, followed by India, the UK and Uganda.

Drivers/enablers and barriers/challenges of VGHEAs
Most papers discussed drivers/enablers (93%, 37/40) and barriers/challenges (98%, 39/40) of VGHEAs (Figure 2, Panel A and B, respectively), which we grouped into 14 categories each. The most common drivers/enablers were strong course content and organization (40%, 16/40 of articles); peer interactions (38%, 15/40 of articles); and activity ease/flexibility (30%, 12/40 of articles). The most common barriers/challenges were challenges to online trainee engagement (unequal participation/engagement or lack of interest/motivation; 48%, 19/40 of articles); issues with virtual platforms/technology or internet connectivity problems (45%, 18/40 of articles); and challenges with time zones or course hours (33%, 13/40 of articles). Unexpected impact of the course (positive or negative) and wider benefits noted: Overall, 58% (23/40) of included articles cited a wider positive impact of the VGHEA beyond what was originally expected. Table 2 presents common themes, such as a wider reach than in-person activities, real world impact, improved existing GH partnerships and activities, and newly identified gaps in GH practices.
Notably, one article (3%, 1/40) cited unanticipated negative consequences of the VGHEA, specifically that uncertainties for ongoing funding and lack of foreign recognition of course credit were unexpected hardships for course participants.

Discussion
To the best of our knowledge, this is the first systematic review to identify and synthesize the recent landscape of VGHEAs, including their enablers and barriers. The findings in this review identify gaps in the literature needing future study and illustrate important themes that GH educators should consider when planning and developing VGHEAs.
Most of the VGHEAs described no cost participation or content, but importantly, most articles implied that participation was linked to university tuition or membership or only available via a GH partnership. These findings highlight the difficulty in accessing VGHEAs should a learner not be affiliated with a university or formal GH program or partnership. Aside from one paper, 40 there was a paucity of information regarding specific costs of the activities, both in terms of host cost (e.g., technology infrastructure, platform subscriptions, salary support, etc.) and trainee costs (e.g., university fees, personal costs, cost of data plans or Wi-Fi to access, etc.). Because GH experiences are linked with increased awareness of health system costs and issues, and because decreased funding for GH activities could lead to negative consequences for education, partnerships, and collaboration (disproportionately affecting LMIC partners), 6, 40-43 more financial information about VGHEAs would be useful to inform the discussion on the costs and benefits of continuing in-person travel for GH activities versus shifting to virtual activities long-term.
In terms of ownership and hosting of VGHEAs, there was a notable lack of both shared hosting between LMIC partners and of LMIC institutions that had sole hosting/ownership of the activity. Regarding participation, the USA was overall the biggest consumer of activities reported, but it was unclear from papers discussing participation in VGHEAs by multiple countries what proportion of participants came from HIC versus LMIC settings. These findings raise multiple questions for future study regarding who is making decisions about content topics, target audiences, and goals of GH activities; whether virtual iterations of activities are appropriate for different audience types; and what barriers the HIC partner can alleviate for the LMIC partner. 40 Regarding authorship, the vast majority of included papers reflected first and last authors from HIC institutions and an overall majority of HIC authors. Although this trend of unequal representation of LMIC authors in the GH literature is documented,44, 45 it is perhaps a call to colleagues involved in GH partnerships to ensure equal ownership and authorship of the VGHEA content and academic outputs.   Regarding targeted audiences, our team found surprisingly minimal information about the trainees in the included papers. Further elucidation of learner types and geographic distribution would be key in future studies to better understand activity uptake and appropriateness, particularly for unique LMIC learners, such as in refugee settings. 46 We also found that HIC audiences made up a larger proportion of targeted trainees. This merits further discussion in terms of how much content should be directed toward HIC consumers (specifically when the education is preparing for HIC trainees for experiences in LMIC settings) versus content focusing on building support for LMIC partners and addressing health disparities. Included articles discussed VGHEA evaluations and measured various outcomes, but details about the evaluation methods were not always well described, nor were outcomes standard even among similar activities. Key gaps in our included literature sample appear to be standard evaluation tools, how to best document VGHEA effectiveness, and critically, how the VGHEA affects relevant communities after trainees completed the activity. Documenting and exploring these topics could have large implications for GH educators seeking concrete guidance on best practices for VGHEA evaluation and quality improvement.
The several enablers and barriers of VGHEAs and key themes identified provide important considerations for GH educators. Certain elements were both enablers and barriers, specifically funding, the need for protected and convenient course timing, and technological support needed for VGHEA implementation. The double mention of these factors highlights their critical importance to the success of VGHEAs; indeed, those articles that mentioned funding, [47][48][49][50][51][52] timing, 40,[53][54][55] and strong technology 40,49,53,54,[56][57][58][59][60][61][62] as facilitators of VGHEAs offer key insights into how to overcome barriers that may prevent successful VGHEA implementation. More research in this area will be important to guide the planning and development of VGHEAs, particularly between HIC/LMIC partners who will have different needs and capacities.
Our group noted several gaps in the available literature that could benefit from future study to better guide GH educators in their virtual program planning. In terms of the VGHEAs described in the 40 unique articles, we found that most papers provided basic, descriptive information only. While this information is useful to document the current landscape of VGHEAs, there was less information regarding best practice recommendations for described activities, specifically in terms of frequency, evaluation, duration, organization, and content. Additionally, included articles addressed a wide range of VGHEAs covering multiple topics. Further discussion is warranted on what types of activities work best in certain contexts and for which type of trainees. Virtual domestic or global-local activities, an important subject mentioned in only two articles, 63,64 likewise merits future discussion. Last, there was a dearth of information on sustained virtual engagements to benefit ongoing GH partnerships, particularly for partners in LMICs. Only one paper 65 mentioned health equity and equitable partnerships as a topic area, specifically suggesting the need to have an indigenous perspective included in the course presentation. In the future, it will be important to discuss who decides on the topics included in each activity, particularly for those in LMIC consuming material made by HIC educators. Over the coming years, these considerations may influence virtual GHE planning and implementation at graduate medical institutions worldwide.
Our review had several limitations. First, authors attempted to identify all relevant VGHEA articles, but many initiatives prompted by the pandemic were most likely underway but not yet published. Second, we only included VGHEAs focusing on health professional trainees; future investigation into how community health workers or health professions engage with VGHEAs could be of benefit. Third, although the grey literature search found no additional articles to be screened after cross-referencing article databases and online repositories, we found but excluded an abundance of GH activities (typically on websites, in conference proceedings and abstracts, and on online discussion forums) without a link to primary literature; a future mapping of these resources would be useful. Lastly, the broad nature of GHE introduces the possibility of bias in how we defined an activity and decided on inclusion. Addressing these limitations in future reviews would further contribute to guidelines for graduate GH educators.

Conclusions
Our systematic review is the first review to identify and synthesize recent VGHEAs and report on the drivers and barriers that exist in the current literature. The field of VGHEA remains heterogenous and few studies aimed to examine best practices in the development of VGHEA. With medical trainees from HIC being the primary consumer of VGHEA, further consideration on how to be meet the needs of LMIC trainees is needed. These insights may provide guidance to GH educators in their planning and implementation of VGHEAs moving forward. Further work is needed on activity preferences, considerations for LMIC learners, best practice recommendations, and how activities could be created, shared, and consumed more equitably by partners from both HIC and LMIC settings. This review contributes meaningful foundational data to guide discussions among GH educators to address these knowledge gaps.

Appendix 1
Virtual global health education activity review inclusion and exclusion criteria Inclusion -Articles in English, Spanish, or French -Content of curriculum/program/activity/content must have a global health focus, meaning must include recognition of disparity in resource level - The topic of global health curriculum/program/activity/content must be focused on research, clinical, or public health work -A primary focus of the article is on the domestic or international administration of global health curricula/programs/activities/content -Curriculum/program/activity/content must be primarily administered virtually (synchronously or asynchronously online, by phone, or by web-based application), not primarily as an in-person activity with a supplementary and secondary virtual component -Target users of curriculum/program/activity/content are any level trainee in any pre-professional or postgraduate medical specialty -Curriculum/program/activity/content must involve regular, longitudinal, and/or ongoing global health activities, not isolated onetime events The ECHO program or on ECHO program audiences o Rural healthcare providers linked to a larger health system without a focus on resource disparities between the rural and referral sites. o

Exclusion
General medical education without a global health focus o Open access online content without a stated objective to reach trainees in under-resourced or LMIC settings o Telemedicine or tele-consult services without a stated objective to provide education or mentorship to trainees in under-resourced or LMIC settings o Non-human global health topics (i.e., veterinary care) o Trainees outside of pre-professional or postgraduate medical specialties (including biomedical or engineering trainees) o Continuing medical education focus directed at professionals who already completed previous training

Google grey-literature search strategy
Grey literature searched included Google search, CORE, OpenGrey, GreyNet International, Science.gov, WHO Interna-tional Global Health Observatory, WorldWideScience, Web of Conferences, the New York Academy of Medicine Grey Literature Report, and Duke University Grey Literature guide.
The following search terms were used to consider activities available through a university or school that was yet to be published.