Evidence-based medicine curricula and barriers for physicians in training: a scoping review

Objectives To describe the published literature on EBM curricula for physicians in training and barriers during curriculum implementation. Methods We performed a systematic search and review of the medical literature on PubMed, Embase, ERIC, Scopus and Web of Science from the earliest available date until September 4, 2019. Results We screened 9,042 references and included 29 full-text studies and 14 meeting abstracts. Eighteen studies had moderate validity, and 6 had high validity. The EBM curricular structure proved highly variable in between studies. The majority of the EBM curricula was longitudinal with different lengths. Only five studies reported using Kern's six-step approach for curriculum development. Twenty-one articles reported on EBM skills and knowledge, and only 5/29 full-text articles used a validated assessment tool. Time was the main barrier to EBM curriculum implementation. All the included studies and abstracts, independent of the EBM curriculum structure or evaluation method used, found an improvement in the residents' attitudes and/or EBM skills and knowledge. Conclusions The current body of literature available to guide educators in EBM curriculum development is enough to constitute a strong scaffold for developing any EBM curriculum. Given the amount of time and resources needed to develop and implement an EBM curriculum, it is very important to follow the curriculum development steps and use validated assessment tools.


Introduction
Evidence-based medicine (EBM) has been defined as the "conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients". 1 EBM involves integrating clinical expertise with the best available evidence in an attempt to bridge the gap between the growing medical literature and patient care while also considering the preferences and values of the patient in the decision-making. 2 EBM is patient-centered, requires active participation by the learner, acknowledges gaps in knowledge, and bases clinical decisions on evidence rather than authority. 3 Understanding the concepts of EBM and mastering the abilities to retrieve, critically appraise, and apply medical literature to patient care are essential skills for clinicians. 4 The Accreditation Council for Graduate Medical Education states that residents must demonstrate the ability to investigate and evaluate their care of patients, appraise and assimilate scientific evidence, and continuously improve patient care based on constant self-evaluation and life-long learning. 5 Each step of the EBM process requires different levels of competency. 6,7 Despite these requirements, most physicians in training have limited knowledge of the EBM process and research methodology, including study design and interpretation of the results. 8 During the training years, learning fulfills all the adult-learning theory requirements, as the physicians in training come to the table with their own set of life experiences and motivations, they can direct their own learning, they learn better by doing and want to apply their learning to concrete situations sooner rather than later. 9 As the EBM process is very complex, the learning efficacy is enhanced when it is based on the adult learning theory and when constant, longitudinal exposure pertains over time to the EBM concepts. 10 Despite the fact that EBM training is now standard in physician training programs, there is still no consensus on how best to ensure that these skills are taught effectively for life-long learning. The most successful teaching methods for EBM are still unknown, and there is little evidence about which teaching methods lead to improved patient outcomes. [11][12][13][14][15][16] It is also extremely important to know of potential barriers that might need to be addressed in order to achieve a successful implementation of the EBM curriculum. [17][18] The objectives of this scoping review are 1. Describe the structure of the EBM curricula and its impact on the residents' attitudes, behaviors, skills and knowledge, and 2. Describe the barriers that were identified during the EBM curriculum implementation process.

Methods
The methodology for this scoping review was based on the framework outlined by Arksey and O'Malley. 19 The review included the following five key phases: (1) identifying the research question, (2) identifying relevant studies, (3) study selection, (4) charting the data, and (5) collating, summarizing, and reporting the results. The optional 'consultation exercise' of the framework was not conducted.
For the purpose of this scoping review, the word "resident" was used interchangeably with "physician in training", as the term "resident" is mainly used in North America and not widely recognized. This review was guided by the question, 'What is the structure of the existent EBM curricula and its impact on the residents' attitudes, behaviors, skills and knowledge and what are the barriers identified during the EBM curriculum implementation process? Given the complexity of the research question, we decided that a scoping review would be the ideal study design to answer our question.

Information sources
The following databases were searched: PubMed, Embase, ERIC, Scopus and Web of Science from the earliest available date for each database until April 25, 2019, and rechecked on September 4, 2019. Gray literature was also searched. No limits or barriers were applied, but only articles written in English were included. We screened the reference lists of retrieved studies for relevant publications.

Search and eligibility criteria
PubMed search strategy is described: step #1 "Evidence-Based Practice" [Mesh:NoExp] OR "Evidence-Based Medicine" [Mesh] OR "evidence-based health care" OR "evidencebased healthcare" OR "evidence-based medicine" OR "evidence-based practice" OR "evidence-based care" OR ebm OR ebp OR (critical* AND apprais*) OR ebhc; step #2 "Education, Medical, Graduate" [Mesh] OR gme OR (graduate AND medical AND education) OR postgrad* OR "post-grad" OR "post-graduate" OR "post-graduates" OR residency OR residencies OR resident* OR trainee*; step #3 "Teaching" [Mesh] OR teach OR teaching OR learn* OR curriculum OR curricula OR workshop* OR instruct* OR educat* OR train* OR "journal club" OR "journal clubs" OR lectur* OR module* OR course*; step #4: #1 AND #2 AND #3.
The Appendix contains the search strategies for each database. To identify articles relating to teaching EBM in residencies in curricular format and barriers to teach EBM during the curriculum implementation, the following search terms and their word variants were used: resident, evidencebased, education, teaching, lecture, curriculum, barriers. Exclusion criteria were: student, practitioner, private. The word 'curriculum' was not an inclusion criterion word because not all the articles that described an EBM curriculum had the word 'curriculum' included in the title or abstract. Because the assessment is part of the curriculum development process, studies that reported on an EBM curriculum but didn't have an assessment were excluded. Studies reporting on a single EBM education session or single journal club were excluded as they were thought not to qualify for the curriculum description if they did not address all of the EBM steps.

Selection of sources of evidence
We aimed to identify all randomized, nonrandomized, and before and after studies that reported on: 1. teaching EBM in a curriculum format to residents and on: 2. barriers during curriculum implementation. To be included, studies had to provide details about the format of their EBM curriculum and report on the evaluation of their curricula, as evaluation is the last step in the curriculum development process. We used the following criteria to screen studies for inclusion in our review: 1. Study design: original studies and abstracts presented at national conferences, quantitative, qualitative, and mixed methods studies were included if they reported on EBM curriculum with or without barriers to teaching EBM to residents; abstracts describing an EBM curriculum and its evaluation were included to avoid the risk of reporting bias if they had been excluded. Systematic reviews on the EBM curriculum and barriers were not included. Their references were manually verified against our search, and the articles not initially included were uploaded for screening; 2. Population: residents of any specialty, working under the supervision of a medical specialist. Medical students and attending physicians were not included as their learning environment, patient care exposure, and motivation are significantly different than for the physicians in training; 3. Outcomes: pertaining to curriculum evaluation, the following categories based on Kirkpatrick's training evaluation model 20 were used to assign the types of outcomes for included studies related to teaching EBM to residents: level 1 (attitudes toward EBM and participants' satisfaction), level 2 (EBM skills and knowledge), level 3 (behavior, the degree to which participants apply what they learned during training) and level 4 (results, the degree to which targeted outcomes occur as a result of the training, patient-related outcomes); barriers reported during curriculum implementation were collected, and categorization was reported.

Data charting process
A detailed excel spreadsheet with all the data points that were planned to be collected was developed through collaboration between 2 of the authors (AH and BH). The form was calibrated through concomitant (both authors) analysis and data extraction from various studies. Data charting was done independently and in duplicate. Potentially relevant article abstracts were independently reviewed by two authors (AH and BH), who then met and agreed on article selection. For each potentially eligible study, the full paper was read and reviewed by the same two authors (AH and BH) independently. They both assessed whether the study fulfilled the inclusion criteria. Studies about which both reviewers had doubt or disagreement about inclusion were thoroughly discussed during three consensus meetings, and a common decision was reached.
Two authors (TD) and (BH) performed the data extractions from the full-text articles included, which were then fully reviewed by another author (AH) and any disagreements on data extractions were resolved by consensus between AH and BH. Two authors (BH) and (EN) performed the data extraction from the abstracts included. A prespecified data extraction form was used to extract and collect information from the included studies on the author, year of publication, type of study design, country, residency type, number of participants, study period, specific details of the EBM curriculum structure, length, teachers, content, outcome measures, evaluation methods, results, and barriers. Because we expected to find no homogenous studies in this area, no pooling of data was attempted.

Critical appraisal of individual sources of evidence
To assess the risk of bias of individual studies, two authors (BH and TD) independently rated the quality of the studies at the study level. All disagreements were reviewed and resol-ved by a third author, AH.

Risk of bias assessment of individual studies
We assessed the quality of the pre-post studies using the National Heart, Lung and Blood Institute Quality Assessment Tool for Before-After (Pre-Post) Studies with no Control Group (Table 1). 21 Maximum points for each study were 12. We added scores for each criterion together and divided them by 12. The risk of bias rating was: low (75-100%), moderate (25-75%), or high (0-25%). For the randomized controlled trials, quality was assessed according to the Cochrane Collaboration recommended methodology. 22 Each criterion was assessed as high, low, or unclear risk of bias. The quality of cohort studies included in this review were assessed using the Newcastle-Ottawa Scale (NOS), 23 which assigns a maximum of 9 points to each study. The assessment scale analyzes three broad perspectives of each study: the selection of the study groups, the comparability of the groups, and the ascertainment of either the exposure or outcome of interest for cohort studies. When a study received 3 or 4 pluses in the selection domain and 1 or 2 pluses in the comparability model and 2 or 3 pluses in the outcome/exposure domain, it was considered to be of good quality (or low risk of bias); if the study received 2 pluses in selection domain and 1 or 2 pluses in comparability domain and 2 or 3 pluses in outcome/exposure domain, it was considered of moderate quality; if a study received 0 or 1 plus in selection domain or 0 pluses in comparability domain or 0 or 1 plus in outcome/exposure domain, it was deemed to be of high risk of bias (Table 1).

Selection of sources of evidence
Nine-thousand-forty-two references were imported for screening, and 214 duplicates were removed. Eighteen extra articles were considered for inclusion from a manual review of the references, and after cross-checking with the articles already included in the initial screening, 8 of them were duplicates and were removed. Four extra articles were suggested by experts in the field and were included for screening. A total of 9064 articles were included after automatically removing the duplicates. Further duplicates were removed manually, and 8842 articles were screened against the title and abstract, and 8742 articles were found irrelevant and were excluded. One hundred articles were assessed for full-text eligibility, and 57 articles were excluded because they didn't meet the eligibility criteria. Only one out of the 18 studies manually added after searching the references was added to this review. One of the 4 studies suggested by the experts fulfilled all inclusion criteria. The final review included 43 articles (29 full-text articles and 14 abstracts) describing the structure of the EBM curriculum. Thirteen of these articles reported on barriers to EBM curriculum implementation. 2. Were eligibility/selection criteria for the study population prespecified and clearly described?
3. Were the participants in the study representative of those who would be eligible for the test/service/intervention in the general or clinical population of interest? 4. Were all eligible participants that met the prespecified entry criteria enrolled?
5. Was the sample size sufficiently large to provide confidence in the findings?
6. Was the test/service/intervention clearly described and delivered consistently across the study population? 7. Were the outcome measures prespecified, clearly defined, valid, reliable, and assessed consistently across all study participants? 8. Were the people assessing the outcomes blinded to the participants' exposures/interventions? 9. Was the loss to follow-up after baseline 20% or less? Were those lost to follow-up accounted for in the analysis?
10. Did the statistical methods examine changes in outcome measures from before to after the intervention? Were statistical tests done that provided p values for the pre-to-post changes?
11. Were outcome measures of interest taken multiple times before the intervention and multiple times after the intervention (i.e., did they use an interrupted time-series design)?
12. If the intervention was conducted at a group level (e.g., a whole hospital, a community, etc.) did the statistical analysis take into account the use of individual-level data to determine effects at the group level?
Prospective cohort study design 23 Selection 1) Representativeness of the exposed cohort 2) Selection of the non-exposed cohort

Characteristics of sources of evidence
Of the 43 studies included in the final systematic review, 29 studies were reported as full text as follows: 18 described the results of quantitative pre-post study design, 5 described the results of quantitative surveys, 4 described prospective cohort studies, two randomized controlled trials, and 14 meeting abstracts. Table 2 and Table 3 present a more detailed description of the studies. Most studies included were performed in countries where English is the primary language: 40 in the United States, 2 in Canada, and one in the Netherlands. Full-text articles included between 4 and 181 residents, and the abstracts included between 11 and 108 residents. One full-text article and two abstracts didn't report on the number of residents enrolled. The specialties included were varied; the majority were internal medicine (12 full studies and ten abstracts). Other specialties included were: emergency medicine (3 full studies 15,16,17 and 2 abstracts 24,25 ), family medicine (7 full text studies), pediatrics (4 full text studies 4,12,17,18 and 1 abstract 26 ), 1 neurology 27, 1 general surgery 28 and 2 obstetrics and gynecology 29,30 full studies. One abstract didn't specify the residency type. 31 The study period reported was as short as six weeks 32 and as long as five years. 32,33 One study was a single point in time survey. 27 One full-text study 34 and 3 abstracts 35-37 didn't report on the study period.

Critical appraisal within sources of evidence
The risk of bias was assessed only for the studies included as full text. The abstracts were not assessed for their quality, as it was appreciated that they contained very limited information about the methodology and possible bias. Also, the full-text surveys were not assessed for risk of bias. The overall quality of the 24 studies evaluated was determined to be moderate or low risk of bias (Appendix 1 and Appendix 2). On a scale of 1 to 10, they rated the influence to include EBCP concepts in their daily clinical practice as high (mean 6.8, S.D. 1.5). All graduates believed the EBCP concepts were useful, but only applied them when time allowed. On a 1-to-10 scale, they rated the influence to include EBCP concepts in their daily clinical practice higher than the residents (mean 8.5, S. D. 1.2). pre 6.9, post 9. Interns significantly increased their confidence over the course of the rotation; scores improved postrotation in all 3 areas tested: EBM terms and concepts 81% to 97%; quantitative skills 51% to 80%; question formulation and searching 71% to 92%, with the total score increasing from 63% to 87%; residents reported applying the EBM skills they learned to patient care (86%) and that these skills were reinforced in the teaching they received outside of the rotation (81%); all residents felt that the EBM curriculum had improved patient care.
Residents demonstrated significant improvement in postcurriculum examination scores for research and statistics items, correct responses increased 27% (p< .001), residents were 5 times more likely to achieve a perfect score on research and statistics items postcurriculum (p< .001).
Overall, residents found the optional technology session helpful, appreciated librarian involvement during EBM conferences, increased their knowledge of library resources, reported improved knowledge and comfort using electronic library resources after the curriculum changes were implemented, and felt that they could integrate knowledge learned during the EBM conference series into meaningful literature searches. Eighteen studies had a moderate risk of bias, and six studies had a low risk of bias. After applying the NOS criteria, 2 of the four observational prospective cohort studies were assessed as having a low risk of bias 38,39 and the other two studies were assessed as having a moderate risk of bias. 40,41 From the randomized controlled trials, Kim and colleagues 42 had a low risk of bias, and Kortekaas and colleagues 43 had a moderate risk of bias. From the 18 pre-post studies, only 3 of them were assessed as being at low risk of bias; 44-46 the rest were considered at moderate risk of bias. None of the studies were evaluated as having a high risk of bias.

Synthesis of results
Details regarding the EBM curriculum structure, length, frequency, teachers, content, outcome measures, evaluation method, and results can be found in Table 2 for the full-text articles and in Table 3 for the abstracts.

EBM curriculum length / frequency / teachers
The majority of the EBM curricula was longitudinal with different lengths, from weekly 1-hour sessions for 4 weeks 49 to 3 year curriculum. 32 Some curricula were reported in blocks of 2 weeks 25,56 7 weeks, 46 8 weeks, 38 two 2-hour sessions, 13 daily EBM teaching rounds for one month, 53 or intensive 30 hours of EBM training during a month. 33 The frequency of the sessions in each curriculum was variable. In the longitudinal curricula, the sessions were as often as weekly, 34,49 monthly, 29,60 or a few months every year. 41 Only two curricula were designed based on the PGY level, reporting different sessions and activities based on the year of training. 4,32 The rest of the curricula were designed for the generalized audience of all physicians in training. There was a multitude of EBM teachers involved in curriculum delivery in most of the studies, faculty physician, faculty librarian, all level residents, learning coach, 55 research program coordinator, 28 nonphysician biostatistician, research assistant 32 and pharmacist. 50 Four out of the 29 full-text articles and 5 out of the 14 abstracts didn't specify their EBM curriculum teachers.

EBM curriculum content
The majority of the reported EBM curricula address all five of the EBM competencies. 1 Bentley and colleagues 45 Konen and colleagues 32 and Schwartz and colleagues 61 had the main focus on critical appraisal skills. Kitchens and colleagues 34 used a series of articles as a basis for critical appraisal teaching. Trickey and colleagues 28 and Windish and colleagues 49 had a major focus on teaching basic and advanced biostatistics.

EBM curriculum development process
Kern's six-step approach to design an experiential curriculum in knowledge translation has been a well-established model for the curriculum development process that has been shown to lead to successful implementation and long-term sustainability. 62,63 The included steps are: 1. problem identification and general needs assessment; 2. needs assessment of targeted learners; 3. goals and objectives; 4. educational strategies; 5. implementation; 6. evaluation and feedback. From the 29 full-text articles, only 5 of them (17%) mentioned that they used Kern's six-step approach for curriculum development. 33,49,50,52,59 Allan and colleagues and Green and colleagues reported on all the steps as well but without mentioning on the approach. 46,48 George and colleagues reported on all the steps except goals and objectives. 55 Nicholson and colleagues reported on all the steps without the needs assessment. 53

Clinical integration of EBM curriculum
We defined clinical integration of the EBM curriculum if the EBM principles, steps, or concepts were taught in the context of patient care. The EBM curriculum was clinically integrated into 29.3% (10 out of 29) of the full-text articles. Ten percent of the EBM curricula were only partially clinically integrated 4,43,44 , and the rest, which was the vast majority, were not clinically integrated.

Outcome measures
Most of the EBM curricula reported on level 2 (EBM skills and knowledge), 45% (13 out of the 29 full-text articles) and 57% (8 out of the 14 abstracts). Fewer articles reported on level 1 (attitude toward EBM and participants' satisfaction): 17% (5 out of 29 full-text studies) and 35.7% (5 out of 14 abstracts). A total of 10 articles (9 full texts and one abstract) reported on both level 1 and level 2. We found 2 full-text articles that also reported on level 3 (behavior, the degree to which participants apply what they learned during training when they are back on the job). Kortekaas and colleagues 43 reported on information-seeking behavior, and Ross and colleagues 59 reported on EBM terminology usage on tape-recorded interactions.

Evaluation method
From the 29 full-text articles, only 4 used the validated Fresno 33,42,44,60 and 1 used the validated Berlin questionnaire 45 to assess outcomes. Out of the 14 abstracts, 5 used the validated Fresno 25,31,36,56,57 and 1 used the validated Berlin questionnaire. 26 Nelson and colleagues 58 reported the usage of a 20 question previously validated questionnaire. The rest of the studies used unvalidated surveys and questionnaires to assess their outcomes. Eighteen studies out of 43 (41.8%) reported on the usage of surveys to assess the residents' attitudes and comfort with EBM concepts, and 24 out of 43 studies (55.8%) used some questionnaire (multiple choice examinations) to assess the EBM knowledge and skills. Only three studies (7%) also used qualitative analysis (focus group) to assess the residents' reaction to the EBM curriculum implementation. 4,44,55

Barriers to EBM curriculum implementation
Barriers reported during the EBM curriculum implementation were categorized into three main subsets: barriers to teach, barriers to learning, and barriers to practice EBM. (Table 4) Most commonly, EBM curriculum articles reported on barriers to teach, with time and difficulty to recruit teachers being the most frequent barriers reported. Time has been as well reported as the main barrier to practice EBM. The heterogeneity of the studies interventions and outcome assessments precludes any attempt to combine the results quantitatively. All the published studies and abstracts, independent of the EBM curriculum structure and the evaluation method used, found an improvement in the residents' attitudes and/or EBM skills and knowledge. Bentley and colleagues 45 reported a 33.4% increase in the Berlin questionnaire posttest. George and colleagues 55 also reported a significant increase of 31% in the post quiz score. Kenefick and colleagues 13 showed a 58% improvement in the posttest regarding PICO question and 53% improvement in the searching skills. Al Jabout and colleagues 25 reported a 23point improvement in the median Fresno posttest, which was statistically significant at p=0.002. Gupta and colleagues 35 had 97% of residents satisfied with the EBM curriculum. There was a 37% improvement in the written posttest in the study by Vogel and colleagues 51 and Walkey and colleagues 36 had an impressive improvement in the Fresno test score of 55%. None of the studies reported on long term, post-residency training, retention of EBM skills, or knowledge.

Summary of evidence
We reviewed a total of 43 articles, from which 29 full-text articles and 14 abstracts reporting on the EBM curriculum.
Overall the body of evidence regarding EBM curriculum structure, efficacy, and barriers is small and of limited quality. The studies included were considerably heterogeneous with respect to their EBM curriculum structure, content, length, evaluation method, and outcome assessment. A common finding within the studies was that multiple teaching strategies were implemented simultaneously. All studies showed an improvement in educational outcomes across different EBM curricula. However, no specific curriculum was found to be superior because there was no direct comparison. The curricular structure in each article included was described in adequate detail that would allow reproduction and generalizability. This is in contrast with what Green and colleagues 64 noticed in their systematic review of EBM curricula, that many of the reports suffer from the incomplete description of the curricula. The vast majority of the curricula contain multiple strategies to teach EBM in multiple settings. The most common integral session in the EBM curricula reported was journal club, followed by small group sessions, resident-led sessions, hands-on interactive sessions, and workshops. Also, the majority of the EBM curricula reported has been comprehensive, addressing all of the EBM competencies. It is known that offering multiple EBM interventions in multiple learning venues throughout residency training will facilitate the reinforcement and retention of key skills and behaviors. 10 Some suggest adopting a spiral approach to EBM teaching, whereby concepts increase in complexity and are reinforced throughout learning experiences. 65 Moving EBM instruction out of the classroom and into the clinical arena is crucial for the application. 4 When residents have the time and opportunity to integrate EBM competencies during real-time patient care, they shift from a paternalistic to a participatory decision-making style. 53  should be considered in future EBM curriculum development. In his review, Coomarasamy and colleagues 66 reported that only integrating EBM teaching with clinical activities was associated with increased EBM competency across all levels (attitudes, knowledge, skills, behaviors) in medical postgraduates.
Our findings are consistent with Green and colleagues 52 regarding lack of attention to curriculum development principles, as only less than a third of the full-text articles reported on the curriculum development and implementation steps. Eight of the 9 studies that explained the EBM curriculum development and implementation process were published in the last 20 years, which can suggest the hypothesis that in the previous 20 years, with the progression of science, the introduction of reporting guidelines and also with the wide availability of the internet, scientists improved their knowledge in curriculum development and reporting. The lack of attention to curriculum development principles could be explained by the lack of knowledge and training in curriculum development of the curriculum developers.
Very few studies reported on the Kirkpatrick' evaluation model level 3, behavioral change, and no studies reported on level 4, results/patient outcomes. These findings are similar to previous systematic reviews data on medical education interventions. 46 This could be due to the fact that assessing level 3 and level 4 has been previously described as difficult to achieve because it is significantly more time-consuming and requires a more demanding evaluation. Level 4 should be viewed as the primary goal of any training program, but it is very challenging to assess whether certain patient outcomes can be linked to the EBM curriculum intervention or not. In a systematic review of 599 research articles published in three major medical education journals, patient outcomes accounted for only 0.7% of all articles. 67 Ilic and colleagues 7 describe criticism of the current state of the evidence in this field as being the lack of a uniform, validated assessment tool that can measure all aspects of the EBM competencies. Our findings are similar, with the vast majority of curricula being evaluated by non-validated tools. A 2006 systematic review identified 104 unique instruments with reasonable validity for evaluating some domains of evidence-based practice and may be targeted to different evaluation needs. 68 The majority (90%) were not high-quality instruments with established inter-rater reliability, objective outcome measures, and three or more types of established validity. To this day, only the Fresno test of competence in evidence-based medicine 69 and the Berlin Questionnaire 70 are high-quality instruments identified as evaluating EBM knowledge and skills across at least 3 of the 5 EBM steps.
Time has been mentioned as the most frequent barrier to teach and practice EBM. This finding is consistent with prior literature. Dijk and colleagues 17 reported in his systematic review on barriers to residents' practicing EBM that limited available time has been the most often mentioned and prim-ary barrier for residents. Another important barrier that has been very frequently reported was the lack of EBM experts/teachers. This barrier has significant implications in the successful implementation of any EBM curriculum and constitutes one of the first steps in curriculum development.

Strengths and limitations
The strengths of our scopic review include systematic reproducible search of the literature, explicit methods which limit bias, reliable and accurate conclusions and the ability to bring a vast body of literature to an update review. One of the main limitations of this review is the likely publication bias. Because negative studies are less likely to be written, submitted or published, especially in medical education, the studies' results might be skewed toward reporting the positive effects of the interventions. We also noticed that the published data on EBM curriculum is geographically limited and thought that it might be due to publication bias and might further bias the results by not fully representing the EBM curricula from many other countries. It is also possible that there are many innovative and effective EBM curricula that have never been reported. Another major limitation is the quality of the study design of the full manuscripts included. The heterogeneity of the interventions and outcomes constitutes a limitation in the ability to draw a conclusion based on this data. One other limitation worth noting is the small sample size in the majority of these studies, as a convenience sample was most often used, and the studies were single centre-none of the studies reported on patient care impact assessment.

Conclusions
This is the first scoping review to answer the question regarding the structure of EBM curricula and barriers found during the EBM curriculum implementation process. EBM competencies are necessary for providing high-quality patient care within physicians in training education. The current body of literature available to guide educators in EBM curriculum development and barriers to implement and teach EBM is enough to constitute a strong scaffold for developing any EBM curriculum. Teaching EBM concepts can use any method and can be integrated into almost any setting. Special attention should be paid to the potential barriers during the curriculum implementation, and ways to overcome them should be identified early in the curriculum development process. Given the amount of time and resources needed to develop and implement an EBM curriculum, it is imperative to follow the curriculum development steps and use validated assessment tools. During that development, medical educators should design rigorous evaluation strategies with more meaningful clinical outcomes that would evaluate Kirkpatrick's learning levels 3 and 4. Further studies should focus on studying long term retention of the EBM concepts, skills and knowledge as they remain unknown for now.