Author (year)	Allan et al. (2008)	Aneese et al. (2019)		Bentley et al. (2018)		Burneo et al. (2006)	Chitkara et al. (2016)	Friedman et al. (2010)
Study design	Pre-post study	Pre-post study		Pre-post study		Survey	Pre-post study	Pre-post study
Country	Canada	USA		USA		Canada	USA	USA
Residency type	Family medicine	Internal Medicine		Emergency Medicine		Neurology	Pediatrics	Emergency Medicine
Number of Participants, n	181	60		53		12	17	24 seniors (PGY-3 and PGY-4)
Study period	3 years	1 year		1 year		single point in time survey	3 years	3 months
EBM Curriculum Structure	Workshop, Family Medicine Desktop, brief evidence- based assessment of the research [BEAR], journal club; teaching on industry–physician interaction, presentation skills, and a quarterly review of the literature	1)Monthly EBM workshop; 2)monthly Journal Club; 3)resident led morning report; 4)teaching rounds		8 self-paced modules		24 clinical topics chosen at beginning of the year; prior to each session, resident chooses real encounter on the topic, develops question, searches, selects best article and distributes to all trainees and faculty. CAT is designed and published to website.	1) first year residents: foundational course as small groups and independent study 2) 2nd year: application, CAT presentation 3) 3rd year: integration (act as mentors to 1st and 2nd)	1 hour EBM online tutorial; 3 dedicated formal literature searches for active management questions; brief EBM teaching round (reviewed the clinical question, method, literature search and the relevant evidence was used to answer the management question)
Length/frequency	1) Workshop: 4 half-days, each consisting of a one-hour lecture followed by a two-hour, small- group session. 2) Each BEAR is presented in approximately 15 minutes; 3- 4/resident; 3) Journal club - 2.5 hours	1) 90 minutes X 4/year; 2)60 minutes X 12/year; 3)1 hour X 1/week; 4)1 hour X 3/week		Monthly		Biweekly (every other week), 90 minutes (total of 24 per year)	Longitudinal, year long, frequency not specified	1 hour tutorial; 3 x 8 hour shifts
Teachers/ facilitators	Faculty physician, faculty librarian, second-year resident	Faculty physician, faculty librarian, second and third year resident		Not reported		Faculty physicians, residents of all levels	Faculty, librarian, third year residents as mentors	Not reported
Clinically integrated (lectures versus point of care)	Yes, partially	Yes, partially		No		No	Yes, partially	Yes
Curriculum content	Knowledge component: 1) Understand the rationale and benefits of EBM 2) Provide a strong foundation in the basic principles of EBM, including: Recognizing and formulating clinical questions; Finding and accessing information, interpreting information, applying information 3) Provide and identify online resources, educational tools, and web-links. Skill component: 1) Learn to identify problems/questions encountered in practice and seek solutions by rapidly answering clinical questions with best evidence 2) Build skills and comfort in knowledge transfer. Attitude component: 1) Facilitate appreciation and enthusiasm for the judicious use of current evidence to optimize patient care, and Maintenance of skills and knowledge through life-long self-directed learning.	Formulating clinical question; Search skills; Critical appraisal skills; Evaluating the evidence skills; Applying the evidence		Critical appraisal skills; EBM concepts based on study design		Formulating clinical question, finding evidence, critical appraisal	1) PICO, search skills, critical appraisal 2) PICO applied, CAT presentation, CAT abstract generation 3) EBM teaching and mentorship	PICO, Formulating clinical question; Search skills; Critical appraisal skills; Evaluating the evidence skills; Applying the evidence
Curriculum development process described	Yes, quite extensive. Comittee formed to investigate concerns regarding previous curriculum, goals and objectives developed	Not discussed		Not discussed		Not discussed	Not discussed	Yes, Kerns six step approach used
Outcome measures	Self-assessed skills, past training, and objectives in EBM	EBM knowledge and skills; Attitude toward EBM		EBM knowledge and skills; attitude toward EBM		Participation during EBCP sessions, applicability to daily clinical practice, and foreseeable future practice, how likely clinical practice was influenced, if trainees were teaching the concepts to other residents/students	a. Knowledge and skills b. Attitudes and behavior	Practice norms and attitudes regarding EBM; knowledge, skills
Evaluation method	pre-curriculum survey, quiz; post survey, quiz	Validated questionnaire (Fresno test); Questionnaire; Focus group		validated questionnaire (Berlin Questionnaire BQ)		questionnaire/survey	a. Pre-post knowledge questionnaire (1st year), b. self assessment survey (years 2 and 3), c. focus groups	survey
Results	Statistically significant improvement in all areas of comfort with evidence-based practice and opinions regarding the curriculum; general opinion (e.g., The practice of EBM results in better patient care) showed improvement as well, albeit to a lesser degree;	Fresno pre-test 110.5, post-test 115 (p=0. 60); pre-post therapy questionnaire (p=0. 006), diagnosis (p=0.006), systematic review (p=0.002), harm (p=0.004)		BQ median pre scores 40%, median post scores 73.3% (p=0.002)		On a scale of 1 to 10, they rated the influence to include EBCP concepts in their daily clinical practice as high (mean 6.8, S.D. 1.5). All graduates believed the EBCP concepts were useful, but only applied them when time allowed. On a 1-to-10 scale, they rated the influence to include EBCP concepts in their daily clinical practice higher than the residents (mean 8.5, S. D. 1.2).	pre 6.9, post 9.1 (p=0.02); institutional practice changes reported, such as creating new care plans; residents were better able to appraise the literature (p=0. 02); improved reported searching skills, appraisal, clinical application, repeated practice in the peer mentor role improved knowledge and skills retention	No significant change in the number of subjects that anticipated further practice changes as a result of their experience; 16.3% change in management by the primary team based on the literature searches performed; the change in management was a perceived change as reported by the subjects and was not reviewed by an attending physician.
Author (year)	Gehlbach et al. (1980)		George et al. (2012)	Green and Ellis (1997)		Halalau et al. (2016)	Keddis et al. (2011)	Kenefick et al. (2013)
Study design	Prospective cohort		Pre-post study	Pre-post study		Pre-post study	Prospective cohort	Pre-post study
Country	USA		USA	USA		USA	USA	USA
Residency type	Family Medicine		Family Medicine	Internal Medicine		Internal Medicine	Internal Medicine	Pediatrics
Number of Participants, n	23 PGY-2 and PGY-3		26 PGY-2	34 (PGY-2 and PGY-3)		23	45 PGY-3, 42 PGY-1	4 x PGY-1
Study period	2 years		2 years	1 year		1 year	5 months	1 month
EBM Curriculum Structure	Seminar sessions, resident presentations		One-on-one 1-hour monthly meetings	Based on adult learning theory, the educational strategy included a resident- directed tutorial format, use of real clinical encounters, and specific EBM facilitating techniques for faculty		Based on adult learning theory; resident-directed teaching, small-group sessions, open discussion sessions and direct one-on-one teaching of EBM concept	PGY-1 residents participate in 8 interactive 1-hour sessions during a 1-month rotation. Residents apply their knowledge by presenting their EBM analysis of a clinical case and pertinent literature to a group of residents and faculty members at a 1-hour weekly conference called Clinical Decision Making Journal Club	2 sessions
Length/frequency	8 x 1 hr weelky sessions		1h monthly meetings (30 min/ meeting reserved for EBM) x 1 year	7-week, weekly 1 h tutorials, for 5-14 residents		Over 10 hours of EBM training each month, for 12 months	8 x 1 hour sessions	2 x 2h sessions during a month
Teachers/ facilitators	Faculty		Learning coach = family physician and recent residency graduate	General medicine faculty, librarian		Faculty, librarians, residents	Not reported	Medical librarian and hospitalist
Clinically integrated (lectures versus point of care)	No		No	Yes		Yes, partially	No	Yes
Curriculum content	First 3 seminars: clinical trials, sensitivity, specificity, predictive value, statistical significance; journal handouts for each session; Next 4 seminars: resident presentations evaluating articles; Last seminar: take-home test		EBM curriculum topics: Introduction to EBM, introduction to online EBM resources, introduction to PubMed, basic biostatistics, Introduction to guidelines, formulating PICO Questions, study design, clinical question search strategies	Each of the seven tutorials guides the residents through a real clinical scenario, representative of one of six prototypical clinical questions: therapy, prognosis, harm, diagnosis, prevention, and decision making. In the seventh, they can choose any question type, but must use a systematic review as evidence.		1) Fundamentals of EBM and their application to patient care; study designs and simple statistics and their interpretation in patient care; 2) sessions on using and searching EBM resources; learn the hierarchy of evidence and to gain searching skills in finding the best evidence to inform decision making for patient care; 3) skills in critical appraisal of any study design; 4) critical appraisal of a study that would answer their clinical question and receive direct peer to peer and EBM faculty feedback; 5) basic study design and statistical analysis interpretation of therapy studies, diagnosis studies, harm studies and systematic reviews and meta- analysis	Sessions address the following topics: the EBM cycle; the patient, intervention, comparison, outcome, type of study design, type of question (PICOTT) format for structuring clinical questions; hierarchy of evidence; study design; and critical appraisal of therapy, diagnosis, prognosis, meta-analysis, and harm articles.	PICO format to help form and organize a clinical question; search of the literature to answer their PICO question, levels of evidence, optimal study types
Curriculum development process described	Not discussed		Yes, minimal components included (needs assessment,  implementation,  evaluation)	Yes, most components (needs assessment, goals and objectives, implementation, assessment)		Yes, fairly robust (questionable needs assessment. Yes implementation, assessment)	Not discussed, pre-existing curriculum	Not discussed
Outcome measures	Skills to interpret medical literature critically		Attitudes regarding EBM; EBM knowledge and skills	Attitudes toward EBM, prior critical appraisal training, self-assessed EBM competence, medical reading habits, and preferences for information sources.		Residents' comfort level with basic statistics, literature search and EBM concepts, residents' satisfaction with the EBM curriculum	Resident confidence with and knowledge of EBM topics	Scores on in-class, independent searches, and posttest cases, each of which were scored independently by three faculty members
Evaluation method	take-home test w multiple choice questions centered around some article samples; a practical problem given to all residents at the in training exam; comparison between 23 PGY 2-3 who took curriculum and 12 PGY-1 who did not (for the in training exam problem)		attitudes - survey; knowledge - 18 item quiz, 6 questions adapted from Fresno test of competence; qualitative - coach ratings after every coaching session	survey of EBM behaviors, a survey of self- assessed EBM competences, and an EBM skills test - modified from Stern et al. (free text responses to questions based on a clinical vignette and a redacted journal article)		27 item survey	Likert- scale type survey (which included questions from the validated Berlin questionnaire)	pre-test and posttest of four cases to evaluate the residents' skill and efficiency in asking a clinical question and finding an appropriate answer
Results	Participants' % scores on the article analysis questions ranged from 42-93%; nonparticipants scored from 34-100%. Participants had a mean score of 73.8 %, nonparticipants had a mean 63.8 % (stat signif). When median percent scores were compared, a larger difference was observed between the resident groups (79 participants, 62 nonparticipants)		Favorable attitudes; significant improvement in quiz scores by 31.8% (p<0.001); using wider range of electronic resources; greater confidence and comfort in finding clinical information; ability to find best medical evidence for patients in real time / point of care; EBM valued; importance of one-on-one format	Scores on the EBM skills test (8.5 to 11.0, p =.001) versus control (8.5 to 7.1, p=0. 09). Posttest scores for case and control subjects, the mean difference was 3.9 points (p =.001, 95% confidence interval [CI] 1.9, 5.9).		Self-reported conceptual understanding improved for: relative risk 14%, odds ratio 14%, confidence intervals 27%, and number needed to treat 12%. Comfort with meta-analysis appraisal improved, from 30% to 38%. Routine appraisal sheet use increased by 31%. A 17% increase in satisfaction with the EBM curriculum was reported.	There was no relationship between confidence with and actual knowledge of EBM topics (PGY-1 pre-curriculum vs PGY- 3). Lower confidence among PGY-3 than among PGY-1 internal medicine residents for several EBM topics. PGY-3 residents demonstrated poor knowledge of several core topics taught during internship.	PICO: pre-curriculum test score 29%, post- curriculum test score 87%; Searching: Pre- curriculum test scores 46%, post- curriculum 99%; average time to complete cases pre-curriculum 12 min, versus post-curriculum 8.10 min.
Author (year)	Kim et al. (2008)	Kitchens and Pfeifer (1989)	Kohlwes et al. (2006)		Konen and Fromm (1990)	Kortekaas et al. (2016)	Lentscher and Batig (2017)
Study design	RCT	Pre-post study	Prospective cohort		Survey	RCT	Pre-post study
Country	USA	USA	USA		USA	Netherlands	USA
Residency type	Internal Medicine	Internal Medicine	Internal Medicine		Family Medicine	General Practice	Ob/Gyn
Number of Participants, n	50 (PGY-2 and PGY-3)	83 (PGY-1, PGY-2, PGY-3)	32 PRIME residents		12	79	17
Study period	1 year	unclear	2001-2004		5 years (1983-1988)	2 years	1 year
EBM Curriculum Structure	EBM teaching workshops (intervention) vs weekly resident journal clubs without a formal or explicit curriculum (control)	Group A - literature-based curriculum in clinical appraisal; Group B - ambulatory care medicine topics	4 components: (1) didactic lecture, (2) frequent journal clubs, (3) work-in-progress sessions, and active mentoring to enable residents to "try out" a clinical research project during residency, (4) research project supervision		PGY-1: 2 seminars, a journal club, quarterly research gatherings; PGY-2: presentation at grand rounds; PGY-3: seminars	Intervention: integrated EBM training and teaching sessions based on dilemmas from actual patient consultations; Comparison: stand-alone EBM training at the institute only	Structured journal club curriculum
Length/frequency	6 x 2h EBM teaching workshops during a month	weekly preclinic conference 30-45 min; 17 weeks for Group A, 8 weeks for group B	(1) 12 didactic lectures in consecutive weekly 90- minute sessions; (2) weekly afternoon small journal clubs; (3) PRIME projects - residents present their projects quarterly in 90 min work-in-progress sessions (4) research project supervision: 1-2 x 1 h meetings/month		3 year curriculum; PGY-1: 2 seminars and a journal club, quarterly research gatherings (length not clear); PGY-2: 1 hour presentation at grand rounds; PGY-3: monthly 2 h seminars	a) Clinical practice: 1 h x 4 days/week; integration of EBM 1/week; critical appraisal 1/month b) institute: 1 day/week; EBM course 5 days/year, 2.5 hours; exchange of last week's experiences in clinical practice 1 h/week, integration of EBM for 15 min	Monthly
Teachers/ facilitators	Medical librarians, resident-led	Faculty preceptors	Members of clinical investigator faculty, program director, program directors, clinician-investigator faculty, faculty		Faculty members, physician research director, nonphysician  biostatistician, research assistant	Experienced GPs	Not reported
Clinically integrated (lectures versus point of care)	Yes	No	No		No	Yes, partially	No
Curriculum content	Formulating clinical questions using PICO; medical librarians tought evidence-based synopses and summary resources; topics of therapy, prevention, diagnosis, and prognosis - led by a resident who identified questions from actual patient encounters and performed literature searches using both bibliographic databases as well as evidence-based summary resources to find articles that addressed their question	Series of articles published by the Department of Clinical Epidemiology and Biostatistics at McMaster University that provides a reader with practical guidelines to help read the clinical literature critically. Each week one of these articles was paired with a current article from the clinical literature. Each resident was expected to read both articles, and one resident was assigned to lead a discussion of the content of the clinical article as well as the methodological strengths and weaknesses of that article	(1) PRIME epidemiology curriculum: A. overview of epidemiology study types, B. How epidemiology data are presented; C. How to design a research question; D. Study design and sampling; E. Issues in measurement; F. Causal inference; G, H. Qualitative biostatistics; I. Computer skills; J. Research ethics; K. Systematic review; L. Meta-analysis (2) Weekly journal clubs: diagnostic test evaluation, case- control studies, cohort studies, randomized control trials, meta-analysis, decision analysis, cost- effectiveness analysis, practice guidelines, and clinical overviews are covered, and articles that they frequently hear quoted on the medical wards.		1) Critically evaluate research studies in the professional literature and translate valid conclusions into medical practice; 2) identify areas of potential research interest; 3) embark on a lifelong program of continuing education and professional growth.	Integration of EBM, critical appraisal of an article, EBM course focusing on translation of evidence into clinical practice, patient-related preassignments and postassignments to perform with supervisor; exchange of last week's experiences in clinical practice; discussion of barriers; presentation of a critically appraised topic; possibility for e-learning, online coaching, participation in research	Monthly curriculum that paired a chapter/topic from 2006 edition of The Lancet Handbook of Essential Concepts in Clinical Research with a contemporary journal article selected to highlight the month's topic: overview of clinical research, descriptive studies, bias and causal associations, cohort studies, case-control studies, finding controls for case-control studies, overview of clinical research, uses and abuses of screening tests, refining clinical diagnosis with likelihood ratios, sample size calculations in RCTs, general allocation sequences in RCT, unequal group sizes in RCT
Curriculum development process described	Not discussed	Not discussed	Not discussed		Not discussed	Not discussed	Not discussed
Outcome measures	EBM knowledge, use of evidence-based resources, performance on web-based clinical vignettes	Basic knowledge of clinical epidemiology	Clinical competence, board passing rates, asked to be chiefs, PRIME program popularity, research output		Attendance (as the best measure of interest); resident ratings (attitudes) of each session	EBM behavior, assessed by measuring guideline adherence (incorporating rational, motivated deviation) and information-seeking behavior; EBM attitude, EBM knowledge	Attitudes and self-rated knowledge; EBM knowledge
Evaluation method	Fresno test pre- and post-intervention. Post intervention, residents twice completed a web-based, multiple-choice instrument (15 items) comprised of clinical vignettes, first without then with access to electronic resources.	clinical epidemiology test 1 month post- intervention for group A and group B; intervention changed for group A and group B (cross over); then test again; test with 22 questions on clinical epidemiology principles derived from the articles by the McMaster University group	clinical competence scores		pre-test taken from Reigelman	1. EBM behavior: information-seeking behavior - logbooks; guideline adherence - validated instrument; 2. EBM attitude: McColl validated questionnaire; 3. EBM knowledge: MF Kortekaas validated questionnaire	survey: 6 initial questions to assess individual attitudes and self-rated knowledge; additional questions to assess individual knowledge related to basic study design, bias, and levels of medical evidence. administered pre- and post-intervention
Results	Posttest scores improved for both groups, but were significantly higher for the EBM teaching group (mean score increase 22 (SD=13.8) for teaching group vs. 12 (SD= 12.2) for control group, p=0. 012). There was more improvement in EBM knowledge (100-point scale) for the intervention group compared to the control group (mean score increase 22 vs. 12, p=0.012).	Group B performed significantly better on the second test (post) than on the first test (pre), 68.5% vs 63.3% (p=0.034), while group A did not improve (64.5% va 65.9% - group who received the intervention from the beginning).	The overall clinical competence scores evaluating the 32 PRIME residents gave an overall average score of 8.23 on a 9.0 point scale. This was significantly better than the average of 8.09 for the rest of the internal medicine program (p<0.001).		Attendance greatly improved after the first year (p<0.001); seminar topic preferences (computers in medical practice 0.99; how to write a research article 0.98; different study designs 0.91; reading a research article 0.88; biostatistics 0.86; conducting a literature review 0.75; clinical trials 0.71)	No significant differences in outcomes between the 2 groups, with relative risks for guideline adherence varying between 0.96 and 0.99 (95% CI 0.86 to 1.11) at the end of the third year, and 0.99 and 1.10 (95% CI 0.92 to 1.25) at 1 year after graduation, and for information-seeking behaviour between 0.97 and 1.16 (95% CI 0.70 to 1.91) and 0.90 and 1.10 (95% CI 0.70 to 1.32), respectively.	There was no significant improvement in resident self-assessed knowledge following curriculum implementation. There was a trend toward improved objective knowledge pertaining to study design and interpretation after curriculum completion, but this was not statistically significant
Author (year)	Letterie and Morgenstern (2000)	Luciano et al. (2016)	Mohr et al. (2015)		Nelson et al. (2017)	Nicholson and Shieh (2005)	Ross and Verdieck (2003)
Study design	Survey	Prospective cohort	Pre-post study		Pre-post study	Survey	Pre-post study
Country	USA	USA	USA		USA	USA	USA
Residency type	Ob/Gyn	Internal Medicine	Emergency Medicine		Pediatrics (PGY-1 and PGY-2)	Internal Medicine	Family medicine
Number of Participants, n	not provided	61	31		60	36	18 - intervention; 30 - control
Study period	1 year	2 months	2 years		1 year	1 year	1 year
EBM Curriculum Structure	Journal club: 2 sets of articles distributed for each session; 1 - literature on topics of epidemiology, biostats, experimental design (2-6 articles); 2 - clinical literature for review (illustrate and emphasize prior concepts)	Patient-centered ambulatory morning report (AMR), which combines the User's Guides to the Medical Literature approach to EBM and the Kolb experiential learning theory	Journal club		Sessions grounded in adult learning theory principles, flaw catching exercises, multiple choice questions using audience responses and interactive review of abstracts and articles	Daily EBM session during rounds with the hospitalist team	Interactive workshops
Length/frequency	One 2h session/ month x 12 months	Morning report: 2-3 months/year, weekdays, 30 min/day; + 5h basic EBM concepts; + 4h interactive didactic EBM content (most of these 9h take place within morning report)	monthly		4 x 90 min sessions	1 month	10 sessions; 1-2 hour sessions, brief lecture (30- 40 min) followed by practical application
Teachers/ facilitators	Resident, staff physician	Residents, facilitators (not specified)	Faculty		Faculty	Hospitalist	Preceptors
Clinically integrated (lectures versus point of care)	No	No	No		No	Yes	Yes
Curriculum content	24 concepts descriptive of topics considered critical in the assessment and evaluation of literature descriptive of clinical practice and patient care: topics of epidemiology, biostatistics, and experimental design (p values -> meta analysis), literature on clinical topics (illustrate and emphasize concepts of experimental design and statistical analysis).	EBM skills: identify a clinical problem, define the problem as a structured clinical question, acquire new knowledge; EBM competencies: evaluate the validity of evidence cited, evaluate the results; apply and use new knowledge; EBM competency: apply the results to the clinical scenario	1) Dedicated 20-min monthly discussion led by a faculty member during monthly journal club focused on an EBM-focused topic. 2) a single faculty member being grouped with three coordinating residents to review selected articles from an EBM perspective prior to journal club to discuss the teaching points of each in detail. 3) a period of peer-to-peer teaching during journal club, 4) a dedicated core of EBM faculty dedicated to directing the journal club curriculum and establishing expertise in EBM concepts		Session 1: class - EBM test, flaw catching exercise, Introduction to MAARIE framework; homework - multiple choice questions covering session 1 information; Session 2: class - review multiple choice questions in teams using audience response systems; introduction to MAARIE framework; flaw catching exercise; homework - read randomized controlled trial; Session 3: multiple choice questions covering session 2, apply MAARIE framework to abstracts; apply MAARIE framework to RCT #1; homework - read RCT #2; Session 4: review MAARIE framework and key concepts; apply MAARIE framework to RCT #2, EBM test.	Definitions and principles of EBM, literature appraisal, EBM internet resources; formulate focused clinical questions for each patient admitted, do literature searches, use EBM principles to evaluate search results, and present findings during subsequent rounds.	1. Multiple-choice pretest, overview and constructing focused questions: interactive formulation of good, researchable EBM questions for patient care; 2. categorizing articles, reading review articles: discussion of recognizing focused reviews, how to sort articles into groups (i.e., therapy, diagnostic tests, reviews); 3. reading review articles: a discussion regarding key features of well-researched and written reviews; 4. evidence for therapy; 5. evidence for diagnostic tests; 6. evidence for history and physical exam; 7. evidence for prevention; 8. evidence for practice guidelines; 9. applying quality improvement
Curriculum development process described	Not discussed	Not discussed	Not discussed		Not discussed	Yes, everything but discussing needs assessment	Yes, Kerns six step approach used
Outcome measures	EBM knowledge and skills	EBM knowledge: learn to use medical literature to guide patient care	EBM skills and knowledge		Baseline knowledge, efficacy of the curriculum in improving knowledge	Learning experience, understanding of EBM terms or practice skills	Knowledge and application of EBM during continuity clinic patient care
Evaluation method	questionnaire	18-question EBM test (of the 18 questions, 1 required formulating a structured clinical question, 6 addressed evaluating validity of an article, 5 addressed evaluating results, and 6 addressed applying results to a clinical scenario).	Fresno test		validated 20 question evidence based medicine multiple choice test was administered on three separate occasions	questionnaire on which they were asked to rate the impact of the curriculum on their understanding of 20 EBM terms or practice skills	a 50-item, multiple-choice examination was administered before and after the workshop series; residents at another FP residency at the same university served as a control group. Resident–preceptor interactions during outpatient continuity clinic were tape recorded prior to and six months following introduction of the curriculum
Results	85% residents wanted to continue the format without much change; 15% wanted to restrict literature to Ob/Gyn content	Average score increased during the training period, most strongly following internship. Mean scores stabilized after internship; however, the range of scores successively narrowed; EBM scores improved in time in the following domains: formulating a structured clinical question, assessing the validity, and evaluating the results	Total test scores did not increase significantly (105.4 vs. 120.9, p = 0.058) in the before–after analysis; the only subscore showing improvement was interpretation of study validity (32.1 vs. 40.4 points, p = 0.03); attendance was significantly associated with Fresno test score, with those attending more than 6/11 sessions (55%) scoring 28.2 points higher (p = 0.003)		Post curriculum, the fall group’s scores improved 23% from baseline (M=10.3, SD=2.4) to (M=12.7, SD=3.0) students (t(26)=-3.29, p=0.0018) while the spring group improved by 41% (M=10.0, SD=2.8) to (M=14.1, SD=2.2) students (t(32)=- 6.46, p<0.0001). There was an association between number of sessions attended and increase in post curriculum score (χ2 (3, N=60) =11.75, p=0.0083).	Results were very positive with average effect of more than 4 (somewhat strong effect/impact) for 16 of the 20 questions.	Pre-intervention multiple-choice test results were similar (control mean 56%, experimental 53%, p>. 22 NS); post-intervention test scores for the experimental group were significantly improved (mean 72%, p<.001); there was no significant improvement in test results among members of the control group (p>.05 NS); in the recorded resident–preceptor interactions, a marked increase in the use of EBM terms indicated awareness and/or use of EBM in the experimental group
Author (year)	Shaughnessy et al. (2012)	Thom et al. (2004)	Trickey et al. (2014)	Windish (2011)	Zeblisky et al. (2015)
Study design	Pre-post study	Pre-post study	Pre-post study	Pre-post study	Survey
Country	USA	USA	USA	USA	USA
Residency type	Family Medicine	Family Medicine	General surgery	Internal medicine	Pediatrics
Number of Participants, n	23	13 PGY-1	40	52	46
Study period	5 years	3 months	6 years	2 years	6 months
EBM Curriculum Structure	1) Block format - intensive instruction over the course of 1 month (30 hours); 2) longitudinal series of ongoing conferences: a modified journal club, group session at the beginning of the year, weekly subspecialist conferences; teaching was also integrated into day-to- day clinical activities via precepting interactions	Individual EBM rotation for interns; online tutorial, workshop, journal club	Lectures, tutorials, practice questions, examples	Journal articles reading, seminars, presentations by learners	1) Technology sessions, 2) increased librarian support for EBM presenters, 3) increased interaction during conferences, 4) standardized curriculum to reflect progression of the academic year, and 5) small group learning;
Length/frequency	1) 30 hours face-to-face intensive EBM education during orientation; 2) longitudinal: unclear length & frequency	2 weeks (3 half-day clinics per week, with the remainder of the time available for EBM)	5 x 1-hour monthly research and statistics lectures	4 x 1 hour weekly sessions	Monthly conference
Teachers/ facilitators	Faculty, residents	Faculty, medical librarian	Research program coordinator	Teacher with expertise in epidemiology and biostatistics	Librarians, pharmacists, others
Clinically integrated (lectures versus point of care)	Yes	Yes	No	No	No
Curriculum content	1) Introduction to Information Mastery; 2) group exercise: is It a POEM; 3) don't panic: statistics you can understand; 4) power reading of journals; 5) expert- based information delivery systems; 6) diagnostic testing: Bayes' Theorem, evaluating studies of diagnostic tests; 7) evaluation of Grand Rounds using the CME evaluation form; 8) obtaining useful information from pharmaceutical representatives; 9) using Clinical Decision Rules; 10) clinical Jazz: harmonizing clinical experience with Evidence-based Medicine	Critical evaluation of articles about diagnosis, therapy, prognosis, meta- analysis and decision making, and how to ask clinical questions. PICO, formulate questions, generate searches, developing searching strategies, high quality EBM resources, information mastery; essential EBM concepts, including clinical question development, levels of evidence search strategies, and appraisal techniques; critically appraised topics - each CAT is structured to include the 'clinical bottom line' answer to the question	Hypothesis testing, variable types, study designs, clinical trial phases, bias and confounding, odds and risk ratios, sensitivity and specificity, 2 x 2 tables, incidence and prevalence, representative descriptive statistic, standard deviation and measures of variability, multiple comparisons, types of statistical tests, power and sample size calculations, number needed to harm/treat, confounding assessment and adjustment, patient safety and quality improvement methods, national surgical quality improvement program, surgical care improvement project measures	1) hypothesis testing and exploratory data analysis; 2) confirmatory data analysis; 3) study designs; 4) each resident had to prepare a hypothetical study in advance which would answer a clinical or research question	1) Introduction and PICO questions; 2) live literature search and analysis of the results; 3) article analysis focusing on assigned key concepts of analyzing different types of articles such as therapy, diagnosis, etc., so that each month focused on a different concept in depth; and 4) article application to patient population; 5) modified journal club format.
Curriculum development process described	Yes, Kerns six step approach used	Not discussed	Not discussed	Yes, Kerns six step approach used	Yes, Kerns six step approach used
Outcome measures	Mastery of residents' evidence-based medicine knowledge and skills as well as their confidence at critically appraising medical literature and using evidence to inform clinical decisions.	Level of confidence in basic EBM skills	Improvement in research and statistics ABSITE scores	Assessment of resident performance, assessment of curriculum	Impact that the curriculum changes (including librarian involvement) had on the residents' perceived abilities to integrate knowledge gained from the EBM conference into meaningful literature searches.
Evaluation method	Fresno Test of Evidence-based Medicine; an attitude questionnaire at the start of the curriculum and then again before graduation	self-assessment of EBM skills (interns) - adapted from Fresno test	ABSITE examinations	20-item multiple-choice knowledge test; written survey about the curriculum	survey
Results	Modified Fresno Test scores significantly improved from 104.0 to 121.5. Using a pass/fail approach, nine residents (40.1%) passed the test at the start of training, increasing to 17 (73.4%) at the end of the intervention. Confidence in critical appraisal scores increased from an average 17.90 (95% CI=16.55–19.25) to 21.10 (95% CI=19.49–22.71), out of a possible score of 25. Attitudes regarding confidence in the use of evidence and a decreased reliance on experts were also improved following the curriculum.	Interns significantly increased their confidence over the course of the rotation; scores improved postrotation in all 3 areas tested: EBM terms and concepts 81% to 97%; quantitative skills 51% to 80%; question formulation and searching 71% to 92%, with the total score increasing from 63% to 87%; residents reported applying the EBM skills they learned to patient care (86%) and that these skills were reinforced in the teaching they received outside of the rotation (81%); all residents felt that the EBM curriculum had improved patient care.	Residents demonstrated significant improvement in postcurriculum examination scores for research and statistics items, correct responses increased 27% (p< .001), residents were 5 times more likely to achieve a perfect score on research and statistics items postcurriculum (p< .001).	Mean±SD proportion of correct answers for the 52 course participants was 58±16%; historical controls – 26 residents in the same programme and 277 residents from other residency programmes – scored less well (48±14%, p=0.01 and 41±15%, p<0.0001, respectively).	Overall, residents found the optional technology session helpful, appreciated librarian involvement during EBM conferences, increased their knowledge of library resources, reported improved knowledge and comfort using electronic library resources after the curriculum changes were implemented, and felt that they could integrate knowledge learned during the EBM conference series into meaningful literature searches.

Int J Med Educ. 2021; 12:101-124; doi: 10.5116/ijme.6097.ccc0