Author (year)		Al Jalbout et al. (2019)		Gupta et al. (2014)		Kao et al. (2012)		Kluesner (2015)	Merritt (2015)
Study design		Pre-post design		Prospective cohort		Pre-post study		Pre-post study	Pre-post study
Country		USA		USA		USA		USA	USA
Specialty		Emergency Medicine		Internal Medicine		Internal Medicine		not reported	Internal Medicine
Number of Participants, n		14		not reported		11		26	108 (10-12/session)
Study period		16 months		Not reported		1 year		2 years	6 months
EBM Curriculum Structure		8 structured interactive online modules and a librarian meeting		5 modules: 1) introduction to EBM; 2) randomized control trials; 3) cohort and case-control studies 4) diagnostic tests. 5) meta-analyses		2 week block rotation for first and second year residents		Journal Club integrated with EBM principles	Interactive lectures and case discussion
Length/frequency		2 weeks		Not reported		2 weeks		18 months	Once a month for 30 minutes
Teachers/ facilitators		Librarian		Physician, EBM expert, instructional designer		Internal medicine faculty, librarians		Teaching resident, core EBM faculty	Not reported
Curriculum content		Process of EMB, study design, validity, and biostatics, PICO and search strategies		1) introduction to EBM. 2)  randomized control trials. 3) cohort and case control studies. 4) diagnostic tests. 5) meta-analyses		Hands-on sessions to teach literature search and critical appraisal		1) peer instruction model and peer to peer discussion coordinated by a teaching resident, 2) dedicated EBM lecture delivered at the begining of each journal club, 3)   identification of	1) assessment of risks and benefits of treatment. 2) rational use of diagnostic testing
Outcome measures		EBM knowledge and skills		Satisfaction with the course, milestones, correctness of responses, number of attempts, need for remediation		EBM skills and knowledge		EBM skills and knowledge	Comfort in understanding of material
Evaluation method		Fresno test		survey		Fresno test; Resident self assessed comfort with EBM resources		Fresno test administered at the end of each year	comfort via 5pt Likert scale; critical risk interpretation test (CRIT)
Results		1) Fresno median pre-test 80, median post-test 103 (p=0.002). 2) PICO question (p=0.009), internal validity (p=0.018), study design (p=0.004); no significant improvement among search strategy or biostatistics knowledge		97% of residents were satisfied or very satisfied with the course		1) Fresno mean pre-test 114.3, mean post-test 154.6 (p<0.001). 2) Likert scale (p<0.001) for: searching and using MeSH terms, advanced limits, subheadings in Ovid, and conducting and presenting a literature review		Fresno pretest 110.16, year 1 posttest 127.82, year 2 posttest 127.07, (p=0.011)	1) "furthered understanding of how to practice EBM" 4.6/5 of Likert. 2) "use skills learned in these sessions during regular practice" 4.5/5 on Likert
Author (year)		Moll (2012)		Mroz and Carroll (2019)		Schwartz et al. (2000)		Vesbianu and Rodriguez (2018)	Vogel (2000)
Study design		Pre-post study		Prospective cohort		Pre-post study		Pre-post study	Pre-post study
Country		USA		USA		USA		USA	USA
Specialty		Emergency Medicine		Internal Medicine		Primary Care		internal medicine	internal medicine
Number of Participants, n		not specified		68		14		30	42
Study period		3 years		1 year		6 weeks		6 months	One year
EBM Curriculum Structure		Modules		Interactive lectures and problem based learning		Self-reported critical appraisal on 13 items, and self-reported electronic searching skills on 17 items		Small group ambulatory didactics	Didactic, with residents giving presentations at the end of the course
Length/frequency		Four 6-month modules		Six 1-hour sessions over one year		60 hours over 6 weeks		1 hour sessions every week	8 weeks
Teachers/ facilitators		Faculty		Not reported		Not reported		Faculty	Unclear
Curriculum content		1) introduction and tools needed; 2) small groups formulate a clinical question, research and  rate literature; 3) implementation module, knowledge translation and set		1) recognition of overconfidence and understanding probability; 2) test interpretation; 3) study design; 4) asking a clinical question; 5) abstract review; 6) feedback and skills		1) critical appraisal 2) electronic searching skills		1) appraise and understand medical literature. 2) incorporate learned EBM knowledge into patient care	1) developing clinical questions from patient encounters 2) finding best- quality information sources 3) critical appraisal
Outcome measures		EBM sources, knowledge translation tool use and retention		EBM skills and knowledge; opinions on the new curriculum		Attitudes towards EBM curriculum		Level of confidence	Survey of EBM attitudes and abilities, computer searching skills, opinions on curriculum, confidence in skills learned
Evaluation method		self-reported questionnaire		Fresno, resident evaluation		survey, 6-item 4pt nihilism scale to address distrust of research methods and applicability of research articles		self-reported EBM skills questionnaire	self-reports, surveys, and written tests
Results		EBM sources are the first query in 67% compared to 50% at the start, and 63% self-identify a change in how they obtain information		1) Fresno post-test 124; 2) resident course rating 7.8 (SD 1.8) on a 10 point Likert scale		1) critical appraisal before study 71%, after study 85%, (p=0.002). 2) electronic searching skills before study 68%, after 83%, (p=0.001).		perceived level of confidence improved from 3.62 to 3.89, (p=0.05)	1st written pretest 49% posttest 86%, 2nd written pretest 42% posttest 57%. (p<0.001) for each
Author (year)	Vom Eigen (2002)		Vu et al. (2017)		Walkey and Fairchild (2006)		Zipkin et al. (2010)
Study design	Pre-post study		Pre-post study		Pre-post study		Pre-post study
Country	USA		USA		USA		USA
Specialty	Primary Care Internal Medicine		Pediatric		primary care		Internal Medicine
Number of Participants, n	46		23		21		18
Study period	3 years		1 year		Not reported		Not reported
EBM Curriculum Structure	interactive didactic small groups		Small groups, didactics		Didactic, interactive		Small, interactive groups
Length/frequency	10-12 weekly sessions for 1 hour over the course of the 3 month curriculum		Monthly conferences		1.5 hr lecture on didactics followed by 15 min residents presentation on		4 half-day sessions over one week
Teachers/ facilitators	Instructor		Residents and faculty		More senior residents residents		Not reported
Curriculum content	Biostatics, informatics, clinical epidemiology, patient care issues, search for relevant research articles, discussion lead by instructor		Formulate a clinical question, selected and appraised a single article, present an overview, and facilitated discussions		1) didactic session, sample patient encounters. 2) patient contact, generating and question and presentation based on encounter. 3) opportunity to teach other residents		Understanding study design, bias and random error, diagnostics, screening, treatments, harm, and prognostics
Outcome measures	Pre/post test of knowledge as well as self-evaluated changes in clinical knowledge, skills, practice, and ability to evaluate medical literature		level of EBM education, ability to appraise articles on Likert scale		Ability to form clinical questions		Knowledge, quality of education, written feedback
Evaluation method	38 item test		Berlin Questionnaire and Assessing Competency in EBM (ACE)		adapted Fresno test (72 pt scale)		25 point multiple choice test
Results	mean score of pre and post tests improved from 65.3% to 73.7% (p<0. 001).		1) Berlin Questionnaire pretest 63%, posttest 76%, (p<0.001). 2) ACE pretest 63%, posttest 67%, (p=0.27). 3) 2.9/5 Likert for quality of education in EBM during medical school. Rated ability to appraise articles pretest was 2.8 and posttest was 3.2 on Likert (p=0.03)		1) Fresno pre/post scores increased by 55% (p<0.0001). 2) key subsections: ability to identify EBM sources (120% increase in correctness, p<0.0001) and ability to form questions in PICO format (36% increase, p=0.03)		pretest mean 14.17 (SD 4), posttest mean 17.11 (2.4), (p=0.008)

Int J Med Educ. 2021; 12:101-124; doi: 10.5116/ijme.6097.ccc0