Table 3. Description of abstracts included in the review of research on residents' evidence-based medicine curricula
Author (year) Al Jalbout et al. (2019) Gupta et al. (2014) Kao et al. (2012) Kluesner (2015) Merritt (2015)
Study design Pre-post design Prospective cohort Pre-post study Pre-post study Pre-post study
Specialty Emergency Medicine Internal Medicine Internal Medicine not reported Internal Medicine
Number of Participants, n 14 not reported 11 26 108 (10-12/session)
Study period 16 months Not reported 1 year 2 years 6 months
EBM Curriculum Structure 8 structured interactive online modules and a librarian meeting 5 modules: 1) introduction to EBM; 2) randomized control trials; 3) cohort and case-control studies 4) diagnostic tests. 5) meta-analyses 2 week block rotation for first and second year residents Journal Club integrated with EBM principles Interactive lectures and case discussion
Length/frequency 2 weeks Not reported 2 weeks 18 months Once a month for 30 minutes
Teachers/ facilitators Librarian Physician, EBM expert, instructional designer Internal medicine faculty, librarians Teaching resident, core EBM faculty Not reported
Curriculum content Process of EMB, study design, validity, and biostatics, PICO and search strategies 1) introduction to EBM. 2)  randomized control trials. 3) cohort and case control studies. 4) diagnostic tests. 5) meta-analyses Hands-on sessions to teach literature search and critical appraisal 1) peer instruction model and peer to peer discussion coordinated by a teaching resident, 2) dedicated EBM lecture delivered at the begining of each journal club, 3)   identification of 1) assessment of risks and benefits of treatment. 2) rational use of diagnostic testing
Outcome measures EBM knowledge and skills Satisfaction with the course, milestones, correctness of responses, number of attempts, need for remediation EBM skills and knowledge EBM skills and knowledge Comfort in understanding of material
Evaluation method Fresno test survey Fresno test; Resident self assessed comfort with EBM resources Fresno test administered at the end of each year comfort via 5pt Likert scale; critical risk interpretation test (CRIT)
Results 1) Fresno median pre-test 80, median post-test 103 (p=0.002). 2) PICO question (p=0.009), internal validity (p=0.018), study design (p=0.004); no significant improvement among search strategy or biostatistics knowledge 97% of residents were satisfied or very satisfied with the course 1) Fresno mean pre-test 114.3, mean post-test 154.6 (p<0.001). 2) Likert scale (p<0.001) for: searching and using MeSH terms, advanced limits, subheadings in Ovid, and conducting and presenting a literature review Fresno pretest 110.16, year 1 posttest 127.82, year 2 posttest 127.07, (p=0.011) 1) "furthered understanding of how to practice EBM" 4.6/5 of Likert. 2) "use skills learned in these sessions during regular practice" 4.5/5 on Likert
Author (year) Moll (2012) Mroz and Carroll (2019) Schwartz et al. (2000) Vesbianu and Rodriguez (2018) Vogel (2000)
Study design Pre-post study Prospective cohort Pre-post study Pre-post study Pre-post study
Specialty Emergency Medicine Internal Medicine Primary Care internal medicine internal medicine
Number of Participants, n not specified 68 14 30 42
Study period 3 years 1 year 6 weeks 6 months One year
EBM Curriculum Structure Modules Interactive lectures and problem based learning Self-reported critical appraisal on 13 items, and self-reported electronic searching skills on 17 items Small group ambulatory didactics Didactic, with residents giving presentations at the end of the course
Length/frequency Four 6-month modules Six 1-hour sessions over one year 60 hours over 6 weeks 1 hour sessions every week 8 weeks
Teachers/ facilitators Faculty Not reported Not reported Faculty Unclear
Curriculum content 1) introduction and tools needed; 2) small groups formulate a clinical question, research and  rate literature; 3) implementation module, knowledge translation and set 1) recognition of overconfidence and understanding probability; 2) test interpretation; 3) study design; 4) asking a clinical question; 5) abstract review; 6) feedback and skills 1) critical appraisal 2) electronic searching skills 1) appraise and understand medical literature. 2) incorporate learned EBM knowledge into patient care 1) developing clinical questions from patient encounters 2) finding best- quality information sources 3) critical appraisal
Outcome measures EBM sources, knowledge translation tool use and retention EBM skills and knowledge; opinions on the new curriculum Attitudes towards EBM curriculum Level of confidence Survey of EBM attitudes and abilities, computer searching skills, opinions on curriculum, confidence in skills learned
Evaluation method self-reported questionnaire Fresno, resident evaluation survey, 6-item 4pt nihilism scale to address distrust of research methods and applicability of research articles self-reported EBM skills questionnaire self-reports, surveys, and written tests
Results EBM sources are the first query in 67% compared to 50% at the start, and 63% self-identify a change in how they obtain information 1) Fresno post-test 124; 2) resident course rating 7.8 (SD 1.8) on a 10 point Likert scale 1) critical appraisal before study 71%, after study 85%, (p=0.002). 2) electronic searching skills before study 68%, after 83%, (p=0.001). perceived level of confidence improved from 3.62 to 3.89, (p=0.05) 1st written pretest 49% posttest 86%, 2nd written pretest 42% posttest 57%. (p<0.001) for each
Author (year) Vom Eigen (2002) Vu et al. (2017) Walkey and Fairchild (2006) Zipkin et al. (2010)
Study design Pre-post study Pre-post study Pre-post study Pre-post study
Specialty Primary Care Internal Medicine Pediatric primary care Internal Medicine
Number of Participants, n 46 23 21 18
Study period 3 years 1 year Not reported Not reported
EBM Curriculum Structure interactive didactic small groups Small groups, didactics Didactic, interactive Small, interactive groups
Length/frequency 10-12 weekly sessions for 1 hour over the course of the 3 month curriculum Monthly conferences 1.5 hr lecture on didactics followed by 15 min residents presentation on 4 half-day sessions over one week
Teachers/ facilitators Instructor Residents and faculty More senior residents residents Not reported
Curriculum content Biostatics, informatics, clinical epidemiology, patient care issues, search for relevant research articles, discussion lead by instructor Formulate a clinical question, selected and appraised a single article, present an overview, and facilitated discussions 1) didactic session, sample patient encounters. 2) patient contact, generating and question and presentation based on encounter. 3) opportunity to teach other residents Understanding study design, bias and random error, diagnostics, screening, treatments, harm, and prognostics
Outcome measures Pre/post test of knowledge as well as self-evaluated changes in clinical knowledge, skills, practice, and ability to evaluate medical literature level of EBM education, ability to appraise articles on Likert scale Ability to form clinical questions Knowledge, quality of education, written feedback
Evaluation method 38 item test Berlin Questionnaire and Assessing Competency in EBM (ACE) adapted Fresno test (72 pt scale) 25 point multiple choice test
Results mean score of pre and post tests improved from 65.3% to 73.7% (p<0. 001). 1) Berlin Questionnaire pretest 63%, posttest 76%, (p<0.001). 2) ACE pretest 63%, posttest 67%, (p=0.27). 3) 2.9/5 Likert for quality of education in EBM during medical school. Rated ability to appraise articles pretest was 2.8 and posttest was 3.2 on Likert (p=0.03) 1) Fresno pre/post scores increased by 55% (p<0.0001). 2) key subsections: ability to identify EBM sources (120% increase in correctness, p<0.0001) and ability to form questions in PICO format (36% increase, p=0.03) pretest mean 14.17 (SD 4), posttest mean 17.11 (2.4), (p=0.008)
Int J Med Educ. 2021; 12:101-124; doi: 10.5116/ijme.6097.ccc0