Table 1. Source of evidence framework according to Messick
Source Description Plan Analysis
Content Ensure that the test content reflects what it is intended to measure Expert determination of content in conjunction with international guidelines
Response process Ensure uniformity and control of the response process and minimalise assessment bias Standardise written information and answer sheet, same instructor for all completions
Internal structure Relationship among data items within the instrument and underlying construct Calculate internal consistency reliability Cronbach's a and intraclass correlation coefficient (ICC)
Relationship to other variables Extent to which assessment results relate to other variables Compare the scores between the groups (novices, intermediates, experienced) ANOVA with Bonferroni correction
Consequences Evidence pertaining to intended and unintended consequences of passing and failing Establish a pass/fail score and explore consequences of this score in terms of false-positives and false-negatives Contrasting-groups method
Int J Med Educ. 2022; 13:267-273; doi: 10.5116/ijme.6321.8e5d