Threats to the validity of locally developed multiple-choice tests in medical education: construct-irrelevant variance and construct underrepresentation.

Full Abstract

Construct-irrelevant variance (CIV) - the erroneous inflation or deflation of test scores due to certain types of uncontrolled or systematic measurement error - and construct underrepresentation (CUR) - the under-sampling of the achievement domain - are discussed as threats to the meaningful interpretation of scores from objective tests developed for local medical education use. Several sources of CIV and CUR are discussed and remedies are suggested. Test score inflation or deflation, due to the systematic measurement error introduced by CIV, may result from poorly crafted test questions, insecure test questions and other types of test irregularities, testwiseness, guessing, and test item bias. Using indefensible passing standards can interact with test scores to produce CIV. Sources of content underrepresentation are associated with tests that are too short to support legitimate inferences to the domain and which are composed of trivial questions written at low-levels of the cognitive domain. "Teaching to the test" is another frequent contributor to CUR in examinations used in medical education. Most sources of CIV and CUR can be controlled or eliminated from the tests used at all levels of medical education, given proper training and support of the faculty who create these important examinations.

Learn Faster Today Improve your study skills

Author information

Author/s: Downing, Steven M (SM);

Affiliation: University of Illinois at Chicago, College of Medicine, Department of Medical Education (MC 591), 808 South Wood Street, Chicago, IL 60612-7309, USA. sdowning(-atsign-)uic.edu

Journal and publication information

Publication Type: Journal Article; Validation Studies

Journal: Advances in health sciences education : theory and practice (Adv Health Sci Educ Theory Pract), published in Netherlands. (Language: eng)

Reference: 2002-; vol 7 (issue 3) : pp 235-41

Dates: Created 2003/01/02; Completed 2003/01/29; Revised 2004/11/17;

PMID: 12510145, status: MEDLINE (last retrieval date: 11/6/2008)

Sourced from the National Library of Medicine. Abstract text and other information may be subject to copyright.

External Links for this article (including full text providers, if available):

Click Electronic Full-text Provider Links to see options for finding the electronic full text links to this article. Note there may be a subscription or fee required for access to the full text. See our FAQ for information on finding FREE full text articles.

This article may also be located in paper journal collections available in many libraries. Use the Journal and Publication Information above to find the full article.

MeSH headings (categories)

This article was linked to the MESH Headings shown below.

Bias (Epidemiology)
Data Interpretation, Statistical
Education, Medical - methods
Educational Measurement - methods; statistics & numerical data

Educational Measurement - methods; statistics & numerical data
Humans
Mental Recall
United States

This article has not been indexed for related articles as yet, however you can still use the live related article search links below.

See 100+ related articles.