Medical students' ratings of faculty teaching in a multi-instructor setting: an examination of monotonic response patterns.

Full Abstract

Realizing that the psychometric properties of a measure may be highly variable is especially relevant in a multi-instructor context, since an implicit assumption is that student ratings are equally reliable and valid for all faculty ratees. As a possible indicator of nonattending (i.e. invalid) responses, the authors examined the effects of monotonic response patterns on the reliabilities of students' ratings of faculty teaching - including how an alternative presentation format may reduce the prevalence of this behavior. Second-year medical and dental students (n = 130) enrolled in a required basic science course during the 1998-99 academic year were randomly assigned to one of two groups - each of which evaluated the teaching of 6 different faculty across 6 distinct dimensions (i.e. overall quality, organization, preparation, stimulation, respectfulness, and helpfulness). Using a 'split ballot' design, two versions of the conceptually equivalent faculty evaluation form were distributed at random to students in each group. Form A contained the 'traditional' items-within-faculty format, while Form B listed faculty-within-item.The number of monotonic forms (i.e. the identical rating of all 6 items) varied measurably across faculty ratees, as did the respective effects on scale reliabilities. Alpha was especially inflated where a sizeable proportion of monotonic patterns were located on response categories that were either very high (> +1.28 z(m) deviations) or very low (< -1.28 z(m) deviations) compared to the group mean. Lastly, the prevalence of monotonic response patterns was significantly (p = < or = 0.01) less when a faculty-within-item format is used (Form B). These findings suggest that monotonic response patterns differentially impact the reliabilities and, hence, the validity of students' ratings of individual faculty in a multi-instructor context.

Learn Faster Today Improve your study skills

Author information

Author/s: Stratton, Terry D (TD); Witzke, Donald B (DB); Jacob, Robert J (RJ); Sauer, Marlene J (MJ); Murphy-Spencer, Amy (A);

Affiliation: Office of Academic Affairs, Division of Testing and Evaluation, University of Kentucky College of Medicine, UKMC Room MN104, Lexington, KY, USA.

Journal and publication information

Publication Type: Journal Article

Journal: Advances in health sciences education : theory and practice (Adv Health Sci Educ Theory Pract), published in Netherlands. (Language: eng)

Reference: 2002-; vol 7 (issue 2) : pp 99-116

Dates: Created 2002/06/20; Completed 2002/08/13; Revised 2004/11/17;

PMID: 12075143, status: MEDLINE (last retrieval date: 11/6/2008)

Sourced from the National Library of Medicine. Abstract text and other information may be subject to copyright.

External Links for this article (including full text providers, if available):

Click Electronic Full-text Provider Links to see options for finding the electronic full text links to this article. Note there may be a subscription or fee required for access to the full text. See our FAQ for information on finding FREE full text articles.

This article may also be located in paper journal collections available in many libraries. Use the Journal and Publication Information above to find the full article.

MeSH headings (categories)

This article was linked to the MESH Headings shown below.

Attitude of Health Personnel
Education, Dental - standards
Education, Medical - standards
Faculty, Medical - standards
Humans
Professional Competence - statistics & numerical data

This article has not been indexed for related articles as yet, however you can still use the live related article search links below.

See 100+ related articles.