This Article Full Text (PDF) References Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Alert me to new issues of the journal Add to Saved Citations Download to citation manager Request Permissions Request Reprints Add to My Marked Citations Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Citing Articles via Scopus Google Scholar Articles by HAYES, J. R. Articles by SILK, C. M. Search for Related Content Social Bookmarking                         What's this?

Does Holistic Assessment Predict Writing Performance?

Estimating the Consistency of Student Performance on Holistically Scored Writing Assignments

Carnegie Mellon University

JILL A. HATCH

Carnegie Mellon University

CHRISTINE M. SILK

Los Angeles, California

This study addressed the question, "How consistently do students perform on holistically scored writing assignments?" Instructors from 13 introductory writing classes at two colleges were asked to provide essay sets written by their students in response to the three to five most important writing assignments in their classes. In all, 796 essays were collected from 241 students. The study drew on a pool of 15 experienced judges to evaluate the essays. Each essay set was scored holistically and independently by 6 of the judges who either ranked or graded the essays in the set. All papers written by a particular student were scored by the same judges. Pairwise correlations of the scores assigned to each essay set were computed for each judge and then averaged across judges. The average of these correlations was 0.16, indicating very low consistency of holistically scored student performance from essay to essay. This result suggests that drawing conclusions from one or even a few writing samples is problematic.

Written Communication, Vol. 17, No. 1, 3-26 (2000)
DOI: 10.1177/0741088300017001001


CiteULike    Complore    Connotea    Del.icio.us    Digg    Reddit    Technorati    Twitter    What's this?

This article has been cited by other articles:

				P. S. Rogers and J. Rymer Analytical Tools to Facilitate Transitions Into New Writing Contexts: A Communicative Perspective Journal of Business Communication, April 1, 2001; 38(2): 112 - 151. [Abstract] [PDF]

				R. L. JOHNSON, J. PENNY, and B. GORDON Score Resolution and the Interrater Reliability of Holistic Scores in Rating Essays Written Communication, April 1, 2001; 18(2): 229 - 249. [Abstract] [PDF]