The effectiveness of Behaviorally Anchored Rating Scales in writing skill evaluation

Anthony VENTOURIS, Aristotle University of Thessaloniki

The majority of the rating scales used in language education are numeric. The numeric scales are almost always ordinal and can provide us information about the classification of the evaluees. Even if someone accept the classification as an output of the evaluation, it will be difficult to disregard the lack of meaningfulness of each score. However, in language education the evaluation is meaningful if communicates specific information to the stakeholders which allow them the proper decision making (Fulcher, 2010:386). The descriptive scales contain a large amount of useful information but their management is difficult and they also need further elaboration to be statistically usable. A multifaceted scale, combining and associating a verbal scale with a numeric one and explaining each value with an accurate description, could probably solve the aforementioned problem (Upshur & Tuner, 1994:4). This is the main research hypothesis of this study, considering the Behaviorally Anchored Rating Scale an appropriate tool to provide suitable and usable information in the context of formative and summative evaluation as well (Aiken, 1996:37). The empirical research conducted, based on a sample of 80 written productions of 16 adult students (18-21), divided in two groups. During an Italian language course of three months (120 hours), the productions of the first group were rated with the use of an arithmetic scale from 1 to 10 and those of the second one with a BARS. The main aim of the research practice was to compare the performance improvement progress of the two groups. The results showed a significant difference in the performances of the two groups, with the experimental group (BARS) to show faster improvement and to achieve higher final scores. At the end of the program a relative questionary was distributed to the students of the two groups, as well as to the teacher. The answers collected showed that the students of the experimental group considered the feedback of their written production very helpful for their improvement, sense that was weak to the control group. The teacher found the BARS scale very informative for both students and teachers and reported the significant difference in the progress of the two groups and in the motivation they got by the feedback of their written productions.