Composing at the Keyboard: An Investigation into the Effects of Mode of Response to Writing Tasks

8:00am - 5:30pm EDT

Please consult the event program for time and location.

Presented at: ECOLT 2015

Under the mandate of No Child Left Behind, all standardized tests must be administered via computer by the start of the 2015-2016 school year. While this transition motivates a number of far-reaching effects, one important question is whether younger students have the keyboarding skills necessary to demonstrate their knowledge and abilities on writing test tasks. This is of particular concern for English Language Learners (ELLs), who enter the classroom with varied backgrounds and educational experiences.

Given these concerns, this paper investigates whether ELL students’ writing samples are comparable using both handwriting and keyboarding administration modes. Fifteen students who were identified by their teacher as being capable keyboarders were tested on two test tasks that assessed English language proficiency in science, one using the computer and one using a traditional paper test booklet. Student responses were double scored by trained raters using a scoring rubric that provides a holistic overview of student proficiency to ascertain whether students were able to produce academic English through writing at the same level across the two administration modes. Results indicate that student scores on the two tasks are similar across the handwritten and keyboarded administration modes; and while some students scored better on the handwritten task, others earned higher scores on the keyboarded items. To further assess comparability across administration modes, responses were also analyzed for characteristics of linguistic stamina, vocabulary use, and language complexity.

This study takes an important first step in providing data and results that inform language testers’ understanding of mode effects on tests of writing. Additionally, it clarifies and corrects claims made about raters’ scoring across keyboarded and handwritten modes, while providing crucial information about ELL students’ keyboarding capacities.