- This event has passed.
LARC 2025 Conference Presentation—Examining Alternate-Form Reliability in Adult ESL Literacy Assessments: A Case Study
April 5
Examining Alternate-Form Reliability in Adult ESL Literacy Assessments: A Case Study
Date & Time: April 5, 2025 | 04:00–04:30 p.m. PDT
Location: California State University, Fullerton
Presenter(s):
- Leah Guo (Center for Applied Linguistics)
- Aubrey Sahouria (Center for Applied Linguistics)
- Leslie Fink(Center for Applied Linguistics)
Description:
| One essential quality of any good assessment is reliability, which describes the consistency of measurement and the stability of test scores when an assessment is repeated upon a population. The Standards for Educational and Psychological Testing (AERA et al., 2014) stress the importance of using assessments with high reliability and provide some factors to consider with different coefficients. This presentation discusses specifically how alternate-form reliability estimates can provide more insight into score interpretation for pre- and post-instruction assessments, especially in English as a Second Language (ESL) programs. This method for assessing reliability is superior to the test-retest method because it reduces the extent to which an individual’s memory can inflate the reliability estimate, as may happen with a single form. This study examines scores from three alternate test forms of a literacy assessment designed for use in an adult English language teaching context in the US. We investigated whether examinees would receive comparable results, all else held constant, when assessed by an alternate test form. Measuring the gains of participants in federally funded adult education programs that are required to report under the National Reporting System (U.S. Department of Education, Office of Career, Technical, and Adult Education, 2017). Data was collected from participants across three adult ESL programs in the US. Each participant completed two of the three possible forms of the assessment within a three-week window. Their raw scores were converted to literacy levels on the National Reporting System (NRS) Levels classification scale based on the results of a previous standard-setting study, and the examinee’s classifications based on each form were compared. Alternate-form reliability was evaluated using bivariate correlations among the different forms administered to examinees. When comparing examinee performance across shared exercises across forms, performance did not change significantly, suggesting minimal prior exposure effects and lending credence to the reliability of the alternate test forms. Results showed reasonably high alternate-form test-retest reliability coefficients and provided good empirical evidence towards examinees obtaining comparable results between forms. Specifically, for NRS Levels classifications for literacy, examinee’s classifications between their two administered alternate forms were either exactly in agreement or had adjacent level agreement. This comparability is a necessary consideration in the context of standard setting and content alignment for literacy assessments. |
