Why an Evaluator's Toolkit?
In this section:
The document Guiding Principles for Dual Language Education is a tool to help schools develop high quality programs. One of the strands in that document is titled Assessment and Accountability. That section of the document provides excellent guidance in indicating that programs should:
- create and maintain an infrastructure that supports an accountability process (e.g., develop a data management system);
- align student assessment with state content and language standards as well as program goals;
- collect a variety of data using multiple measures; and,
- analyze and interpret their data appropriately.
While these guidelines are helpful, most school staff have no idea how to adhere to these principles.
The purpose of this digital Toolkit is to assist staff in programs that serve ELL students (with a particular reference to two-way/dual language programs) to understand how to meet assessment and accountability guidelines. This Toolkit assumes no prior knowledge of data collection, data management or data analysis on the part of the user. It is truly "A Beginner's Toolkit." It speaks to personnel in local dual language programs who want and need to conduct a program evaluation but may not have access to an evaluation department or other expert resources who can advise them on what data to collect, how to maintain it, how to analyze it, and how to present it—this describes many projects in large and small districts. Small districts may not have the resources. Large districts may have the resources, but the technical expertise is dedicated to priorities that do not include the dual language program.
Who is the Audience?
Dual Language vs. Two-way Immersion vs. Developmental
Bilingual
In this Toolkit, we use the term "dual language" to encompass both two-way immersion programs and late-exit maintenance, or developmental bilingual programs. Both of these programs serve ELL students with the purpose of developing full bilingual proficiency. However, while most of this Toolkit can serve developmental bilingual programs, it focuses more on two-way immersion programs (both 90/10 and 50/50), in that it also discusses evaluation of and data on native English-speaking students.
What This Toolkit DOES
This Toolkit contains:
- Specific guidelines on developing an appropriate evaluation plan, with goals and objectives for the type of dual language program being implemented
- A discussion of data collection instruments, with an examination of multiple measures—with links to various instruments that have been developed or used in dual language programs
- Information about how to set up a data management system, with simple-to-follow instructions, and downloadable Excel spreadsheets and codebooks
- Information about how to analyze, interpret, and present data, including step-by-step instructions for typical analyses using the statistical software SPSS and for setting up common graphs in PowerPoint
- A glossary of terms and acronyms used in the Toolkit (and which are common in language education)
- Resources for program evaluation activities:
- Examples of a program description, goals and objectives, and data collection plan and timeline
- Sample questionnaires, spreadsheets, and codebooks
- Research data from various research projects on two-way programs to allow schools to make comparisons with larger datasets
What This Toolkit DOES NOT Do
This Toolkit does not provide the spreadsheet or statistical software needed to maintain or analyze data. The user will need to acquire separate software for that. Many school computers come with spreadsheet software such as Excel already installed. Many project personnel and/or their support staff know how to use these. This Toolkit merely gives guidance in how to use them in preparation for appropriate data analysis, and we do provide sample spreadsheet layouts in Excel.
Users may find it necessary to obtain a statistical software program. Many are available, and this Toolkit does not recommend any particular one. It uses the Statistical Package for the Social Sciences (SPSS) for its examples because it is widely used among educational evaluators and researchers. The best program is the one that is easiest for you. Any statistical program will do the simple kinds of analysis most commonly needed for program evaluation as described in Section 6.
