Section 2: How to Frame Evaluation Questions:
What Questions Do You Need to Investigate?
In this section:
What to Evaluate?
One of the most important concerns in evaluation is determining what to evaluate. Most evaluation and research questions in dual language fall into three broad categories:
- Outcome questions
- What kind of progress have students in the different groups made in their oral and written proficiency in each language?
- What are the academic achievement outcomes for students from the different language groups at different points in time?
- Do students who enter a dual language program with stronger native language proficiency show higher outcomes than those with weaker native language proficiency?
- What attitudes do students and their parents demonstrate toward their participation in the program?
- What attitudes do students and their parents demonstrate toward bilingualism and biculturalism?
- What are teachers' perceptions of the effects and benefits of the program?
- Implementation questions
- Is the program implemented as intended?
- In what ways does the implementation differ from the plan?
- Does the program as implemented provide equality to all students, or does it meet the needs of one group more effectively than another?
- Participation questions
- Who are the students who participate?
- Are the language groups represented in the intended ratios?
- Are there significant demographic differences among the language groups, e.g., ethnicity, socio-economic status (SES)?
- How proficient are the students in their native or second languages when they enroll?
- What is the attrition rate? Does it differ by language, ethnicity, or SES category?
While the above questions and many others are important research and evaluation questions, most evaluation questions tend to be outcome-oriented, and based on the goals at the school site.
Variations in community and administrative needs mean that schools have different goals for implementing dual language programs. However, three major goals in many dual language programs include variations of the following:
- Students will become bilingual and biliterate.
- Students will achieve at grade level in the content areas measured in both languages.
- Students will develop positive cross-cultural attitudes.
Some programs also embrace goals of school restructuring, professional development, curriculum development, and parent education.
How to Develop Goals and Objectives
Once the goals have been determined, it is important to establish objectives. An objective is a precise definition of the goal so that the goal can be measured. This objective is specific in stating the outcome: what score is expected, what instrument will be used, in what language, and at what point in the program.
For example, the goal for a student to become bilingual does not specify the level of bilingualism, at what grade level the student should be expected to demonstrate that s/he is bilingual, or how bilingualism will be measured.
Objectives must be stated in a very precise way. Let's use as an example the goal that students will become proficient in their second language. Some individuals might write an objective that states something like "Students will score high on the XYZ Test in the second language."
While this objective is helpful in that it has been determined that language proficiency will be measured in the second language with the XYZ Test and that high scores are expected, it is not clear what "high" means. A more specific objective would be: "Students will score at least 25 on the XYZ Test in their second language after four years in the program."
Accountability and state-level requirements specify that students reach grade-level expectations. While goals and objectives should reflect high expectations, they should also be reasonable and attainable—for the students who are participating in your program. Thus, the goal for students to achieve at (or above) grade-level in the content areas is a goal that is attainable according to research. However, the research shows that students reach grade level at different points in their L1 and L2, and depending on their language and socio-economic background (actually, level of parental education and literacy in the home). Some research on two-way programs is presented in the resource section to help you set reasonable goals and objectives.
Some examples of goals and objectives are presented in the SUGGESTIONS & EXAMPLES section below.
In the process of determining what the objectives should be, there are a few other important issues that need to be considered as well:
- How much individualized assessment and group testing can be conducted?
- What instruments are available in the appropriate languages?
- Which audiences need to know what information?
- Who is available to help interpret the evaluation data?
The answers to these questions will help in establishing the objectives, what instruments will be used, and timelines for testing. We will address these questions in Section 3 of this Toolkit.
SUGGESTIONS & EXAMPLES: Goals and Objectives
If you have not developed goals for your program or you are unclear of your goals, now is a good time to develop or even re-examine them. Put this information in your Evaluation Notebook.
- View/download a template for your goals and objectives in PDF or Word
- View/download an example of goals and objectives in PDF or Word
