Promoting education and achievement of adults learning English

Briefs

Using the ESL Program Standards to Evaluate and Improve Adult ESL Programs

Kirsten Schaetzel and Sarah Young

Joy Kreeft Peyton
Center for Adult English Language Acquisition
May 2005

Background on Programs for Adult English Language Learners

Adult ESL (English as a second language) programs serve adults whose first language is not English. The primary objective of these programs is to enable adult learners who are not fully fluent and literate in English to become proficient in communicating in English, so that they can meet their personal, employment, community, and academic goals.

Providers of adult education for English language learners include

  • K-12 public schools and districts (local education agencies) that offer adult ESL instruction

  • Community colleges

  • Community-based (CBO), volunteer, and faith-based organizations

  • Family literacy programs

  • Correctional institutions

  • Libraries

  • Workplaces

The types of programs offered include

  • General English language development programs, which focus on developing skills in listening, speaking, reading, and writing. These programs might include content-based ESL classes, which concentrate on a subject area (e.g., civic participation) and, at the same time, develop English language skills related to the subject. They serve a range of learners, from those who are not literate in their native language or in English to those who are highly literate in their native language and are learning English language and literacy.

  • Family ESL literacy, which focuses on knowledge and skills that parents need to help their children succeed in U.S. schools

  • Citizenship preparation, which prepares learners to fulfill the U.S. naturalization requirements

  • Vocational ESL (also VESOL or VESL), designed to prepare learners for job training or employment in specific occupational areas

  • Workplace ESL, which focuses on language and communication skills needed for success in the workplace. Workplace ESL classes are commonly supported by an employer and offered at a work site.

  • English for specific purposes (or ESP), which focuses on developing language and communication skills needed for professional fields of study such as business, agriculture, or medicine

  • Pre-academic ESL, which prepares learners for further education and training in postsecondary institutions, vocational education classes, or ABE and GED classes

Audience for This Brief

This brief is written for the following audiences:

  • Adult ESL program administrators and coordinators

  • Teachers, tutors, and counselors in adult ESL programs

  • Program funders

  • Program partners

  • Program evaluators

Standards in Education: Background

Education standards describe for all stakeholders (program administrators, teachers and other school and district staff, parents, students, and policy makers) the goals, teaching and learning conditions, and expected outcomes of education programs. Education standards include program standards, content standards, and performance standards. This brief focuses on program standards in adult education for English language learners. Therefore, the examples of types of standards are those that pertain to programs that serve adults learning English.

  • Program standards can be used for program development and self-review. They describe the components and features of a program that should be in place for the program to be effective and to provide context and resources that support student learning and achievement (Teachers of English to Speakers of Other Languages, 2003). They also list examples of the evidence that can be used to demonstrate the existence of these essential feature. The standards and evidence guide

    administrators and funders in articulating the goals, objectives, and expected outcomes of the program and the resources that support it;

    teachers and other instructional staff in examining how their instructional practice fits into and supports the goals, objectives, and expected outcomes of the program; and

    learners in identifying the contexts and learner support systems that allow them to meet their goals and needs.

  • Content standards define what students should know and be able to do in different content and skill areas as a result of instruction (Stites, 1999; Teachers of English to Speakers of Other Languages, 1997; U.S. Department of Education, Office of Vocational and Adult Education, 2003a). Several states (e.g., Arizona, Florida, Maryland, New York, Tennessee, and West Virginia) have developed content standards, curriculum frameworks, and resource guides that provide guidance to local programs in developing effective curriculum and instruction. The Adult Education Content Standards Warehouse is being developed to allow users to search for and review adult education standards in ESL, mathematics, and reading (www.adultedcontentstandards.org).

  • Performance standards specify how well students perform at different levels of content knowledge and language proficiency and the measures used to demonstrate how well they perform (National Council on Educational Standards and Testing, 1992; Stites, 1999; U.S. Department of Education, 1994). The English-as-a second language model standards for adult education programs (California Department of Education, 1992) include performance standards for listening, speaking, reading, and writing at different English proficiency levels (e.g., Listening, ESL Beginning, Low – “The learner will demonstrate comprehension of simple words in the context of common, everyday situations.” p. 64).

Program Standards in Adult Education

Attention to program standards in adult education grew out of both the interest of adult education professionals and legislation. As early as 1986, the Mainstream English Language Training (MELT) Project described the conditions that affect the movement of learners from one Student Performance Level (SPL) to another (U.S. Department of Health and Human Services, 1995). The conditions included program-related factors such as intensity of instruction (number of hours of instruction per week); entry/exit procedures and policies; and curriculum, instruction, and assessment practices.

Interest in program standards in adult education has developed in tandem with the development of standards in K-12 education. A focus on program standards in K-12 education began in the 1980s with the National Education Goals Panel and the Goals 2000: Educate America Act. The National Education Goals included a pledge that by the year 2000, all American students would demonstrate competency in challenging subject matter. Subsequently, the U.S. Department of Education, other federal agencies, and foundations made grants to major professional organizations and academic institutions to develop model standards in different subject areas. Although much of the standards work has focused on content standards -- what students need to know and be able to do in different subject areas including math, arts, civics and government, foreign languages, geography, and history – the National Research Council (1999) also has focused attention on the conditions under which students learn and on professional development for teachers.

The Adult Education and Literacy Act of 1991 required the U.S. Department of Education to develop indicators of program quality to assist states and local adult education service providers in assessing the effectiveness of their programs. The legislation specifically called for indicators in the areas of student recruitment, retention, and educational gains. A quality program indicator was defined as a variable reflecting effective and efficient program performance. (See Van Duzer & Berdán, 2000, for discussion.)

The U.S. Department of Education developed examples of quality indicators for ABE programs in general, but did not provide examples specifically related to adult ESL programs (U.S. Department of Education, Office of Vocational and Adult Education, 1992). Subsequently, Title II of the Workforce Investment Act of 1998 (H.R. 1385, Pub. L. No. 105-220), also known as the Adult Education and Family Literacy Act, included both ABE and ESL programs. The act required adult education agencies to establish core indicators of program quality and of learner performance related to educational gain, placement and retention in employment, participation in postsecondary education or training, and high school completion (Adult Education and Family Literacy Act, Sec. 212; U.S. Department of Education, Office of Vocational and Adult Education, Division of Adult Education and Literacy, 2001; 2003b).

Recognizing the need for quality standards specifically for programs serving adult English language learners, Teachers of English to Speakers of Other Languages, Inc. (TESOL) convened a task force to review the accountability requirements in federal adult education legislation and existing program quality indicators and develop a set of standards. The resulting document, Standards for Adult Education ESL Programs (Teachers of English to Speakers of Other Languages, 2003), here referred to as TESOL program standards, has become a centerpiece in the development of program standards in adult ESL education. In addition to the TESOL program standards, other program standards or program quality indicators have been developed by various states and other organizations. (See a list of some of these at the end of this document.)

Research Base

Although most states have included indicators of program quality in their monitoring and evaluation plans, there has been no evidence-based research carried out to demonstrate the role and impact of specific program standards in adult ESL education. There is some research on standards in K-12 education, primarily on the impact of teacher preparation (e.g., Allen 2003). One document (Apthorp, Dean, Florian, Lauer, Reichardt, Sanders, & Snow-Renner, 2001) summarizes the research on standards-based education in grades K-12, to inform efforts to improve low-performing schools and create or sustain standards-based, high-performing learning communities.

Program Standards in Adult ESL

The Standards for Adult Education ESL Programs developed by TESOL provide a framework to guide program staff in analyzing the strengths and weaknesses of their program and in developing a program that will effectively meet their goals for educating adults learning English. These standards are not meant to be prescriptive in any way, but rather to provide a structure and guidance for this analysis. The standards are designed for use in many different types of programs. Programs may differ in size; educational goals; connections with larger institutions and programs; funding sources, requirements, and restrictions; learner goals and characteristics; resources; and staffing.

Standards are grouped into the following nine categories:

1. Program Structure, Administration, and Planning
2. Curriculum and Instructional Materials
3. Instruction
4. Learner Recruitment, Intake, and Orientation
5. Learner Retention and Transition
6. Assessment and Learner Gains
7. Employment Conditions and Staffing
8. Professional Development and Staff Evaluation
9. Support Services

Depending on the goals of the program and the focus of the review, a group of reviewers may choose to work on some or all of the standards. Program narratives describe how staff in a large, institution-based program and a smaller, volunteer-based program used the standards to review their program and develop an action plan.

A program self-review section of the TESOL program standards has a page for each standard. An example page from the self-review instrument is shown in Figure 1.

After the statement of the standard, each page has the following sections:

  • Measures – Measures describe the criteria for determining the extent to which the standard is in place and give examples of the many ways that the standard is implemented. Staff (or external evaluators) mark those program features that are in place and list others.

  • Sample Evidence – Sample evidence lists specific items that demonstrate that the standard is in place. Staff mark those items that are in place and list others.

  • Score - A score is given based on the measures and evidence marked. Scores range from 0, not in place; to 3, in place and well developed. For some standards, it is recommended that all of the measures be in place for a score of 2 or 3. For some, one or more of the measures should be in place. For others, those measures marked with a * should be in place.

  • Priority – Staff mark whether this standard has high or low priority for the program.

  • Comments - Based on the measures, evidence, and score, staff write comments about the status of this standard in the program.

  • Action Plan/Next Steps – Staff describe the next steps that will be taken related to this standard.

Appendix 1 provides a template so that a score, priority, comments, and action plan for each standard can be completed in an electronic file.

Once this work has been accomplished for each standard under review, staff complete the Summary Scores and Action Plan Chart by filling in the scores given for each of the standards and writing an action plan, or continuous improvement plan, for each standard category (e.g., a plan for the category of Program Structure, Administration, and Planning). When this chart is complete, program staff can work together (and with others if appropriate) to evaluate the strengths and weaknesses of the program and develop and implement the plan for the program. States and local programs can use the plan as part of their ongoing program monitoring and evaluation.

The first page of this chart is shown in Figure 2. A pdf version of this chart, for use by program staff, can be downloaded from www.tesol.org/s_tesol/sec_document.asp?CID=281&DID=1839. In addition, Appendix 2 provides an electronic version of the chart that can be completed in an electronic file.

The TESOL program standards can be used by different stakeholders for different purposes. They might be used by

  • Program staff seeking to improve their program; align their program’s mission, goals, and outcome measures with federal legislation; or demonstrate the quality of their program to funders

  • Funders seeking to identify effective programs or to help improve programs they are funding

  • Program advocates seeking to start a new program (to identify what features merit putting in place) or to augment areas in which a program is weak (e.g., salaries and full-time positions for teachers or effective assessment procedures)

  • State-level educators seeking guidance for the development of their own standards, aligned with their specific local context and needs. For example, the Maryland State Board of Education developed the Maryland Adult ESL Program Standards for this purpose. To view these standards, go to www.umbc.edu/alrc/ESLstand.html. Although not as detailed as the TESOL standards, the Maryland program standards clearly articulate goals in specific areas, such as “The program supports retention through an enrollment policy that reflects program goals, requirements of funding sources, and demands on adult learners,” with sample measures to demonstrate that procedures to achieve the goals are in place (e.g., “The program documents learner retention and transition through use of Literacy Works MIS”) (Maryland State Board of Education, n.d., p. 18). In a parallel effort in another country, Myers (1999) describes the process of developing adult ESL program standards for the province of Alberta, Canada.

  • Program evaluators seeking to formally assess the effectiveness of a program (e.g., in a program review for accreditation) or informally document the features of a program. For example, an external evaluator used the TESOL program standards to review a mid-sized, urban adult ESL program that was loosely structured and poorly documented (McCartan, 2005). The standards guided his interviews with teachers, administrators, and students; observations of classroom instruction; and review of program documents. As a result of the evaluation, the program had data to support its claims regarding success and to guide the development of its program improvement plans.

If a program is undergoing accreditation review, all program staff (including administrators, teachers, and support staff) as well as external reviewers might complete the Program Self-Review Instrument and Summary Scores and Action Plan Chart, with the goal of reviewing implementation of all of the standards. If the focus is on one aspect of the program (e.g., instruction), the staff might review only that area, which consists of 13 specific standards, ranging from activities that are aligned with principles of adult learning and language acquisition to activities that prepare learners for formal and informal assessments.

Example of Use of the TESOL Program Standards

As an example of how staff and other stakeholders associated with an adult ESL program might use the TESOL program standards, let’s imagine that a large program in a community college wants to go through a program review and improvement process.

  • The program director, financial office staff, and representatives from the primary funding agencies might focus on the standards for Program Structure, Administration, and Planning. Standard 1.C. (one of 10 standards) states that “The program has sound financial management procedures to collect and maintain fiscal information, guide program budgeting, ensure continuity of funding, and meet reporting requirements.”

  • Curriculum developers and teachers might go through the same process and focus on the standards for Curriculum and Instructional Materials. For example, standard 2.D. states that “The curriculum specifies measurable learning objectives for each instructional offering for learners and is appropriate for learners in multilevel classes.”

  • Teachers, classroom aides, and professional developers working in or with the program might focus on the standards for Instruction. Standard 3.E., for example, states that “Instructional activities integrate the four language skills (listening, speaking, reading, and writing), focusing on receptive and productive skills appropriate to learners’ needs.”

  • Program administrators, counselors, and intake staff might work on the standards for Learner Retention and Transition, for which standard 5.E. states, “The program provides learners with appropriate support for transition to other programs.”

  • Assessment specialists and test administrators might focus on the standards for Assessment and Learners Gains, for which standard 6.D. states that, “The program has procedures for collecting and reporting data on educational gains and outcomes.”

In short, the entire community of stakeholders could get involved in evaluating specific components of the program, to identify strengths and areas in which there are weaknesses and gaps and to develop an action plan to improve the program. At the end of this process, when all of the groups have completed the work in their area, a representative group could get together to review and complete the Summary Scores and Action Plan Chart, develop an improvement plan for the program, and develop a timeline based on high- and low-priority areas. When the improvement plan has been implemented, the outcomes can be evaluated according to the standards.

Conclusion

Program quality standards can help program staff and other stakeholders develop, improve, and maintain programs that are consistent with their goals, objectives, and expected outcomes; state and federal requirements; and learners’ goals and needs. Working together to develop program standards or to review progress toward meeting standards can help the entire staff – administrators, coordinators, teachers, aides, and counselors -- have a common understanding of the components of their program; the activities, resources, and funding needed to accomplish its objectives; and their roles in the endeavor. Evaluation of success in meeting program standards can inform stakeholders of program gaps and weaknesses and guide the development of a continuous improvement plan that all involved can implement.

References

Allen, M.B. (2003). Eight questions on teacher preparation: What does the research say? Denver, CO: Education Commission of the States.

Apthorp, H.S., Dean, C.B., Florian, J.E., Lauer, P.A., Reichardt, R., Sanders, N.M., & Snow-Renner, R. (2001). Standards in classroom practice research synthesis. Aurora, CO: Mid-continent Research for Education and Learning. (Available: www.mcrel.org)

California Department of Education. (1992). English-as-a second language model standards for adult education programs. Sacramento, CA: Author.

Maryland State Board of Education. (n.d.). Maryland Adult ESL Program Standards. Baltimore, MD: Author. Available: www.umbc.edu/alrc/ESLstand.html

McCartan, W. (2005, March). Standards-based evaluation of adult ESL programs. Paper presented at the annual meeting of Teachers of English to Speakers of Other Languages, San Antonio, TX.

Myers, C. (1999). Developing program standards for adult ESL. TESL Canada Journal, 16(2), 77-85.

National Council on Educational Standards and Testing. (1992). Raising standards for American Education: A report to Congress, the Secretary of Education, the National Education Goals Panel, and the American People. Washington, DC: US Government Printing Office.

National Research Council. (1999). Testing, teaching, and learning: A guide for states and school districts. R.F. Elmore & R. Rothman (Eds.), Committee on Title I Testing and Assessment, Board of Testing and Assessment, Commission on Behavioral and Social Sciences and Education. Washington, DC: National Academies Press.

Stites, R. (1999, September). A user’s guide to standards-based educational reform: From theory to practice. Focus on Basics, 3C. Available: www.ncsall.net/?id=31

Teachers of English to Speakers of Other Languages. (1997). ESL standards for pre-K-12 students. Alexandria, VA: Author. (Available for purchase from www.tesol.org)

Teachers of English to Speakers of Other Languages. (2003). Standards for adult education ESL programs. Alexandria, VA: Author. (Available for purchase from www.tesol.org)

U.S. Department of Education. (1994). High standards for all students. Washington, DC: Author. (Available: www.ed.gov/pubs/studstnd.html)

U.S. Department of Education, Office of Vocational and Adult Education. (1992). Model indicators of program quality for adult education programs. Washington, DC: U.S. Department of Education.

U.S. Department of Education, Office of Vocational and Adult Education, Division of Adult Education and Literacy (2001, March). Measures and methods for the National Reporting System for Adult Education: Implementation guidelines. Washington, DC: Author. (Available: www.nrsweb.org/reports/implement.pdf)

U.S. Department of Education, Office of Vocational and Adult Education. (2003a, June). A blueprint for preparing America’s future. The Adult Basic and Literacy Education Act of 2003: Summary of major provisions. Washington, DC: Author. (Available: www.ed.gov/policy/adulted/leg/aeblueprint2.doc)

U.S. Department of Education, Office of Vocational and Adult Education, Division of Adult Education and Literacy (2003b). NRS data monitoring for program improvement. Washington, DC: Author. (Available: www.nrsweb.org/download/NRSDataMonitoringGuideFinal.pdf)

U.S. Department of Health and Human Services, Social Security Administration, & Office of Refugee Resettlement. (1995). Mainstream English Language Training Project (MELT) Resource Package. Washington, DC: Author.

Van Duzer, C.H., & Berdán, R. (2000). Perspectives on assessment in adult ESOL instruction. The Annual Review of Adult Learning and Literacy, 1, 200-242. San Francisco, CA: Jossey-Bass.

Program Quality Standards: Example Documents

See the Adult Education Content Standards Warehouse, being developed to provide information about adult education content standards developed by states. (www.adultedcontentstandards.org).

English as a second language model standards for adult education programs. (1992). Sacramento, CA: California Department of Education. (Available: Bureau of Publications, Sales Unit, California Department of Education, P.O. Box 271, Sacramento, CA 95812-0271)

English for speakers of other languages (ESOL) and citizenship programs. Technical assistance paper. (2000). Tallahassee, FL: Florida Department of Education Division of Workforce Development. (Available: www.firn.edu/doe)

Family literacy standards. (2000). Saskatoon, Canada: Saskatchewan Literacy Network. (Available from the Sasketchewan Literacy Network, www.sk.literacy.ca/)

Pennsylvania’s family literacy indicators of program quality. (2002). U.S. Department of Education Even Start State Initiative Program, 200-2002. Bureau of Adult Basic and Literacy Education. Available: www.pafamilyliteracy.org)

Standards for adult education ESL programs. (2003). Alexandria, VA: Teachers of English to Speakers of Other Languages. (Available for purchase from www.tesol.org)

Appendices

These appendices provide templates that allow staff to complete parts of the Program Self-Review Instrument and the Summary Scores and Action Plan Chart in electronic files. They were developed by William McCartan (McCartan, 2005).

Appendix 1. Program Self-Review Instrument: Electronic File

Appendix 2. Summary Scores and Action Plan Chart: Electronic File

This document was produced by the Center for Adult English Language Acquisition (CAELA) at the Center for Applied Linguistics (4646 40th Street, NW, Washington, DC 20016 202-362-0700) with funding from the U.S. Department of Education (ED), Office of Vocational and Adult Education (OVAE), under Contract No. ED-04-CO-0031/0001. The opinions expressed in this report do not necessarily reflect the positions or policies of ED. This document is in the public domain and may be reproduced without permission.

back to top