Abstract
Purpose. The purpose of this study was to describe the process that California's School Assistance and Intervention Teams (SAIT) use to improve their services to low-performing schools and to determine to what degree summative evaluation was part of this process. Methodology. SAIT providers on California Department of Education's February 16, 2005, Approved SAIT Providers List qualified to be a qualitative, phenomenology case study. Twenty-eight providers participated in a telephone survey concerning SAIT providers' value, use, and familiarity with formal evaluation tools including checklists, standards, guiding principles, and meta-evaluation. Open-ended questions asked for the most effective practice(s) and greatest barrier(s) to improving SAIT providers' service to low-performing schools. Findings. SAIT providers highly value evaluation tools. When SAIT providers utilized evaluation tools, they were self-developed. SAIT teams from higher institutes of education, educational foundations, and private consultants used evaluation tools more often than county offices of education. SAIT providers are unfamiliar with specific evaluation tools in the evaluation field. The CDE's process was considered an effective evaluation tool; however, half used an additional self-developed, well-defined protocol which enhanced their services over competing SAIT organizations. Pronounced entrepreneurial competition existed between county offices of education and other organizations. Relationship with the client, the team's processes, and the internal team practices were viewed as effective. Barriers consisted of lack of time, the CDE process, the schools/districts as clients, and competition between SAIT providers. Conclusions. SAIT providers value formal evaluation tools and would benefit from collaboration time to reflect on their most effective practices/common barriers, followed by time to adjust their protocols. SAIT providers are "accidentally trained," and finding additional trained, skillful leaders was difficult. Training in formal evaluation tools and increased business skills would benefit SAIT providers. Recommendations. Establish an institutionalized meta-evaluation checklist to improve SAIT providers' evaluation service. Provide training from professional organizations/universities in educational evaluation. Develop collaboration opportunities among providers and adopt guiding principles and standards. Prepare individuals for the multifaceted requirements of this developing field through a degree/credential in education program evaluation. Establish an award for outstanding education evaluation from education organizations.