Abstract
Purpose. (1) To describe the data management practices used by original BTSA programs that provide feedback about program quality/effectiveness and guidance for program change/revision/adjustment (useful, limited and not helpful practices); and (2) to describe the most significant program changes/refinements/adjustments made within the past five years (elements that prompted decisions as well as impeded making program changes/refinements/adjustments). Methodology. This qualitative study used descriptive research. A combined method design, a "between methods" approach, and purposeful criterion sampling were used. The sample consisted of directors from original BTSA programs. Individual in-depth interviews guided by an interview guide served as the primary method of data collection. Secondary methods included a questionnaire and document review. Findings. The design, implementation, and use of an information and analysis system is labor-intensive work, but is essential for providing feedback for continuous improvement. A combined methods practice to gathering data was identified as most useful for providing program feedback. The statewide annual survey was found to be of limited use in providing specific feedback especially for consortiums. The most significant program changes/adjustments had to do with group session design/delivery. Major factors prompting this change were: increased program size, meeting grade level/collaboration needs, providing flexibility, and improving attendance. Growth was a factor influencing two other significant changes: (1) increasing human resource support and (2) building program leadership capacity. For many programs, a shift to CFASST was significant. Bureaucratic processes and parameters were identified as barriers that hinder program improvements. Conclusions and recommendations. Programs that collect, analyze, and use data to make decisions are better able to effectively measure what is working and how to drive improvements. Information and analysis work hand in glove with ongoing program improvement, and is a critical component in planning for and determining the effectiveness of change. A framework/set of expectations related to assessing a program's approach, implementation and outcome with respect to information and analysis should be provided.