|
The view of accountability that emerges from our research is that schools vary in their responses to external pressure depending on their level of internal accountability, the way they manage their internal structures, the ways in which they define and distribute leadership, and the ways in which they address the knowledge and skill requirements of the new demands of policy. Capacity inheres not in the presence or absence of these factors, but in the relationships among them. Accountability policies produce variable responses among schools based both on the initial capacities of schools and on whether schools increase or improve their capacities as they are responding to the requirements of new policies.
An important finding of our work, as noted by Debray, et al. is that school capacity seems to dominate policy in determining effects on student performance. While, as Carnoy, Loeb, and Smith indicate, the aggregate effect of accountability policies on individual students seems to be generally positive, in terms of student performance and retention in school, there is considerable variability among schools in their capacity to produce good outcomes. This view of the relationship of capacity and accountability raises one troubling issue about existing accountability policies. If it is true, as our case studies seem to indicate, that state accountability policies are based on the working theory that external pressure for performance is designed to mobilize existing capacity, rather than to create new capacity, then it is possible that the long-term effect of accountability policies, other things being equal, could be to increase the gap in performance between high and low capacity schools. The relative absence in our case studies of evidence of deliberate, systematic efforts to influence capacity by states and localities makes this a troubling issue. How one would intervene to increase and equalize capacity in schools is a complex issue. The answer depends on how the pieces of capacity fit together. Our research suggests that internal accountability is a necessary condition that precedes any effective response to external accountability policies. Variation in internal accountability is likely to be the main variable explaining variability in schools' performance against external accountability measures. But internal accountability is not a static or stable characteristic of schools, it is a continuous variable, and it is the creation of active problem solving inside schools. Schools that have high internal accountability are likely to have it because they have worked hard as organizations to create it. This in turn means that these schools are more likely to have the capacity to manage the various structural problems posed by external accountability policies, and that they are more likely to treat structures instrumentally than to try to adapt their responses to fit into existing-often dysfunctional-structural arrangements. But working out these solutions requires a particular bias on the part of school leaders toward the creation of internal accountability and toward the use of structures. And working through these solutions requires a particular view of how leadership might be defined and distributed in the organization. In doing this work, people in schools are likely to run up against the limits of their own knowledge and skill, creating the need for access to new sources of information, curriculum, and instructional practices. To the degree that the school is successful in mobilizing and focusing the existing knowledge, skill, and commitments of it members, and augmenting it, one would expect to see systematic improvements in performance. One thing that stands out in our case studies is that schools that try to respond to external pressure by doing what they are already doing at a higher level of efficiency and effectiveness typically don't produce substantial improvements in either practice or performance. The fact that the schools that seem to fit this pattern are disproportionately-but not entirely-target schools is troublesome. This finding suggests that the work that would be most productive in improving the responsiveness of schools to external accountability measures is work that focuses on increasing school-wide coherence and agreement on expectations for student performance and instructional practice, rather than focusing exclusively on focused, incremental work in tested subjects. This is daunting, difficult work in schools that are often cautious, intimidated, risk-averse because of their poor performance in the past. And it requires school leaders with a different understanding of their work than one typically finds in low-capacity, low-performing schools. Nor is it possible to think of this kind of work happening without substantial external support and resources at the local and state level designed to build the pool of people with the knowledge and skill to carry out this work in schools-not just principals but subject matter specialists, lead teachers, coaches, and so on. It is also possible that what we observed in the target and low-capacity schools that were attempting to comply with external accountability systems without fundamentally changing their internal accountability systems was, in fact, the early stages of a longer-term improvement path. External accountability systems may force attention to issues of internal accountability and capacity in general by holding schools to external standards and scrutiny. In general, aggregate data on student performance of the kind in the Carnoy, Loeb, and Smith study seem to indicate that external pressure does produce significant effects on student academic performance. All of the target schools in our Texas case studies were able successfully to increase test scores. The issue that needs further scrutiny is the degree to which schools can manage the improvement process, from compliance-oriented decisions to major changes in their capacity, by themselves without major support. The basic point for accountability policy, however, is that powerful responses to accountability policies require school-wide capacities that, at the moment, are highly variably distributed in the population of existing schools. To the degree that policies continue to operate on the assumption that they call forth existing capacities, rather than that they create and distribute new capacities, the effect of these policies is likely to be considerable variation in school performance.
|
