Reviewing our progress and looking ahead: A message from C-SAIL’s director

As the Center on Standards, Alignment, Instruction and Learning (C-SAIL) begins its third year of work with five years of funding from the Institute of Education Sciences, I wanted to reflect briefly on our progress to date and what lies ahead. Our focus has been from the start on the implementation and effects of standards-based reform.  In the past few years, the amount of change and flux in the design and instrumentation of standards based reform has been enormous, especially in comparison to the first 20 years of standards-based reform including the years of No Child Left Behind (NCLB).

Since we submitted our proposal to IES only three years ago, the Common Core State Standards (CCSS) have become a political hot potato, toxic to many.  In 2014, the CCSS were being newly implemented in 46 states and the District of Columbia. Today, nine states have revised or rewritten the standards as “state” college- and career-readiness standards. The new federally funded assessment consortia, PARCC and Smarter Balanced, were hard at work designing and building assessments aligned to the CCSS; as of 2016, only 20 states remained in either consortium. On top of that, the Every Student Succeeds Act (ESSA) was passed with bipartisan support restoring considerable state autonomy in contrast to the days of NCLB. While we did not anticipate this turmoil at the time we submitted our proposal, our focus on implementation and effects is as relevant today as it was three years ago.

Our Framework

Let me start with a brief overview of the framework we use for investigating the quality of implementation because you will likely find it useful in your own work in education. From our earlier research on standards-based reform and our investigations of how teachers make decisions about what to teach their students, my colleagues and I have developed the Policy Attributes Framework. High-quality implementation of standards-based reform requires the following: specificity in stating the content goals of teaching and learning; consistency/alignment among the policies and practices put in place to pursue those goals (e.g. content standards, student achievement tests, performance standards, curriculum materials, professional development, accountability); authority/legitimacy to those charged with implementation; power through rewards and sanctions; and stability—when policies, practices, and leadership are in a state of flux, it is difficult to take each new thing that comes along seriously.

Study Updates

Now let me bring you up to date on our major lines of work: the Measurement Study, Implementation Study, Longitudinal Outcomes Study, and building and testing the effects of our Feedback on Alignment and Support for Teachers (FAST) Program Study.

Our first major contribution was produced early on in our first year of work through the work of the Measurement Study. Under the leadership of Morgan Polikoff at USC, we completed a detailed description of what content means in the areas of K-12 English language arts and mathematics. Drawing upon earlier work that produced the Surveys of Enacted Curriculum (SEC), those definitions of content were revised and expanded to better reflect the new content emphases of college- and career-readiness standards (e.g. the CCSS, the Texas content standards). The tools provide careful and replicable descriptions of the content teachers teach and an objective measure of alignment between content standards and assessments, taking much of the mystery out of the increasingly ambiguous term of alignment. These C-SAIL tools are being used by C-SAIL as we continue our work.

The entire C-SAIL program of research places its bets on the importance of understanding the quality and quantity of efforts to support the implementation of new and more challenging content standards required by ESSA. These college- and career-readiness standards include but are in no way limited to CCSS. Our Implementation Study PI is Laura Desimone at PennGSE. Much of the work focuses on our partner states which for the first two years included California, Kentucky, Massachusetts, Ohio, and Texas, reflecting much of the variation among all 50 states in approaches to standards-based reform. The findings from our surveys at the state, district and school levels and our interviews at the state and district levels are too many to summarize here but can be found in our state reports. Using our new tools for measuring instructional content, we have found that elementary math and high school ELA teachers are providing instruction that is better aligned to the new standards in their state than are elementary ELA teachers and high school math teachers. Even so, there is plenty of room for more aligned instruction for most teachers in both mathematics and ELA at all grade levels. Especially challenging is for teachers to provide instruction that captures the new demands concerning text complexity in ELA and mathematical practices in mathematics.

In all C-SAIL work, we pay special attention to content standards implementation for students with disabilities (SWD) and English language learners (ELLs). We find that ELL teachers report coverage very similar to general education teachers but that SWD teachers quite consistently report substantially less well-aligned instruction, especially reflecting content below grade level. Despite the greater state autonomy provided by ESSA, our findings are very similar across states, the one notable exception being ELL where there are substantial differences among states in approaches to implementation.

Our Longitudinal Outcomes Study is led by Mengli Song at AIR and focuses on changes in National Assessment of Education Progress (NAEP) student achievement, high school completion, and college going levels and trajectories post-state adoption of the new college- and career-ready standards. Song’s work also focuses on explaining any differences found using measures of quality and amount of support for implementation as framed by our policy attributes theory. Since virtually all states adopted new and more challenging college- and career-readiness standards in the same year, 2010, we are not able to use differences among states in times of adoption as a way to estimate effects. We did ask whether states with less rigorous content standards prior to adoption than states with more rigorous content standards prior to adoption had steeper post-adoption trajectories, with our hypothesis being that adoption represented a bigger change for the states having previously less rigorous content standards. Of course, it could be that states with more rigorous prior content standards are states best positioned to continue to profit from higher content standards. In any event, we found small negative contrasts in ELA and no effects in mathematics when we compared states on degrees of rigor of prior content standards. Going forward, we will focus on estimating the effects of amount and quality of support for implementation when estimating differences among states.

Toni Smith at AIR is leading our efforts to build a program that supports teachers in high-quality instruction well aligned to the new and challenging standards, called the Feedback on Alignment and Support for Teachers (FAST) program. The program was designed and built in year one, piloted in year two and is being formally tested in a randomized control design in years three and four. The program consists of individualized coaching from a content expert; school level collaborative academic study teams; formalized specific feedback to teachers on how well aligned the content of their instruction is to their state standards; analysis of videos of their teaching; and, an online library of resources. While we are just beginning our formal test of the program, I can report from our pilot that teachers are enthusiastic about their work with the individualized content expert coaching and are surprised to find how much better they understand the intent of their state content standards after working in the program for the better part of the year.

---

We find that most states, but not all, are making a good-faith effort to support the implementation of standards-based reform. At the same time, state departments of education are too distant from and lack the resources to provide the type and quantity of support teachers need. The constant change at the federal level and in most states is having negative effects upon implementation of the challenging college- and career-readiness standards. Hopefully, there will be greater stability in the next few years. A particular challenge lies with SWD students. We find too many educators at all levels believe the standards are not appropriate for these students.

I hope you explore our website to better understand our findings to-date and to keep up with new findings as they occur. Also, keep an eye out for our annual conferences aimed at engaging educators, policymakers, and researchers in conversation about the implications of and future directions for our work. Our next conference is scheduled for April 27 in Washington, D.C.

Andy