AERA Distinguished Contributions to Education Research Award Lecture: Standards-based reform and C-SAIL

Andy Porter
Tuesday, April 19, 2016

C-SAIL director Andy Porter received the American Education Research Association’s highest honor last year when he received the Distinguished Contributions to Education Research Award. Award recipients give a featured lecture at the AERA annual conference the following year. Last week, Andy used that opportunity to talk about the history of standards-based reform and C-SAIL’s contributions and place in that history. In this post, Andy briefly recaps his lecture.

Standards-based reform has its origins in the leadership of a small handful of states in the 1980s, most notably California and Kentucky. The basic idea of standards-based reform is to make clear, through content standards, the content that students are to master and that teachers are to teach. In 1991, Smith and O’Day published a highly influential conceptual piece on this idea which they called “systemic reform.” The idea, as foreshadowed in the leadership of states, is to build a coherent system of supports to facilitate the implementation of instruction aligned to the content standards: student achievement tests, performance standards, accountability, align curriculum materials, aligned teacher education and professional development. Smith convinced Basam Shakashiri, then director of the Education and Human Resources Directorate at the National Science Foundation (NSF) to use the concept of systemic reform for a grants competition for states to take on building standards-based reform in K-12 science. This and subsequent competitions at NSF went a long way toward bringing attention to and implementation of standards-based reform. In 1994, Smith was Undersecretary of the US Department of Education and in that position served as the chief architect of the ESEA reauthorization. No surprise, Smith used standards-based reform as the core idea.

As a precursor to standards-based reform, I built and led a small team of researchers at the Institute for Research on Teaching at Michigan State University and funded by the National Institute of Education (a precursor of today’s Institute for Education Sciences). Called the Content Determinants group, we studied what factors influence teachers in deciding what to teach and what not to teach to their students. This is, of course, the central question for standards-based reform. In the 1980s and before, however, education research and policy focused almost exclusively on pedagogy while ignoring the content of instruction. The Content Determinants group work led to something now called “the Policy Attributes Theory”. This theory hypothesizes five potential policy attributes that give a policy weight in influencing teacher’s content decisions: specificity, consistency, authority, power and stability. Specificity refers to how clear the content message is in the policy. Consistency refers to how aligned that policy content message is to the content message in other policies. Authority and power are concepts taken from Max Weber, German sociologist. Power refers to the use of rewards and sanctions related to policy compliance while authority refers to how persuasive the content message of the policy is. Authority can be built through law, consistency with norms, based on expertise and promoted by charismatic individuals. Stability describes the duration of the policy without change.

The key concept in standards-based reform is consistency, typically referred to now as alignment. In my 2001 AERA presidential address, I described a methodology for defining and measuring content and for measuring the degree of alignment between the content message of content standards and the content actually taught in the classroom. This methodology, now known as Surveys of Enacted Curriculum (SEC), has become a commonly used methodology in education research and to some extent in education policy building. Briefly, content is defined by a content language at the intersection of topics (e.g. linear equations) and cognitive demand (memorize, solve novel problems). Overtime content languages have been defined in mathematics, English language arts, science and social studies. The number of topics varies across subjects from something like 250 to 350 and the number of cognitive demands is five.

20 years after Smith and O’Day and standards-based reform as the framework for ESEA, the National Governors Association (NGA) and the Council of Chief State School Officers (CCSSO) published Common Core State Standards (CCSS). In the same year, the U.S. Department of Education funded two multistate testing consortia to build systems of assessments in mathematics and English language arts that are aligned to the CCSS: Partnership for Assessment of Readiness for College and Careers (PARCC) and Smarter Balanced Assessment Consortium (SBAC). I used the SEC to investigate how much change from the then-current state content standards the CCSS represented. The answer is that not only was there a lot of content variation among states in their standards but also each set of state standards differed in content quite dramatically from the CCSS. This was true for both mathematics and language arts. Further, while the SEC alignment was low when measured at each grade level the alignment was higher but remained low when alignment was investigated at the aggregation across grades three through eight. In that study we also investigated alignment of current state assessments to the CCSS. Those alignments were quite low, as well, in part because assessments represent samples of items not an entire domain of content as do content standards but still low.

Because the alignment of a state’s student achievement tests is notoriously low to that state’s content standards despite ESEA requiring that they be highly aligned, my students and I developed a test construction algorithm that uses the SEC to build aligned assessments rather than just to measure the degree of alignment of an assessment to its content standards after the fact. The algorithm selects items for a test form from a larger set of items such that the first item selected is the item most highly aligned to the target standards. After each item selection, the standards target is re-centered, removing the content of the items selected from the target, and another item is selected as the one most aligned with the target from the remaining pool of items. The resulting alignment, which is measured on a scale from 0.0 to 1.0, has proven to be surprisingly high, from .4 to .5 for tests of length 18 to 25 items in comparison to .25 for much longer state assessments to their content standards.

Moving up to the present in this history of standards-based reform, the Institute for Education Sciences funded the Center on Standards, Alignment, Instruction and Learning (C-SAIL) on July 1 of 2015. I am excited to be the director of that center which is investigating the implementation and effects of college- and career-readiness standards and designing an intervention to facilitate valid implementation of standards-based instruction in classrooms. Policy attributes theory guides the work, the SEC is being revised and updated to better capture the content of college- and career-readiness standards (already completed) and used to provide feedback to teachers participating in our FAST intervention, and to build student achievement assessments highly aligned to college- and career-readiness standards to test the effects intervention.

While we are still in our first year, much has already been accomplished. The SEC revisions are complete. Implementation studies in our partner states are underway with first results available late this coming summer. The longitudinal study, which uses National Assessment of Education Progress data to estimate state effects, will produce its first results this summer. We have completed the profile of all 50 states the adoption of college- and career-readiness standards and aligned assessments (look for our interactive map coming soon!). Our FAST intervention, which is designed to take standards-based reform through the classroom door to students, is being built now, will be piloted in year two and will be tested with a randomized control trial in years three and four.

Later in the same year that C-SAIL was funded the U.S. Congress passed the new reauthorization of ESEA, “Every Student Succeeds Act”. The act retains standards-based reform as its major framework, requiring standards and assessments in grades three through eight and high school. At the same time the act strips the U.S. Department of Education of its power to force states to use certain approaches to standards-based reform (i.e. the U.S. Department of Education can no longer use rewards and sanctions). Nevertheless, states are free to use all the power and authority they want as well as the other policy attributes to build the strongest standards-based reform systems possible. It is too early to say where states will go with their new freedom. Will they continue their past practices in standards-based reform or will they go in a new direction? Will they exercise rewards and sanctions in their approach to standards-based reform or will they rely on authority alone. What is the future of the two multistate testing consortia? It looks like PARCC as failing and SBAC is challenged. What is the future of Common Core State Standards which have become increasingly controversial in several states? Will the C-SAIL FAST intervention be as successful as we hope and become the savior of standards-based reform? Stay tuned.