High-Need Students and California’s New Assessments

First-Year Results Indicate a Wider Test Score Gap

The much anticipated first-year Smarter Balanced assessment (SBAC) test results were largely unsurprising: a smaller share of California students scored at or above the benchmark for the new Common Core State Standards (CDE 2015; EdSource 2015). However, the results for English Learner and economically disadvantaged students suggest that high-need students are further behind than educators may have thought.

The overall outcome was widely anticipated, and California is not alone-many other states that are implementing the Common Core standards and new assessments saw lower shares meeting their new standards than their old standards (Butrymowicz and Felton 2015; Harris 2015). There are many reasons for the lower scores: the new standards being taught changed and are being implemented unevenly across school districts (Warren and Murphy 2014; McLaughlin, Glaab and Carrasco 2014, Harrington 2016); the definition of having met the standards changed; and the testing method changed (London and Warren 2015).¹ While it is true that these assessments are in many ways not comparable (indeed, legislation passed in 2013 prohibits the CDE and local education agencies from doing so),² it is useful to understand which districts and schools are doing consistently well on both tests, and whether districts doing well on the SBAC English language arts (ELA) also do well on the SBAC math.

However, the drop in scores was particularly pronounced among high-need students. For example, only 12 percent of 4th-grade ELs met the SBAC ELA standard, compared to 30 percent the last time the CST was administered. Similarly, 26 percent of economically disadvantaged 4th graders met the SBAC standard, while more than half (53%) met the standard the last time the CST was administered (2012-13). As a result, the test score gaps between high-need students and white students are larger on the SBAC than they were on CST for both math and ELA (Figure 1).³ In particular, the gap in math between EL students and white students was 80 percent on the SBAC, compared to 38 percent on the CST-in other words, the share of EL students who met the standard for the SBAC was 80 percent lower than the share of white students who met those standards. For the ELA test, the SBAC score gap was 79 percent, rather than 61 percent on the CST. These gaps were larger for EL students than any other group. Gaps between economically disadvantaged students and white students are larger as well. For math, the gap between economically disadvantaged students and white students was 59 percent on the SBAC and 22 percent on the CST. For the ELA test, the gap was 54 percent on the SBAC and 31 percent on the CST.

Not only are these gaps larger in the first year of SBAC testing, but the ELA gaps for English Learner and economically disadvantaged students are now similar to the math gaps whereas previously the gaps were larger for ELA. This is not entirely unexpected: educators were expecting that some students would find math especially challenging because Common Core math requires more ELA proficiency than California’s old standards.⁴ But it does suggest that, as measured in the first year of the SBAC, high-need students have farther to go-perhaps further than the old standards and assessments indicated.

To better understand the test score gap, we explored the relationship between the share of students who are high-need in a district (or school) and the share of EL and economically disadvantaged students who met the SBAC standards. We explored the same relationship with the CST in order to identify the largest gaps for high-need students with SBAC and whether they are the same districts and schools with low scores on the CST. Before the SBAC results were released, there were cautions issued against comparing the test results. As we-ve already noted, the SBAC test assesses students according to different (arguably more difficult) standards that are in the early phases of implementation, and the test itself is very different. However, highlighting the districts and schools that have done well with both tests may prove useful for districts and schools looking to improve their results. Similarly, districts and schools that did well with the CST but are now struggling with the SBAC may need to change course.

We focus on English language arts results in the 4th grade both because early test scores have proven useful in predicting later educational outcomes (see an example in Zau and Betts 2008) and because so many English Learners are reclassified at this grade level (Hill, Weston, and Hayes 2014). The decision to move students out of EL status, which has historically depended heavily on scores on the ELA portion of the CST, has important implications for long-term student success. We include comparable findings for math in the appendix.

Assessing Test Scores at the District Level

In school districts with higher percentages of high-need students, smaller shares of students meet or exceed either the new Common Core standards as measured by the SBAC, or the old standards as measured by the CST. This is why the Local Control Funding Formula (LCFF) directs more funding (on a per pupil basis) to districts with a higher proportion of high-need students (Rose and Weston 2013). Along with the extra funding, districts have more spending flexibility. But they also have more responsibility for improving all student outcomes and closing achievement gaps for EL, economically disadvantaged, and other students.

District-designed accountability plans (known as LCAPs) include a variety of measures, including test scores. These plans are meant to help local stakeholders assess school and district progress, but the accountability mechanism is not yet clear. Policymakers at the state level are designing a replacement for the state’s old accountability metric (the API), but what it will include is still uncertain. While the SBAC is just one measure of academic achievement, it is an important one and will likely be a part of the state’s new accountability measure.

The solid diagonal lines in Figure 2 illustrate the relationship between the share of high-need students and the share meeting the standards at school districts, with orange for the CST and teal for the SBAC.⁵ We find a statistically measurable difference in the strength of the relationship between SBAC and CST scores and the district-wide percentage of high-need students. This is additional evidence that many districts-particularly those serving large high-need populations-had a difficult time adjusting to the new standards.

In Figure 2, the curved lines provide a sense of how this relationship varied across districts.⁶ The lines indicate the upper and lower bounds for the shares of 4th graders who met the ELA standard in 68 percent of demographically similar school districts.⁷ Overall, the scores were lower in districts with larger shares of high-need students, but in some districts student performance was either better or worse than expected, based on the shares of high-need students.⁸ The orange dots (for the CST) and the teal dots (for the SBAC) represent the 20 school districts that were furthest above or below expectations-these dots are mostly outside the curved lines.

These findings are important for districts that are performing below expectations for two reasons. First, these districts may not be aware of how far below average their high-need students are; and second, struggling districts may be able to learn from districts that are outperforming expectations.

In some districts, high-need students performed similarly on both tests. In Figure 2, we indicate that a district that is a top-20 performer on both the CST and the SBAC with a line that connects its test score dots. This shows that even in top performing school districts, a much lower share of students met the standard on the new test. Six the top 20 districts on the CST also performed in the top 20 on the SBAC. Five of the bottom 20 districts on the CST were also in the bottom 20 for on SBAC.¹⁰

We further explore the relationship in performance across the two tests, still focusing on high-need students, but analyzing the results separately for EL and economically disadvantaged students, both of which are treated separately in LCAPs and accountability measures.

Economically disadvantaged students

We find among the top 50 districts for SBAC ELA performance for economically disadvantaged students, 30 percent of these districts were also top performers on the CST ELA.¹¹ Nearly one-third of districts that exceeded expectations under the old standards are doing much better than expected with the new standards, as measured by these tests. Fifty-eight percent of districts in which economically disadvantaged students did well on the ELA portion of the SBAC were also likely to have good results on the math portion of the SBAC. More than a fifth (22%) of districts with good ELA results were also top performers on the CST ELA-as well as in SBAC math. At the other end of the spectrum, we found that 38 percent of districts among the lowest 50 performers were also at the bottom on the CST ELA. More than half of the districts with performance that was worse than expected on the SBAC ELA also fared poorly on the SBAC math (52%).

English Learner students

To better understand the SBAC performance of EL students at the district level, we explored the degree to which districts with relatively high English Learner performance on the SBAC ELA overlap with top performers on other exams. Overall, 58 percent of the high-performing districts on the SBAC were also top performers on the CST ELA.¹² Nearly three-quarters (72 percent) of the 50 top performing districts on the SBAC ELA were also top performers on the SBAC math. More than two-fifths (44 percent) of districts that were high performers on the ELA portion were also high performers on the CST ELA and in SBAC math. Among the 50 districts in which the EL population performed much worse than expected, 38 percent also underperformed on the CST ELA and 54 percent were underperformers on the SBAC math test.¹³ In 20 school districts, no 4th-grade EL students scored at or above the state standard on the ELA assessment. This means that many students will fall short of the standards for reclassification-as recently as 2012-13, 30 percent of school districts required scores of “proficient” on the CST in order for ELs to be reclassified (Hill, Weston, and Hayes 2014). Our finding implies that 4 percent of school districts would have no students eligible for reclassification based on that cut-off. In other words, some school districts have a great deal of work to do to improve EL student outcomes.

Despite differences in standards, assessments, testing methods, and benchmarks for meeting the standards, there is some overlap between high-performing districts on the SBAC and CST. However, more than half of high-performing districts on the SBAC were not high performing on the CST.¹⁴ And for many districts, what worked well in the past does not appear to be working as well now. Future research could help low-performing high-need districts understand how other districts with similar shares of high-need students managed to exceed expectations.

Assessing Test Performance at the School Level

LCFF dollars flow to districts, and districts have both the responsibility and opportunity for improving all student outcomes and closing achievement gaps for students at their district schools. Annual results from the Smarter Balanced assessments should help districts evaluate progress toward those LCAP goals that rely on SBAC scores and evaluate whether their LCFF funding is being directed toward students and schools in the most beneficial manner. Here, we examine the performance of economically disadvantaged students and EL students at schools throughout the state.¹⁵ As with the district analysis, we focus on English language arts results for 4th graders. We find that as the share of economically disadvantaged or EL students increases at a school, those students exhibit lower test scores, on average, and we examine the performance of EL and economically disadvantaged students separately at their schools.

Economically disadvantaged students

Our analysis of school-level test results indicates that as the share of economically disadvantaged students in a school increases, the share of those students scoring at or above the level of the standard falls on both the CST and the SBAC. In Figure 5, this is illustrated with the solid, downward trending line. As we did at the district level, we find that some schools exceed or fall far below expectations. Because districts are responsible for improving educational outcomes, it is important to identify the schools that may need the most help. These struggling schools may benefit from the experience of schools that are faring relatively well on the SBAC.¹⁶

Four of the 20 schools in which economically disadvantaged students scored better than expected on the SBAC were also in the top 20 performers on the CST.¹⁷ Conversely, three of the 20 schools with scores that were much worse than expected on the SBAC were also low performers on the CST.¹⁸

When we compare the results from top and bottom performing schools on the SBAC ELA to their experiences with the CST, we find that 26 of the top 100 SBAC performers were also among the 100 schools that exceeded expectations on the CST the most. Almost half (47%) of the top performing schools on the ELA portion of the SBAC were also top performers on the math portion. At the other end of the spectrum, we find that 18 of the bottom 100 performing schools were also among the bottom performers on the CST. Just 29 percent were on the bottom-performing lists for both the SBAC ELA and math tests.¹⁹

English Learner students

EL students are the least likely of all student groups to meet the state standards as measured by the SBAC-only 12 percent of 4th-grade ELs did so in the first year of testing. This means that in hundreds of schools, no EL students are meeting the state ELA standard. In Table 1, we list the schools with EL test scores in the top 20 on the SBAC (on the left) and the CST (on the right). Only three schools (Brentwood Academy in Ravenswood City Elementary School District, Otay Elementary in Chula Vista, and Robert Louis Stevenson in Wright Elementary School District) are top performers on both the CST and SBAC. (These schools are highlighted in the table.)

The biggest overachiever in terms of EL scores on the SBAC was Newhall Elementary in Newhall School District. Based on the school’s demographic profile, 6 percent of EL students would have been expected to meet or exceed the state ELA standard, but 52 percent actually did so.

Four of the top 20 performers are in Westminster School District in Orange County, two are in San Diego Unified School District, and two are in San Francisco Unified School District.

Of the 1,653 schools with at least 100 EL students taking the SBAC, 273 (or 17%) had no 4th-grade EL students who met the ELA standards. The shares of EL students in these schools ranged from about 25 percent to 84 percent. We found no relationship between EL test scores (as opposed to the percentage who met the standard) and shares of EL students at these schools. Mean ELA scaled scores for ELs in these schools range from 2,307 to 2,396 (out of 2,663 possible points) at the lowest achievement level (“Standard Not Met”). When we include all schools with enough tested ELs (10 or more) to have their scores reported by CDE, we find that in 740 out of 3,464 schools (21%), no 4th-grade ELs who met the state ELA standard; in 748 of these schools, no ELs met the math standard.

The schools with no EL students meeting the SBAC standards are distributed across the state (Figure 6). In counties with at least 10 schools that administered tests to 100 or more ELs, the shares of schools in which no 4th-grade ELs met the state ELA standard are indicated by shading.

In more than 30 percent of the schools in Madera and San Joaquin Counties that administered tests to 100 or more EL students, no ELs met the SBAC standards. The district with the largest number of schools in which no EL students met the ELA standards is Los Angeles Unified School District: in 182 of the 343 schools with more than 100 EL students, no students met the standards. There were 21 such schools in Oakland Unified (out of 34); and there were 31 schools (out of 45) with 100 or more ELs in which no ELs met the standards in Fresno Unified, 22 (out of 31) in Stockton, and 12 (out of 25) in Ontario Montclair.²⁰

Overall, 30 percent of SBAC high-performing schools and 17 percent of SBAC low-performing schools are similarly ranked for the CST. In addition, there is a high degree of overlap in over-performers on the SBAC ELA and math tests (53%).²¹ However, there is much less overlap-just 8 percent-in the bottom performing lists for both ELA and math.

While there are some schools with high concentrations of successful EL students, many more are falling far short-in more than 700 schools, no tested EL students met the state standard. It will be important for these schools to find ways to help their EL students master the standards, especially because scores on the SBAC are likely to be an important criterion by which EL students will be evaluated for reclassification.

Policy Implications

With the advent of the Common Core standards and the SBAC assessments, the bar for “meeting the standard: is clearly higher. The first-year test results indicate that English Learners and economically disadvantaged students have the most ground to cover. However, in some districts and schools these students performed relatively well, including some whose students performed well on the CST, but also some in which students did not.

As California’s new accountability framework is being developed, educators at the district and school levels have a unique and important opportunity to examine their performance, and to learn from schools and districts that are having more success with the SBAC. The state should play a role in facilitating this process; the CORE districts’ peer matching program may prove to be a useful model. The CORE districts, a collection of 10 districts representing more than one million students, pair “schools of distinction” with demographically similar low-performing schools to encourage collaboration between their teachers and administrators. The State Board of Education could also facilitate research and encourage or require the California Collaborative for Educational Excellence and the county offices of education to enlist peer schools to provide technical assistance to struggling districts and schools.²²

There is strong evidence that English Learner students have the farthest to go in meeting the new standards. In the past, the state ELA test was a critical factor in reclassification decisions. If this carries over to the new reclassification policy, the SBAC may become a large barrier to reclassification for many EL students. State policymakers should consider the findings we have presented when implementing the new EL standards and deciding how EL students will be reclassified.

Accountability in California is still evolving-there are many questions about the LCAP, the LCAP rubrics, the state replacement for the Academic Performance Index (API), and the federal Every Student Succeeds Act (ESSA). But, however these questions are resolved, the goal of the LCFF is clear: to improve outcomes for all students and close achievement gaps for high-need students. The first-year SBAC results provide an important call to action for districts and schools that are struggling to educate high-need students.