The following report includes results comparing STEMscopes and non-STEMscopes districts on the 2020-2021 California Science Test (CAST™) for 5th grade. Districts were identified as STEMscopes districts if they had a subscription to STEMscopes for the students in the tested grades (i.e., 5th grade) and showed any usage of STEMscopes in that grade based on analytics data. The state of California creates proficiency benchmarks in science and identifies students as not meeting the standard, nearly meeting the standard, meeting the standard, and exceeding the standard. The percentage of students who “meet and exceed” the grade level standard is used by the state as the district passing rate. The 2020-2021 school year also occurred during the global Covid-19 pandemic. Many California school districts (469 of 883 that submitted 5th-grade CAST data) reported 0s in the “students tested” column, leaving a sample of 413 districts. Of these 413, an additional 117 presented masked data, leaving only 296 with usable 2021 passing rate percentage scores (~33% of 883 submitted). Masked data occurs when too few students have taken the assessment (and indeed, among the masked cases, districts reported fewer than 10 students assessed), leaving many districts without publicly available 2021 CAST data.

The state average passing rate for all California school districts that include 5th-grade scores (N = 296) was 29.74%. Of these districts, 43 districts (14.5%) used the STEMscopes science curriculum, and the other 253 districts used either a district-created or purchased science curriculum. Before evaluating group differences between the STEMscopes and non-STEMscopes districts, we tested whether districts differed in background characteristics to ensure we were making an equitable comparison. Specifically, we tested for differences in previous levels of district science achievement (based on the 2019 CAST passing rate), percent attendance average, whether a district included charter schools, as well as the percent of students served by a district who were reported as: economically disadvantaged, requiring special education services, English language learners and percent of students served from different racial/ethnic backgrounds. As can be seen in Table 1, there were two statistically significant background differences: We found that fewer African Americans were served in STEMscopes districts and that consistent student attendance was higher in STEMscopes districts than non-STEMscopes districts. Please note no other differences between districts were found; of particular importance, we found STEMscopes and non-STEMscopes baseline science achievement in 2019 did not significantly differ. However, we only present our full model, with all covariates, given that some differences were found.

Table 1

Baseline Characteristics

This table demonstrates that STEMscopes and non-STEMscopes districts generally were similar to each other in aggregate background characteristics.

Background Characteristic Total % STEMscopes non-STEMscopes t-value p-value
Baseline District 2019 Science Proficiency (5th Grade) 31.72% 34.81% 30.02% 1.39 0.17
District Attendance (Students Not Considered Truant) 84.90% 88.51% 84.47% 3.16 0.002*
District Includes Charter Schools 33.42% 35.71% 33.15% 0.33 0.75
% Economically Disadvantaged Students 59.35% 56.24% 59.71% 0.76 0.45
% Black/African American Students 3.15% 1.96% 3.30% 2.68 0.008*
% Latino/Hispanic Students 43.55% 45.18% 43.36% 0.35 0.73
% White/Caucasian Students 40.79% 38.04% 41.12% 0.66 0.51
% Multi-Racial Students 5.54% 4.81% 5.63% 1.24 0.22
% English Language Learners 14.53% 18.80% 17.42% 0.53 0.60
% Special Education Students 13.51% 12.04% 13.68% 1.85 0.07

Follow-up Analysis on Elementary Results

We used multiple regression analysis to calculate potential differences in the CAST district level passing rates between STEMscopes and non-STEMscopes districts. We also accounted for (included) 2018-2019 science passing rates as well as whether the district included charter schools, district attendance rate, and demographic information of students (i.e., race/ethnicity, socioeconomic status, ELL status, SPED status). By accounting for other important variables in the model (including the two with baseline differences), we provide a stringent test of the effect of STEMscopes. Put another way, this provides a weighted effect. Results are presented in Table 2 and Figure 1 below. The results indicate that once other important variables are accounted for, the districts that used STEMscopes have significantly higher overall 5th-grade science proficiency rates compared to non-STEMscopes districts. Specifically, there is a weighted increase of 3.79% more 5th graders meeting the California science passing benchmark for STEMscopes versus non-STEMscopes districts.

Table 2

MULTIPLE REGRESSION RESULTS

“Meets or Exceeds”
grade level standard

Predictors of CAST B (SE) p-value
Intercept 27.43 (0.87) <0.01
STEMscopes 3.79* (1.94) 0.05
2018-2019 Passing Rate 8.23* (1.10) <0.01
Charter District -3.35 (1.33) <.0.05
Students % Black -0.60 (0.73) 0.41
Students % Latino -1.47 (1.17) 0.21
Students % Multi-Racial -1.78 (1.10) 0.11
Students % Econ. Disadv. -5.61 (1.38) <.0.01
Students % SPED -0.22 (0.95) 0.82
District Attendance Rate 1.80 (1.33) <0.05

Figure 1

WEIGHTED 5TH GRADE SCIENCE PROFICIENCY IN STEMSCOPES AND NONSTEMSCOPES SCIENCE DISTRICTS

27.43%
Non-STEMscopes Districts
31.22%
STEMscopes Districts

Table Note: The model above includes the Students % Latino variable, but does not include Students % White or Students % English Language learner because these variables were correlated above +/- 0.80 (numbers close to -1 or +1 represent a very strong relationship). Background variables with too strong of a relationship can bias model results. We ran additional models where only %White was included, and only %ELL was included. The result pattern was similar, thus we chose to present the above analyses with %Latino.

Conclusion

This report provides evidence that districts that used STEMscopes Science in the 2020-2021 school year had higher 5th grade passing rates than districts that did not use STEMscopes when controlling for previous year achievement, and several important district and student demographic variables. This evidence is also consistent with our previous study evaluating the 2019 data. Specifically, STEMscopes districts increased the “meets and exceeds grade level standard” passing rate an estimated 3.79% in STEMscopes districts, resulting in approximately 640 additional students passing among the students who were tested. We interpret these data cautiously given the lower number of districts that reported standardized data in 2021 due to the Covid-19 pandemic. Even within districts that provided scores, the number of students with valid scores was considerably lower than the 2019 test. We also interpret findings optimistically as these results are very similar to the full data analyses results from 2019. Taken together, these findings provide consistent support for the effectiveness of the STEMscopes Science Curriculum.
Students Collaborating 350px

Empower Your Students With Accelerate Learning Today

Designed to make measurable impacts on student achievement, Accelerate Learning Inc. offers you a customized STEM solution for all your students. Learn how you can preview or demo our products for free.

Explore More Research & Case Studies