The following report includes results comparing STEMscopes and non-STEMscopes districts on the science component of the 2020-2021 State of Texas Assessment of Academic Readiness (STAAR™) for 5th grade. Districts were identified as STEMscopes districts if they had a subscription to STEMscopes for the students in the tested grades (i.e., 5th grade) and showed usage of STEMscopes based on analytic data. The state of Texas creates proficiency benchmarks in science and identifies students as not proficient, approaching grade-level proficiency, meeting grade-level proficiency, and mastering grade-level proficiency. The percentage of students in each of these categories is used to determine the district’s achievement in science. The percentage of students who approach grade-level performance is used by the state as the district passing rate. The 2020-2021 school year also occurred during the global Covid-19 pandemic. Many Texas students began the school year remotely.

The state average passing rate for all Texas school districts that include 5th grade (N = 1162) was 61%. Of these districts, 608 districts used the STEMscopes science curriculum, and 554 (52.7%) districts used either a district-created science curriculum or purchased a different science curriculum. The rates of proficiency for these two groups of districts are found in the table below. The average passing rate for the STEMscopes districts was 62.7%, and the average passing rate for the non-STEMscopes districts was 61.8%. This was not significant. However, following the example of the state, we reclassified districts based on whether they reported that >50% of their students were remote or not in October 2020. When we look at the group with more than 50% of students remote, we see very little difference among STEMscopes districts (47.2%) and non-STEMscopes districts (48.1%) in the percentage of students who approach grade level proficiency. In the group of districts with less than 50% remote (more than 50% of students in face-to-face instruction), we see a significant difference in the percentage of students who approach grade level proficiency in STEMscopes (67%) versus non-STEMscopes (64%) districts. STEMscopes districts also had significantly higher rates of students who met and mastered grade-level proficiency, with differences of about 2 percentage points for both proficiency categories. Overall, the COVID-19 pandemic drastically decreased student science proficiency (from 74% to 61% overall), but this decrease was worse for students who were remote longer. We can also note that for students whose science knowledge was near the approaches benchmark, the impact of being out of school had a negative effect on how well they were able to learn science either from the STEMscopes science curriculum or any other curriculum. This may suggest that for students with lower levels of proficiency (compared to peers who were able to meet or master grade level proficiencies), it is more beneficial to be in face-to-face instruction and even more beneficial still to be in face-to-face instruction with STEMscopes Science.

Table 1

  Approaches

Approaches
 (>50% remote)

 Approaches
(<50% remote)		 Meets Masters
STEMscopes Districts (n = 608) 62.7% 47.2% 67.0% 30.1% 11.7%
Non-STEMscopes Districts (n = 554) 61.8% 48.1% 64.0% 28.0% 9.7%

 

In addition, achievement for specific subgroups of students was examined (with the overall “approaches benchmark” as this is the passing benchmark for the state. We did not account for time spent remotely in this analysis). In the table below, STEMscopes districts had higher passing rates for African American and Latino students. Significant differences ranged between 2.5 - 7 percentage points.

Table 2

  STEMscopes Districts Non-STEMscopes Districts B Standard Error p-value
Economically Disadvantaged Students 56% 55% 0.71 1.11 0.52
African American Students 47% 40% 7.19* 1.93 <0.01
Latino Students 58% 55% 2.51* 1.21 0.04
LEP Students 47% 49% -2.11 1.68 0.21

 

Follow-up Analysis on Elementary Results

We conducted a follow-up study to further evaluate differences across STEMscopes and non-STEMscopes districts after accounting for other important variables that influence student achievement. Given the differences among remote students and from our subgroup analysis,  we were interested in accounting for the effect of % of remote students (in October 2020), as well as important district demographics. We also wanted to account for the 2018-2019 passing rate. We used multiple regression analysis to recalculate these passing rates taking into  account 2018-2019 science passing rates as well as % remote in October 2020, district size, whether the district was a charter school district, average teacher experience, district attendance rate, and demographic information of students (i.e., race/ethnicity, socioeconomic status, LEP status, SPED status). Results are presented in the table below and include the approaches, meets, and masters grade level benchmarks. The results indicate that once these important variables are accounted for, the districts that used STEMscopes have significantly higher overall science passing rates for each benchmark level compared to districts that did not use STEMscopes. The largest difference was for the meets grade level benchmark. With STEMscopes, districts have a 2.36% higher passing rate than non-STEMscopes districts. Across levels, STEMscopes districts had between 1.5% - 2.4% more students meeting benchmarks compared to districts that did not implement STEMscopes science curriculum.

Table 3

  Approaches Grade Level Meets Grade Level Masters Grade Level
Predictors of STAAR B (SE) p-value B (SE p-value B (SE p-value
STEMscopes 1.54* (0.63) 0.05 2.36*(0.74)  <0.01 1.95*(0.43) <0.01
Students % Remote: 10/2020 -4.08* (0.59) <0.01 -2.66*(0.54) <0.01 -1.01*(0.32) <0.01
2018-2019 Passing Rate 7.15* (0.53) <0.01 5.73*(0.49)  <0.01 2.61*(0.29)  <0.01
Number of Students 0.05 (0.42) 0.91 0.24 (0.38) 0.53 0.25 (0.23) 0.27
Charter District -4.23 (1.70) 0.01 -1.48 (1.56) 0.34 -1.33 (0.92) 0.15
% Black Students -1.62* (0.55) <0.01 -0.67 (0.51) 0.19 -0.29 (0.30) 0.34
% Latino Students -0.31 (0.64) 0.63 -0.61 (0.59)  0.30 -0.58 (0.35) 0.09
% Econ. Disadv. Students -4.63* (0.62) <0.01 -5.05*(0.57) <0.01 -3.11 (0.34) <0.01
% LEP Students 0.53 (0.59) 0.36 1.01 (0.54) 0.06 0.88*(0.32) <0.01
% SPED Students -0.44 (0.46) 0.34 0.08 (0.42) 0.85 0.16 (0.25) 0.52
District Attendance Rate 3.58* (0.79) <0.01 2.14*(0.72) <0.01 0.56 (0.42) 0.18
Avg. Teacher Exp. in District 0.35 (0.55) 0.52 0.96 (0.51) 0.06 0.19 (0.30) 0.53

Conclusion

Across every year that has included standardized testing since 2015, districts that used STEMscopes science has had higher 5th grade passing rates than districts that did not use STEMscopes when controlling for previous year achievement, several important demographic variables, and (new this year due to the pandemic)  percent of students remote in the late fall. STEMscopes districts increased the “approaches grade level” passing rate of their students by 1.54%, resulting in an additional 2,657 students passing the state of Texas ‘passing’ benchmark. In addition to these passing rates, STEMscopes districts increased the “meets grade level” proficiency rate of their students by 2.36%, and the masters grade level rate by 1.95% resulting in an additional 2,000 students meeting grade level and 688 students mastery grade level proficiencies in these districts compared to non-STEMscopes districts. Results also showed that passing rates for African American and Latino students were higher in STEMscopes districts than non-STEMscopes districts. These findings show continued evidence that STEMscopes science curriculum is associated with increases in student science achievement.
Students Collaborating 350px

Empower Your Students With Accelerate Learning Today

Designed to make measurable impacts on student achievement, Accelerate Learning Inc. offers you a customized STEM solution for all your students. Learn how you can preview or demo our products for free.

Explore More Research & Case Studies