The following report includes results comparing matched STEMscopes and non-STEMscopes districts on the science component of the 2020-2021 State of Texas Assessment of Academic Readiness (STAAR TM ) for 8th grade. Districts were identified as STEMscopes districts if they had a subscription to STEMscopes and showed usage of STEMscopes Science in 8th grade for at least 50% of middle schools in a given district, based on analytic data. Once we had identified STEMscopes districts, they were matched with non-STEMscopes districts. Matching was based on the Texas’ Education Agency’s district types (e.g., urban, major suburban), then district size, and via a district’s percent of students classified as economically disadvantaged (i.e., free and reduced lunch).

On the STAAR assessment, the state of Texas creates proficiency benchmarks that identify students as ‘not proficient’, ‘approaching’, ‘meeting’, or ‘mastering’ grade-level proficiency. The percentage of students who ‘approach’ grade-level performance is used by the state as the district passing rate and we used this as the outcome in the study. We also included the percentage of 5th grade students who approached grade level science proficiency from the 2017-2018 STAAR assessment. We used the 2017-2018 assessment because the students who contributed data to the 8th grade proficiency rate in 2021 would have been in 5th grade in 2018. The 5th grade proficiency rate provides a baseline science achievement level across study districts.

Middle School STAAR Results

After matching, our sample included 758 districts (379 STEMscopes and 379 non-STEMscopes districts). As can be seen in Table 1, the STEMscopes districts and the non-STEMscopes districts did not statistically differ in regards to district size; students racial/ ethnic make-up; special education, gifted, or LEP status nor by average teaching experience in the district. Importantly, there were no differences in baseline 5th grade proficiency between STEMscopes (73.27% proficiency) and non-STEMscopes districts (72.13%). It is important to establish that the STEMscopes districts and the non-STEMscopes districts did not differ on these key variables, particularly baseline science proficiency as this ensures that potential differences in 8th grade science proficiency across districts are comparable (and are not due to previous differences). For the 8th grade STAAR science assessment in 2021, the state average proficiency rate for all Texas school districts ( N = 1146) was 67%. Within our study (n = 758), the unweighted total average proficiency rate was 67.84%. The unweighted average proficiency rate for the STEMscopes districts was 69.07%, versus 66.61% in non-STEMscopes districts. This was a significant difference (b = 2.45, p < .05).

Follow-up Analysis on Middle School Results

We conducted a follow-up analysis to further test differences across STEMscopes and non-STEMscopes districts after accounting for other important variables that influence science proficiency. Importantly, we wanted to account for 2018 5th grade science proficiency as previous science proficiency tends to be a strong predictor of future science proficiency. By including this and other demographic variables in the model, we provide a stringent test of the effect of STEMscopes. Put another way, this provides a weighted effect. We used multiple regression to recalculate proficiency rates taking into account 2017-2018 5th grade science proficiency as well as district size, average teacher experience, and district demographic make-up (i.e., percent of district students via race/ethnicity categories, socioeconomic status, LEP status, SPED status and gifted and talented status). Results are presented in Table 3 and Figure 1 below. The results indicate that once other important variables are accounted for, the districts that used STEMscopes still have a significantly higher overall 8th grade science proficiency rates compared to non-STEMscopes districts. Specifically, there is a weighted increase of 2.29% in 8th grade science proficiency for STEMscopes versus non-STEMscopes districts

Subpopulation Analyses

In addition, weighted proficiencies for specific subgroups of students were examined. Specifically, among economically disadvantaged students, STEMscopes districts (63.04% proficient) had higher passing rates than non-STEMscopes districts (60.57%) controlling for baseline 5th grade science proficiency and all other demographic variables, (b = 2.47, p < .05). There were no other significant differences among sub-populations, although results numerically favored STEMscopes districts.