The following report includes results comparing STEMscopes and non-STEMscopes Math schools in Dallas ISD on the math component of the 2020-2021 State of Texas Assessment of Academic Readiness (STAAR™) for 5th grade. Schools were identified as STEMscopes Math schools if they had a subscription to STEMscopes Math for the students in the tested grades (i.e., 5th grade) and showed usage of STEMscopes Math based on analytic data. STEMscopes Math rolled out in Texas in 2020-2021. Dallas ISD was an early adopter of STEMscopes Math.

The state of Texas creates proficiency benchmarks in math and identifies students as not proficient, approaching grade-level proficiency, meeting grade-level proficiency, and mastering grade-level proficiency. The percent of students in each of these categories is used to determine a school’s achievement in math. The percentage of students who approached grade-level performance is used by the state as the school passing rate. The 2020-2021 school year also occurred during the global Covid-19 pandemic.

Elementary School STAAR Results

The Dallas district average passing rate for 5th grade (N = 150) was 69%. Of Dallas ISD schools, 36 schools (24%) used the STEMscopes Math curriculum and 114 (76%) schools used either a district-created math curriculum or purchased a different math curriculum. The unweighted rates of proficiency for these two groups of schools are found in Table 1 below. The average passing rate for the STEMscopes schools was 72.3%, and the average passing rate for the non-STEMscopes schools was 68.1%. This was not significant. However, please see follow-up analyses below where differences are significant once accounting for school and student demographic differences between STEMscopes and non-STEMscopes schools in Dallas ISD. Across all benchmarks (approaches, meets, masters) the average rate was numerically higher in STEMscopes Math schools compared to non-STEMscopes Math schools.

In addition, achievement for specific subgroups of students was examined (with the overall “approaches benchmark” as this is the passing benchmark for the state). In Table 2 below, STEMscopes schools had higher passing rates for African American students when compared to African American students in non-STEMscopes schools. There was also a trend level finding for the Hispanic student sub-group. The “p-value” in Table 2 tells us how confident we feel about whether an association is true and trustworthy versus possibly occurring by chance. The typical p-value used by researchers is p < .05 which means we are 95% confident that the association is not by chance. In this case, we are only 92% confident that the association is not by chance. No other differences approached significance. The differences for African American students was particularly remarkable, with an increase of over 12 points for African Americans in STEMscopes Math schools relative to African Americans in non-STEMscopes Math schools. Please note this difference remained even when accounting for school and student demographics.

Follow-up Analysis on Elementary Results

We conducted a follow-up study to further evaluate differences across STEMscopes and non-STEMscopes Math schools in Dallas ISD after accounting for other important variables that influence student achievement. We used multiple regression analysis to recalculate and weight passing rates taking into account 2018-2019 math passing rates as well as average 5th grade class size by school, average teacher experience, average teacher-student ratio for a school, and demographic information of a school’s students (i.e., race/ethnicity, socioeconomic status, LEP status). Results are presented in Table 3 and Figure 1 below. The results indicate that once these important variables are accounted for, the schools that used STEMscopes Math have a significantly higher overall math passing rate compared to the schools that did not use STEMscopes Math.