Key Finding

STEMscopes Math schools demonstrated significantly greater mathematics learning gains than matched comparison schools in all three grades. Adjusted differences ranged from 2.47 to 3.08 percentage points. No statistically significant differences were found in overall mathematics achievement or exploratory science achievement.

Main Effects On Mathematics Learning Gains

 

Grade Matched School Pairs Adjusted Difference (Pp) 95% Confidence Interval Hedges' G | P
3 156 +2.99 [0.76, 5.22] 0.246 | .009
4 149 +3.08 [0.91, 5.24] 0.262 | .005
5 148 +2.47 [0.14, 4.80] 0.207 | .038

 

Study Design

The independent, third-party evaluation used a quasiexperimental design with grade-specific propensity score matching (1:1, without replacement) and analysis of covariance (ANCOVA). Schools were matched on prior-year mathematics achievement, economic disadvantage, minority enrollment, charter status, and Title I designation. Postmatching balance was good across all five measured characteristics (absolute standardized mean differences below 0.10). Analyses adjusted for prior-year achievement; Hedges’ g used pooled unadjusted outcome standard deviations.

Exploratory Implementation Findings

Among higher-use STEMscopes schools compared with their original matched non-STEMscopes schools, mathematics learning gains were higher in Grade 3 for total clicks (+4.34 pp; g = 0.409; p = .002) and unique scopes (+5.02 pp; g = 0.463; p = .003), and in Grade 4 for total clicks (+4.58 pp; g = 0.396; p = .002). Grade 3 unique-scopes comparisons also favored STEMscopes for science achievement (+3.59 pp; g = 0.186; p = .023). No significant Grade 5 median-split differences or continuous dosage associations were identified. One Grade 4 implementation-tertile omnibus test was significant, without a consistent linear pattern.

Interpretation and Limitations

Findings provide consistent, promising evidence of an association between STEMscopes Math participation and mathematics learning gains. Because schools were not randomly assigned, unmeasured differences may remain despite matching. Implementation intensity was not randomly assigned, so exploratory dosage findings should not be interpreted as causal effects. The study has not been assigned an Evidence for ESSA evidence tier.

Evaluator

Dr. Matthew Foster

 

Dr. Matthew Foster is an associate professor in the University of South Florida’s Department of Child and Family Studies. His research focuses on children’s development and academic achievement, particularly in STEM, across diverse backgrounds and abilities. He partners with Accelerate Learning as an external evaluator, applying his statistical and developmental expertise to examine how programs such as STEMscopes Math support student learning.