Background

The Math Nation program aims to boost students' math skills and help more students reach high achievement levels on state exams. This report examines how 10th-grade students in Kentucky using Math Nation performed on the state’s Kentucky Summative Assessment (KSA) in math, compared to students in schools that used different math programs. The focus was on the percentage of students who achieved the top "Distinguished" score. Additionally, the study explored how this effect varied among different groups of students, like those from economically disadvantaged backgrounds and those not needing additional support for English language learning.

Methods

To understand the impact of Math Nation, we compared the test scores of students from schools using Math Nation with those from schools using other math programs. We wanted to make sure that the schools we compared were similar in many ways, so any differences in scores would be due to Math Nation and not other factors. We matched schools based on several characteristics, like the percentage of students from different ethnic/racial backgrounds, the overall size of the school, grade-level enrollment, and the percentage of students receiving special services.

Key Findings

The main finding was that students in Math Nation schools were more likely to reach the Distinguished level in math than students in non-Math Nation schools:
  1. Overall Performance: Schools using Math Nation had nearly 2% more students reaching the Distinguished level in math compared to schools without the program, and this difference was statistically significant. While this difference is modest, it points to a positive trend for schools using Math Nation.
  2. Economically Disadvantaged Students: Students from economically disadvantaged backgrounds in Math Nation schools showed small but encouraging gains in performance, with a slight increase (almost 1%) in the number of students reaching the Distinguished level, although this difference was not statistically significant.
  3. Female Students: Girls using Math Nation showed statistically significant improvement, with nearly 2% more achieving higher proficiency levels than their peers in non-Math Nation schools.
  4. Non-English Language Learners (Non-ELL): Among students who did not require extra language support, significantly more in Math Nation schools (+1.52%) achieved Distinguished proficiency compared to similar students in non-Math Nation schools.
  5. Students without Individualized Education Plans (IEPs): These students also saw benefits from Math Nation, with a significant 1.38% boost in the number reaching higher proficiency.

2024 Math Nation Kentucky Figure 1

Figure 1. Distinguished proficiency rates for 10th grade students at Math Nation vs. non-Math Nation schools

Conclusion

These findings suggest that Math Nation may be beneficial in helping Kentucky students excel in math. Even though some differences were modest, the program's impact was positive across various groups, hinting that it could be a useful tool for improving math outcomes for a broad range of students.

In terms of impact on equity, Math Nation seemed to offer particular benefits for students traditionally underrepresented in high achievement categories, like those from economically disadvantaged backgrounds. However, the study suggests that larger samples might be needed to confirm some of these trends with more certainty.