This report presents the evidence that the Math Nation program is effective at raising student achievement on the End-of-Course (EOC) Algebra exam in South Carolina. We used a post facto quasi-experimental design (QED) with a matched control group to evaluate potential associations between Math Nation usage and EOC achievement for algebra students in South Carolina. QEDs with matching attempt to overcome the barriers of “non-random” assignment. The report presents four rigorous studies using diverse methodologies that provide strong evidence of the curriculum’s effectiveness in improving student and school achievement while demonstrating its equity in serving a diverse student population.
Study 1 investigates whether schools using Math Nation performed better in Algebra EOC exams compared to those that did not. The study compared test scores from 120 schools, ensuring they were similar in size, past performance, and demographics to isolate the impact of Math Nation. Our analysis showed that students in Math Nation schools scored almost 3 points higher on average than those in schools using other math programs. See Figure 3.
Study 2 reveals that Math Nation schools positively impacted student performance, particularly for Black/African American and White/Caucasian students, who showed higher test scores and proficiency levels compared to their peers in non-Math Nation schools. Female students also demonstrated promising trends, with slight improvements noted for students in poverty and English language learners.
To evaluate school performance, schools were categorized into four clusters based on community income, location, and performance ratings. This method allowed for fair comparisons and assessment of curriculum effectiveness across diverse contexts. Math Nation schools showed improved test scores and proficiency levels, particularly in Cluster 1, with notable gains in all clusters, especially Cluster 3.
Study 4 aimed to determine which activities are crucial for student success by examining engagement with three curriculum activities: Check Your Understanding (CYU) Questions, Study Expert Videos, and Test Yourself. Using a regression tree to analyze student performance and past scores, the study found that completing more CYU questions was the strongest predictor of success, suggesting that active engagement with practice questions is highly beneficial.
Overall, the evaluation findings indicate that Math Nation is an effective and engaging tool for enhancing students’ math learning and performance. Across multiple subgroups of students, those using Math Nation consistently outperformed their peers. These findings support the potential for Math Nation as an effective tool in helping students, especially those in underserved groups, improve their math performance.
