Introduction

Math is crucial for academic success and future career opportunities. Without mastering high school-level algebra, students face challenges in college admissions and many job training programs. Algebra 1 is essential for students' progress in STEM-related courses, which can increase their chances of attending college and securing better career prospects. Research shows that students who complete Algebra 2 are more likely to graduate from college than those who don't.

However, by middle school, less than 40% of U.S. students are proficient in math, lacking the foundation needed for algebra. Many students struggle with basic algebra problems, emphasizing the need for effective teaching strategies and tools to support those entering Algebra 1.

Purpose of the Study

While previous research highlights the positive impact of Math Nation, we need to understand its effectiveness more comprehensively. This study uses a meta-analysis to evaluate the impact of Math Nation on student test performance during the 2022-2023 school year across multiple states and grades. The evaluation includes schools with varying tenures of using Math Nation, with most being in their first year of adoption.

What is a Meta-Analysis?

In this study, we used a meta-analysis to comprehensively evaluate the impact of Math Nation across multiple grade levels, states, and demographic groups during the 2022-2023 school year. A meta-analysis is a method that combines results from multiple studies to provide a clearer picture of a curriculum’s effectiveness. By combining all of the findings from the individual studies of Math Nation into one comprehensive analysis, we can draw larger conclusions.

Think of it like lifting a heavy object. Is it easier to lift something heavy by yourself, or with the help of a group? Naturally, it’s easier with a group. Similarly, pooling the results of many studies into a meta-analysis gives us a much larger amount of data to work with, making our conclusions more reliable and applicable to a broader audience.

In our study, we combined all of our efficacy studies of Math Nation during the 2022-2023 academic year to do the “heavy lifting” of answering our biggest questions. By analyzing data from the studies comprehensively, we can identify overall trends and determine how well a program works in different contexts. This approach helps us see the bigger picture and make more informed decisions about Math Nation.

Methods

We included 13 evaluation studies of Math Nation from the 2022-2023 school year in this meta-analysis. These studies compared Math Nation users to non-users, ensuring comparable groups through statistical matching. Data on Math Nation usage, student performance, and demographics were collected from various sources.

Results

The meta-analysis showed that Math Nation has a small but significant positive impact on student test performance even in the earliest stages of adoption, z = 2.28, p = 0.02, d = 0.10. The program's effectiveness was consistent across different states and grade levels, p > .05, suggesting that it benefits all students regardless of location or grade.

Conclusion

This study provides a comprehensive look at Math Nation's impact on student math performance. It is important to note that all but one included study was of states that were in their first year of using Math Nation. Even still, Math Nation had a small but significant positive impact on student test performance. The consistent impact across states and grades supports the curriculum's generalizability.

We expect that Math Nation’s impact will become stronger with continued use, as teachers and administrators become more confident in implementing a new program. As more data becomes available, ongoing evaluation will provide deeper insights into how various factors influence the curriculum's effectiveness. Overall, Math Nation is a promising tool for improving student math proficiency among middle and high school students.