Research Study
The following report includes results comparing STEMscopes and non-STEMscopes districts on the science component of the 2020-2021 State of Alabama Comprehensive Assessment Program (ACAP TM) for 4th grade. Districts were identified as STEMscopes districts if they had a subscription to STEMscopes for the students in the tested grades (i.e., 4th grade) and showed usage of STEMscopes based on analytic data. The state of Alabama just introduced the ACAP test in 2021. It is a computerized criterion-referenced assessment that measures student progress on the Alabama Courses of Study Standards. Proficiency benchmarks identify student science understanding across four levels: minimal understanding of grade-level science standards, partial understanding of grade-level science standards, strong understanding of grade-level science standards, and advanced understanding of grade-level science standards. The percent of students in each of these categories is used to determine the district’s achievement in science. The percentage of students who have a strong and/or advanced understanding of grade-level science is used by the state as the district proficiency rate. The 2020-2021 school year also occurred during the global Covid-19 pandemic. Many Alabama students did not receive a full year of in-person instruction.
The state average passing rate for all Alabama school districts that include 4th grade (N = 138 plus 3 Charter schools with 4th grade science results) was 34%. Of these districts, 23 districts (16.3%) used the STEMscopes science curriculum in 4th grade and 118 districts (83.7%) used either a district-created science curriculum or purchased a different science curriculum. The rates of proficiency for these two groups of districts are found in the table below. The average unweighted passing rate for the STEMscopes districts was 38.4%, and the average passing rate for the non-STEMscopes districts was 29.7% (an 8.7% difference). This was a significant finding. We also evaluated passing rates across all four levels of student science understanding. What we can see is that a lower percentage of students in STEMscopes districts were in levels 1 and 2 (below science standards) compared to non-STEMscopes districts, while a higher percentage of students in STEMscopes districts were in level 3 and particularly level 4 (advanced) compared to students in non-STEMscopes districts.
| Variables | Level 1 (minimal) | Level 2 (partial) | Level 3 (strong) | Level 4 (advanced) | State proficiency (levels 3 and 4) |
| STEMscopes Districts (n = 23) | 15.3% | 46.3% | 26.7% | 11.5% | 38.4% |
| Non-STEMscopes Districts (n = 118) | 18.4% | 51.9% | 23.2% | 6.5% | 29.7% |
In addition, achievement for specific subgroups of students was examined (with the state proficiency benchmark). In the table below, STEMscopes districts had higher passing rates for African American, Latino and students receiving special education services (SPED). Significant differences ranged between 4.6 - 7.6 percentage points.
| Variables | STEMscopes Districts | Non-STEMscopes Districts | b | Standard Error | p-value |
| Economically Disadvantaged Students | 24.8% | 21.1% | 3.73 | 2.58 | 0.15 |
| African American Students | 18.7% | 14.0% | 4.69* | 2.30 | <0.05 |
| Latino Students | 29.3% | 22.6% | 6.73* | 3.33 | <0.05 |
| Limited English Proficiency Students | 15.0% | 12.2% | 2.81 | 3.32 | 0.40 |
| Sped Students | 22.5% | 14.8% | 7.68* | 2.18 | <0.01 |
We conducted a follow-up study to further evaluate differences across STEMscopes and non-STEMscopes districts after accounting for other important variables that influence student achievement. We used multiple regression analysis to recalculate (weight) these proficiency rates taking into account district size, percent of experienced teachers in the district, and district students’ aggregate demographic information (i.e., race/ethnicity percentages, socioeconomic status, district SPED percentage). Please note the percentage of students classified as Limited English Proficient (LEP) was not included because it was highly correlated with the percent of Latino students in a district with a correlation of 0.89 (in other words, in Alabama, almost all students who were classified as LEP were also Latino). Due to the high degree of correlation, these variables cannot be included as covariates in the same analysis. Results are presented in the table below, with the state proficiency level as the dependent (outcome) variable. The results indicate that once other important variables are accounted for, the districts that used STEMscopes still have a significantly higher overall science proficiency rate compared to districts that did not use STEMscopes, with a 5.07% weighted increase in average district proficiency rate for STEMscopes districts compared to non-STEMscopes districts. This equates to approximately 991 more 4th-grade students meeting the proficiency level in these STEMscopes districts.
Outcome: Alabama Comprehensive Assessment Program Science Proficiency Rate
| Predictors of ACAP | B (SE) Adjusted Rates | p-value |
|---|---|---|
| STEMscopes | 5.07* (0.81) | 0.02 |
| Number of Students in District | 0.42 (0.76) | 0.58 |
| Students % Black | -9.13* (1.19) | <0.01 |
| Students % Latino | -3.24* (0.81) | <0.01 |
| Students % Econ. Disadv. | -5.78* (1.00) | <0.01 |
| Students % SPED | -2.71* (0.85) | <0.01 |
| Avg Teacher Exp. in District | -0.37 (1.14) | 0.75 |
Designed to make measurable impacts on student achievement, Accelerate Learning Inc. offers you a customized STEM solution for all your students. Learn how you can preview or demo our products for free.
