STEMscopes Districts Had 4% Higher Rates on the 2018 Oregon Assessment

Overview and Key Findings

The ALI research team examined achievement results on the 2018 Oregon state science assessment in 5th grade for districts that used STEMscopes NGSS compared to districts that did not.

The key findings include:

Districts that used STEMscopes had a proficiency rate 4% higher than districts that did not use STEMscopes, even after controlling for important demographic variables and previous year achievement.
The 4% increase in proficiency rates associated with STEMscopes translates to an additional 371 5th grade students considered proficient in science.

Background

Oregon adopted the NGSS in 2014 and has been phasing these new standards in to their school districts over the past few years. This transition will be complete when a new statewide NGSS assessment becomes operational in 2019. Until then, Oregon students are continuing to be assessed on previous state science standards adopted in 2009. The state has worked to align these two standards to ensure continuity of instruction and assessment throughout the transition. Several districts in Oregon have chosen STEMscopes NGSS as their curriculum for these new standards.

Fifth Grade Results

The state average proficiency rate for all Oregon school districts that reported 5th grade assessment results (N = 174) was 63%.1 Of these districts, 27 (serving 9,297 5th grade students) used STEMscopes NGSS during this school year, and 147 districts (serving 37,245 5th grade students) used either a district-created science curriculum or purchased a different science curriculum. The rates of proficiency for these two groups of districts are found in the table below. The average proficiency rate for the STEMscopes districts was 67%, and the average proficiency rate for the non-STEMscopes districts was 62%. In addition, districts that used STEMscopes improved their proficiency rates by 2 percentage points from 2017 to 2018, while districts that did not use STEMscopes decreased their proficiency rates by 2 points.

Table 1

	2017	2018	Change
STEMscopes Districts (n=27)	65%	67%	+2
Non-STEMscopes Districts (n=147)	64%	62%	-2

Follow-up Analysis on Proficiency Differences

Follow-up analyses were conducted to determine whether these differences remained statistically significant after accounting for other important variables that influence student achievement. Specifically, multiple regression analysis was utilized to recalculate these proficiency rates, taking into account 2017 proficiency rates as well as important district demographic variables, including the size of the district and demographic information of students (i.e., race/ethnicity and socioeconomic status).

Results showed that, even when accounting for these important variables, districts that used STEMscopes continued to have significantly higher overall proficiency rates compared to districts that did not use STEMscopes. Specifically, STEMscopes districts had a weighted proficiency rate of 67%, and non-STEMscopes districts had a weighted proficiency rate of 63%. In other words, using STEMscopes NGSS increased proficiency rates by 4 percentage points.

Conclusion

Districts in Oregon that used STEMscopes had higher 5th grade proficiency rates than districts that did not use STEMscopes, controlling for previous year achievement and several important demographic variables. The 4-point increase in proficiency rates associated with STEMscopes translates to an additional 371 5th grade students considered proficient in science. While the Oregon science assessment is based on previous science standards, these results provide promising evidence for the effectiveness of STEMscopes NGSS.