Summary

In a sample of schools that used STEMscopes, this study examined the use of STEMscopes in middle school as a predictor of achievement on the Florida Statewide Science Assessment (SSA). In Florida, middle school students are assessed on the SSA in the 8th grade year. Utilizing the SSA data, Florida school demographic data, and analytics data from the STEMscopes platform, the research team found a positive association between teachers’ use of STEMscopes and student proficiency in science at their school. 

Study Participants

In the 2018–2019 school year, 95 middle schools in 10 different counties used STEMscopes for 8th grade. These schools served a total of 99,707 students (15,839 of which were 8th grade students) and represented a range of sizes and percentage of minority and economically disadvantaged students (see Table 1). On average, these schools had over 1,000 students, and were about 75 percent minority and 65 percent economically disadvantaged. On average, science proficiency in these schools did not change between the 2018 and the 2019 school year, while the state average decreased by 4 percentage points.

In the 2018–2019 school year, 95 middle schools in 10 different counties used STEMscopes for 8th grade. These schools served a total of 99,707 students (15,839 of which were 8th grade students) and represented a range of sizes and percentage of minority and economically disadvantaged students (see Table 1). On average, these schools had over 1,000 students, and were about 75 percent minority and 65 percent economically disadvantaged. On average, science proficiency in these schools did not change between the 2018 and the 2019 school year, while the state average decreased by 4 percentage points.

Demographics

n=96 Range Average
# Studets  56 - 2,299 students 1,050 students
% Minority 18% - 100% 75%
% Economically Disadvantaged  1-100%  63%
2018 Science Proficiency Rate 0% - 94%  46% 
2019 Science Proficiency Rate 0%-90% 46%
STEMscopes Teacher Usage 12 - 5,455 visits 1,030 visits

Study Design

Data for this study came from three sources. First, schools that used STEMscopes for 8th grade in the 2018–2019 school year were identified through the STEMscopes analytics platform. Each school was assigned a usage score comprising the number of times STEMscopes online components were accessed or visited by teachers during this school year. These usage data served as a measure of dosage or intensity of implementation at each STEMscopes school.

Second, school performance on the SSA was accessed through the Florida Department of Education website. The 2019 school proficiency rate, as measured by the percent of students who were deemed proficient in science based on their performance on the SSA, was used as the outcome variable. The 2018 school proficiency rate was used as a control variable; STEMscopes usage was examined as a predictor of gains in student proficiency from 2018 to 2019. Third, demographic information from the schools was obtained from Florida’s Education Information Portal.

Analyses and Results

Multiple regression analyses were conducted to examine the association between STEMscopes usage and 2019 science proficiency rates, controlling for the 2018 science proficiency rate, percentage of minority students, and percentage of students identified as economically disadvantaged. Results showed that use of STEMscopes predicted gains in science proficiency from 2018 to 2019 (see Table 2).

Table 3

Proficiency Rates of Low, Medium, and High STEMscopes Users

  # Schools 2018 Passing Rate 2019 Passing Rate Change
Low 32 46% 44% -3
Medium 32 44% 41% -3
High 31 49% 54% +5

Conclusion

The results of this study showed that frequency of STEMscopes use, as measured by how often teachers accessed STEMscopes components online, predicted their gains in science proficiency. That is, schools that used STEMscopes more frequently were more likely to experience gains in science proficiency. In the 2018–2019 school year, schools that used STEMscopes did not show a change in science proficiency, on average. However, when taking into account frequency of STEMscopes use, schools that were considered high users of STEMscopes increased an average of 5 percentage points from 2018 to 2019, compared to the state average decrease of 4 percentage points. These results support previous research showing that inquiry-based science instruction has a cumulative effect on student science achievement. The more exposure students had to inquiry-based science through STEMscopes, the better they performed on the Florida state science assessment.