Florida Schools Using STEMscopes Science Score 3% Higher on 2018 SSA

Summary

In a sample of schools that use STEMscopes, this study examined the usage of STEMscopes in elementary school as a predictor of achievement on the Florida Statewide Science Assessment (SSA). In Florida, elementary students are assessed on the SSA in the 5th grade year. Utilizing the SSA data, Florida school demographic data, and analytics data from the STEMscopes platform, the research team found a positive association between teachers’ usage of STEMscopes and student proficiency in science at their school.

Study Participants

In the 2017-2018 school year, 133 elementary schools in 14 different counties used STEMscopes for 5th grade. These schools served a total of 12,468 5th grade students and varied widely in size and representation of minority and economically disadvantaged students (see Table 1). On average, these schools had over 600 students, who were about 50% minority, and predominantly economically disadvantaged. In addition, these schools increased their science proficiency by an average of 3 percentage points in the 2018 school year (slightly less than the 4-point average increase in proficiency across the state). In these schools, usage of STEMscopes varied significantly. The purpose of this study was to examine how varying usage of STEMscopes might predict science proficiency. The average STEMscopes school visited a component of STEMscopes online 299 times throughout the school year, which translates to 1-2 visits per day for every day of the school year. However, this varied from 1 visit for the entire school year to 3,513 visits throughout the school year (approximately 20 visits per day).

Table 1

Demographics

n=133 schools	Range	Average
# Students	237 - 1,890 students	663 students
% Minority	5%-99%	48%
% Economically Disadvantaged	15%-100%	75%
2017 Science Proficiency Rate	11% – 98%	50%
2018 Science Proficiency Rate	7% – 98%	53%
STEMscopes Usage	1 – 3,513 visits	299 visits

Study Design

Data for this study came from three sources. First, schools that used STEMscopes for 5th grade in the 2017-2018 school year were identified through the STEMscopes analytics platform. Each school was assigned a usage score based on the number of times STEMscopes online components were accessed or visited by teachers during this school year. These usage data served as a measure of dosage or intensity of implementation at each STEMscopes school.

Second, school performance on the SSA was accessed through the Florida Department of Education website.1 The 2018 school proficiency rate-as measured by the percentage of students who were deemed proficient in science based on their performance on the SSA-was used as the outcome variable. The 2017 school proficiency rate was used as a control variable; therefore, STEMscopes usage was examined as a predictor of gains in student proficiency from 2017 to 2018. Third, demographic information from the schools was obtained from the Florida PK-20 Education Information Portal.2

Analyses and Results

Multiple regression analyses were conducted to examine the association between STEMscopes usage and 2018 science proficiency rates, controlling for the 2017 science proficiency rate, school size, and percentage of students identified as minority or economically disadvantaged. The 2017 passing rate and STEMscopes usage were significant predictors of science proficiency (see Table 2). Schools that had a higher passing rate in 2017 also had a higher passing rate in 2018. In addition, usage of STEMscopes predicted gains in science proficiency from 2017 to 2018.

Table 2

Results from the Multiple Regression Analyses

Predictor	B	Standard Error	p-value
STEMscopes Usage	0.005*	0.002	0.008
2017 Passing Rate	0.616*	0.074	<0.001
# Students	0.001	0.003	0.670
% Minority	-0.039	0.043	0.365
% Economically Disadvantaged	-0.097	0.049	0.052

*Indicates statistical significance with p <.05

To illustrate the finding that STEMscopes usage predicted gains in science proficiency, the 133 schools were separated into low, medium, and high users. Their proficiency rates from 2017 to 2018 are displayed in Table 3. Schools that were considered high STEMscopes users experienced the largest gains in proficiency rates (6%) compared to low and medium users (2% and 1%, respectively).

Table 3

Proficiency Rates of Low, Medium, and High STEMscopes Users

	# Schools	2017 Passing Rate	2018 Passing Rate	Change
Low	44	51%	52%	2 percentage points
Medium	44	52%	53%	1 percentage point
High	45	47%	54%	6 percentage points

Conclusion

The results of this study show that the extent to which these schools used STEMscopes, as measured by how often teachers accessed STEMscopes components online, predicted their gains in science proficiency. In other words, schools that used STEMscopes more frequently were more likely to experience gains in science proficiency. In the 2017-2018 school year, schools that used STEMscopes improved their science proficiency by 3 percentage points on average. However, when taking into account how much schools used STEMscopes, schools that were considered high users of STEMscopes saw an average increase of 6 percentage points in proficiency from 2017 to 2018 (higher than the state average of 4-point increase). These results support previous research showing that inquiry-based science instruction has a cumulative effect on student science achievement. The more exposure students had to inquiry-based science through STEMscopes, the better they performed on the Florida state science assessment.

Florida Elementary Schools with High STEMscopes Science Usage Have Higher Science Achievement on the 2018 SSA