PUTTING STUDENTS TO THE TEST

Matthew Pietrafetta and his team at Academic Approach have been closely monitoring the trends and outcomes related to the transition to all-digital SAT and PSAT for Illinois high school students. A premier tutoring and test preparation organization in the Chicago region, Academic Approach stays abreast of any changes to the SAT and ACT, and this was a big one. It’s the first time a written test was not offered, and students had to adjust.

“There was a burst of social media chatter initially that the test seemed a lot harder than all the tests the College Board had released in its digital app called Bluebook,” Pietrafetta says. “The scores are coming back now; we’re gathering, analyzing, and surveying students to see if there is more variability to scores than we’ve seen since the SAT was last revised in 2016.”

Pietrafetta says there are always concerns when you roll out a new test because the College Board, which creates the test, will produce a new scoring scale, not unlike a curve, that will heavily impact student scores. It takes time to validate and normalize a scale after a new assessment is released, so the students who were the first to take the new test served as a sort of sample group, and their results could be impacted.

Another quirk with the online SAT is that it’s predictive—each item is weighted differently to predict a student’s aptitude.

“The way the ACT works is each question is worth one raw score point, and those points are scaled onto the 36-point scale. The digital SAT utilizes item response theory (IRT) where the questions are not equally weighted the same,” says Pietrafetta.

Large public schools on the North Shore are moving rapidly to adapt to the new testing realities. As the statemandated exams, the SAT and PSAT will be administered to high school freshmen, sophomores, and juniors in April. High school freshman took the digital PSAT earlier this month, which will provide data that can help shape the future of the test moving forward. Pietrafetta and the Academic Approach team are eager to gather student data and feedback. Student feedback, though, is something Pietrafetta measures carefully.

“Like all of us, students are subject to recency bias: you tend to recall and judge an entire experience by what is most recent,” Pietrafetta says. “The last thing you tend to recall from a test is the most challenging item you were wrestling with last, which can color your view of the entire test. That’s natural, like when we love a movie for an hour and 58 minutes but hate the two-minute ending and then, naturally, hate the whole movie. So, ultimately, it’s going to take few months of gathering and analyzing data to tell the whole story of this new digital SAT.”

In addition to this uncertain change to the digital SAT, there is another potential uncertainty brewing. From 2001 to 2015, the state of Illinois used the ACT as its mandatory test for high schoolers, but in 2016 switched its testing requirement from the ACT to the SAT. According to Chalkbeat Chicago, an education news site, the state is strongly considering a switch back to the ACT, which submitted a much lower bid in response to an RFP from the Illinois State Board of Education to offer the test. Yo-yoing back and forth between tests, Pietrafetta says, creates challenges for both educators and students.

“It’s a whiplash moment, potentially going back to the ACT, and while we have done and continue to do lots of work with the ACT, it’s not great for schools that have done so much work to implement the SAT,” he says. “I was talking with a long-standing charter school partner of ours today, and they said that if Illinois does switch back to ACT, they are going to stick with the SAT for another year because they’ve invested so much time and effort adjusting their instruction to SAT standards and norms.”

Analyzing how well students are doing requires a standard set of data from year-to-year, and switching back to the ACT means educators must, in effect, start all over again.

“The beauty in working with so many CPS and Illinois public high schools from 2001 to 2015 was that when we worked with a school or network of schools, we had years of data to look at trends over time and in detail. When you change tests so frequently it’s really hard to rebuild that,” says Pietrafetta. “Analyzing what’s working best in student instruction and creating stable systems of accountability for administrators and teachers become difficult because you’re comparing apples to oranges.”

With colleges returning to standardized testing and schools eager to assess the academic progress of students since learning loss from remote schooling, it’s important to support families and schools in navigating these changes in standardized tests.

The good news is that Academic Approach has been committed to this work since its founding in 2001, which means it has already weathered plenty of testing changes and is well-equipped to jumpstart data collection and analysis, thereby turning uncertainty into certainty for students and educators alike.

For more information about Academic Approach, visit academicapproach.com or call 888-610-4437.