Math and Science Partnership Scoring High Grades

by David Wheeler
Print-Friendly Version      Download Article as PDF

It’s 10:40 a.m. at Lewis County High School in northeastern Kentucky, and that means it’s time for Shawna Voiers’ Chemistry II class to begin. “We’ve been balancing redox reactions for a few days now,” says Voiers, an energetic teacher with curly blond hair and glasses. “Today we’re going to perform what we call a redox titration,” she adds, tossing the scientific phrase off as if it’s commonplace. After Voiers explains the assignment, the students put on their goggles, find a partner and head to their chemistry stations. At first glance, it looks like an activity from a typical high school chemistry book. But what’s different about this experiment is the follow-up assignment, which Voiers calls “Making Meaning.” The students conduct a college-level analysis of the experiment—what went right, what went wrong and why it matters. You won’t find this assignment in a textbook because it was developed by Voiers herself at a professional development session.

As a testament to the effectiveness of this advanced curriculum, Morehead State University offers college chemistry credit to students who make a high enough score on the final exam in Voiers’ Chemistry II class. Voiers attributes the higher-level classroom experience to resources and innovations made possible by the Appalachian Math and Science Partnership (AMSP). [To find out how this project began, go to www.research.uky.edu/odyssey/spring03/primenumber.html.]

A $24 million program funded by the National Science Foundation, AMSP was developed to address the stubborn problem of academic underperformance in central Appalachia, where students consistently score below state averages in math and science. With UK as the lead institution, AMSP created a partnership among 56 central Appalachian school districts, 10 higher education institutions, and the Kentucky Science and Technology Corporation. The program, launched in 2003, emphasizes professional development as one of its main components.

More than 400 schools from four states participate in AMSP. Ask a science or math teacher at any one of these schools for an assessment of the program, and you’ll hear lengthy success stories, peppered with examples of eureka moments for students in the classroom. You’ll also get positive reviews from Inverness Research Associates, an independent educational research organization hired by the National Science Foundation to evaluate AMSP. But those reviews deal with qualitative data—primarily verbal evidence. Meanwhile, quantitative data on AMSP has been more difficult to find, because the hard-and-fast numbers that show an improvement in student test scores have been elusive. That is, until now.

Numbers Tell the Story

You won’t see any test tubes in Genia Toma’s office in Patterson Office Tower, but there’s a lot of science going on in there—social science, specifically. “I’ve been studying the economics of education my entire career,” says Toma, a Kentucky native and professor in UK’s Martin School of Public Policy and Administration, “and this is a project where I’m bringing those tools to Kentucky schools—and that’s fun.”

Toma explains that it has become clear to academics and policymakers that certain approaches to solving the problem of underperformance in schools—such as across-the-board increases in school expenditures and teacher salaries—simply weren’t working. “We spent a lot of time thinking that we should just change expenditure levels, but this approach hasn’t panned out,” says Toma, who holds a Ph.D. in economics from Virginia Tech. “There is a lot of quantitative evidence, however, showing that teacher quality affects student achievement.”

Thus, two of AMSP’s components address teacher training: pre-service training (before teachers enter the classroom) and professional development (for teachers already in the classroom). But it will be a long time before the effects of such training are known; after all, it might take decades before the teachers undergoing AMSP’s pre-service training constitute a majority of the teaching workforce in Appalachia. In the meantime, Toma’s pilot project looked at the effectiveness of AMSP-sponsored professional development.

How can you tell if professional development works? For the layman, it might seem that the answer could be found simply by looking at school test scores before and after those teachers participated in the AMSP-sponsored training. If scores went up, the professional development succeeded—right?

Well, test scores certainly have improved, but because so many human variables are involved it’s difficult to be sure what actually made the difference. Therefore, Toma used a technique inspired by economics in order to control for variables that tend to increase or decrease student performance. Her preliminary research showed that, when other factors were accounted for, Kentucky schools with teachers participating in AMSP programs exhibited statistically significant improvements in math and science scores.

“The proof is in the stats,” Toma says, burrowing into one of the half dozen or so stacks of paper in her office and pulling up a chart. “We found improvement for seventh-grade science scores, eighth-grade math scores, and 11th-grade math and science scores.” Among the more specific findings: “We found that after controlling for other factors, if the percentage of teachers participating in AMSP professional development within a school is increased by 1 percent, math scores of 11th-graders rise by .29 points.”

With such promising preliminary research, Toma received an additional $1.5 million NSF grant to further substantiate these gains. The new study will conclude in September 2011.

The “Nanotechnology” of Educational Research

Toma’s study did indeed find a correlation between professional development and higher math and science scores. One question she hopes to answer by continuing to research the effects of AMSP is, which professional development activities work best? The answer requires more specific data, Toma says adjusting her black-framed glasses. The pilot project used school-level data, showing information such as the percentage of teachers participating in AMSP professional development and the average test scores for certain grade levels. The new study will provide data on which professional development workshops specific teachers attended, which students were in which classes, and how those individual student test scores changed after the student took a class from that teacher. To protect privacy, teacher and student data is coded, and names are removed.

 “With the new study, we will know specifically which types of the AMSP activities were more effective than others. We’re actually going to be able to delineate those by going down to a very micro level.” She pauses, smiling. “It’s kind of like nanotechnology. We’re going down to the smallest unit possible.
“We’ll be able to match students enrolled in a class to the teacher who taught the class. So if there is an effect of AMSP on the participating teacher, we should see it directly from those students in the classrooms of the teachers who participated in AMSP.”

While the techniques Toma uses are known in economics, they are much less used in education research. Toma explains that in this work she treats AMSP as an intervention, “kind of like a medical treatment,” she adds. “We realized that the intervention was not done randomly. You don’t put heart stents in both healthy people and unhealthy people. So one of the things we have hypothesized in this study is that there is a selection bias, or non-randomness, that describes which teachers choose to participate in these professional development programs.” Some teachers might feel they need the extra help, others might be self-motivated to try these specific sessions, and still others might be encouraged to attend by the principals at their schools.

Toma also focused on another variable: changes at the schools from year to year. “Test scores may have changed because the socioeconomic composition of the classrooms changed,” she explains. “So we control for those sorts of things. And now that we’re going down to the student level, we’ll actually be able to identify these students’ particular characteristics, and we’ll know certain characteristics of the teacher.” For example, as you would hope, most teachers with five years of experience in the classroom are more effective than teachers who are brand new. “So we’ll be able to control for those other things that may influence the effectiveness of a specific teacher.”

“A True Partnership”

The AMSP project participants and coordinators were thrilled with Toma’s innovative method and preliminary results. Several are hoping her method will be used to study the effectiveness of AMSP in other states and nationwide.

When AMSP began six years ago, the UK team included Wimberly Royster, Stephen Henderson, Ron Atwood, Paul Eakin, and Carl Lee. Royster, the original AMSP project director, says he is pleased with the rising test scores, but points to another significant result of the program. “We have developed relationships and a better understanding of what partnering is,” says Royster, referring to the collaboration among teachers, professional development leaders and AMSP project leaders.

John Yopp, the current AMSP project director, echoes Royster’s sentiments. “What satisfies me the most about AMSP is that it is a true partnership; it’s not a top-down model,” says Yopp, who is also UK’s associate provost for educational partnerships and international affairs. “The testimonies that we hear are just glowing. It is the antithesis of the stereotypical image of Appalachia. Good things are happening in Appalachian classrooms.”

Anyone unconvinced of the teachers’ enthusiasm need only talk to Voiers at Lewis County High School. At the first mention of the initials AMSP, she lights up.

“Oh, my gosh,” Voiers says. “AMSP has been absolutely wonderful. They have given us so much training and help with technology. For example, in our redox probe reactions experiments, where the students analyze a solution, the AMSP folks provided us with a logger pro. This computer-like device allows an interface with a laptop where data from the experiment can be collected and analyzed. The big picture is, you’re able to improve the curriculum. It’s a huge step forward—for teachers and students.”

AMSP student

Lewis County High School senior Cory Collins and foreign-exchange student Sarah Straub mix two solutions together to determine the endpoint of a redox reaction—oxidation, in which electrons are lost or transferred, and reduction, in which electrons are gained. An everyday example of a redox reaction is rust.

Enlarge Photo


 

Genia Toma

Genia Toma, a professor in UK’s Martin School of Public Policy and Administration, has focused her career on studying the economics of education. “This is a project where I’m bringing those tools to Kentucky schools,” she says.

Enlarge Photo