Students of UTeach Graduates Learn More

by Michael Marder, Executive Director, UTeach Austin

[Originally posted on the UTeach Blog on January 9, 2017.]

During the 20 years UTeach has prepared teachers, many have asked for evidence that high school students learn more if their teachers come from UTeach. Two recent studies from Texas show that students of UTeach graduates do learn more.

There are many reasons why preparing math and science teachers through UTeach makes sense. UTeach graduates get a teaching certificate along with their degree and do not have to pay for extra coursework or years in school. UTeach raises the number of math and science teachers coming from the universities where it has been established [see graph]. UTeach graduates stay in teaching longer than those prepared by alternative certification programs [see graph]. But for many thoughtful observers, the most important question has been whether students of UTeach graduates learn more. Now we can answer the question. The answer is yes.

The first study, from UTeach, examines the differences in student outcomes of university prepared and alternatively certified teachers in Texas (pdf), using scores in Algebra I and Biology from the 2011–2012 academic year compared with math and science scores in 2010–2011. We looked at the general question of whether graduates of standard university or alternative certification programs produce more student learning because rapid growth of for-profit teacher certification may be the most significant change taking place now in the way teachers are prepared in the United States.

Students of UTeach-prepared teachers had an advantage of 9 months of schooling in both Algebra I and Biology for Gifted students, and 5 months of schooling in Biology for Economically Disadvantaged and Hispanic students.

We concluded that students of university-prepared teachers in Texas gain around one more month of schooling than comparable alternatively certified teachers in ninth-grade Algebra I and 0.7 months in ninth-grade Biology. Focusing specifically on UTeach-prepared teachers, we found significant advantages of around 9 months of schooling in both Algebra I and Biology for Gifted students, and 5 months of schooling in Biology for Economically Disadvantaged and Hispanic students. There were no subgroups of students for whom alternatively certified teachers obtained significantly better student learning.

The second research group (Ben Backes, Dan Goldhaber, Whitney Cade, Melissa Dodson, and Kate Sullivan of the American Institutes for Research) looked exclusively at UTeach. They made use of all the math and science tests given in public secondary schools in Texas from 2011–2012 to 2014–2015. They found that, relative to non-UTeach teachers in the state, graduates of both the UTeach founding program at UT Austin and six other UTeach programs in Texas universities are more effective than comparison teachers as measured by their ability to raise student test scores in math and science. The difference was 2–3 months of schooling for UTeach graduates overall in Texas and 4–6 months of schooling for Austin UTeach graduates, similar to the difference between novice teachers and those with 10 or more years of experience.

Graduates of Texas UTeach programs are more effective compared to other teachers in the state, as measured by their ability to raise student test scores in math and science.

These research results became possible because of recent improvements in how the Texas Education Agency collects and distributes student data. When a research group (Mellor, Lummus Robinson, Brinson, and Dougherty) first attempted to measure Texas student learning gains due to educator preparation program in the mid 2000s, they had to contact 400 school districts one at a time, and their main conclusion was that “limitations of most state data and assessment systems, including the one in Texas where our study was conducted, make this kind of research difficult.” Six years later, when statewide data were finally available in a central location, a team led by Paul von Hippel tried again and concluded, “In Texas we find that TPP [Teacher Preparation Program] estimates consist mostly of noise.” The passage of a few more years means that more years of data are available, and errors in the datasets have been removed as multiple research groups use them. It also helps that effects for secondary math and science in general and for UTeach in particular seem to be larger than for the elementary teachers that dominated previous studies.

The very first research group that tried a decade ago had not been wrong. Research of this sort is difficult. The math and science tests that secondary students take were not designed for the purpose of evaluating teachers or their preparation programs. What the tests measure is a slow and steady accumulation over time of basic skills in mathematics, science, and reading. Students advance by limited amounts from year to year no matter who their teachers may be; the tests probe material mastered over many years. The contribution of any given teacher in one year only makes a difference in classroom average exam scores at around the 10% level. So, while saying that students gain a month of schooling may sound like a large effect, this corresponds to one chance in five of getting one additional question right on an exam with 50 questions. More on this in a later post.

Papers referenced:

Michael Marder and Caitlin Hamrock, Math and Science Outcomes for Students of Teachers from Standard and Alternative Pathways in Texas

Ben Backes, Dan Goldhaber, Whitney Cade, Melissa Dodson, and Kate Sullivan, American Institutes for Research, Can UTeach? Assessing the Relative Effectiveness of STEM Teachers

Teasing conclusions out of small effects from exams not designed for the purpose requires lots of data, patience to remove errors from the data, and complicated methods of analysis. That is why it is so important that UTeach and AIR carried out the analyses independent of each other, that AIR conducted this study without any funding from or communication with UTeach, and that the analyses, done in somewhat different ways, are consistent with each other on all points where they overlap.

Student test scores provide only one way to evaluate teacher preparation programs. There is such a shortage of math and science teachers that preparing more of them is of value in itself. But finding that UTeach graduates increase student learning is great news. This is a new, important reason for UTeach to spread further across the nation.