What PISA Can't Teach Us

February 10, 2016
Education Week

By Martin Carnoy, Emma Garcia, & Tatiana Khavenson

The results of the Program for International Student Assessment, or PISA, the test that is used to do an international ranking of students' academic performance, will be trumpeted later this year, and the responses will undoubtedly repeat those of previous years: U.S. officials will wring their hands and lament that American student achievement is stagnant, that it is lagging woefully behind our economic competitors', and that we therefore need to import features of schooling from higher-scoring countries.

This edu-masochism—a distinctly American way of focusing on our educational shortcomings—can be traced at least back to the late 1950s and the Soviet Union's launch of Sputnik. Perhaps the nation's early trailblazing successes in establishing mass schooling and developing a uniquely excellent system of higher education have left U.S. educators particularly vulnerable to charges that other nations are surpassing us.

These fears unfortunately have led education policymakers astray. They focus on what other nations are achieving, and they fall prey to those urging us to copy Finland's, Singapore's, or South Korea's schooling practices.

But why compare national student performance in the United States with average scores in other countries, when U.S. students attend schools in 51 separate education systems responsible to states and the District of Columbia, not the federal government? The U.S. education system is a construct that does not exist operationally. What's more, a well-respected test, the National Assessment of Educational Progress (NAEP) provides a state-by-state picture of our schools that is much more relevant than either PISA or other major international tests, such as the Trends in International Mathematics and Science Study (TIMSS).

...

Click here to read full article originally posted by Education Week.

Mentioned Publications

Martin Carnoy

,

Emma García

,

Tatiana Khavenson

Since its inception in 2000, the Program for International Student Assessment (PISA)—an international test of reading, math, and science—has shown that American 15-year-olds perform more poorly, on average, than 15-year-olds in many other developed countries. This finding is generally consistent with results from another international assessment of 8th graders, the Trends in International Mathematics and Science Study (TIMSS).

International test rankings have come to dominate how politicians and pundits judge the quality of countries’ education systems, including highly heterogeneous systems such as that of the United States. While international tests and international comparisons are not without merit, international test data are notoriously limited in their ability to shed light on why students in any country have higher or lower test scores than in another. Policy prescriptions based on these test results therefore risk being largely descriptive, based on correlational evidence that offers limited and less-than-convincing proof of the factors that actually drive student performance.

Indeed, from such tests, many policymakers and pundits have wrongly concluded that student achievement in the United States lags woefully behind that in many comparable industrialized nations, that this shortcoming threatens the nation’s economic future, and that these test results therefore demand radical school reform that includes importing features of schooling in higher-scoring countries.

This report challenges these conclusions. It focuses on the relevance of comparing U.S. national student performance with average scores in other countries when U.S. students attend schools in 51 separate education systems run not by the federal government, but by states (plus the District of Columbia). To compare achievement in states with each other and with other countries, we use newly available data for student mathematics and reading performance in U.S. states from the 2011 TIMSS and 2012 PISA, as well as several years of data from the National Assessment of Educational Progress (NAEP).4 In particular, we use information on mathematics and reading performance of 15-year-olds from the PISA data, information on mathematics performance in 8th grade from the TIMSS data, and information on mathematics and reading performance of students in 4th and 8th grade from the NAEP data.

You are here