Tuesday, August 7, 2012

The Unreliability of Cross-Country Comparisons

One of the common criticisms levelled against American health care is that we supposedly spend too much money for such poor results. Typically, critics will also cite international comparison to show that the US fares poorly relative to other countries on a variety of supposedly objective metrics.

However, this recent paper from AEI by Frech, Parente, and Hoff provide some important context: "US health care: A reality check on cross-country comparisons".

In particular, they note that we have to be extremely careful about using standard OECD metrics such as infant mortality, life expectancy, and "premature mortality".

A few excerpts from their article:
For example, babies who are not viable and who die quickly after birth are more likely to be classified as stillbirths in countries outside the United States, especially in Japan, Sweden, Norway, Ireland, the Netherlands, and France. This is especially likely for babies who die before their birth is legally registered...
In the United States, however, nonviable births are often recorded as live births, making the US infant mortality rate appear misleadingly high. In a detailed study of medical records and birth and death certificates in Philadelphia, Gibson and colleagues found that infant mortality had been overstated by 40 percent, merely as a result of these nonviable births that were recorded as live births...
There is another problem with using infant mortality to represent health care efficacy. US physicians often go to great efforts—at the prenatal and postnatal stages—to save a baby with poor survival chances. The additional prenatal care an American doctor provides may improve the odds of the live birth of a baby with poor survival chances, who is then likely to require extensive neonatal care.
Accordingly, the US uses substantially more neonatal intensive care units (NICU) than other industrialized countries. In this case, the additional health care may actually worsen reported infant mortality rates and misleadingly suggest poor care in the United States.
In other words, our willingness to expend greater resources on expensive neonatal care is held against us as a "proof" of our "inefficiency"!

Similarly, life expectancy comparisons can be skewed by differences in lifestyle as well as differences in accidents/trauma.  But even though such differences can be important, they don't necessarily reflect on the quality of a country's medical care system once a patient becomes ill.

The authors also point out this problem with mortality statistics:
Mortality data are an inadequate proxy for health system performance for another reason: they measure years of life, but do not reflect the quality of that life...

The OECD report, however, treats all years of life as the same, regardless of health status... The OECD report sticks with raw LE [Life Expectancy] —rather than using quality-adjusted versions—because of the wider availability of unadjusted LE data, but at the expense of conceptual accuracy.
As a result, the OECD report attributes no value to expenditures that permit people to enjoy a better life by, for example, being able to work or to be functional longer; it correlates expenditures only with mortality. Thus, money spent on knee replacements, for instance, would appear to be inefficient in that it does not decrease mortality, despite the obvious advantages of improved mobility and prevention of falls. Therefore, it is difficult to see mortality alone as an accurate measure of health system efficiency.
The US health care system does have some genuine problems as a result of decades of government interference.  And there is some validity to the argument that we spend money inefficiently -- a problem arising in large part by the government-driven third party payment system which creates perverse incentives to overconsume.  But these problems can be addressed by free-market reforms such as Health Savings Accounts.

And we should be especially careful not to base health policy on flawed statistical comparisons that paint us in a falsely bad light relative to other countries.

(Read the full text of "US health care: A reality check on cross-country comparisons".)