Manhattan Institute: The Limits of Comparative Effectiveness Research

President Obama has touted "comparative effectivness research" (CER) as a way to save money under his health care plan. In his words:

If there's broad agreement [that] the blue pill works better than the red pill, ...and it turns out the blue pills are half as expensive as the red pill, then we want to make sure that doctors and patients have that information available to them.

Although this sounds good at first glance, there are serious problems using this approach in a one-size-fits-all government health system.

Philipson and Sun of the Manhattan Institute discuss this topic in depth in their latest article, "Blue Pill or Red Pill: The Limits of Comparative Effectiveness Research".

In particular, they note that this approach can cause tremendous problems when policy-makers then treat patients as some sort of global "average" while neglecting their individual differences:

..[I]t is the drug or treatment with the larger average effect on an entire population that "wins." In the president's hypothetical, the blue pills are "just as effective" as the red ones because, on average, they do as much good for patients. But the average patient is not the same as any particular individual patient. Declaring a treatment most effective based on an average is a medical and an economic error, for two reasons.

First, individuals differ from one another and from population averages. Therefore, what may be on average a "winning" therapy may simply not work for a large number of patients. Conversely, a drug that is less effective on average may still be the best, or only, choice for a sizable proportion of patients.

The second reason is the variance in dependence in patient responses across therapies. Dependence, for any individual patient, is the degree to which response to one treatment predicts response to another. Dependence varies from illness to illness and from drug to drug but is often an important aspect of finding treatments that work. One cannot know in advance, as a general rule, that Drug A's failure guarantees the failure of Drug B. Yet a reimbursement policy based on CER could well make this error: by refusing to reimburse Drug B on the grounds that Drug A is "more effective," such a policy assumes that failure with Drug A will predict failure with Drug B.

The bulk of their paper discusses how this CER methodology would fail if applied to real-world antipsychotic drugs, where there are seeming equivalents of Obama's "red pill" and "blue pill".

Because of the factors mentioned above, a policy based on CER that restricted doctors' ability to prescribe (and patients' ability to receive) drugs deemed by the government as insufficiently "cost effective" would actually cost more money in the long run, as well as inflict tremendous and unnecessary hardship on patients.

Of course, CER data can be valuable to practicing physicians who treat patients as individuals. But it can be a dangerous tool when fallaciously misused by policy-makers who treat patients as a collective or an "average".

(Read the full text of "Blue Pill or Red Pill: The Limits of Comparative Effectiveness Research" or download the PDF version.)