Dr. Sandeep Jauhar has a nice piece in the 7/22/2015
New York Times, "
Giving Doctors Grades".
He discusses some of the negative unintended consequences of "report cards" that supposedly rate a surgeon's quality based on patient mortality and complication outcome data. In essence, they discouraged surgeons from taking on the tougher cases and instead created a perverse incentive for them to "cherry pick" only the healthiest patients.
He cited experience from New York state, which has used such report cards since the 1990s:
They
often penalized surgeons, like the senior surgeon at my hospital, who
were aggressive about treating very sick patients and thus incurred
higher mortality rates. When the statistics were publicized, some
talented surgeons with higher-than-expected mortality statistics lost
their operating privileges, while others, whose risk aversion had earned
them lower-than-predicted rates, used the report cards to promote their
services in advertisements.
This
was an insult that the senior surgeon at my hospital could no longer
countenance. “The so-called best surgeons are only doing the most
straightforward cases,” he said disdainfully. Research since then has largely supported his claim...
And:
Surgical
report cards are a classic example of how a well-meaning program in
medicine can have unintended consequences. Of course, formulas have been
developed to try to adjust for the difficulty of surgical cases and
level the playing field. For example, a patient undergoing coronary
bypass surgery who has no other significant diseases has an average
mortality risk of about 1 percent. If the patient also has severe kidney
dysfunction and
emphysema,
the risk of death increases to 10 percent or more. However, many
surgeons believe that such formulas still underestimate surgical risk
and do not properly account for intangible factors, such as patient
frailty.
The
best surgeons tend to operate at teaching hospitals, where the patients
are the most challenging, but you wouldn’t know it from mortality
statistics. It’s like high school students’ being penalized for taking
Advanced Placement courses. College admissions officers are supposed to
adjust grade point averages for difficulty of coursework, but as with
surgical report cards, the formulas are far from perfect.
The
problem is compounded by the small number of operations — no more than
100 per year — that a typical cardiac surgeon performs. Basic statistics
tell us that the “true” mortality rate of a surgeon is not what you
measure after a small number of operations. The smaller the sample, the
greater the deviation from the true average.
It's not that quality metrics are completely useless. But they can be dangerously misleading. And they can create perverse incentives that harm both patients and physicians. (Via H.R.)