He discusses some of the negative unintended consequences of "report cards" that supposedly rate a surgeon's quality based on patient mortality and complication outcome data. In essence, they discouraged surgeons from taking on the tougher cases and instead created a perverse incentive for them to "cherry pick" only the healthiest patients.
He cited experience from New York state, which has used such report cards since the 1990s:
They often penalized surgeons, like the senior surgeon at my hospital, who were aggressive about treating very sick patients and thus incurred higher mortality rates. When the statistics were publicized, some talented surgeons with higher-than-expected mortality statistics lost their operating privileges, while others, whose risk aversion had earned them lower-than-predicted rates, used the report cards to promote their services in advertisements.And:
This was an insult that the senior surgeon at my hospital could no longer countenance. “The so-called best surgeons are only doing the most straightforward cases,” he said disdainfully. Research since then has largely supported his claim...
It's not that quality metrics are completely useless. But they can be dangerously misleading. And they can create perverse incentives that harm both patients and physicians. (Via H.R.)Surgical report cards are a classic example of how a well-meaning program in medicine can have unintended consequences. Of course, formulas have been developed to try to adjust for the difficulty of surgical cases and level the playing field. For example, a patient undergoing coronary bypass surgery who has no other significant diseases has an average mortality risk of about 1 percent. If the patient also has severe kidney dysfunction and emphysema, the risk of death increases to 10 percent or more. However, many surgeons believe that such formulas still underestimate surgical risk and do not properly account for intangible factors, such as patient frailty.The best surgeons tend to operate at teaching hospitals, where the patients are the most challenging, but you wouldn’t know it from mortality statistics. It’s like high school students’ being penalized for taking Advanced Placement courses. College admissions officers are supposed to adjust grade point averages for difficulty of coursework, but as with surgical report cards, the formulas are far from perfect.The problem is compounded by the small number of operations — no more than 100 per year — that a typical cardiac surgeon performs. Basic statistics tell us that the “true” mortality rate of a surgeon is not what you measure after a small number of operations. The smaller the sample, the greater the deviation from the true average.
