Anaesthetists Hymn

It is rare to find something on You-Tube that summarizes your work in 2:14.

The Tyranny of Statistics

I attended an evening retreat of senior management of my hospital last Tuesday. Another 3 hours of my life that I won't get back. At least they served dinner and we were allowed to have a glass of wine before dinner.

The topic of the retreat was quality. Of course we are all for quality. It was only yesterday and today that I realized that the purpose of retreat was to prepare us for the release today of the hospital statistics from the CIHI which compares hospitals according to a number of indices. Traffic on the website has been heavy so I haven't been able to look at my hospital's performance nor have I been able to look at other hospitals. I am assuming by the 5 memos of talking points I just received that we might have some problems but that is just speculation.

About a month ago, I watched Moneyball which I thoroughly enjoyed. Moneyball is of course how statistics can be used to get the same performance using less expensive players. The most interesting fact with Moneyball was how different statistics from what had always been traditionally used were the important statistics, things like on base percentage rather than batting average, pitch count rather than ERA. Hospitals, health systems and human beings are way more complicated than Major League Baseball and I wonder whether anybody has even tried to look at intangibles.

The retreat started out with the concept of a dashboard. This is a chart where 8 or so variables are displayed, the idea being to track changes and motivate staff. For example in an operating room we might track on-time starts, sick days, cancellations etc. The two administrators from our sister operating room had actually brought a copy of their dashboard for our table to view. They were quite proud of it. I was sitting next to our head nurse and asked her whether we had a dashboard. She told me that in fact we had one the past which was posted but she stopped doing it because it made absolutely no difference to performance, the numbers went up and down randomly.

As things went on, the red wine and the buffet meal facilitated the mental effect of the boring presentation that was going on and I started to day-dream. I always ask applying residents whether medicine is an art or a science. (I am not the only person who does obviously because most of them have a pat non-answer.) I am more and more beginning to think of what I do at work is more art than science. Sure there is some science but how we apply it is an art. As we learn more about physiology it is more and more apparent how different rather than similar we all are. Add in surgical insults, time of day etc and we are talking art. Then I drifted off to how could we measure the performance of my colleagues and me. Do we measure on-time starts, turnover (important to admin and surgeons), recovery room times, average post-op pain scores, average nausea scores. Does all of that make a difference. If my patients are discharge ready early, is that any use if they just end up waiting for a porter longer? If my drug costs are lower is that just because my patients awake in pain?

I then drifted off to the band practice I was missing. Music can be very mathematical, there is a lot of science in it, especially in the area of instrument design but no one would argue that it is anything but an art. If you play something too mechanically you will be told to be more "lyrical". I never play a piece the same way twice, not the same tempo (they tell me I rush, but I think everybody else drags), volume or tone; not intentionally that is just how I do it. Multiply that by 50 people and think of the various permutations that come out in the final product. Throw in variations like the acoustics of the room which of course change from the empty hall you rehearse in to the hall full of relatives who were strong-armed into watching you play. Playing outside is a whole different experience. I have played about 7 outdoor concerts as an adult, every single one of them has been entirely different. In someways it is the minor imperfections in music that make it a joy to listen to.

Try putting a number to all of that. At the end of the piece, the concert or the rehearsal, I can think whether I played really good, okay or badly. I could have a dashboard; average variance from tuning, tempo, mistakes etc. but I don't. If we liken a hospital to a huge really complicated orchestra you can see how limiting appraisal of its performance to a snap-shot of 20 or so statistics is going to completely miss the boat.

The other issue is of course that as soon as you introduce statistics into the game, you introduce gaming. Most statistics are subjective so it is possible for a smart hospital to "upcode" its patients and procedures to improve their "batting average". (At the retreat we were advised to sort of do this, not in an illegal of course). Further it opens the door for hospitals to refuse to accept patients that they know will affect their "batting average". In the 1990s our province financially penalized hospitals whose average length of stay was outside the norm. This resulted one hospital refusing to accept palliative care patients because their length of stay was so long. This was by the way the regional palliative care centre.

Soon we will be told to practise medicine in a way that enhances the hospital's stats rather than what is necessarily the best practice. As we computerize more we can mine more data and we can get individual dashboards of all kinds of stats. So we have a bunch of Type A personalities who now have access to stats comparing their "performance" to their peers. Do we become like the teacher who knows what is going to be on the standardized test at the end of the year and teaches to it. Good marks but did the students really learn anything?

And of course statistics lead to cheating such as the case of hospitals who "discharge" and "readmit" patients the same day in order to change a single long stay into two medium stays. Readmissions are tracked which of course gives hospitals a disincentive to discharge patients for fear of worsening their readmission stats. Infection stats are often more a function of a hospital's honesty or the aggressiveness of their ID department than any deviation from quality of care. Sometimes what looks statistically good may not be better for the patients. Length of stay is frequently used stat in Canada. As someone else pointed out this is a stat with a diminishing return if we look at the global picture not just at the hospital.

The more I am exposed to the spurious "science" of medicine, the more I appreciate music.