Tuesday, 9 March 2010

An example of a powerful metric in use

Mortality ratios

Yesterday’s posting about performance metrics in the UK’s NHS system provoked much comment and discussion. The BBC’s Panorama investigated the discrepancies between certain hospitals’ own assessment of their performance, and independent assessments of their performance. In rather too many cases the discrepancies were significant.

One metric that caught my attention was discussed by Professor Brian Jarman of Imperial College, London. The hospital standardized mortality ratio compares the number of deaths expected and the number of in-patients at a hospital who actually die. Professor Jarman’s response to the interviewer’s questions was particularly instructive. When asked about the relevance of the metric he replied that it could indicate:
  1. Data had been incorrectly entered
  2. Something was wrong with the calculation
  3. There is a problem with patient care
And he listed these possibilities in that order. In other words, the metric was an indicator that highlighted the need for further investigation. It was not an indicator that provided cut and dried proof of a problem. He went on to say that in one instance the ratio had highlighted a problem for ten years.

This is how performance metrics are designed to be used:
  • An alert to a potential problem
  • A need to investigate further
  • Trends (upwards, downwards, consistently bad, consistently good) tell you more than any one single metric.
From my own experience I know that the NHS is not the only example of this mis-match between reported performance and actual performance. If there is any justice for tax payers this will run and run.

9 comments:

  1. Nyallpurposepaving.com offers Paving Contractor, Parking Lot Repairs, sealing, striping, Oil and Stone Driveways at Riverhead NY, Quogue NY, Hamptons NY, Southampton NY, Sag Harbor NY, Green point NY, Wading River NY, East Moriches NY, Center Moriches NY, Mannorville NY, Miller Place NY , Mt. Sinai NY,Greenport NY

    ReplyDelete
  2. Diskdatadestruction.net offers hard drive destruction, Paper shredding at Hauppauge NY, Suffolk County NY, Long Island NY, Ronkonkoma NY, Melville NY, Huntington NY, Garden City NY, Farmingdale NY, Westbury NY, Bohemia NY, Stony Brook NY,
    and Nassau County NY

    ReplyDelete
  3. geffnerkersch.com Divorce Attorney Nassau County NY, Prenuptial Agreements Nassau County NY. Call us now: (516) 268-9898

    ReplyDelete
  4. Atopcommercialpaving.com offers Commercial Paving, Asphalt Contractor, and Parking Lot Resurfacing at Queens NY, Nassau County, Bayside NY, Jamaica NY, Middle Village NY, Ozone Park NY, Howard Beach NY, Little Neck NY

    ReplyDelete
  5. Jettonsgrading.com offers Grading, Excavation and Paving Contractor, Land Clearing Services, Septic Systems, Retaining Walls and Driveways, Drainage Systems, Construction Site Clean-Up, Demolition Contractor at Walhalla SC, Seneca SC

    ReplyDelete
  6. reliablecommercialpaving.com offers Parking lot paving, walk way construction, Commercial paving, asphalt, Parking Lot Repairs, Parking Lot Striping, Commercial Blacktop, Parking Lot Repairs, Commercial Paving Contractor, Commercial Asphalt Contractor at New Jersey at Hanover NJ, East Orange NJ, Newark NJ, Morristown NJ, Summit NJ , Union NJ, Hackensack NJ, Patterson Nj

    ReplyDelete
  7. Bedbugprocpc.com offers Bed Bug Exterminator, Bed Bug Inspections, Bed Bug Elimination, Bed Bug Inspection for Hotels and Residential and Commercial Bed Bug at Palo Alto CA, San Mateo CA, San Jose CA, Belmont CA, San francisco CA, Redwood City CA, Sunnyvale CA

    ReplyDelete
  8. Complete-pest-control.com offers Pest Control, Exterminator, Rodent Control, Rodent Elimination, and Pest Elimination at Palo Alto CA, San Mateo CA, San Francisco CA, Redwood City CA, Sunnyvale CA, and Belmont CA

    ReplyDelete
  9. Davispestmanagement.com offers Pest Control, Exterminator, Bed Bug Removal, Inspections, Rodent, Pest Elimination, Commercial, Residential, Professional, Licensed Exterminator , Scorpion, Bee Hive, Hornet, Fire Ant Removal by Service Area, Las Vegas NV

    ReplyDelete