DO WE MEASURE WHAT MATTERS?
This is another discussion piece in my Summer and Autumn of work around local health and social care services.
Politics can sometimes be dominated by political ping pong on the release of a new batch of health or social care statistics.
Does the latest data indicate performance is up or down on the previous month, is the change a “scandal” (a term used far too frequently), or is it an indication of a seasonal/temporary blip in service or does the latest change really reflect a real concern at the overall direction of travel?. The need for, and purpose of data collection and performance measurement is important.
In October 2017 Cabinet Secretary Vaughan Gething issued a Written Statement:
“The NHS and social services across Wales gathers a wide range of data that is essential to provide safe, effective health and care services. This vast collection of data means that it is important that we not only ensure that there are robust systems in place to manage data securely, but also make better use of data to improve decision making, plan change and drive improvements in quality and performance.
Building on the intentions for improvement and innovation outlined in Informed Health and Care, Welsh Government’s digital strategy for the NHS and social services, I am pleased to publish this Statement of Intent, which outlines four key areas for action that will support our aim of making better use of health and care data for safe, effective and efficient services in Wales.
The Statement of Intent is available on the Welsh Government website at:
(my emphasis in bold type above to stress this is a ‘vast’ activity but it must impact on ‘quality and performance’ improvements – otherwise why do it?).
As the Statement of Intent states:
“By making better use of available data we can improve decision making, plan change and drive improvements in quality and performance. Beyond supporting the immediate care of individuals, the sharing and use of data is essential as the basis for creating information and intelligence to help those commissioning and delivering health and care services to learn from what has happened in the past, understand what is happening today, and to plan for the future”.
On 25th June 2018 the Welsh Government published a “Welsh health and care statistics mapping tool” – you can find copy here. It is an attempt to open up the availability of data by different geographies and subjects. This is no bad thing given the ‘vast’ resource that is used in assembling this data.
Yet as a member of the Assembly’s Health, Social Care and Sports Committee I also ask myself whether we actually spend sufficient time measuring what matters?
To be clear this question probably applies across a range of activity in the public and private sector not just health and care, and I recognise a journey is already underway.
The data we gather, and how it is gathered, is critical.
An insight to sources of data in the Welsh NHS can be found here.
It can be clearly evidenced how this data relates to key “targets” e.g. waiting times for specific treatments or to frame a view on the performance of elements of the service e.g. ambulance response times. Data collection and management is a whole industry in itself so in short this piece I just offer some thoughts for discussion while asking the question – do we actually measure what matters?
For me this thought process stems back to an earlier debate in which I was involved around ambulance response times. At that point, some five years ago, I was a trade union official and we could see that the, then, measure of “success” (then the 8 minute response time target) was a blunt instrument that could act as a disincentive, and was in fact having a demoralising impact on hard working front line staff. That was what I called a perverse effect that made little sense and was hindering important discussions about developing a modernised and more responsive service. That specific debate is now well rehearsed and was part of the move to change how we measure success.
Of course this change does not remove the challenges facing the service, and the nature of the critical decision now to be made about categorising incoming calls. However in refining the performance measurement, we have helpfully moved beyond a rather blunt measure to something more meaningful.
I have seen somewhat parallel experiences in A+E which is subject to 4 hour waiting time target. On some of my visits to A+E it is clear the unit has been very busy yet, in reality, with some people who did not need to be in A+E. Once again I felt it was a little perverse to include in an A+E performance measure those people who did not really need the service of A+E. Surely the best measure of the performance of this service would be how it deals with those people truly in need of accident or emergency treatment? The other people reflect an imbalance in the service, not the performance of true A+E activities.
As a result of the new social services legislation there has been a widespread review of the “Performance management framework for local authorities” with a purpose that includes to “enable people to understand the quality of social services”.
“Well-being is to be measured nationally by the national outcomes framework for people who need care and support and carers who need support. This will contain national outcome indicators of well-being that all services and people themselves will contribute to. These national outcome indicators will be reported on by the Welsh Government and will provide greater transparency on whether care and support services are improving national well-being outcomes for people in Wales”.
“In relation to social services, qualitative data will detail a person’s experience, including their satisfaction with care and support. The measures developed include asking people about their experience of social services and whether this has contributed to improving their well-being”.
Inputs, Outputs and Outcomes
We gather data to help inform our decisions and assess progress/change over time.
It can take many forms but in my experience the political debate has traditionally tended to primarily focus on ‘inputs’ and ‘outputs’, and only more recently to the critical issue of outcomes. The inputs and outputs have probably been a focus as they are critical indicators of how money is spent. In a financially driven system how many nurses, carers, police, teachers etc are employed has been seen as a key indicator, and a determinant of output (number of operations performed, crimes tackled etc.)
But that is less useful data in assessing the outcome from all this activity. Critically for me, if we want to build our health and care system around the principles of being patient/client centred then the ‘outcome’ they experience – their well-being is surely THE most critical thing. It is possibly also the most difficult thing to manage when the focus is squeezing activity out of restrained budgets.
Thinking aloud – tell me more
This piece is really some thinking aloud about how we measure success in health and care? It recognises that we are on an important journey that is bringing outcomes and well-being to the central of our measures.
It has half an eye to recent changes in legislation and performance frameworks but also reports like “Futures for Wales” which set out the scale and nature of the challenges ahead. I reflects a fear that lots of incremental data measures distract us from the big changes and improved outcomes we must achieve.
In generating data to satisfy often short term financial measures are we missing a bigger picture?
I’d be interested to hear more thoughts about my theme for this piece – Do we measure enough of what matters?