Friday 15 February 2013

A Stab in the Dark Anyone?

With just seven days left to get responses in to the Ministry of Justice in relation to their plans for privatising the Probation Service, a lot of attention is turning to the method by which contractors will be paid. It's called Payment by Results and the government are absolutely beside themselves at the prospect it brings of 'reforming' public services. 

In order to avoid any doubt 'reform' is a code term, a bit like 'handy size' when your favourite choc bar shrinks in size but costs the same, or 'new fares' when the bus company only puts the prices up. There are lots of examples of this mealy-mouthed public relations con-trick sort of language and for 'reform' read more for less. It's supposed to mean 'efficiencies' of course, but even kids aren't impressed by that kind of argument if pocket money stays static or reduces.

I'm increasingly coming to the conclusion that the battle for the future of the Probation Service will hinge upon PbR and whether it has any chance of working. Now, as it happens, in our work with offenders we know that past behaviour is often an important indicator for future behaviour. Actually this is as good a rule of thumb for life generally. If you burnt your hand on a hot oven last time, the same will happen if you touch it next time. 

Extending the analagy, PbR can hardly be said to have been a roaring success in the DWP's Work Programme. In fact according to this Guardian article, "it's a dangerous idiocy that makes staff tell lies." Written by Toby Lowe, visiting fellow at Newcastle University business school, it's utterly damning of PbR as experienced so far in public service:- 


"Payment by results is a simple idea: people and organisations should only get paid for what they deliver. Who could argue with that? If your job is to get people back to work, then find them a job dammit.
Plenty of people working in local government and public services are already starting to realise this is nonsense, and a pernicious, damaging nonsense at that. The evidence is very clear: if you pay (or otherwise manage performance) based on a set of pre-defined results, it creates poorer services for those most in need. It is the vulnerable, the marginalised, the disadvantaged who suffer most from payment by results.
Here's why: payment by results does not reward organisations for supporting people to achieve what they need; it rewards organisations for producing data about targets; it rewards organisations for the fictions their staff are able to invent about what they have achieved; it pays people for porkies.
We know that common things happen when people use payment by results, and other outcomes-based performance management systems. There have been numerous studies that show that such systems distort organisational priorities and make organisations focus on doing the wrong things – and they make people lie.
This lying takes all sorts of different forms. Some of them are subtle forms of deception: teachers who teach to the test or who only enter pupils for exams they know they are going to pass; employment support that helps only those likely to get a job and ignores those most in need; or hospitals that reclassify trolleys as beds, and keep people waiting in ambulances on the hospital doorstep until they know they can be seen within a target time. In the literature, this is known as gaming the system."
There is absolutely no reason to think things will be any different in a privatised Probation Service where contractors will be paid by PbR. In fact it's fascinating to read what the proponents of the wonder solution are saying. Here we have the fourth post in a series on the Russell Webster website by consultants Aylesbury Partnerships. Entitled 'Getting the right information system for payment by results' these selective quotes give a flavour of the shenanigan's involved:-
When your organisation’s revenue is linked to outcomes that are achieved months or even years after delivering its service, understanding the factors that lead to payable outcomes is more important than ever. The more innovative your intervention is, the more uncertain the link between the activities you perform today and the outcomes you achieve.
Monitoring the intermediate steps that lead to your outcomes will improve your estimate of the eventual outcomes. Initially these estimates may be a stab in the dark, but as the first outcomes of your contract are achieved, the data you have collected will improve your forecasts substantially.
We can get excited about using Monte Carlo analysis to understand contract risk on an ongoing basis but often it is the simplest measures, properly illustrated that have the greatest impact. If the information is stored and accessible within a well-designed system, customised reports can be provided to each stakeholder in a visually appealing format.
The most compelling evidence when bidding for contracts or seeking investment shows that your organisation has ample experience in delivering the services in question – the track record. Adding detail and interesting statistics to the track record you have built up will increase the impact of that information, convey the impression that you really understand the intervention and that you can pinpoint what has worked and what has not.
So there we have it. It could all be "a stab in the dark" using a "Monte Carlo Analysis" and based on "detail and interesting statistics (that) convey the impression that you really understand the intervention and that you can pinpoint what has worked and what has not." 
I will leave the last word to Toby Lowe concerning an up-coming conference on PbR:-   
I will be taking part in a public conversation with others who are asking questions about payment by results, and seeking alternatives to outcomes-based performance management at a conference in Manchester on Wednesday 6 March. If you've been forced into gaming the system or just plain telling porkies in order to meet daft results targets, I'd love to hear your stories. If you get it off your chest you help contribute to changing the system. Let's make this change; it's important.
Sign the No 10 petition here. 

   

No comments:

Post a Comment