One Year of Transforming Rehabilitation Payment by Results

One Year of Transforming Rehabilitation Payment by Results

Last week, the Government announced the early termination of the CRCs’ contracts. Various bodies have criticised the new probation arrangements and some private companies have made substantial losses because the number of sentences they were asked to manage was lower than anticipated. We expect, therefore, the CRC’s reoffending rates to be poor given this context. My previous blogs after 3 and 6 months of results found the binary rate of reoffending to be down, but the frequency of reoffending to be up (both compared to 2011). This blog reviews what happened after 12 months of payment of by results.

Reoffending rate and frequency

Between Oct 2015 and September 2016, the CRCs collectively supported approximately 108,000 offenders who qualified for payment by results and 45.4% of these reoffended. Of those that reoffending, on average they committed 4.7 offences each. These results compare to an expected reoffending rate of 47.5% (the 2011 baseline rate) and an expected average number of offences of 4.2 (the average recorded 2011). In other words (and as previously concluded), there are fewer offenders but they are committing more offences. Only a handful of CRCs have performed differently to this overall picture.

Figure 1 below describes the reoffending rate recorded in each CRC. The grey circle indicates that CRC’s 2011 baseline reoffending rate.

Figure 1: Adjusted reoffending rate in each CRC (commencements Oct 2015 to Sept 2016)

Chart to show adjusted reoffending rate in each CRC (commencements Oct 2015 to Sept 2016)

Source: MoJ Proven Reoffending Statistics

Seventeen of 21 CRCs recorded a reoffending rate lower than the 2011 baseline. Some of the reductions were large – such as the 7% reduction in reoffending recorded in Cumbria and Lancashire. Four CRCs recorded reoffending rates higher than the 2011 baseline (indicated in red in Figure 1). The worst performing CRC was Warwickshire and West Mercia where the reoffending rate was 3% higher than the 2011 baseline.

Figure 2 below describes the average number of reoffences in each CRC (the grey circle indicates the baseline average).

Figure 2: Average number of reoffences in each CRC (commencements Oct 2015 to Sept 2016)

Chart to show average number of reoffences in each CRC (commencements Oct 2015 to Sept 2016)

 

Source: MoJ Proven Reoffending Statistics

The general pattern in Figure 2 is the opposite of that shown in Figure 1: Nineteen CRCs recorded an average number of re-offences worse than the 2011 baseline (Durham Tees Valley and South Yorkshire substantially so). Just two CRCs beat the baseline average: Merseyside and Northumbria (the same two CRCs were the only ones to beat the baseline after 6 months).

Actual and expected re-offences

Contrasting results were recorded for the two reoffending indicators used for payment by results – the reoffending rate and average number of reoffences. Only two CRCs [1] recorded results better than 2011 for both indicators. How can we therefore assess the overall performance of the CRCs? A good method is to compare the actual total number of reoffences to the expected total number of reoffences. Across all the CRCs, the actual number of reoffences was 221,220 compared to an expected number of 214,618 (if performance was the same as 2011) – a difference of 6,602 and an increase of 3%. This suggests that overall reoffending performance was slightly worse under Transforming Rehabilitation. Figure 3, however, describes the percentage reduction and increase in the number of offences at each CRC and there was a large range in performance.

[1] Merseyside and Northumbria

Chart to show the percentage reduction and increase in the number of offences at each CRC

Nine CRCs recorded fewer re-offences than were expected. Merseyside was the best performer with a 29% reduction in offences. Northumbria (19%), Cheshire & Greater Manchester (12%) and Cumbria & Lancashire (10%) also recorded large reductions. The remaining 12 CRCs recorded more offences than expected. The increase was as high as 41% in South Yorkshire and 28% in Durham Tees Valley.

Conclusion

The macro trend across all CRCs was for fewer re-offenders but those that did were likely to commit more re-offences than previously. This meant that the CRCs’ recorded a small overall increase in the number of reoffences compared to 2011. Given the difficulties with transforming rehabilitation, the overall results suggest that these probably have not resulted in large increases in reoffending. Local CRC performance can influence results, and that might be why we see wide variation in the number of recorded reoffences compared to the expected number, but these effects were probably small compared to the macro effects. My experience suggests that the police forces are the biggest influence on local reoffending rates and in response to reduced resources many have prioritised high harm and priority offenders – this could explain the macro trends. Any CRC – current or under the future arrangements –  should therefore be fully aware of their local police force’s performance and the PCC’s crime plan.

Blog originally guest posted on http://www.russellwebster.com 13th August 2018.

Offender/Offender Manager Relationships - image of paper people holding hands

Positive Relationships in Offender Management

We are delighted to welcome Prof Darrick Jolliffe, Professor of Criminology at the University of Greenwich as a guest blogger. Prof Jolliffe has worked with GtD on our offender management work and here he discusses the role of relationships in that work.

Positive relationships have the power to change us. This is not just the catchy slogan of the International Coaching Federation, but something that resonates with all of us.  The constructive relationships that we have, or have had with our parents, teachers, friends and colleagues not only define our interactions with them, but our interactions with all others, while defining who we see ourselves to be. Translating this ‘common sense fact’ into the management of those who have committed offences has an extensive history, and some would argue is the bedrock on which the probation service was founded. However, like many things that are common sense, such as we only use 10% of our brains, and carrots provide good eyesight, the suggestion that if ‘offenders’ have a positive relationship with those providing their supervision they will have more positive outcomes, the actual relationship might actually be a bit more complicated. 

I was invited by Get the Data Director Jack Catell to speak their event, Transforming Rehabilitation – Learning from the PbR results, about my experience of attempting to capture the evidence that positive relationships between offenders and offender managers results in definable benefits. This was something that I first worked on over a decade ago, having first been commissioned by the Home Office to develop an Offender Management Feedback Questionnaire, which was a series of questions that asked offenders about their engagement with their named offender manager. I was subsequently commissioned to work on revised versions, and also helped develop a mirror version which asked about how offender managers viewed the relationship that offenders had with probation.

Some findings fit with expectation, for example, female offenders consistently had more positive relationships with offender managers than males, and the longer that an offender had spent on probation the more they felt they had developed positive relationships and acquired useful skills.  However, some findings were very far from expected, for example, those who reported greater engagement with probation were not less likely to reoffend, and even more concerning, or at least confusing, those who reported greater acquisition of skills while on probation were significantly more likely to reoffend.

How do we reconcile these findings with what we know about the power of positive relationships?  Well, I don’t think there is an easy answer here.  I suspect that positive relationships with offender managers are important for offenders, but they may not be important enough to have an impact on a blunt measure such as reoffending over and above the other issues that many of these people are facing in their lives.  We put the finding, that those who reported greater acquisition of skills were more likely to reoffend, down to offender’s overly optimistic view of how easy it would be to continue to stay crime free.  When we looked more closely at the scores of the questionnaires, it was the items which referred specifically to acquiring skills which would reduce later reoffending that those who reoffended tended to endorse more strongly – almost as though they were trying to convince themselves of their ability to stay on the straight and narrow.

Reflecting back on this work, I am really proud of what we did achieve, despite not getting the common sense result we all expected.  I think there would be a lot of mileage in revisiting the offender/offender manager relationship as a potential desistence tool, and if I was going to do this again I would be looking to measure changes in relationships (so administering items more than once) and see how these changes might relate to more short-term outcomes (e.g., offender’s motivation), as well as longer-term outcomes (e.g., reconviction). 

Make Social Impact Your Goal! - GtD and Street Soccer Academy

Make Social Impact Your Goal!

I am delighted to be working again with Street Soccer Academy (SSA) to put evidence of their social impact at the heart of their work with ex-offenders. This important work has been made possible by a grant from the Access Impact Foundation whose mission is to make charities and social enterprises more financially resilient and self-reliant, so that they can sustain or increase their impact. Without the generous funding from the Access Impact Foundation we would not able to provide Street Soccer Academy with our powerful data analysis.

Street Soccer Academy is a great client to work with. They use professionally organised sports-based programmes in the rehabilitation and reintegration of people from some of the nation’s hardest to reach groups, including ex-offenders. Our task is to prove that SSA’s pro-social models are affecting the attitudes and thinking of the men and women with whom they work, with particular emphasis on their relationships and roles in society.

In the coming months we will be using our rigorous social impact analytics to contribute to the knowledge of what makes ex-offenders desist from crime. Our previous evaluation of the academy’s prison to community service, produced evidence of SSA’s excellent engagement of ex-offenders into their programme. With the foundation’s funding we will build on that by using our advanced statistical analysis to identify who benefits from the programme, how and in what circumstances. These analyses will assist the academy to identify the most effective practice and allow it to develop its professional programmes. To ensure that it has the right information at the right time we will be building a dashboard to communicate this data to those delivering the programme, their managers and the funder.

Not only will this improve their practice, but it has important implications for government’s Transforming Rehabilitation agenda. That agenda depends on organisations like SSA being commissioned to deliver services through the private community rehabilitation companies. However, the participation of such organisations has been low as they struggle to demonstrate their impact on reoffending pathways and desistance from crime. Access Impact is helping to overcome those obstacles and by funding our work, will enable SSA to attract further funding and make the systemic changes that are essential to support men and women to desist from crime.

To find out more about our analytics services and how we can help your organisation demonstrate your impact definitively, contact us on 020 3371 8950 or email jack.cattell@getthedata.co.uk

Join our LinkedIn Transforming Rehabilitation Group

Join Our Transforming Rehabilitation LinkedIn Group

We have created a LinkedIn group dedicated to the topic of Transforming Rehabilitation. It is a forum for discussion, for individuals who work or have an interest in, the fields of probation, offender rehabilitation and Transforming Rehabilitation.

Our idea to set up the Transforming Rehabilitation LinkedIn group follows our recent Transforming Rehabilitation: Learning from the PbR results event, which was attended by professionals working across all aspects of the probation service. At the event, attendees were able to network, and share their experiences and learnings, which was highly beneficial. We wanted the discussions and new connections made, to continue beyond the event and the group is the ideal way to help facilitate this. Everyone working in this sector has valuable insights and their contributions to shared learning will ultimately help us all to improve the outcomes for those entering the justice system.

The first discussion topic is The importance of the worker-service user relationship

Establishing a good relationship between the offender managers and their clients/service users is an important part of quality offender management. Putting this into practice might be more difficult. Can you share some of your experiences of good practice in involving offenders in their sentencing planning, and what obstacles were in your way and how they were overcome?

Please do take a look at the conversation and comment if this is a topic on which you’re able to share some knowledge. We will regularly post questions, but also invite members to post your own questions for discussion. You can request to join via this link – Transforming Rehabilitation

Magnifying glass over the word analyse to illustrate analysis of TR PbR Figures

TR PbR Figures – Context and Drivers

This blog post summaries a presentation I gave at our recent event: Transforming Rehabilitation: Learning from the PbR results.

 

Overall results  

In this blog, I will discuss the data I presented at our recent event, ‘Transforming Rehabilitation: Learning from the PbR Results’. The event was held shortly after the publication of the first reoffending figures which showed that reoffending in the first cohort had been reduced by 1.9%. My colleague Jack Cattell has already discussed these figures in an excellent blog, so this blog is about the how external drivers across the criminal justice system might be affecting these results.  

By CRC? – PbR Figures, who is in control of reoffending? 

In order to make conclusions from these results, one must be aware that there are many factors that can influence reoffending. When faced with (improved or reduced) performance results – it’s too easy to fall into the trap of thinking that the reasons for this were outside of your control, or that the results were dominated by one particular driver.  

In understanding the reoffending rates by Community Rehabilitation Companies (CRCs), there are many possible drivers across the CJS, and they can be complex to understand. So, could the CRCs have achieved the difference in performance we have seen under TR? Internal factors such as CRC procedures and workflows, and offender profiles will have an effect.  But so too will a myriad of external influences from other agencies of the CJS, for example police positive outcome rates, court conviction rates and timeliness of police & court procedures. Since these factors are outside a CRC’s control, I wanted to investigate and determine whether any might have had an effecting change in the reported reoffending rates.  

National context 

Chart showing - No. of offences thousands

First off, it is important to bear in mind the national context for the period from baseline to first cohort results, which was a time when the total estimated crime fell and police recorded crime remained constant.

 Chart showing proportional change from 2011

Further, positive outcome rates, specifically charges and cautions had been reduced – in general police arrest rates were down and police had been using other disposals (such as community resolutions) to reduce the number of people – particularly young people – in custody.

 

Chart showing days from offence to completion

And individuals were being processed more quickly during the baseline: on average court processes are now taking 10-15 days longer.

Give this context, the national reoffending rate was reduced by 1.8%. That’s a surprisingly resistant figure, given the changes in context nationally. Large changes in this binary rate will not be seen regardless of policy or systematic change in CRCs.

Chart showing reoffending rate

(Of course, the full picture of the reoffending stats has the rate down, but the frequency up – suggesting a shrinking group of more prolific reoffenders. But that is for another post).

Differences in CRCs?

I will now look at these potential drivers of reoffending rates at the CRC level and determine if contexts really were different or helped to drive different performance. To highlight any of these potential differences, I will look at two CRCs who were at opposite ends of the reoffending rate changes.

What is surprising, is that there are no large, obvious differences between those two CRCs when comparing police outcome rates, court effectiveness and court timeliness measures. For example, the police positive outcome rates were reasonably different at the start of the period, but by the end of 2015 had converged to be broadly similar – the best performing CRC had reduced reoffending while police positive outcomes fell by a significant amount.

Chart showing positive outcome rate

Similarly, the lowest performing CRC had raised rates against the baseline despite court timeliness both increasing, and being higher than other CRCs.

So What?

We’ve seen that reoffending rates are complex measures dependent on many factors, from individual, to regional to national level, and the interaction between them. The rates don’t change very much, even though the contexts can differ wildly.

Even comparing the CRCs at either end of performance spectrum there were not huge differences in the police and court factors. This can be seen in different results for CRCs operating in widely similar contexts.

So, my advice is this: don’t fall into the trap of feeling the results are not within your ability to affect. Understanding the wider picture is helpful to contextualise local results. In the following blogs, my colleagues will be focusing on what is in your control, and my colleagues will be identifying what works and using that to inform good practice.

Get involved in the conversation by joining our LinkedIn group.

TR: PbR results - speakers who presented to Community Rehabilitation Companies and other justice sector employees

Community Rehabilitation Companies: PbR Results Event

Transforming Rehabilitation is the UK government’s programme of outsourcing probation services to new community rehabilitation companies. In a radical move, the government is now paying these new companies by the reduction in reoffending results they achieve. GtD is at the forefront of this by providing our cutting-edge social impact analytics to Sodexo Justice Services who manage a number of these new companies.

The first PbR figures were published last month and GtD has been active in informing the debate on their significance. As part of this debate, we recently hosted a sell-out event for senior management and practitioners working in community rehabilitation companies and the justice sector.

An expert panel comprising Prof. Darrick Jolliffe of Greenwich University (above left), Dr Sam King of Leicester University (above right) and GtD’s own Jay Hughes (above centre left), considered the initial findings and what to do next, with Jack Cattell (above centre right) setting out a new vision of how predictive analyses can be used by practitioners to improve performance.

Prof. Darrick Jolliffe – University of Greenwich

If you were unable to attend but would like to learn more about how GtD could support you in evaluating your social impact outcomes or for a free predictive analytic roadmap for your CRC, contact Jack Cattell  The event presentations can also be viewed via the link below:

Transforming Rehabilitation – Learning from the PbR results presentations

Dr Sam King – University of Leicester

 

Jack Cattell – GtD

 

We’ve also set up a LinkedIn group as a forum for shared learning and discussion, for individuals who work or have an interest in, the fields of probation, offender rehabilitation and Transforming Rehabilitation. Click here to request to join – Transforming Rehabilitation

Scales of Justice representing Transforming Rehabilitation

Transforming Rehabilitation: Payment by Results Figures

Last week saw the release of the Transforming Rehabilitation (TR) Payment by Results figures for the October to December 2015 cohort.

The overall result was encouraging, and defy the view that Transforming Rehabilitation’s radical changes to probation, and the ensuing problems, would result in increased reoffending – though it is very important that I point out that this is just the first set of results of many and overall judgement should be reserved for at least a year. The reoffending rate for all CRCs was 45.6% compared to a 2011 baseline rate of 47.5%. I had to make some (conservative) assumptions to estimate the baseline rate but I think it is also safe to say that the difference was statistically significant, suggesting reoffending rates have reduced under TR. Please see the the note at the end of this blog to understand better how I completed the analysis.

 

Transforming Rehabilitation – CRC Performance

The chart below describes each CRC’s reoffending rate in relation to the baseline 2011 rate. The grey line represents the range of reoffending rates that would indicate no change from 2011 (the baseline confidence interval). If the CRC’s rate is outside this range, we are confident in statistically terms to state that the CRC’s performance was either better or worse than the reoffending rate achieved in 2011. The green bars represent the reoffending rates of CRCs that outperformed 2011, the orange bars represent those that performed the same as 2011 and the red bars present those that performed worse than 2011.

Chart to show CRC Performance

Source: Ministry of Justice Final Proven Reoffending Rates TR (Oct to Dec 2015 cohort).

Thirteen of the CRCs beat the baseline rate. The best performing CRC was Cumbria and Lancashire, which beat the baseline rate by 8.2% (49.9% to 41.7%). The nest best was Hampshire and the Isle of Wight which beat the baseline by 5.4% and the third best was Northumbria with a better rate by 4.3%. Two of the CRCs performed worse than the 2011 baseline. Warwickshire and West Mercia recorded a reoffending rate 3% worse than the baseline rate, and South Yorkshire’s rate was 2.8% worse. With most CRCs, however, outperforming the reoffending rate form 2011, the figures are a promising set of results.

 

Transforming Rehabilitation – Comparing CRC performance

Now that the baseline rates have been published, we can better understand how well each area was performing in 2011 and whether a CRC is now being asked to better good or bad performance achieved in that year. The chart below describes the difference between the actual baseline rate and the 2011 baseline’s OGRS score (in other words their expected rate of reoffending). A negative result in the chart means the area performed better in 2011 than the OGRS score expected.

Chart showing the difference between a CRC's actual baseline rate and the 2011 baseline’s OGRS score

Source: Ministry of Justice Final Proven Reoffending Rates TR (Oct to Dec 2015 cohort).

The charts highlights that six of the CRCs are being asked to beat better than expected performance in 2011 (in other words to be better than good). Whereas other CRCs, notably London and Wales, are being asked to outperform potentially poor performance in 2011. It it interesting that South Yorkshire and, Warwickshire & West Mercia – the two areas that recorded poor performance for TR – are being asked to beat good performance from 2011. Merseyside and Cheshire & Greater Manchester, however, are equally being asked to beat good performance from 2011 and were able to do so for the October to December 2015 cohort. The OGRS score does not allow for area effects, which will exist and could explain the differences between the OGRS score and the baseline rate. It not possible now to conclude whether payment by results will be easier in some areas than others, but, going forward, I will monitor the impact of whether a CRC is being asked to perform better than good or poor performance from 2011 on their ability to achieve payment by results bonuses.

 

Notes on analysis

The latest Ministry of Justice bulletin released more data than was previously available and I was able to complete a statistical analysis of the impact of TR. This could only be completed with making conservative assumptions that would make finding a statistically significant result less likely. The following actions were taken:

  • I assumed the spread of offenders across CRCs in 2011 was exactly the same as it was in the October to December 2015 cohort. This would not be the case but any analysis would want to weight the two samples so they represented each other so the impact of this assumption is minimal.
  • The 2011 sample size was assumed to be the same as that of the October to December 2015 cohort. The 2011 sample will be considerably bigger, so this assumption meant the standard error used for the analysis was larger than it should be.
  • A t-test with unequal variances assumed was used to test the difference between the cohort’s and the baseline’s reoffending rate. The t statistic result was 4.6.

Blog originally guest posted on http://www.russellwebster.com 31st October 2017 with additional commentary from Russell Webster.