Posts

Influence through Data

“Yeah, Says Who?” – Influence Through Data

You know you’ve achieved results – the data tells you so – but how do you influence sceptics to believe it?

It can be a rude awakening to take the findings of a study outside your own team or organisation, where trust and mutual support are more or less a given. In front of a wider audience of funding providers or other stakeholders, you will inevitably in my experience find yourself being challenged hard.

This is as it should be – scrutiny is a key part of a healthy system – but, at the same time, it’s always a shame to see an impactful project or programme struggle purely because its operators fail to sell it effectively.

Fortunately, while there are no black-and-white rules, there are some things you can do to improve your chances.

Confidence = Influence

When I present findings I do so with a confidence that comes with experience and from really understanding the underlying mechanics. But if you’re not a specialist and don’t have that experience there are things you can do to make yourself feel more confident and thus inspire greater confidence in your audience.

First, make sure you have thought through and recorded a data management policy. Are you clear how often data should be entered? If information is missing, what will you do to fill the gaps? What are your processes for cleaning and regularising data? Is there information you don’t need to record? A professional, formalised approach to keeping timely and accurate data sends all the right signals about your competence and the underlying foundations of your work.

Secondly, use the data as often as possible, and share the analysis with those who enter your data so that they can understand its purpose, and own it. Demonstrating that your data is valued and has dedicated, accountable managers hugely increases its (and your) credibility.

Thirdly, take the initiative in checking the reliability and validity of your own tools. If you use well-being questionnaires, for example, take the time to check whether they are really measuring what you want to measure in most instances. In other words, try to find fault with your own approach before your stakeholders so that when they find a weak point you have an answer ready that not only reassures them but also underlines the objectivity with which you approach your work.

Own Your Data’s Imperfections

Finally, and this might feel counterintuitive, you should identify the weaknesses in your own data and analysis and be honest about them. All data and analysis has limitations and being clear about those, and the compromises made to work around them demonstrates objectivity which, again, reinforces credibility.

In conclusion, the better you understand your own data and analysis, flaws and all, the more comfortable and confident you will feel when it, in turn, comes under scrutiny.

Social impact webinar

Webinar: Make 2020 the Year of Social Impact

On both sides of the Atlantic, our clients are working hard to think of new ways to deliver vital services to their clients: the homeless, the refugee, offenders.  They are often the people who are most likely to be at the margins and their needs are urgent.  So, now is a time for adapting services to the weeks of lock-down and shelter in place orders. And at GtD we are adapting the ways in which we are delivering our services and helping our clients.

While we wait for the pandemic to subside, the needs of our clients and yours remain pressing, and that will continue long after CoVid19 becomes yesterday’s news. However, I doubt that we will go back to work just as we did before we heard about social distancing and self-quarantining.  When we do go back to work, we will be asked to do things in new ways, to answer new needs and to find new partners to work with.

2020 is GtD’s “Year of Social Impact” when we are encouraging non-profits and charities to think about new ways of demonstrating their social value – definitively. We started out with a roadshow of events in Atlanta and then London, meeting with leaders from the nonprofit sector, listening to their needs and telling them about how our “Measure”, “Learn” and “Prove” packages can assist them.

We were due to take the roadshow to Manchester on April 28th.  Instead, we will be delivering the event as a webinar and inviting people to attend from across the UK and the USA.  This is bound to be a fascinating meeting as we share experiences across the Atlantic and hear more about how GtD’s social impact analytics are helping non-profits on both sides of the ocean.

The webinar is free and we have made it easy to join.  To register, please use this link and learn more about how to attend.

At times like this, we need to keep going as best we can and re-evaluate our priorities. Today, I received some words of wisdom from a valued client and jazz aficionado.  He and I were talking about how best to address the needs of those who have no medical care in rural Georgia. We had to suspend some work and address the new priorities thrown up by lockdown, and so I would like to share his words with you:

“What we’ve been doing lately reminds me of an announcement Louis Armstrong made during an interruption in one of his concerts, “And while they’re fixing the mike, we are going to lay on you our very best version of ‘Tin Roof Blues’”. Getting important things done during times like these can be helpful both to our projects and to our morale”.

Jack and I hope to meet you at the webinar but please contact me on alan.mackie@gtd-us.com if you have any questions. In the meantime, we wish you all the best as you address your new priorities and redouble your efforts to serve your clients.

Learn from data

What You Can Learn From Your Data

In a world driven by data, it’s becoming increasingly important for social organisations to prove their impact – not just through what they say, but through cold, hard facts.

While many already collect information on the work they’re doing, there remains a gap.

There’s a limit to what you can understand from the numbers alone, and simply reporting the data can feel far removed from the real-world and the day-to-day work done by the workers in your organisation.

For your information to become truly valuable, you need to start unpicking it. By applying the right methods of data analyses, you should be able to learn from your data and to use the insights you draw out of it to make practical changes within your organisation. This approach to data analysis will allow you to optimise your resources and maximise your results.

More than that, you’ll be able to extract the story behind your successes, and use that to show the world the valuable social impact your organisation is achieving.

Empower your staff

Looking at the data alone, it can be difficult for practitioners to know where to concentrate their efforts, and where changes to their service are most needed. But with the right analysis, the data you already have can be transformed into immensely valuable information to guide practitioners’  decisions, and highlight areas for improvement.

Learning from your data can give your practitioners insights on their individual clients, on how the team is performing, and on the organisation as a whole. It means they’ll know exactly what you need to target, and how to fine-tune your service to achieve the best possible results.

Understand your beneficiaries

No doubt you have worked in your sector long enough to know that your clients are not all the same. By applying the appropriate analytical techniques, you can understand the differences between clients and tailor your services to their varying needs.

Methods like segmentation and cluster analysis allow you to pick out patterns across different groups of people with different characteristics, and identify not only “what works?”, but “what works for whom?”.

Predict future outcomes

Your data can tell you not just what’s happened to date, but the outcomes that your organisation could produce in the future.

This can be done by applying scenario and simulation analysis, to predict the potential results of a variety of situations. These predictions should update dynamically, changing over time as other conditions change.

What’s really valuable about these predictions is that you can use them to optimise your resources, and to expend them in a way that will have maximum effect. That’s important for any organisation to know, but particularly helpful for non-profits and social enterprises whose resources are often limited.

Know your social impact, and prove it

Learning from your data gives you the confidence of knowing – not just hoping – that what you’re doing works. It lets you know that you’re making a real impact in your field, but perhaps more importantly, it offers assurance to the people who are looking for that proof.

Your funders and stakeholders need to have confidence in what your organisation is achieving, and by analysing your data you can give them just that.

To take a  youth service as an example, a potential funder might want to know how much of a difference the organisation is making to youth unemployment in the area. In answer, it could use examples from the analysed data, and effectively demonstrate its improvement in comparison with a control group.

By telling the story of its impact, backed up with concrete evidence and analysis, the youth service would be able to provide a much more convincing argument than it would be, using the raw data alone.

Learn from your data with GtD

Get the Data works with organisations in the UK and the US that want to prove their impact on society. We provide social impact analytics in the form of three main packages: Measure, Learn and Prove.

Our Learn service sits in the middle, making it a perfect next step for organisations that have collected data on their service but need to understand it better in order to use it. “Learn” can also be used in combination with our “Measure” and “Prove”  services, in any order that suits you.

Beginning with a strategic review of your data, we’ll put together an insight strategy and follow up with regular reports to monitor the results – integrating those reports with your own systems so they’re dynamic, useful, and convenient to use.

If you know you could be doing more to learn from your data but you’re not sure where to start, take the first step and contact us today.  We will get you started on your social impact journey.

All Aboard for a Social Impact Journey Get The Data

All Aboard for a Social Impact Journey

Jack, Jay and I were delighted to host our London social impact seminar last week. We are very grateful to all who took the time to join us and contribute their thoughts and experiences.  As with every good seminar the learning was a two-way process. So, we talked about the social impact journeys we have been travelling on with our clients but also learned from our guests’ experiences of making a social impact.

All Aboard for a Social Impact Journey Get The Data

In introducing the seminar I noted that everyone in the room had a common cause: to improve the lives of the most vulnerable people in society. Most of the attendees represented national and London-based organisations that are concerned with mental health, homelessness and young people.  As always, Jack, Jay and I were struck by the commitment and deep knowledge that our guests brought to their work.  So what, asked one attendee, was our interest in working with these groups? A great question and my answer was straightforward: my colleagues and I simply want to provide our analytical expertise to demonstrate the value of our clients’ work.

Talk about data and analyses can often seem abstract and there is a danger that it might reduce vital work with vulnerable people to a calculus of efficiency and effectiveness. To avoid that, we were able to tell stories of how our work has helped other organizations to measure their impact, learn how to improve their services and demonstrate the success of their programmes.  Sometimes our work is used to demonstrate the value of a policy change or justify funding, but often our work simply helps managers and front-line workers think about their practice and keep track of the various services and activities that they deliver.

A valid concern that arose was how to avoid treating a vulnerable person as a unit of analyses rather than an individual with real needs and wants. This concern included how best to engage vulnerable people in the analyses to ensure that they can provide their voice and experiences.  This is a very current topic in the practice of evaluation.  For GtD’s part, we ensure that our work with vulnerable people is undertaken sensitively and in accordance with ethical standards, and that the processes of data collection and analyses do not further marginalize individuals and groups who have been excluded by institutions and society at large.

All Aboard for a Social Impact Journey Get The Data

 

In concluding the seminar, I reflected that the best social impact journey is one that is taken together.  The success of the journey will require trust and mutual respect between the analysts, workers and clients. I believe that the stories we told last week provide good illustrations of how GtD forge working relationships within which to deliver expert analyses and insight.

Join us for a social impact journey seminar

Book Your Winter Getaway – Now!

The holiday season seems to be a time for journeying. As the school nativity plays remind us, the first Christmas saw the little donkey carry Mary and Joseph to Bethlehem and the “star of wonder” led the Three Wise Men to the stable. Today, thousands of British and American travelers will drive and fly home to be with family for the holidays.  And, no sooner is the festive period over, than hundreds of us will be heading to the ski slopes and beaches for a dose of much-needed sunlight. Winter is, indeed, a time for journeying.

At this time GtD’s London-based team invite you to join us on a journey. Unlike the Winter Getaways you see advertised in the press and television, we are offering to take you on a journey to understand your organization’s impact on the communities and people it serves.  At GtD we call this your “social impact journey” and we will travel with you to gather data, unlock its meaning and use it to demonstrate the effectiveness of your work.  Each of these is a stopping point on a single journey.

If you are interested in learning more, then you are warmly invited to join a free seminar on Wednesday, 8th January in London. For more details and how to book your place for this event, please go to http://bit.ly/SocialImpactSeminar.

GtD’s “Winter Getaway” doesn’t promise you golden beaches or pristine snow, but it will give you a good start to the new year as you and your organization embark on your social impact journey.  We look forward to seeing you there.

Get the Data and Gideon’s Promise Team Up for Innovative Metrics Get The Data

Get the Data and Gideon’s Promise Team Up for Innovative Metrics

Over the past year, GtD has been proud to be work with Gideon’s Promise to develop measures of culture change that will transform the American criminal justice system. We are delighted to welcome Ilham Askia, Executive Director of Gideon’s Promise to guest blog about our partnership.

Get the Data and Gideon’s Promise Team Up for Innovative Metrics Get The Data

Ilham N. Askia
Executive Director
Gideon’s Promise

Gideon’s Promise is a U.S.-based, non-profit whose mission is to transform the criminal justice system by building a movement of public defenders who provide equal justice for marginalized communities. Gideon’s values-based approach uniquely trains public defenders to use a “client-first” method to defense practice. The impact that this approach can have on culture change in public defense will ultimately transform an entire criminal justice system. Since January 2017 we have been working with GtD to create an innovative approach to measure the effects of culture change in public defense systems.

While most experts focus on case outcomes and court processing to determine the effectiveness of a public defense system, Get the Data and Gideon’s Promise have designed a way to measure the effects of a values-based approach to training and supporting public defenders. Gideon’s Promise believes that, in order to have equity in the criminal justice system, the lawyers who represent the accused must be excellent in their profession but also care about the dignity of their clients. For over 80% of the people who are charged with a criminal offense, a public defender has the enormous responsibility to not only defend a charge but also illustrate the humanity of a defendant. If administrators of justice could view the accused more humanely, then the treatment of people warehoused in jails, prisons and courtrooms would be different. Because the criminal justice system does not effectively function to rehabilitate offenders, the dignity of the human spirit is stripped away as soon as an individual is accused of an offense. The culture of the system must change from viewing people as case numbers and files and more like human beings with lives that have value. Public defenders tell the stories of those who are deemed unworthy of support. Public defenders remind the system about the value of human lives.

Get the Data has designed metrics to discern whether the culture of an office, the confidence of a public defender, and environment play into how defendants feel about their representation. These metrics also measure whether the attitudes of Gideon’s Promise public defenders affect the public defender offices where they work, the courtrooms where they practice and the relationships they have with their clients. While no one else is measuring the impact of culture change in the criminal justice system, Get the Data and Gideon’s Promise are on the verge of a transformative approach to repairing a broken criminal justice system by using public defenders as anchors to reform and explaining their importance through quantitative data that measure qualitative relationships.

Although Gideon’s Promise is a national organization, it primarily focuses its work in the southeast region of the U.S. where the highest concentration of inequity exists. There are six public defender sites piloting these metrics. The Defender Value Spectrum Survey (DVSS) and the Client Evaluation Survey (CES) are being completed by a sample of Gideon’s Promise influenced public defender offices and clients who received services from those offices. Our goal is to conclude whether a values-based trained attorney positively correlates with how people view public defenders, clients and how both are treated in the court system. Are there sets of core values to public defense training and mentoring that systemically changes the culture? Is Gideon’s Promise’s curriculum aligning with the goal to transform the culture of public defense? Does a caring lawyer matter? These are all questions this study will answer. Our hope is that we can replicate this evaluative process across the country to not only inform Gideon’s Promise’s programming but also encourage public defense systems to adopt our model while providing data that our culture change model works. Caring, competent and committed lawyers are essential to true criminal justice reform.

This partnership is crucial to capturing a values-based approach to criminal justice reform. Gideon’s Promise is truly grateful to be working with Get the Data on this ground-breaking work.

Magnifying glass over the word analyse to illustrate analysis of TR PbR Figures

TR PbR Figures – Context and Drivers

This blog post summaries a presentation I gave at our recent event: Transforming Rehabilitation: Learning from the PbR results.

 

Overall results  

In this blog, I will discuss the data I presented at our recent event, ‘Transforming Rehabilitation: Learning from the PbR Results’. The event was held shortly after the publication of the first reoffending figures which showed that reoffending in the first cohort had been reduced by 1.9%. My colleague Jack Cattell has already discussed these figures in an excellent blog, so this blog is about the how external drivers across the criminal justice system might be affecting these results.  

By CRC? – PbR Figures, who is in control of reoffending? 

In order to make conclusions from these results, one must be aware that there are many factors that can influence reoffending. When faced with (improved or reduced) performance results – it’s too easy to fall into the trap of thinking that the reasons for this were outside of your control, or that the results were dominated by one particular driver.  

In understanding the reoffending rates by Community Rehabilitation Companies (CRCs), there are many possible drivers across the CJS, and they can be complex to understand. So, could the CRCs have achieved the difference in performance we have seen under TR? Internal factors such as CRC procedures and workflows, and offender profiles will have an effect.  But so too will a myriad of external influences from other agencies of the CJS, for example police positive outcome rates, court conviction rates and timeliness of police & court procedures. Since these factors are outside a CRC’s control, I wanted to investigate and determine whether any might have had an effecting change in the reported reoffending rates.  

National context 

TR PbR Figures – Context and Drivers Get The Data

First off, it is important to bear in mind the national context for the period from baseline to first cohort results, which was a time when the total estimated crime fell and police recorded crime remained constant.

 TR PbR Figures – Context and Drivers Get The Data

Further, positive outcome rates, specifically charges and cautions had been reduced – in general police arrest rates were down and police had been using other disposals (such as community resolutions) to reduce the number of people – particularly young people – in custody.

 

TR PbR Figures – Context and Drivers Get The Data

And individuals were being processed more quickly during the baseline: on average court processes are now taking 10-15 days longer.

Give this context, the national reoffending rate was reduced by 1.8%. That’s a surprisingly resistant figure, given the changes in context nationally. Large changes in this binary rate will not be seen regardless of policy or systematic change in CRCs.

TR PbR Figures – Context and Drivers Get The Data

(Of course, the full picture of the reoffending stats has the rate down, but the frequency up – suggesting a shrinking group of more prolific reoffenders. But that is for another post).

Differences in CRCs?

I will now look at these potential drivers of reoffending rates at the CRC level and determine if contexts really were different or helped to drive different performance. To highlight any of these potential differences, I will look at two CRCs who were at opposite ends of the reoffending rate changes.

What is surprising, is that there are no large, obvious differences between those two CRCs when comparing police outcome rates, court effectiveness and court timeliness measures. For example, the police positive outcome rates were reasonably different at the start of the period, but by the end of 2015 had converged to be broadly similar – the best performing CRC had reduced reoffending while police positive outcomes fell by a significant amount.

TR PbR Figures – Context and Drivers Get The Data

Similarly, the lowest performing CRC had raised rates against the baseline despite court timeliness both increasing, and being higher than other CRCs.

So What?

We’ve seen that reoffending rates are complex measures dependent on many factors, from individual, to regional to national level, and the interaction between them. The rates don’t change very much, even though the contexts can differ wildly.

Even comparing the CRCs at either end of performance spectrum there were not huge differences in the police and court factors. This can be seen in different results for CRCs operating in widely similar contexts.

So, my advice is this: don’t fall into the trap of feeling the results are not within your ability to affect. Understanding the wider picture is helpful to contextualise local results. In the following blogs, my colleagues will be focusing on what is in your control, and my colleagues will be identifying what works and using that to inform good practice.

Get involved in the conversation by joining our LinkedIn group.

TR: PbR results - speakers who presented to Community Rehabilitation Companies and other justice sector employees

Community Rehabilitation Companies: PbR Results Event

Transforming Rehabilitation is the UK government’s programme of outsourcing probation services to new community rehabilitation companies. In a radical move, the government is now paying these new companies by the reduction in reoffending results they achieve. GtD is at the forefront of this by providing our cutting-edge social impact analytics to Sodexo Justice Services who manage a number of these new companies.

The first PbR figures were published last month and GtD has been active in informing the debate on their significance. As part of this debate, we recently hosted a sell-out event for senior management and practitioners working in community rehabilitation companies and the justice sector.

An expert panel comprising Prof. Darrick Jolliffe of Greenwich University (above left), Dr Sam King of Leicester University (above right) and GtD’s own Jay Hughes (above centre left), considered the initial findings and what to do next, with Jack Cattell (above centre right) setting out a new vision of how predictive analyses can be used by practitioners to improve performance.

Community Rehabilitation Companies: PbR Results Event Get The Data

Prof. Darrick Jolliffe – University of Greenwich

If you were unable to attend but would like to learn more about how GtD could support you in evaluating your social impact outcomes or for a free predictive analytic roadmap for your CRC, contact Jack Cattell  The event presentations can also be viewed via the link below:

Transforming Rehabilitation – Learning from the PbR results presentations

Community Rehabilitation Companies: PbR Results Event Get The Data

Dr Sam King – University of Leicester

 

Community Rehabilitation Companies: PbR Results Event Get The Data

Jack Cattell – GtD

 

We’ve also set up a LinkedIn group as a forum for shared learning and discussion, for individuals who work or have an interest in, the fields of probation, offender rehabilitation and Transforming Rehabilitation. Click here to request to join – Transforming Rehabilitation

Image to illustrate no data can mean no voice and no idea

No Data? No Voice, No Idea – The Importance of Data

The collection and analysis of data must never be allowed to fall by the wayside – it’s a founding stone, not a ‘nice to have’.

Of course, I would say that, wouldn’t I? But here are six concrete reasons why data is important, drawing on Get the Data’s recent work with the Advice Services Alliance (ASA) as a case study.

  1. Data Is the Great Persuader

There is no more powerful tool for influencing stakeholders than data, as I explained in more detail in this recent blog post on using data to influence stakeholders.

ASA works with its members – various associations who provide advice services – to capture and analyse data from the front-line. This gives weight to conversations with all of those who have an interest in ASA’s direction of travel, reassuring them that strategic decisions are being made in response to changing needs, and reinforcing the professionalism that underlies the work they and their partners do.

 

  1. Data Means Funding

In particular, data is invaluable in drawing in new sources of funding, and persuading potential funding providers. Faced with a choice of projects or programmes in which they might invest, with ever-tighter budgets, funding bodies will regard convincing data as a good reason to choose your work over others. ASA will use data to add weight to its funding applications for this reason.

 

  1. Data Gives You the Power to Lead the Debate

ASA seeks to lead thought, representing its members in national discussions by highlighting issues affecting the people who use advice services. The body of data and analysis to which ASA will refer confers authority and allows the organisation to direct the debate and steer collective thinking around youth issues.

 

  1. Data Defines Good (and Bad) Practice

Using data provided by its members ASA will be able to identify areas for improvement in front line practice and also to pinpoint what is working especially well so that good practice can be shared across the community. It informs training programmes, service improvements and helps determine how resources should be employed for maximum impact.

 

  1. Better data and measurement development 

Good data leads to better data. Use of data means we learn its limitations and how to outcome those. We also learn how to measure the right outcomes, better; particularly in how to measure informal outcomes along said formal attainment.

 

  1. Data Is Cheaper and Easier Than Ever

Cloud-based databases are cheap and easy to implement compared to the cumbersome systems of the past. They make it easier for people to enter data and share it. So there’s really no excuse for failing to collect and analyse data in this day and age.

If you would like to find out more about our cutting-edge approach to data capture and analysis please get in touch.

Scales of Justice representing Transforming Rehabilitation

Transforming Rehabilitation: Payment by Results Figures

Last week saw the release of the Transforming Rehabilitation (TR) Payment by Results figures for the October to December 2015 cohort.

The overall result was encouraging, and defy the view that Transforming Rehabilitation’s radical changes to probation, and the ensuing problems, would result in increased reoffending – though it is very important that I point out that this is just the first set of results of many and overall judgement should be reserved for at least a year. The reoffending rate for all CRCs was 45.6% compared to a 2011 baseline rate of 47.5%. I had to make some (conservative) assumptions to estimate the baseline rate but I think it is also safe to say that the difference was statistically significant, suggesting reoffending rates have reduced under TR. Please see the the note at the end of this blog to understand better how I completed the analysis.

 

Transforming Rehabilitation – CRC Performance

The chart below describes each CRC’s reoffending rate in relation to the baseline 2011 rate. The grey line represents the range of reoffending rates that would indicate no change from 2011 (the baseline confidence interval). If the CRC’s rate is outside this range, we are confident in statistically terms to state that the CRC’s performance was either better or worse than the reoffending rate achieved in 2011. The green bars represent the reoffending rates of CRCs that outperformed 2011, the orange bars represent those that performed the same as 2011 and the red bars present those that performed worse than 2011.

Transforming Rehabilitation: Payment by Results Figures Get The Data

Source: Ministry of Justice Final Proven Reoffending Rates TR (Oct to Dec 2015 cohort).

Thirteen of the CRCs beat the baseline rate. The best performing CRC was Cumbria and Lancashire, which beat the baseline rate by 8.2% (49.9% to 41.7%). The nest best was Hampshire and the Isle of Wight which beat the baseline by 5.4% and the third best was Northumbria with a better rate by 4.3%. Two of the CRCs performed worse than the 2011 baseline. Warwickshire and West Mercia recorded a reoffending rate 3% worse than the baseline rate, and South Yorkshire’s rate was 2.8% worse. With most CRCs, however, outperforming the reoffending rate form 2011, the figures are a promising set of results.

 

Transforming Rehabilitation – Comparing CRC performance

Now that the baseline rates have been published, we can better understand how well each area was performing in 2011 and whether a CRC is now being asked to better good or bad performance achieved in that year. The chart below describes the difference between the actual baseline rate and the 2011 baseline’s OGRS score (in other words their expected rate of reoffending). A negative result in the chart means the area performed better in 2011 than the OGRS score expected.

Transforming Rehabilitation: Payment by Results Figures Get The Data

Source: Ministry of Justice Final Proven Reoffending Rates TR (Oct to Dec 2015 cohort).

The charts highlights that six of the CRCs are being asked to beat better than expected performance in 2011 (in other words to be better than good). Whereas other CRCs, notably London and Wales, are being asked to outperform potentially poor performance in 2011. It it interesting that South Yorkshire and, Warwickshire & West Mercia – the two areas that recorded poor performance for TR – are being asked to beat good performance from 2011. Merseyside and Cheshire & Greater Manchester, however, are equally being asked to beat good performance from 2011 and were able to do so for the October to December 2015 cohort. The OGRS score does not allow for area effects, which will exist and could explain the differences between the OGRS score and the baseline rate. It not possible now to conclude whether payment by results will be easier in some areas than others, but, going forward, I will monitor the impact of whether a CRC is being asked to perform better than good or poor performance from 2011 on their ability to achieve payment by results bonuses.

 

Notes on analysis

The latest Ministry of Justice bulletin released more data than was previously available and I was able to complete a statistical analysis of the impact of TR. This could only be completed with making conservative assumptions that would make finding a statistically significant result less likely. The following actions were taken:

  • I assumed the spread of offenders across CRCs in 2011 was exactly the same as it was in the October to December 2015 cohort. This would not be the case but any analysis would want to weight the two samples so they represented each other so the impact of this assumption is minimal.
  • The 2011 sample size was assumed to be the same as that of the October to December 2015 cohort. The 2011 sample will be considerably bigger, so this assumption meant the standard error used for the analysis was larger than it should be.
  • A t-test with unequal variances assumed was used to test the difference between the cohort’s and the baseline’s reoffending rate. The t statistic result was 4.6.

Blog originally guest posted on http://www.russellwebster.com 31st October 2017 with additional commentary from Russell Webster.