Driving Improved Outcomes for the Juvenile Justice System

It was a great pleasure to attend the Coalition of Juvenile Justice’s annual conference in Washington D.C. last week. Under the title, “Redefining leadership: engaging youth, communities and policy makers to achieve better juvenile justice outcomes”, the conference convened a broad coalition of policy makers, practitioners, advocates, researchers and, importantly young people themselves.

Improving outcomes for our young people was a good theme for the conference. And it comes at a time when juvenile justice is undergoing substantial reform in states across the U.S. This includes my home state of Georgia which is considered to be in the vanguard of reforming states.

The motivation for reform appears to be driven by a humanitarian desire to end the use of custody for young people and improve conditions in jail.  It is also clear that the reform agenda is driven by fiscal realities and the need to reduce the cost to the taxpayer. The Department of Juvenile Justice in Georgia estimates that it costs upwards of $90,000 a year to house just one juvenile offender in one of its facilities.

Humanitarian and costs concerns are legitimate concerns in public policy, not least how we treat our vulnerable young people. However, we must guard against unfounded good intentions and the danger of delivering cut-price justice. Good juvenile justice outcomes should be about increasing protective factors, ending the “school to prison pipeline”, improving relations with the police and education authorities and – of course – reducing recidivism.  So it was heartening that the conference was committed to improving outcomes for the juvenile offenders, their families and communities as well as the wider juvenile justice system.

The key to achieving good outcomes lies in understanding the data, and I attend several seminars on data-driven decision making, with presenters demonstrating the use of data to model how their juvenile justice system was reformed to achieve improved outcomes. It was also good to learn more about current evidence based interventions that are successful in addressing the underlying factors related to offending behaviour. My own contribution was to present a poster on how evaluation provides empirical evidence of how and why an intervention has achieved its outcomes and what can be done to improve them.

Juvenile justice reform should be led by an empirical analysis of the data: what is effective and cost beneficial. Evaluation has a key role in this and as the current reforms are rolled out they should be evaluated and scrutinized to determine whether they were successful, how they could be improved – or even whether a particular reform should be reversed or altered.

If you would like to learn more about how evaluation can help you with your local reforms, please contact me at alan.mackie@getthedata.co.uk

 

Impact Evaluation and Social Impact Bonds

How Rigorous Impact Evaluation Can Improve Social Impact Bonds

Social Impact Bonds

In recent years, Social Impact Bonds are being increasingly used by the British government to deliver public services via outcomes based commissioning. They are also becoming increasingly common in the U.S. By linking payments to good outcomes for society, SIBs are used not only to provide better value for money, but also as a driver of public sector reform. In the words of guidance published by the British government’s Cabinet Office:

“[Social Impact Bonds] are … designed to help reform public service delivery. SIBs improve the social outcomes of publicly funded services by making funding conditional on achieving results. “

While SIBs are not without their critics, their proponents argue that the bonds are a great way to attract private investment to the public sector while focusing all partners on the delivery of the desired social outcomes. This new way of commissioning services also encourages prime contractors to subcontract delivery of some service to the community and voluntary organisations, who bring their own experience, expertise and diversity to the provision of social services.

GtD have completed evaluations that have helped shape social impact bonds, and through our work we have identified five key questions that should be asked by anyone thinking of setting up a SIB or is looking to improve the design of their SIB:

1. Will it work?

Some services delivered by SIBs fail before they start because the planned intervention cannot plausibly achieve the desired outcome. In other words, just because an intervention reduced the number of looked after children entering the criminal justice system doesn’t mean it should work for all young people at risk of offending. That said, if the evidence base around a particular intervention is weak that does not mean one should not proceed – but it should promote a SIB design that includes an evaluation that can state quickly whether the SIB is delivering the hoped for outcomes.

2. Who can benefit from this intervention and who can’t?

We all want to help as many people as possible. However, we can quickly lose sight of who we are seeking to help when we are simply meeting output targets. In other words, if public services are funded by the number of clients they see, then providers could be tempted to increase numbers by accepting referrals of people for whom the service was not intended. So to achieve your outcomes – and receive payments – it’s vital to monitor intelligently the profile of your beneficiaries and ask yourself, “If my targeting were perfect are these the clients I would want to work with to deliver my intended outcomes?”

3. Are we doing what we said we were going to do? And does it work?

Interventions can fail simply because they don’t do what you said you were going to do. If, for example, you are working with young people to raise career prospects and your operating model includes an assessment of need (because the evidence base suggests that assessments increase effectiveness) then it should be no surprise that you did not meet your outcome targets if an assessment was completed with only half of your clients. Identifying your key outputs, monitoring their use and predicting outcomes based on their use can give you much greater confidence in achieving your intended impact.

4. What do we need to learn and how do we learn quickly?

All SIBs we have seen collate a lot of data about their beneficiaries and the service provided but few use those data to their full potential. With predictive analysis, we can monitor who appears to respond best, who is not benefiting and what form of service delivery is the most effective. In other words, is one-to-one work or group work more cost effective? As such you can learn how to define your referral criteria better or learn how to improve your operating model, even within the first months of a SIB.

5. How can we build a counterfactual?

A counterfactual is an estimation of what outcomes would have been achieved without the SIB. Comparing your SIB’s outcomes to the counterfactual can highlight some areas for learning and how to improve over time. Consideration should be given to the counterfactual at the commencement of the SIB. Full advantage can be taken of the publically available data sets to construct the counterfactual: for example, the National Pupil Database, Justice Data Lab or the Health Episode statistics. (Top tip: consent from beneficiaries to use these data sources is generally required).

To discuss how GtD’s impact evaluation can help improve your organisation’s SIB, please contact Jack Cattell, jack.cattell@getthedata.co.uk

Non profits in the criminal justice system

Non-profits in the Criminal Justice System & the Role of Social Impact Analytics

Non-profit and commercial organisations play a vital role in assisting offenders to desist from offending, whether working in partnership with community rehabilitation companies or directly with offenders. This week CLINKS, the national organisation that supports voluntary organisations working with offenders and their families, published its annual “State of the Sector” survey. Overall, the survey found that sector remains innovative, flexible and resilient, with organisations having developed and delivered new services to respond to the changing needs of service users and to fill gaps in existing provision.

In acknowledging the importance of the voluntary sector, the survey also found that individual organisations were finding it difficult to keep up with pace of radical policy reform and the effects of continuing austerity. This is not surprising as the majority of organisations that participated in the survey were small, employed fewer than 10 members of staff, operated locally and had an annual turnover of less than £1m. As a consequence, the survey also found that these organisations were spending an increasing amount of time fundraising, sometimes at the expense of client services, while losing their focus on their core purposes. For a few, these challenges threaten their very existence. In the words of Anne Fox, CEO of CLINKS, “The picture painted by this survey is of a sector battling against significant odds but continuing to do essential work and innovate to support offenders and their families with increasing needs”.

As a company, GtD is committed to working in the criminal justice sector, and is currently providing social impact analytics to all levels in the criminal justice system: central and local government, a community rehabilitation company and non-profits. Whether we are providing sophisticated predictive analyses or delivering a performance management framework, the benefits of our social impact analytics are clear: they offer a clarity on where to deploy scarce resources to the greatest effect, and provide definitive information to manage an intervention efficiently and demonstrate its effectiveness to funders.

An effective criminal justice system needs the inputs from a diverse, innovative and healthy voluntary sector. Social impact analytics will define and measure these inputs, and assist organisations remain focused on their core purposes and make an effective contribution.