Our social impact analytics are being used to prove the impact of Street Soccer Academy’s custody to community programme. We demonstrated that the academy was working with the nation’s hardest to reach individuals, and that 75% of this group completed the programme and showed strong engagement upon release. Importantly the programme’s impact on reoffending translates to a £3.8m saving to the prison service. For more information on how GtD can prove your organisation’s impact, contact email@example.com
Jack Cattell will be presenting at the Public Performance and Management Reporting Network Conference at Rutgers University later this month. Addressing the theme “Data Driven Decision Making: Navigating the Data Glut”, Jack will focus on GtD’s work in rationalising an organisation’s data to provide clear performance indicators. The conference has attracted speakers from across North America, Europe and Asia and takes place on the 22nd and 23rd September.
Evaluation is often divorced from an organisation’s day-to-day operation, seen simply as a retrospective assessment of the impact of an intervention or the measurement of a client group’s outcomes. While there remains a place for that traditional approach, my colleagues and I at GtD believe that evaluation should be integral to shaping an organisation’s operations, by both looking back over past performance as well as predicting how to achieve real impact in the future.
In developing cutting edge social impact analytics, GtD’s novel approach to evaluation is being delivered to governments, non-profits and commercial organisations in the U.K. and the U.S. Typically, our clients are working to reduce reoffending, resettle refugees, provide shelter to the homeless and help disengaged young people achieve improved education outcomes. By providing definitive analyses we are enabling our clients to monitor what they are now doing; learn how they can improve their future performance and – ultimately – prove that they had an impact on a client group or wider society.
So, what are our social impact analytics, and what’s the value in our approach to evaluation?
First our Impact Management is helping managers and board members think about what they are seeking to achieve and how they will do that with the resources at their disposal. In our experience, managers, board members and funders should be concerned that their service is well run. By monitoring resources, inputs and outputs, our Impact Management can produce measures of a service’s economy and efficiency.
Second, our Predictive Analyses are helping organisations to deliver more effective services. Our analyses are helping practitioners to identify what will work best for their clients, and managers are using the information to improve interventions and predict future impact, and in the case of social impact bonds, future income.
Finally, organisations that commission GtD are working with some of the most vulnerable people in society. We value their work and are committed to using quantitative methods of evaluation to determine their impact. We are proud that our Impact Evaluations are not only delivering definitive reports on the impact of their work, but can also be used to provide a highly persuasive case to funders or a press-release as part of a media or funding campaign.
To find out how your GtD’s Social Impact Analytics can help your organisation make a difference contact me at firstname.lastname@example.org
Pomp and pageantry came to Westminster this week, with the Queen’s Speech setting out the British government’s legislative agenda for the coming Parliamentary session. But amid the ermine and jewels was a call for hard, empirical data.
The centre piece of the ‘Gracious Address’ was a major shake-up of the prison system in England. Legislation will be brought forward to give governors of six “reform prisons” unprecedented autonomy over education and work in prisons, family visits, and rehabilitation services. With this autonomy will come accountability and the publication of comparable statistics on reoffending, employment rates on release, and violence and self-harm for each prison.
Further details of the government’s prison reforms were contained “Unlocking Potential”, Dame Sally Coates’ review of prison education in England that was published this week. The review includes recommendations to improve basic skills and the quality of vocational training and employability, and also greater personal social development. Echoing the government’s move to devolve greater autonomy to prison governors, Dame Sally’s review also endorsed the need for governors to be held to account for the educational progress of all prisoners in their jails, and for the outcomes achieved by their commissioning decisions for education.
Improved education outcomes for individual prisoners will be supported by improved assessment of prisoners’ needs and the creation of Personal Learning Plans. However, Dame Sally’s review also made a call for greater performance measurement not only for the sake of accountability, but also for the planning and prioritisation of education services.
As noted before, this is an exciting time for prison reform on both sides of the Atlantic. However, reform must be made on evidence and supported by the hard data. Devolving decision making to those who know best is a bold move but with autonomy comes accountability and transparency. As Dame Sally’s report recommends, accountability and transparency are well served by,
“Developing a suite of outcome measures to enable meaningful comparisons to be made between prisons (particularly between those with similar cohorts of offenders) is vital to drive improved performance”.
As the pace of reform continues, GtD looks forward to supporting those reforms with our expertise of outcome measurement and social impact analytics.
It was a great pleasure to attend the Coalition of Juvenile Justice’s annual conference in Washington D.C. last week. Under the title, “Redefining leadership: engaging youth, communities and policy makers to achieve better juvenile justice outcomes”, the conference convened a broad coalition of policy makers, practitioners, advocates, researchers and, importantly young people themselves.
Improving outcomes for our young people was a good theme for the conference. And it comes at a time when juvenile justice is undergoing substantial reform in states across the U.S. This includes my home state of Georgia which is considered to be in the vanguard of reforming states.
The motivation for reform appears to be driven by a humanitarian desire to end the use of custody for young people and improve conditions in jail. It is also clear that the reform agenda is driven by fiscal realities and the need to reduce the cost to the taxpayer. The Department of Juvenile Justice in Georgia estimates that it costs upwards of $90,000 a year to house just one juvenile offender in one of its facilities.
Humanitarian and costs concerns are legitimate concerns in public policy, not least how we treat our vulnerable young people. However, we must guard against unfounded good intentions and the danger of delivering cut-price justice. Good juvenile justice outcomes should be about increasing protective factors, ending the “school to prison pipeline”, improving relations with the police and education authorities and – of course – reducing recidivism. So it was heartening that the conference was committed to improving outcomes for the juvenile offenders, their families and communities as well as the wider juvenile justice system.
The key to achieving good outcomes lies in understanding the data, and I attend several seminars on data-driven decision making, with presenters demonstrating the use of data to model how their juvenile justice system was reformed to achieve improved outcomes. It was also good to learn more about current evidence based interventions that are successful in addressing the underlying factors related to offending behaviour. My own contribution was to present a poster on how evaluation provides empirical evidence of how and why an intervention has achieved its outcomes and what can be done to improve them.
Juvenile justice reform should be led by an empirical analysis of the data: what is effective and cost beneficial. Evaluation has a key role in this and as the current reforms are rolled out they should be evaluated and scrutinized to determine whether they were successful, how they could be improved – or even whether a particular reform should be reversed or altered.
If you would like to learn more about how evaluation can help you with your local reforms, please contact me at email@example.com
Social Impact Bonds
In recent years, Social Impact Bonds are being increasingly used by the British government to deliver public services via outcomes based commissioning. They are also becoming increasingly common in the U.S. By linking payments to good outcomes for society, SIBs are used not only to provide better value for money, but also as a driver of public sector reform. In the words of guidance published by the British government’s Cabinet Office:
“[Social Impact Bonds] are … designed to help reform public service delivery. SIBs improve the social outcomes of publicly funded services by making funding conditional on achieving results. “
While SIBs are not without their critics, their proponents argue that the bonds are a great way to attract private investment to the public sector while focusing all partners on the delivery of the desired social outcomes. This new way of commissioning services also encourages prime contractors to subcontract delivery of some service to the community and voluntary organisations, who bring their own experience, expertise and diversity to the provision of social services.
GtD have completed evaluations that have helped shape social impact bonds, and through our work we have identified five key questions that should be asked by anyone thinking of setting up a SIB or is looking to improve the design of their SIB:
1. Will it work?
Some services delivered by SIBs fail before they start because the planned intervention cannot plausibly achieve the desired outcome. In other words, just because an intervention reduced the number of looked after children entering the criminal justice system doesn’t mean it should work for all young people at risk of offending. That said, if the evidence base around a particular intervention is weak that does not mean one should not proceed – but it should promote a SIB design that includes an evaluation that can state quickly whether the SIB is delivering the hoped for outcomes.
2. Who can benefit from this intervention and who can’t?
We all want to help as many people as possible. However, we can quickly lose sight of who we are seeking to help when we are simply meeting output targets. In other words, if public services are funded by the number of clients they see, then providers could be tempted to increase numbers by accepting referrals of people for whom the service was not intended. So to achieve your outcomes – and receive payments – it’s vital to monitor intelligently the profile of your beneficiaries and ask yourself, “If my targeting were perfect are these the clients I would want to work with to deliver my intended outcomes?”
3. Are we doing what we said we were going to do? And does it work?
Interventions can fail simply because they don’t do what you said you were going to do. If, for example, you are working with young people to raise career prospects and your operating model includes an assessment of need (because the evidence base suggests that assessments increase effectiveness) then it should be no surprise that you did not meet your outcome targets if an assessment was completed with only half of your clients. Identifying your key outputs, monitoring their use and predicting outcomes based on their use can give you much greater confidence in achieving your intended impact.
4. What do we need to learn and how do we learn quickly?
All SIBs we have seen collate a lot of data about their beneficiaries and the service provided but few use those data to their full potential. With predictive analysis, we can monitor who appears to respond best, who is not benefiting and what form of service delivery is the most effective. In other words, is one-to-one work or group work more cost effective? As such you can learn how to define your referral criteria better or learn how to improve your operating model, even within the first months of a SIB.
5. How can we build a counterfactual?
A counterfactual is an estimation of what outcomes would have been achieved without the SIB. Comparing your SIB’s outcomes to the counterfactual can highlight some areas for learning and how to improve over time. Consideration should be given to the counterfactual at the commencement of the SIB. Full advantage can be taken of the publically available data sets to construct the counterfactual: for example, the National Pupil Database, Justice Data Lab or the Health Episode statistics. (Top tip: consent from beneficiaries to use these data sources is generally required).
To discuss how GtD’s impact evaluation can help improve your organisation’s SIB, please contact Jack Cattell, firstname.lastname@example.org
Non-profit and commercial organisations play a vital role in assisting offenders to desist from offending, whether working in partnership with community rehabilitation companies or directly with offenders. This week CLINKS, the national organisation that supports voluntary organisations working with offenders and their families, published its annual “State of the Sector” survey. Overall, the survey found that sector remains innovative, flexible and resilient, with organisations having developed and delivered new services to respond to the changing needs of service users and to fill gaps in existing provision.
In acknowledging the importance of the voluntary sector, the survey also found that individual organisations were finding it difficult to keep up with pace of radical policy reform and the effects of continuing austerity. This is not surprising as the majority of organisations that participated in the survey were small, employed fewer than 10 members of staff, operated locally and had an annual turnover of less than £1m. As a consequence, the survey also found that these organisations were spending an increasing amount of time fundraising, sometimes at the expense of client services, while losing their focus on their core purposes. For a few, these challenges threaten their very existence. In the words of Anne Fox, CEO of CLINKS, “The picture painted by this survey is of a sector battling against significant odds but continuing to do essential work and innovate to support offenders and their families with increasing needs”.
As a company, GtD is committed to working in the criminal justice sector, and is currently providing social impact analytics to all levels in the criminal justice system: central and local government, a community rehabilitation company and non-profits. Whether we are providing sophisticated predictive analyses or delivering a performance management framework, the benefits of our social impact analytics are clear: they offer a clarity on where to deploy scarce resources to the greatest effect, and provide definitive information to manage an intervention efficiently and demonstrate its effectiveness to funders.
An effective criminal justice system needs the inputs from a diverse, innovative and healthy voluntary sector. Social impact analytics will define and measure these inputs, and assist organisations remain focused on their core purposes and make an effective contribution.