Non-Profits and the Measurement of What Matters

Tell Me What You Want, What You Really, Really, Want

Non-Profits and the Measurement of What Matters

Hard to believe that it is 25 years since the Spice Girls recorded “Wannabe”, and with it the launch of the “girl power” phenomenon. A catchy tune for all of us “Dad Dancers” out there, but what has it to do with (the more serious) topic of social impact measurement and non-profits?

Well, at the time Posh, Ginger, Sporty, Scary and Baby were storming the charts, I was settling into my job as a government researcher with the British civil service. 1995 was just before the dawn of New Labour’s managerial approach to public services, but already government was demanding greater efficiency and effectiveness. So, while the rest of the world was going all “zig-a-zig-ah”, I was co-authoring my first research paper, the rather ponderous “Managing the Courts Effectively: the reason for adjournments and delays in the magistrates’ courts in England & Wales”.

For the next 20 years, much of the evaluation work that my colleagues and I completed was framed by this managerialism. While working our way through the elusive “what works?” agenda, it became clear that the answers were of interest to government bean counters rather than the general public. So, “what works?” tended to be about government agencies doing things more swiftly, reducing costs to the Exchequer or moderating the politically troublesome issues of youth crime and anti-social behaviour.  While reducing crime in a neighbourhood, and bringing offenders to justice more quickly, remain laudable aims, were they what was “really, really wanted”?

When launching GtD in the US five years ago, we deliberately sought to partner with local and national non-profits. Not surprisingly they see things differently than government, and with that their focus is entirely on the people they serve. I have said it before, but our best non-profit clients know what they are about, and I have remarked that on visiting their offices there is a real energy to their business and commitment to their clients.  That is not to say that there is no tension between a non-profit’s wider goals and the narrower commitments made to a funder, but the best non-profits jealously guard their own ethos and interests.

I saw this first when working with the International Rescue Committee in Atlanta. Its Connect to Success program does vital work in helping young refugees get into employment, training, or education. However, it was clear that for a humanitarian organization the measures of success had to be more than the number of jobs found or college places achieved.  So, our evaluation also measures the wellbeing of the young people and their general satisfaction with their lives in the USA. Similarly, for the public defender organization Gideon’s Promise their metrics of success did not include reductions in convictions or fewer custodial sentences, but something more radical: the quality of the relationship between the public defenders and the clients they represent. Jack and I are currently getting to grips with a new rural health initiative in Georgia. The initiative seeks to tackle the social determinants of health, but it is important that local people should have a say in what good health outcomes look like for their own community.

Of course, non-profits must be diligent in demonstrating their economy, efficiency and effectiveness, but at GtD we also want to measure what our clients believe to be most important to them. This is all part of what we call our “social impact journey”. Before we start measuring the more obvious outcomes, we will take the time to investigate what an organization really wants to achieve, even when it finds that difficult to articulate clearly.  This is all part of what we call the client’s theory of change and it forms the basis for us to find new and exciting ways to measure a client’s true outcomes. If you want to learn more about what your organization can do to measure “what it really, really wants”, why not join our free webinar on October 29th. We will tell you more about GtD’s social impact journey, our packages of “Measure”, “Learn” and “Prove”, and what the next steps are to measure your social impact – definitively.

Over-representation of BAME young people in custody

The Disproportionate Representation of BAME Young People in Custody

GtD specialises in youth justice and we are aware of the disproportionate representation of BAME young people in custody as was identified in the Lammy Report.

We commissioned an article in partnership with CYPN to understand why BAME young people are over-represented in the custody population. Jack carried out analysis of court sentencing and remand decisions and the results show that the custodial rate for BAME children is consistently higher than for white children. Using data to keep track of decision making has a key role in correcting this.

Jack recommends 3 key action points:

  1. YOTs use data to avoid a “negative cycle” of disproportionate outcomes whereby the more punitive a child’s first disposal or sentence, the more likely they are to receive an even more punitive sentence if they reoffend.
  2. YOTs to work with the police to understand any biases at all stages of the criminal justice system.
  3. That the police include ethnicity when publishing data on the decision to prosecute.

You can read the full article here >> https://bit.ly/33jkFHm and if you would like to discuss the approach we used for analysis, and obtain a copy of how it was carried out, please contact us at info@getthedata.co.uk

Youth Justice: Evidence Based Policy in 2017? Get The Data

Youth Justice: Evidence Based Policy in 2017?

This month has seen the publication of Lord Taylor’s Review of the Youth Justice System in England & Wales.

In his comprehensive report, Taylor makes a number of recommendations “to transform the youth justice system in which young people are treated as children first and offenders second, and in which they are held to account for their offending”.

Of particular interest to me was Taylor’s emphasis on diverting children out of the justice system, where possible, and directing the police, local authorities and health authorities to operate these schemes jointly. Taylor identifies such multi-agency leadership as one of the principles of good practice in diversion that includes, proportionality, speed, sensitivity to victims, light touch assessment and access to other services.

If these recommendations are implemented, then there are reasons to be cheerful about the future direction of youth justice in England and Wales. But, have we not been here before? It is nearly 20 years since Tony Blair formed the ‘New Labour’ administration in 1997, with its commitment to ‘evidence-led’ policy. Nowhere was this more evident than in the 1998 youth justice reforms and the creation of multi-agency Youth Offending Teams. “Tough on crime; tough on the causes of crime” was the slogan of the day, and I cut my evaluation teeth in the boom of criminological research and evaluation, and the search for “what works?” in youth justice.

Within a decade, however, the laudable attempt to re-set youth justice by informed policy had become jaded. Traditional law and order politics were reasserting themselves, something that Barry Goldston recognised in his excellent article “The sleep of (criminological) reason: Knowledge–policy rupture and New Labour’s youth justice legacy“. Giving his retrospective, Goldson identified a “trajectory of policy [that] has ultimately moved in a diametrically opposed direction to the route signalled by research-based knowledge and practice-based evidence”. In other words, the knowledge-base was telling policy makers to be doing one thing, but they appeared to be doing just the opposite.

In his article Goldson identified five areas where the rupture between the policy and practice was most evident: research tells us that young people committing crime is relatively ‘normal’ (but the response is to be intolerant of this); the evidence is that rates of youth crime are relatively low (but politicians tend to amplify it and “define it up”); evaluation shows that diversion is effective (but the response is for earlier intervention and ‘net widening’); universal services of welfare, education and health are effective (but punishment becomes ascendant while welfare is in retreat); decarceration is known to be cost-effective (but the use of custody increases).

The publication of the Taylor report provides us an opportunity to reset youth justice and its recommendations seek to repair the “knowledge-practice rupture”. Are we seeing an awakening of criminological reason? I trust we are and look forward to continuing to play my part in providing evidence of what works to policy makers and practitioners.

Driving Improved Outcomes for the Juvenile Justice System Get The Data

Driving Improved Outcomes for the Juvenile Justice System

It was a great pleasure to attend the Coalition of Juvenile Justice’s annual conference in Washington D.C. last week. Under the title, “Redefining leadership: engaging youth, communities and policy makers to achieve better juvenile justice outcomes”, the conference convened a broad coalition of policy makers, practitioners, advocates, researchers and, importantly young people themselves.

Improving outcomes for our young people was a good theme for the conference. And it comes at a time when juvenile justice is undergoing substantial reform in states across the U.S. This includes my home state of Georgia which is considered to be in the vanguard of reforming states.

The motivation for reform appears to be driven by a humanitarian desire to end the use of custody for young people and improve conditions in jail.  It is also clear that the reform agenda is driven by fiscal realities and the need to reduce the cost to the taxpayer. The Department of Juvenile Justice in Georgia estimates that it costs upwards of $90,000 a year to house just one juvenile offender in one of its facilities.

Humanitarian and costs concerns are legitimate concerns in public policy, not least how we treat our vulnerable young people. However, we must guard against unfounded good intentions and the danger of delivering cut-price justice. Good juvenile justice outcomes should be about increasing protective factors, ending the “school to prison pipeline”, improving relations with the police and education authorities and – of course – reducing recidivism.  So it was heartening that the conference was committed to improving outcomes for the juvenile offenders, their families and communities as well as the wider juvenile justice system.

The key to achieving good outcomes lies in understanding the data, and I attend several seminars on data-driven decision making, with presenters demonstrating the use of data to model how their juvenile justice system was reformed to achieve improved outcomes. It was also good to learn more about current evidence based interventions that are successful in addressing the underlying factors related to offending behaviour. My own contribution was to present a poster on how evaluation provides empirical evidence of how and why an intervention has achieved its outcomes and what can be done to improve them.

Juvenile justice reform should be led by an empirical analysis of the data: what is effective and cost beneficial. Evaluation has a key role in this and as the current reforms are rolled out they should be evaluated and scrutinized to determine whether they were successful, how they could be improved – or even whether a particular reform should be reversed or altered.

If you would like to learn more about how evaluation can help you with your local reforms, please contact me at alan.mackie@getthedata.co.uk