Five Lessons for Impactful Data

Five Lessons for Impactful Data

A client has just told me that our data collection and analysis has helped them secure substantial funding from central government. I wanted to share the lessons I took from that work for impactful quantitative data:

  1. Start simple – the data collection and data analysis were consciously kept very simple to start. This made it easier for staff to collect data and to produce clear messages for internal and external consumption.
  2. Put yourself in their shoes – with the client, we thought long and hard about what different stakeholders would want to know from the data, and the client was able to respond to the Covid-19 pandemic as well.
  3. Be confident  no data are 100% perfect, and there is no need to apologise for one’s data. Instead, be confident by knowing what it can say and be on top of its limitations and how to correct those in the future.  
  4. Visualise the data – not everyone is comfortable with numbers, so after identifying the data story with our client, we took our time to identify the best charts and colours to tell that story.  
  5. Anticipate –the data system was set to so it would be relatively straight forward to collect new data and add new analysis to the current system. This means our client can respond speedily to funder requests.

If you can apply some or all these principles to your data collection and analysis, then you will start making a bigger impact.

Non-profit social impact webinar

Measuring Non-Profit Social Impact During the Pandemic

In changing with the times, we see non-profits that are finding novel ways of interacting with their clients; revising measures to reflect their new activities; and even offering new services to deal the impacts of Covid-19 on the lives of the people they serve.

A time of disruption is undoubtedly challenging, but it is also a time of opportunity. At GtD we are helping non-profits to navigate these challenges and support them to exploit new opportunities.  We believe that our social impact packages of “Measure”, “Learn” and “Prove” are now more important than ever.

As part of our “Year of Social Impact”, our next webinar is entitled, “Measuring Non-Profit Social Impact During the Pandemic”. This webinar will take place on October 29th at 12noon EST/4 pm. It is free to register and highly recommended for leaders from the not-for-profit sector on both sides of the Atlantic. So, join us and discover how we can help your non-profit prove its impact – definitively.

You can learn more about the event and book here >> https://bit.ly/SocialImpactWebinarOct

 

Theory of Change

Theory of Change – the Starting Point for Impact Measurement

In our work, we like to develop impact measurements from a fully articulated theory of change (ToC). Our clients, however, have approached us to show them how to measure something accurately and rigorously, and sometimes are unsure why we first need to spend the time talking to them about causal mechanisms, context and their long-term goal. And would prefer it if we went straight to the end and told them how to measure wellbeing, employability etc.

If you are thinking of improving your social impact measurement, here are 5 reasons to start with a theory of change:

  1. It identifies what is to be evaluated. Organisations are not always sure what of their many services is in scope for social impact measurement and the ToC helps them to set the boundaries on what they will and will not take responsibility to measure.
  2. It makes complexity understandable. Some charities and social enterprises are complex and deliver many different projects, in different ways, for different funders. A current client of ours has 26 unique projects. It can seem impossible to identify common threads and common measurement requirements, but the ToC helps our clients to see the wood for the trees and understand meaningful impact measurement is possible.
  3. It reminds you of what you’re trying to achieve. During day-to-day operations, it is easy to forget what your organisation is trying to achieve. The ToC brings the long-term aim front and centre and puts the pieces together to show how that is achieved.
  4. It tells a compelling story. The ToC describes how impact measurement data can be used to tell a compelling story for how you contribute to your long-term outcomes. By identifying your short term outcomes – those achieved during an intervention – and your end of programme outcomes, the ToC shows you how to use your data to evidence your impact in a compelling and persuasive way.
  5. It can tell you something you don’t know. In preparing a ToC we identify the evidence base that either confirms or amends your projects’ designs and presents any weaknesses in their current delivery.

The development of a ToC is part of our Measure package. Please click here for more information.

Impact Evaluation and Social Impact Bonds

How Rigorous Impact Evaluation Can Improve Social Impact Bonds

Social Impact Bonds

In recent years, Social Impact Bonds are being increasingly used by the British government to deliver public services via outcomes based commissioning. They are also becoming increasingly common in the U.S. By linking payments to good outcomes for society, SIBs are used not only to provide better value for money, but also as a driver of public sector reform. In the words of guidance published by the British government’s Cabinet Office:

“[Social Impact Bonds] are … designed to help reform public service delivery. SIBs improve the social outcomes of publicly funded services by making funding conditional on achieving results. “

While SIBs are not without their critics, their proponents argue that the bonds are a great way to attract private investment to the public sector while focusing all partners on the delivery of the desired social outcomes. This new way of commissioning services also encourages prime contractors to subcontract delivery of some service to the community and voluntary organisations, who bring their own experience, expertise and diversity to the provision of social services.

GtD have completed evaluations that have helped shape social impact bonds, and through our work we have identified five key questions that should be asked by anyone thinking of setting up a SIB or is looking to improve the design of their SIB:

1. Will it work?

Some services delivered by SIBs fail before they start because the planned intervention cannot plausibly achieve the desired outcome. In other words, just because an intervention reduced the number of looked after children entering the criminal justice system doesn’t mean it should work for all young people at risk of offending. That said, if the evidence base around a particular intervention is weak that does not mean one should not proceed – but it should promote a SIB design that includes an evaluation that can state quickly whether the SIB is delivering the hoped for outcomes.

2. Who can benefit from this intervention and who can’t?

We all want to help as many people as possible. However, we can quickly lose sight of who we are seeking to help when we are simply meeting output targets. In other words, if public services are funded by the number of clients they see, then providers could be tempted to increase numbers by accepting referrals of people for whom the service was not intended. So to achieve your outcomes – and receive payments – it’s vital to monitor intelligently the profile of your beneficiaries and ask yourself, “If my targeting were perfect are these the clients I would want to work with to deliver my intended outcomes?”

3. Are we doing what we said we were going to do? And does it work?

Interventions can fail simply because they don’t do what you said you were going to do. If, for example, you are working with young people to raise career prospects and your operating model includes an assessment of need (because the evidence base suggests that assessments increase effectiveness) then it should be no surprise that you did not meet your outcome targets if an assessment was completed with only half of your clients. Identifying your key outputs, monitoring their use and predicting outcomes based on their use can give you much greater confidence in achieving your intended impact.

4. What do we need to learn and how do we learn quickly?

All SIBs we have seen collate a lot of data about their beneficiaries and the service provided but few use those data to their full potential. With predictive analysis, we can monitor who appears to respond best, who is not benefiting and what form of service delivery is the most effective. In other words, is one-to-one work or group work more cost effective? As such you can learn how to define your referral criteria better or learn how to improve your operating model, even within the first months of a SIB.

5. How can we build a counterfactual?

A counterfactual is an estimation of what outcomes would have been achieved without the SIB. Comparing your SIB’s outcomes to the counterfactual can highlight some areas for learning and how to improve over time. Consideration should be given to the counterfactual at the commencement of the SIB. Full advantage can be taken of the publically available data sets to construct the counterfactual: for example, the National Pupil Database, Justice Data Lab or the Health Episode statistics. (Top tip: consent from beneficiaries to use these data sources is generally required).

To discuss how GtD’s impact evaluation can help improve your organisation’s SIB, please contact Jack Cattell, jack.cattell@getthedata.co.uk

Image of data being analysed

There’s no Magic Way of Measuring Impact

Wouldn’t it be great if there was a way of measuring your social impact across multiple projects using a single dependable statistic? Well, I’ve got some bad news, and some good.

I was recently talking to a charity who wanted to know how if they could go about measuring and reporting the overall impact of the organisation on children and families. With multiple strands each aiming to achieve different things, they asked if a single outcome measure – one accurate, reliable number – to sum up the impact of the whole organisation was either possible or desirable.

First, here’s the bad news: it’s very unlikely – I might even be so bold as to say impossible – that any such thing exists. You might think you’ve found one that works but when you put in front of a critic (or a nitpicking critical friend, like me) it will probably get ripped apart in seconds.

Of course, if there is a measure that works across multiple projects, even if not all of them, you should use it, but don’t be tempted to shoehorn other projects into that same framework.

It’s true that measuring impact requires compromise but an arbitrary measure, or one that doesn’t stand up to scrutiny, is the wrong compromise to make.

The Good News

There is, however, a compromise that can work, and that is having the confidence to aggregate upwards knowing your project level data are sound. You might say, for example, that together your projects improved outcomes for 10,000 families, and then give a single example from an individual project that improved service access or well-being to support the claim. In most situations that will be more meaningful than any contrived, supposedly universal measure of impact.

Confidence is the key, though: for this to work you need to find a reliable way of measuring and expressing the success of each individual project, and have ready in reserve information robust enough to hold up to scrutiny.

Measuring Means Data

In conclusion, the underlying solution to the challenge of measuring impact, and communicating it, is a foundation of good project level data. That will also make it easier to improve performance and give you more room to manoeuvre. Placing your faith in a single measure, even if you can decide upon one, could leave you vulnerable in a shifting landscape.

 

Social impact webinar

Webinar: Make 2020 the Year of Social Impact

On both sides of the Atlantic, our clients are working hard to think of new ways to deliver vital services to their clients: the homeless, the refugee, offenders.  They are often the people who are most likely to be at the margins and their needs are urgent.  So, now is a time for adapting services to the weeks of lock-down and shelter in place orders. And at GtD we are adapting the ways in which we are delivering our services and helping our clients.

While we wait for the pandemic to subside, the needs of our clients and yours remain pressing, and that will continue long after CoVid19 becomes yesterday’s news. However, I doubt that we will go back to work just as we did before we heard about social distancing and self-quarantining.  When we do go back to work, we will be asked to do things in new ways, to answer new needs and to find new partners to work with.

2020 is GtD’s “Year of Social Impact” when we are encouraging non-profits and charities to think about new ways of demonstrating their social value – definitively. We started out with a roadshow of events in Atlanta and then London, meeting with leaders from the nonprofit sector, listening to their needs and telling them about how our “Measure”, “Learn” and “Prove” packages can assist them.

We were due to take the roadshow to Manchester on April 28th.  Instead, we will be delivering the event as a webinar and inviting people to attend from across the UK and the USA.  This is bound to be a fascinating meeting as we share experiences across the Atlantic and hear more about how GtD’s social impact analytics are helping non-profits on both sides of the ocean.

The webinar is free and we have made it easy to join.  To register, please use this link and learn more about how to attend.

At times like this, we need to keep going as best we can and re-evaluate our priorities. Today, I received some words of wisdom from a valued client and jazz aficionado.  He and I were talking about how best to address the needs of those who have no medical care in rural Georgia. We had to suspend some work and address the new priorities thrown up by lockdown, and so I would like to share his words with you:

“What we’ve been doing lately reminds me of an announcement Louis Armstrong made during an interruption in one of his concerts, “And while they’re fixing the mike, we are going to lay on you our very best version of ‘Tin Roof Blues’”. Getting important things done during times like these can be helpful both to our projects and to our morale”.

Jack and I hope to meet you at the webinar but please contact me on alan.mackie@gtd-us.com if you have any questions. In the meantime, we wish you all the best as you address your new priorities and redouble your efforts to serve your clients.

Learn from data

What You Can Learn From Your Data

In a world driven by data, it’s becoming increasingly important for social organisations to prove their impact – not just through what they say, but through cold, hard facts.

While many already collect information on the work they’re doing, there remains a gap.

There’s a limit to what you can understand from the numbers alone, and simply reporting the data can feel far removed from the real-world and the day-to-day work done by the workers in your organisation.

For your information to become truly valuable, you need to start unpicking it. By applying the right methods of data analyses, you should be able to learn from your data and to use the insights you draw out of it to make practical changes within your organisation. This approach to data analysis will allow you to optimise your resources and maximise your results.

More than that, you’ll be able to extract the story behind your successes, and use that to show the world the valuable social impact your organisation is achieving.

Empower your staff

Looking at the data alone, it can be difficult for practitioners to know where to concentrate their efforts, and where changes to their service are most needed. But with the right analysis, the data you already have can be transformed into immensely valuable information to guide practitioners’  decisions, and highlight areas for improvement.

Learning from your data can give your practitioners insights on their individual clients, on how the team is performing, and on the organisation as a whole. It means they’ll know exactly what you need to target, and how to fine-tune your service to achieve the best possible results.

Understand your beneficiaries

No doubt you have worked in your sector long enough to know that your clients are not all the same. By applying the appropriate analytical techniques, you can understand the differences between clients and tailor your services to their varying needs.

Methods like segmentation and cluster analysis allow you to pick out patterns across different groups of people with different characteristics, and identify not only “what works?”, but “what works for whom?”.

Predict future outcomes

Your data can tell you not just what’s happened to date, but the outcomes that your organisation could produce in the future.

This can be done by applying scenario and simulation analysis, to predict the potential results of a variety of situations. These predictions should update dynamically, changing over time as other conditions change.

What’s really valuable about these predictions is that you can use them to optimise your resources, and to expend them in a way that will have maximum effect. That’s important for any organisation to know, but particularly helpful for non-profits and social enterprises whose resources are often limited.

Know your social impact, and prove it

Learning from your data gives you the confidence of knowing – not just hoping – that what you’re doing works. It lets you know that you’re making a real impact in your field, but perhaps more importantly, it offers assurance to the people who are looking for that proof.

Your funders and stakeholders need to have confidence in what your organisation is achieving, and by analysing your data you can give them just that.

To take a  youth service as an example, a potential funder might want to know how much of a difference the organisation is making to youth unemployment in the area. In answer, it could use examples from the analysed data, and effectively demonstrate its improvement in comparison with a control group.

By telling the story of its impact, backed up with concrete evidence and analysis, the youth service would be able to provide a much more convincing argument than it would be, using the raw data alone.

Learn from your data with GtD

Get the Data works with organisations in the UK and the US that want to prove their impact on society. We provide social impact analytics in the form of three main packages: Measure, Learn and Prove.

Our Learn service sits in the middle, making it a perfect next step for organisations that have collected data on their service but need to understand it better in order to use it. “Learn” can also be used in combination with our “Measure” and “Prove”  services, in any order that suits you.

Beginning with a strategic review of your data, we’ll put together an insight strategy and follow up with regular reports to monitor the results – integrating those reports with your own systems so they’re dynamic, useful, and convenient to use.

If you know you could be doing more to learn from your data but you’re not sure where to start, take the first step and contact us today.  We will get you started on your social impact journey.

Use Data to Tell an Effective Story Get The Data

Use Data to Tell an Effective Story

Over the past eight years, GtD has been providing social impact analytics to tell our clients’ stories. Effective stories of what their interventions do, who they work with, and the difference they are making to the people they serve: the homeless, young people, public defenders, refugees and immigrants. But, while we are primarily using numbers rather than words to monitor and evaluate our clients’ work, we need to take care that we are following the golden rule of story-telling.

Golden Rule of Story Telling

The golden rule is that a story should have a beginning, a middle and an end. I’ve just finished Sir Ernest Shackleton’s “South”, the Edwardian explorer’s account of his trans-Antarctic exhibition. The story opens in the months just before the First World War, with the preparations for the expedition, all the charting and planning and the resources being loaded on to his ship “Endurance”. The middle of the story is about the embarkation from Plymouth and the ship sailing into the South Atlantic, and then Shackleton leading his crew across the mountains and glaciers of Antarctica with the constant charting and re-charting of their route to safety. The tale ends with the crew’s rescue and return home. While Shackleton did not achieve his stated objective, “South” provides a compelling account of how a leader at times must respond to adversity by altering course and deploying available resources to the best effect.

Many organizations that work to make a social impact use words to tell the value of their work. This is, often done in the form of case studies that tell their clients’ stories. I recently came across such a case study in the annual report of a London-based homeless charity. It tells a compelling story about the effectiveness of their work with a client referred to as “David”. David came to London looking for a job, but soon became homeless and was found by the charity’s outreach worker sleeping rough. Having set out the “beginning” of David’s story, the case study then provided an account of the outreach worker’s role in helping David find hostel accommodation and registering him with a doctor and the benefits agency. The story ended well for David, as he found employment, a home and was returned to good health. A simple but effective story that informed the reader about the charity and how it benefits its clients.

The Descriptive Power of Numbers

Like an epic adventure or a compelling case study, we can use numbers rather than words to tell an effective story. However, before we do so, we need to get our story straight and give some thought to which data should provide the “beginning”, the “middle” and the “end”. If we get this right, then the story can follow the golden rule. So, in telling David’s story with data, we should start at the beginning and identify all those things that contributed to the delivery of the outreach service. We call these “inputs”, and they might include the number of outreach workers employed, their shifts, and the training they received.

Next, we will want to find the numbers that describe the services that David received. These are known as “outputs” and will include the number of contacts with the worker, the time taken to make referrals to the general practitioner and the benefits agency, and the contacts with the local hostel. Finally, we need to find the numbers that will describe the “outcomes” that were achieved for David, which will include changes to his employment, accommodation and health. Of course, we will be collecting these input, output and outcome data for David and every other client seen by all the outreach workers.  This will become an aggregated database of information that will tell a much broader and compelling story of who the charity helps, how it helps them and what outcomes their clients achieve.

Telling your story with GtD

Telling an effective story is essentially what GtD’s “Measure” package is all about.  It will help your organization monitor its activities and evaluate its outcomes. Contact us today if you want to learn more about how “Measure” will help your organization realize the potential of its data. “Measure” will assist you in deciding what to measure and what data to collect. It will provide you with analyses to solve problems and achieve better outcomes. “Measure” doesn’t just give you data, it gives you data to tell an effective story.

If you’d like to learn more about social impact analytics and how it can benefit your organisation, come along to our free event in Manchester on 28th April. Visit our events page to learn more and book your place.

A New Year’s Resolution?

A New Year’s Resolution?

New Year’s Eve, and time to take a moment to make a resolution for 2020.  Last year, according to the Statistica website, 71% of Americans resolved to diet and eat better in 2019 with a further 65% intending to exercise more.  It is not unexpected, perhaps, that 8% of those making resolutions failed to keep it within hours of the New Year being rung in, but a whopping 16% claimed to have kept their resolution for two to four months!

New Year’s resolutions are about making changes and adopting better habits. They are often personal in nature, but have you thought of making a New Year’s resolution for your organization? In the course of 2019, have you ever wondered how to measure the difference your organization is making on society?  As we peer into 2020, are you curious to see how your organization can improve the services it delivers to your clients? Is it now time to get the evidence to demonstrate your organization’s impact – definitely?

If the answer to any of those questions is “yes”, then come along to our free seminar on Wednesday, 8th January in London and learn more about “Measuring” your activities; “Learning” how to make a real difference; and, “Proving” your impact definitively.  For more details and how to book your place for this event, please go to http://bit.ly/SocialImpactSeminar.

Resolve to make 2020 the year you and your organization improves its social impact – and be sure that GtD will be with you every step of the way so that this will be one resolution you will keep.

Join us for a social impact journey seminar

Book Your Winter Getaway – Now!

The holiday season seems to be a time for journeying. As the school nativity plays remind us, the first Christmas saw the little donkey carry Mary and Joseph to Bethlehem and the “star of wonder” led the Three Wise Men to the stable. Today, thousands of British and American travelers will drive and fly home to be with family for the holidays.  And, no sooner is the festive period over, than hundreds of us will be heading to the ski slopes and beaches for a dose of much-needed sunlight. Winter is, indeed, a time for journeying.

At this time GtD’s London-based team invite you to join us on a journey. Unlike the Winter Getaways you see advertised in the press and television, we are offering to take you on a journey to understand your organization’s impact on the communities and people it serves.  At GtD we call this your “social impact journey” and we will travel with you to gather data, unlock its meaning and use it to demonstrate the effectiveness of your work.  Each of these is a stopping point on a single journey.

If you are interested in learning more, then you are warmly invited to join a free seminar on Wednesday, 8th January in London. For more details and how to book your place for this event, please go to http://bit.ly/SocialImpactSeminar.

GtD’s “Winter Getaway” doesn’t promise you golden beaches or pristine snow, but it will give you a good start to the new year as you and your organization embark on your social impact journey.  We look forward to seeing you there.