Five tips for evaluating social innovation in response to coronavirus

“Never let a good crisis go to waste” – Winston Churchill

The silver linings from the COVID19 recession are delicately emerging around the thundering clouds of this crisis.

Social innovation is blossoming

All over the world, community mutual aid groups are forming in the absence of institutional actors.  Supply chains are adapting and familiar brands like Burberry, Dyson and Teslar are reacting swiftly to produce much needed medical equipment.

Some countries like Ethiopia are opening the financial sector to enable more mobile money.  Arts, culture, education, sports and leisure services are fast adapting to provide online services.

Our charities, social enterprises and social services are also feeling the imperative for innovation.  Reduction in income streams from funders delaying or withdrawing, from household cutbacks and from an immediate halt from subsidiary income streams like room rental or paid services mean many will move into survival mode.

This comes at a time when demand for their services will be greater than ever.  The challenge facing the social sector is to create more positive impact with less investment.  It is a time to embrace flexibility and to allow for new learning.

Institutional donors are being asked to permit changes in contract deliverables, timelines and reporting.  Emergency funding should hinge on trust and regular communication.

Impact management and impact measurement are part of innovating

It is tempting to assume that impact reporting can also be moved to the backburner.  However, impact management is just (if not more) important during periods of innovation than established delivery.

Quick and timely feedback will be critical to ensure emerging social needs are being met.  Similarly, funders will be making difficult decisions during this time about effectiveness.  They will be looking across their portfolios and asking which services provide essential impact.

The good news is that impact reporting can be nimble and combined with innovation to make sure we do not end up adrift during this period.  Two methodologies are worth borrowing from during this time: Developmental Evaluation  and Most Significant Change (MSC).   I have pulled five key tips from the two approaches to bake into any COVID19 adaptations to social services:

1.       Gather timely data

During periods of innovation you want to be collecting less impact data, more frequently.  This means your reporting period could be a week – it could even be a day.  You should collect short-term feedback and quickly analyse and consolidate the learnings.

The diagram below illustrates how the action learning cycle, which might take a year or more for an established programme, should be sped up for rapid feedback during the innovation phase.  As your project matures with more consistent features, the length of time for a typical review period and the sophistication of evaluation methods should both increase.

Remember to keep good records in a database and note when changes in program design took place.

2.       Short qualitative data points

The data you will be collecting needs to be easy to manage.  I would advise focussing on a small number of open-ended questions. These should gather feedback on change (outcomes) and why or how those changes are happening.  Try to sketch up a rapid Theory of Change to guide your questions and return to this as you gather data.

To give an example of what the questions look like, for an education service, the question could be as simple as “How much did you know about X topic before the intervention, and how has that changed now?”.

Prepare outcome questions on a few focal areas, then cycle between these with different stakeholders.  This way you reduce the burden of questions on each stakeholder or interaction.  It is worth noting that lighter data collection does not mean you can go light on rigour.  Consider bias in terms of your sample and question structure.

MSC invites the collection of stories to understand what is changing and why it matters.  This matters to avoid feedback becoming soundbites.  You can create mini case-studies to preserve context.  You might cover the background of the interviewee, what happened for them and the difference it has made.

3.       Hit the telephones

Physical distancing during the coronavirus lockdowns means speaking to stakeholders face-to-face should be minimised.  This is not a reason to avoid collecting data.  Service users may even be more receptive than ever to forming a contact and more inclined to provide feedback.

Collect phone numbers or provide videoconference invites to connect remotely with users to conduct short interviews or surveys.  Remember to take time to establish a connection and build trust before diving into feedback as this can sometimes be a little harder over the phone.

4.       Focus on what is working

Evaluation calls for the consideration of positive and negative impact.  When you are innovating it is important to acknowledge what is not serving your stakeholders, but this is not the time to measure it.

Be guided by what is creating positive change for the most stakeholders.  Dig into why it made a difference and how you can amplify that positive outcome.

5.       Involve everyone

Innovation gurus tell us the best ideas come from everywhere.  The same goes for evaluation insights.

Both MSC and Developmental Evaluation emphasise the importance of involving your stakeholders in not just the data collection, but also the data analysis.  They both encourage you to chew over the findings with people from every level within your organisation.  If you have a close funder, talk it over with them too.  This is the way you will generate new solutions, scale what is working and build new collective solutions with partners.

Final thoughts: five tips for evaluating social innovation in response to coronavirus

The social sector will be under immense pressure over the next few months.  It will take a collective effort from all directions to safeguard the wellbeing of those most vulnerable.  Guesswork is not a smart approach to creating impact during normal times and it should not be relied upon now.  Simple, light impact measurement techniques can be built into even the earliest stages of social innovation.