Storytelling as an Impact Measurement Tool
PMEL methods often rely on predefined and rigid quantitative metrics. These can be helpful to track and report on social change in a simple, standardised approach.
However, measuring real change is far from a simple, linear, or predefined line. This is where quantitative data and indicators often miss out on the nuances of an organisation’s impact.
In many cases, focussing only on quantitative indicators and measures of progress comes at the expense of insights from qualitative data. This can include ideas and insights from your communities' lived experiences. It's rare to prioritise specific communities' voices and make their experiences and stories central to an impact measurement matrix. Here is where storytelling as a methodology can be a wonderful way to embed these insights into your M&E frameworks to help create and track dynamic indicators of success.
It's likely that you and your partners already have many case studies, stories, and examples that can speak to your impact and your projects' results.
But it's also likely that these aren't being used as systematically or efficiently as they could be to further your PMEL. Adding them to your frameworks, though, can help us understand the complex realities of our work and the impact trajectories of our projects. Stories can provide us with meaningful information for decision making.
Here are three practical tips for how you can use this kind of data more effectively:
1. Define your sample, and build a data collection strategy
Collect stories of impact grounded in people's realities.
Before collecting stories, you need to take a step back and plan. Here's where you can begin with a sampling and data collection strategy, linking it with your mission, strategy, needs, and resources. This will help you ensure a diverse and representative set of voices are presented in your work.
There is no one-size-fits-all approach to sampling or data collection, but my preferred methodology of participatory data collection targets vulnerable people. These groups are often left out and silenced, like marginalised groups, children, those with literacy barriers, etc.
For instance, during participatory storytelling workshops, creating a safe space and selecting the right questions can ensure everyone feels safe, engaged, invited, and compelled to speak up and be heard. If managed correctly, these workshops can offer a safe space for everyone to share, listen and reflect, empowering participants and organisations.
Prioritising their voices can also help ensure you create safe spaces for them to share their perspectives.
This also means that crafting deliberate and thoughtful questions can help you get the most meaningful information. Make sure that whatever you are asking these groups relates to the intervention you're evaluating and the contributions of different actors to a change process. For example, an education project's impacts will need to ask questions about how the interventions have improved teachers' work, the children's work and education levels. If relevant, it has also gathered more community support.
This can help turn your data collection into a meaningful personal reflection process and support your PMEL systems.
Sometimes, you may not even need to collect stories right away. In some cases, you may already have data that is lying dormant in your cloud folders - hidden in grantee or company reports, emails, or even meeting notes!
2. Create a story and outcome database
This can transform voices into indicators to help show aggregate level impact trends.
The next step is to create an outcome dataset, gathering all the stories to bring an analytical lens to your work. You can code the data yourself or as part of a team, connect the dots, and unearth deeper trends to tell your own stories of impact.
You can choose either a deductive, inductive or mixed approach to develop your indicators or metrics.
Inductively, when you’re coding your text or story data, participants (those whose stories you gather) create their own outcome indicators based on the changes they identify in what’s been shared.
This is a great bottom-up approach to indicator development and relies on communities defining what success looks like to them and developing their own indicators.
This can really show an interesting shift in power dynamics in monitoring and evaluation and impact management.
This may not always be relevant or appropriate, so that you may prefer a deductive approach to coding. This relies on you as the implementors or evaluators taking a set of pre-existing indicators and metrics and forming these as the core of your coding methodology.
These indicators could come from your Theory of Change, logframe, KPIs, or a standardised framework like the SDGs or IRIS metrics. This is also a great place to add indicators from your project or programme or even organisation-level outcomes and objectives.
A mixed approach uses both approaches and is the best of both worlds combining deductive and inductive techniques. This gives you a lot of flexibility to code data according to pre-determined indicators like the SDGs and provides some room to code according to any unexpected insights that arise from the process.
Start coding by analysing the stories with a focus on identifying the positive or negative changes, shifts, or instances of maintaining past gains described within and strategies that led to success.
Unearth trends by coding these shifts according to your framework and aggregating the overarching insights across various stories.
This gives the frequency of outcome trends across a range of different stories - providing an actionable dataset that can be analysed in various ways: over time, across regions within a grant, impact investment, project, etc.
You will need to decide how to keep track of this data, though I prefer Microsoft Excel or Google Sheets. Both options are easily shared across teams to code and analyse your data within your teams and with your partners.
3. Communicate these impact trends
Share exemplary stories and more nuanced case studies.
Once you’ve coded your data, you will have a set of trends and changes that are the most frequent across the stories.
You can add this to a common story database, updated at quarterly or semi-annual intervals and added to publications, donor reports, and social media too.
These are a great source of rich material that can serve as a public advocacy tool for your audiences.
You can use these to highlight overall trends affecting your context while also getting into specific detail about certain experiences and projects.
Think about the audience here and their needs – what is the best way to share your stories with them?
To ensure stories are heard and acted on, it is important to communicate and report strategically. Who are your audiences? Who do you want to move and how? What are some of the most compelling ways to share this data?
I hope this is useful for you! If you’d like to hear more about how storytelling can help your existing PMEL frameworks and be a key part of your communications strategy, join my storytelling workshop!
An earlier version of this article was written for ImpactMapper