• Sanjukta Moorthy

How to Judge the Quality of Evidence

I came across this rubric for how to judge the quality of evidence within your data set. It's a useful five-standard guide to help you ensure that the data underpinning your reports and claims to social change. Thomas Aston wrote it.

The first component is plausibility. The data you present should have a clear and logical thread, explaining the relationships between what was done and its change.

The second is uniqueness in this connection. Proving how a specific intervention led to or helped lead to a specific change is strengthened by showing a unique link between the two. The more confident you are in showing that a specific action led to a specific outcome, the more this component is strengthened, and the more your evidence is strengthened.

The third is triangulation. This is a fairly straightforward principle since it's a central component of validating your data. This step allows you to look at multiple lines of evidence, perspectives, and data sources to ensure that your evidence is accurate. Triangulating data also helps to ensure consistency and prevent biases.

The fourth is transparency. This is not only external - sharing your data with all your audiences - but also internally ensuring that you and your team are confident in your data, you know where it comes from and who and how collected it. Examining these sources, you can also review the limitations of the data and what questions about your project it does not, or cannot, answer.

The final component is independence. Look at whether your data is self-reported, which is largely seen as lower in quality due to its inherent bias, or whether it comes from external and independent sources such as the communities or local governments involved in the project. These actors are more likely to be more independent, and the data generated, therefore, more likely to be seen as reliable.

Please take a look at the guide, which Thomas and his team formulated through triangulation and verification! I hope you find it a useful checklist for your data cleaning!

Quality of Evidence Rubrics
Download PDF • 227KB