Three key challenges to progressing outcome assessment

Challenge 1:

---

Working out what outcomes you care about and how your activities help to achieve them.

Like many organisations, you most likely have some outcomes that are important to you expressed in your mission, programme plan, or agreed with funders. But do you know how your activities contribute to achieving these outcomes? This requires serious thinking about how your activities engage with the people and communities you care about, how they react, and what learning and capacity building is needed for change to happen. Your staff teams, service-user representatives, funders and stakeholders all need to agree on key activities, and pathways to key outcomes.

We work with organisations to map their outcomes to their activities. Often there is agreement and pathways are clear. This is especially so for new projects or programmes, or ones where work has been done to express clearly a theory of change. However, where an organisation has been working for many years for multiple funders, or where there is greater diversity in staff and stakeholders, or where there is less clarity about key organisational activities, then it can take some time for everyone to agree and be happy with a map of these outcomes, and with how their work fits.

In some cases outcome mapping may throw up issues about whole areas of activity and whether or not these fit with the organisation’s mission.

Having a clear picture of how activities map to outcomes is the framework we use as the basis for assessment, so for many organisations discussion is needed to get to a point where there is enough clarity to move on to thinking about how progress will be tracked.

Challenge 2:

Reviewing what data and information you have about progress and whether it’s a good fit.

Once clear pathways from activities to outcomes have been mapped, it’s time to take a look at what data you’re collecting and how this fits with your outcomes framework. In our experience this always throws up a lot of issues. Ask yourself this:

• Do you know where all the relevant data and information about progress is, who collects it, and how often?
• Do you have many different, overlapping data, information and feedback systems?
• Are you counting numbers of engagements, but not measuring the quality of these engagements?
• Do your existing outcome measures have limited information in them?
• Do you have really rich data in case notes, but no system for aggregating these for learning purposes?
• Are there different feedback mechanisms in different parts of the organisation?

If you answered yes to one or more of the above – you are not alone. We commonly find these issues and we support organisations develop a plan for resolving them that creates the grounds for a robust outcomes assessment process.

Once clear pathways from activities to outcomes have been mapped, it’s time to take a look at what data you’re collecting and how this fits with your outcomes framework. In our experience this always throws up a lot of issues.

Our approach is pragmatic – there is no point in searching for perfect outcome measures. Instead we look at creating robust programme logic backed up by a mix of routinely collected and research-based measurement alongside specially designed data collection that together can tell a strong story of impact.

Challenge 3:

Finding the capacity in your organisation for tracking outcomes.

Having enough capacity to carry out outcomes assessment is an issue for many organisations. Some have internal research or analytical capacity that can support the process, but for many it is a case of trying to identify that capacity within their teams.

The practicalities of delivering programmes often absorbs most of the organisational capacity leaving little time for outcome assessment. It can also be challenging to find the time and space to think about evidence and how to analyse it to understand progress in busy delivery-focused organisations. We find these pressures are eased when organisations recognise the potential of outcome assessment for learning, improvement and effectiveness, as well as being more able to report to and attract funding, and prioritise it. It can also be helpful to start small – building capacity and apply learning from one part of the organisation to another.

We established Matter of Focus because we wanted to support organisations through these challenges. We work with organisations to map outcomes, help with data auditing and designing new feedback collection mechanisms, and our software OutNav helps structure and store relevant data, gives a framework for analysis.