Seven steps to understanding success

Through our work with public service organisations we have identified seven steps that organisations need to take to understand how successful a project or programme is. These steps are also important to help the processes of planning, learning, reflecting and monitoring. They can also provide a foundation for ‘telling the story’ of how change happens, to funders, service users or other stakeholders.

1. Clarify the outcomes that matter to you, the people you support and your funders

Understanding the what kinds of outcomes you are seeking and why is essential. This means interrogating and thinking through the outcomes that are important to your programme or organisation and why. What are the outcomes for? How do they help focus the work of the organisation or programme? (Take a look at our earlier post Why focus on outcomes?).

2. Map how activities contribute to outcomes

Next you need to understand how the activities your organisation or programme offers lead to these outcomes. This is key to achieving them and judging their success. We have built on work by Steve Montague¹ to understand the change process. Our approach to mapping a contribution to outcomes means understanding how the people who have engaged with a programme or organisation feel, what they learn and gain, and how this leads to changes in behaviour and practice, or increased capacities. This approach builds a solid basis from which to understand and judge improvement in outcomes for people and communities.

3. Understand what has been successful in the past

Learning, improvement and innovation are all important in public service delivery and development. It’s important to build on what is known to be successful. That knowledge could come from existing research, from evidence developed through practice, or by looking at what other similar organisations or programmes have done in the past. Both ‘what works’ type evidence and thematic reviews can be really helpful for this (for example the Centre for Research on Families and Relationship’s About Families project). This knowledge can help refine the thinking behind the overall approach, and to adapt it if necessary.

4. Audit the evidence you already have

Are you drowning in data and evidence collected through routine activities? Many organisations are; data about service-users, referrers, exit data or evaluation. Bringing this together to tell the story of how your organisation or programme makes a difference for people and communities can be challenging. Assembling existing evidence against an outcome map is a good basis for building this story.

5. Understand gaps in your evidence

It’s often the case that existing evidence can say something about how many people have used a service, but less about what difference it has made for those people’s lives, or how change has happened. Four questions are important here:

• What are the key gaps in evidence in understanding the change process?
• Where is evidence weaker or stronger, and how can you improve weaker evidence?
• Are there opportunities to triangulate (see definition) different evidence types or collection methods to improve the robustness of the process?
• Are you collecting data or evidence that is not helping you learn, improve or tell the story of the difference you make?

6. Make a plan and fill evidence gaps

Once you understand the gaps in evidence, address them to create a more complete and robust picture of how change happens. At Outcome Focus we believe that any new data collection approaches should reflect the perspective of the people who have the most insight into how it works. That would include service-user feedback that is collected and used in a way that’s appropriate and proportionate to the people and programme/organisation in question. But importantly it would also include thinking about how to formalise the insights of practitioners who deliver the service. One of the tools we have developed is a reflective impact log that captures practitioner thinking, alongside a group process to refine that evidence and use it more formally.

7. Embed a process of learning, reflection, discussion and adaptation

In a time of shrinking budgets and entrenched inequalities, the tide seems to be flowing away from external, standalone evaluations that are often quickly forgotten about and left on the shelf. Instead, organisations and programmes are becoming more adept at self-reflection, and attuned to learning as they develop to enhance the chance of success. This needs to be supported by organisational structures that enable evidence gathering, reflection and discussion, and processes to ensure everyone has opportunities and clear criteria on which to reflect and judge progress. These processes of internal evidence-use can help to ensure that working in an outcomes-focused way is embedded into organisational culture.

At Matter of Focus we offer strategic support to understand and map outcomes, alongside our cloud-based software OutNav that can help organisations embed this way of working in their culture in order to be able to plan, learn, reflect and report on their outcome based work.

Notes

¹Steve Montague (2008) Results logic for complex systems. Performance Management Network Inc. pdf download