When public service organisations focus on relationships — work that educates, empowers, inspires, supports, encourages or influences people — then simple measures to understand change just don’t work. They often hide the complexity of what’s going on, and sometimes undermine the ability of these vital approaches to make the case for their work.
This challenge is at the heart of the work we do at Matter of Focus.
We have worked with over 150 organisations and know many more face the same challenges, which is why Matter of Focus Directors Sarah Morton and Ailsa Cook brought together learning from this experience alongside a guide to our approach in a new practical handbook ‘How Do You Know If You Are Making a Difference?’
To mark the publication of the book, we brought people from across public services together for an afternoon of discussion and reflection on the importance and challenges of knowing that you are making a difference, which we summarise in this post.
Why is it important to know you are making a difference?
Mental health support is all about relationship building. That’s why Penumbra, a pioneering mental health charity, have been working with our approach.
Stephen Finlayson, Penumbra’s Head of Innovation and Improvement, and Michael Burns, their Chief Executive, opened the event by telling the story of their journey to focusing on outcomes.
The Matter of Focus approach is based on setting out a theory of change in an ‘outcome map’. As Stephen pointed out, every organisation has a theory of change — that is, they hold beliefs and assumptions about how and why the work they do makes a difference to people — it’s just that these are not always made explicit. As Penumbra found, setting out an outcome map means everyone can be clear about what is important.
Using our six simple headings helped Penumbra set out their theory of change.
Focusing on outcomes in this way is essential if we are to meet the demands for services in the current environment, Michael argued. If outputs are constantly scrutinised rather than outcomes, we just keep needing to deliver more and more:
“We need to move away from activities to an outcome approach. Budgets will run away from us if we keep focusing on what we deliver rather than outcomes for people.”
Michael Burns, Chief Executive, Penumbra
A storm of pressure in the current crisis
People at the core of public services, designing and delivering work that makes a difference to people, are facing some acute challenges. They often see the difference their work makes but know that funders are looking for short term gains — and asset-based, person-centred, preventative work often can’t demonstrate this. The time invested in developing and nurturing effective relationships and partnerships is completely invisible to the system. Worries about cuts to budgets, the impact of the cost of living crisis, how staff are coping and what’s happening to the climate are dominating lots of people’s thinking.
This storm of pressure is in some ways predictable. The Christie Commission is more than ten years old, but set out the need for public services to innovate, focus on what matters to people and develop partnership and relational responses to complex problems. The current crisis has only accelerated this need. Knowing that you are making a difference is even more important and performance management approaches are failing even faster.
A shift in thinking is needed for this new environment.
An alternative approach to understanding change
“It is possible to show you are making a difference.”
Stephen Finlayson, Head of Innovation and Improvement, Penumbra
Acknowledging that multiple factors influence change and moving away from reductionist approaches is essential to understanding this work. Sarah Morton set out the ways that this can be done using complex system approaches, valuing relationships and using contribution analysis.
Sarah and Ailsa explained how we support people through a process of analysing the context for their work, mapping outcomes, bringing together data and evidence against the outcome map, and collective analysis.
All of these processes are part of sensemaking — people having the time and space to work together to reach common understanding of whether and how they are making a difference, what their data is telling them, where they can learn and improve and what they might do differently.
Bringing together learning for improvement alongside tracking impact for external reasons and outcome focused reporting helps organisations build a learning culture, deliver better, and tell a powerful story of the difference they make.
Our approach in action
Six examples of our approach in action were shared during the event:
Outcome mapping in action: Making Children’s Rights Real
The Observatory of Children’s Human Rights Scotland and Public Health Scotland needed to understand how adopting and implementing the United Nations Convention on the Rights of the Child (UNCRC) would make a difference to children in Scotland. We took a participatory approach to map this out, involving policy makers, practitioners, children and young people and third sector organisations. We drew on formal evidence along with evidence from experience.
We set out four pathways for change, seen below. This was a systems change project, so it was important to consider the supporting measures beyond the direct implementation of the system of information, advocacy and legal redress.
Tracking progress in action: Health Technology Wales
Health Technology Wales was an early adopter of our approach. They set out outcome maps for their main areas of work, and we supported them to improve their data with some simple approaches, including a pop-up on their website, capturing informal feedback more systematically, and implementing a simple annual stakeholder survey.
Working in this way led to Health Technology Wales winning the David Hailey Award for the Best Impact Story at the International Health Technology Awards.
Reporting in action: East Renfrewshire Health and Social Care Partnership
Reporting on outcomes in a meaningful way is the cherry on the cake from working in this way.
When East Renfrewshire Health and Social Care Partnership launched a new peer support service, they knew it was the right thing to do, but in the context of tight resources they needed solid evidence to justify the commitment of resources to the project. They went with us on a journey of mapping outcomes, tracking data and with our help produced an outcome focused report that showed how this service was contributing to improved outcomes for people.
The report highlighted that relationships between peers and the people they supported were key, and showed evidence of real improvement in outcomes for people. This led to the partnership committing to funding the service for another three years.
Tracking change across multiple initiatives: Transforming Local Systems
The Technology Enabled Change (TEC) Pathfinders project supports local areas to address a health challenge that is important to them: encouraging local innovation, cross-sector problem solving, and a focus on what matters to people locally. This created a challenge for the Scottish Government, as shared by Transforming Local Systems Lead Margot White, in understanding the contribution to the overall change.
Working with OutNav (the software we have developed to hold our approach) enabled each place in the programme to set out and track the difference they made, focusing on what mattered locally, while also creating congruence across the projects.
“Focusing on what mattered to people enabled us to change the programme – we could see what people wanted and make changes to meet that.”
Margot White, Transforming Local Systems Lead
Bringing the project teams together for virtual meetings to work on data analysis collectively was also great for team building despite the geographical spread, and for building collective understanding of the approach and what was happening in each place.
Owning your organisation’s outcomes: WHALE Arts
Like many third sector organisations with multiple funders, leaders at community arts organisation WHALE Arts felt that there was outcome drift as they flexed to meet these requirements.
Working with OutNav over a few years, initially as part of our collective support for Edinburgh Community Health Forum, WHALE Arts has gone through several iterations of their outcome maps in order to realign their outcomes to what matters to the people they support.
“OutNav has helped us become experts in our own outcomes, to approach funders with robust evidence of what we do, and to secure funding for the things that matter to us.”
Kirsty Frankland, Arts and Wellbeing Manager, WHALE Arts
Kirsty explained some of challenges to embedding the approach, through a process of choosing outcomes, defining them, learning and refining. Bringing together multiple forms of evidence into OutNav has been important for WHALE to highlight how they make a difference to people. Some challenges remain, and their strategy is to refine with a clear focus on what matters to people
Making the case for a new way of working: Future Pathways
Future Pathways was a new person-centred approach to supporting survivors of in-care abuse in Scotland. Putting people at the centre of support seemed important but had not been tried in this way before. This led to a lot of scrutiny not only from the funders, but also from survivors themselves who were understandably mistrustful of services, and cared deeply about how any government money was being spent.
“Outcome mapping allowed us to test assumptions early on, and take an iterative approach to delivery,” said Flora Henderson, Alliance Manager for Future Pathways. The fact that the Matter of Focus approach focuses on relationships was a great fit. Using evidence helped Future Pathways evolve and change their offer, and demonstrate impact.
Embedding the approach created space for managers and staff to look at data together, so that learning and improvement was embedded in the work. They were able to successfully make the case for more funding.
“Bringing quality improvement and a focus on impact together from the start has helped to build a quality offer with clear benefits to people.”
Flora Henderson, Alliance Manager for Future Pathways
Key themes from the discussions
It was great to launch the book and share these discussions with more than 60 people who brought massive expertise, experience and commitment to knowing that they are making a difference. Some strong themes emerged during group discussions, including:
The importance of sensemaking.
Many people described how taking the time to really think through change processes and learn from emerging evidence had been transformative. There were lots of great examples in the room of organisations creating spaces for reflective practice. People recognised that these kinds of sensemaking spaces can be the first things to go when initiatives are under pressure.
The importance of understanding the data culture you are working in.
Participants reflected on the diversity of data cultures that they worked with and how rarely assumptions around data were surfaced through day to day work. This could be very problematic, and participants described times when unreasonable expectations around data and measurement had got in the way of efforts to make change. Several people said they would approach working across data cultures differently in the future.
The importance of making time and space to learn how to know you are making a difference.
Each presenter reflected that their approach to knowing they were making a difference had evolved over time. In every case the process had taken more time than they anticipated, but had yielded more learning than they thought along the way.
This really resonated with participants’ experiences of transforming their organisations in different ways.
We are really grateful to our presenters and everybody that joined us for such an inspiring and interesting afternoon of discussion. It certainly cemented our belief that taking the time to come together really does make sense.
Join us for our virtual launch of How Do You Know If You Are Making a Difference?
12 January 2023, 4 – 5 pm (GMT)