Valuing Different Perspectives (April 2014 – November 2015)
The project sought to evaluate previous co-produced interventions produced by researchers and community organisations in the neighbourhood of Wester Hailes, west Edinburgh. In doing so, it also built in a comparison of academic-led and co-produced approaches to evaluating the outcomes of the previous co-produced outputs. This enabled a comparison of the usefulness of different types of knowledge that were produced.
Additional outcomes included a co-produced report on evaluating research as well as the two project evaluations.
Project report: Doing and evaluating community research: A process and outcomes approach for communities and researchers (2015)
Project evaluations:
Engaging Wester Hailes – Findings from the Valuing Different Perspectives Community Evaluation
Keywords: evaluation, community development, social history, food, methodology
Contributed by: Peter Matthews, University of Stirling + Voluntary sector organisations in west Edinburgh
Measuring the Mountain: what really matters in social care to individuals in Wales? (2018-2019)
Wales’ co-produced Social Services & Wellbeing Act (2014) is based on the principles of voice and control, requiring commissioners and providers to take a relationship-based approach to the delivery of social care in Wales. This co-productive approach provided the starting point for the Measuring the Mountain evaluation project. A Citizen Jury was created in order to better understand social care provision through the eyes of the people who need it, and the carers who help provide it. This project brought together experts in policy, research, public engagement, co-production and social care, and put the expertise of people who are receiving social care support, along with their families and carers, at its heart.
SenseMaker® technology was used to enable volunteer Listeners to have hundreds of conversations with the diverse communities of Wales, recording people’s experiences of social care. The stories revealed where the Act is working well and where things could be done differently. Findings from these stories informed a three-day Citizens’ Jury event with citizens and expert witnesses looking more deeply at key aspects of the Act’s implementation.
The Jury’s findings will contribute to Welsh Government’s understanding of both the implementation of the Act and the shaping of its future.
See also: Evaluation of the Social Services and Well-being (Wales) Act 2014
See also: post on Measuring the Mountain
Final report: http://mtm.wales/ckfinder/userfiles/files/MtM%20Final%20Report%20March%2019%202019.pdf
Citizens’ Jury Report: http://mtm.wales/resources
Website: http://mtm.wales/
Key words: Citizens’ Jury, citizen stories, co-production, social care, policy evaluation, SenseMaker®
Contributed by: Rachel Iredale (University of South Wales), Katie Cooke (RCT Interlink), Beth Smith (RCT Interlink).
Patient Experience and Evaluation in Research (PEER) Group (ongoing)
The Patient Experience and Evaluation in Research (PEER) Group operates within Swansea University and aims to provide a valuable patient/carer perspective at an early stage in the research process.
The name PEER was chosen by the group members and reflects our identity as patient voices. We are members of the general public providing a patient/carer perspective. We promote the notion of active patients engaging in discussions about the design and direction of health and social care research in order to add value to the College’s research environment.
The Group meets on a monthly basis to discuss and make recommendations about proposals submitted by researchers. Submissions can be at a final draft stage or at pre-proposal stage and all researchers receive both written and verbal feedback intended to strengthen the value and impact of the research.
For examples of PEER-supported research see:
- other Michael Coffey entries
- Amy Pritchard
- Ed Lord
Keywords: patient experience, carer experience, research evaluation
Contributed by: Michael Coffey – Swansea University
A participatory evaluation of the work of Cartrefi Cymru using the ‘Most Significant Change’ approach. (2018 – ongoing)
This is a participatory evaluation of the changes brought about by the work of Cartrefi Cymru Cooperative. The evaluation is based upon the “Most Significant Change” approach and reflects the co-operative and participatory ethos of the Cooperative who support people with learning disabilities across Wales
The learning which is emerging from the evaluation is the consequence of an iterative process of action followed by reflection, involving significant stakeholders, at key stages.
Expressed in simple terms the Most Significant Change approach involves eliciting stories from key stakeholders, which they feel say something important about the changes brought about by a project or other intervention. These stories are then ‘valued’ by other stakeholders through a systematic process of discussion and prioritisation which allows an exploration of why some stories are valued more highly than others. This process allows significant outcomes to be identified, and underlying values to be explored and made explicit.
For Cartrefi the structure is as follows:
- The overall process is steered by an evaluation Board made up of Cartrefi staff and an external facilitator/researcher.
- Staff, people we support, and community supporters meet at three regional forums across Wales and tell their story in response to the question: ‘During the past year, what is the most important change you experienced as a result of Cartrefi Cymru Cooperative’.
- A total of six stories from staff/community members, and six from people we support, then proceed to the next stage.
- The twelve stories are considered by a group consisting of a member of staff and a person we support. Four stories are selected to go forward to the Cartrefi Board for further discussion. This process happens three times across Wales. In addition, a content analysis is carried out of all the stories to draw out the most important themes.
The process results in the following:
- Members are able to tell their stories to each other and celebrate success. This helps them feel heard and valued as well as enabling them to express what it is that they value.
- The structured process of discussion and selection of stories allows Cartrefi to identify what outcomes are most valued. This allows the core values of Cartrefi to surface.
- The content analysis is more systematic and allows a more quantitative expression of the main outcomes achieved by Cartrefi.
Keywords: participatory evaluation, most significant change, valuing changes, value surfacing
Related publications: The Most Significant Change (MSC) technique, a Guide to Its Use 2005, Rick Davies, Jess Dart
Contributed by: Alain Thomas – Independent facilitator; Martin O’Neill – Cardiff University
Defining, achieving, and evaluating collaborative outcomes: a theory of change approach (2017)
Governments have repeatedly claimed that collaboration improves public service outcomes. However, defining, achieving, and evaluating collaborative outcomes is often problematic. Analysis of multi-sectoral projects in Wales, which were supported by the European Social Fund, exemplifies these challenges. Shifts in policy discourses and the interplay between national and local agendas produced complex and contested understandings of outcomes which made difficult to evaluate the projects’ achievements. We argue that the pursuit of collaboration needs to be understood not simply as an attempt to improve public service effectiveness but also ‘cultural efficacy’. The conclusions offer reflections relevant for theory and practice.
Keywords: collaboration, public service outcomes, cultural efficacy, evaluation
Contributed by: Valeria Guarneros-Meza – De Montford University; Steve Martin, James Downe – Cardiff University.
User and Community Co-Production of Public Services: What Does the Evidence Tell Us? (2016)
International Journal of Public Administration, 2016
Tony Bovaird, Elke Loeffler
Much of the current discussion of user and community co-production makes strong claims for its potential to improve outcomes. How much is actually known about the level, drivers, and potential effects of co-production? In this article, some of the key claims made for co-production are examined and an assessment is made of how they stack up against the empirical evidence. In particular, some areas are identified in which practice must be cautious about the potential contribution of co-production, and where further research is needed.
Keywords: community co-production, co-producer characteristics, co-production drivers, user co-production
International Review of Administrative Sciences, 82:1 2016
Tony Bovaird, Gerry Stoker, Elke Loeffler, Tricia Jones and Monica Pinilla Roncancio
Previous research has suggested that citizen co-production of public services is more likely when the actions involved are easy and can be carried out individually rather than in groups. This article explores whether this holds in local areas of England and Wales. It asks which people are most likely to engage in individual and collective co-production and how people can be influenced to extend their co-production efforts by participating in more collective activities. Data were collected in five areas, using citizen panels organized by local authorities. The findings demonstrate that individual and collective co-production have rather different characteristics and correlates and highlight the importance of distinguishing between them for policy purposes. In particular, collective co-production is likely to be high in relation to any given issue when citizens have a strong sense that people can make a difference (‘political self-efficacy’). ‘Nudges’ to encourage increased co-production had only a weak effect.
Keywords: citizen activation, community co-production, co-production correlates, influence strategies, nudge
Contributed by: Tony Bovaird – University of Birmingham, INLOGOV (Institute of Local Government Studies), Elke Loeffler – Governance International
Skip to content