Research School Network: Monitoring to Improve Not Prove Monitoring a range of implementation outcomes to support effective implementation

Implementation outcomes are essential indicators that help us understand whether a new program or approach is being delivered effectively. These outcomes serve as benchmarks for progress, providing valuable insights into what is working, what needs adjustment, and how to sustain improvements over time. By focusing on both final outcomes and implementation outcomes, we can enhance the likelihood of achieving meaningful and long-lasting impact. These outcomes are the stepping stones that support us when REFLECTING on areas for improvement. 

The EEF Guide to Effective Implementation outlines a range of implementation outcomes that we may consider:

How do we design a monitoring system to capture implementation outcomes?

To effectively track implementation outcomes, we need a structured yet flexible system. We might consider the following:

• UNITE around a culture of continuous improvement:

We know that the culture and climate of an organisation has an impact on implementation. The climate of an organisation can be damaged if we don’t spend the time working with teams to UNITE them around why monitoring is important and how data will be collected. This is about a UNITING around a shared mission of continuous improvement to improve outcomes for all learners. Capturing implementation outcomes around acceptability and feasibility can be just as important for ensuring the protection staff workload and well being. 

• ENGAGE stakeholders:

Staff should be actively ENGAGED in deciding what types of data will be useful, at what time points, and why. This can also support distributed ownership of implementation activity avoiding a sense of division between the data collectors and the classrooms/​corridors where ​‘data’ is generated. Where possible, ENGAGING students and parents when designing monitoring systems can allow schools to understand how an intervention is impacting those it is intended to support.

• Set clear goals:

When establishing implementation outcomes we need a clear sense of what will indicate success in the short, medium and long term. For example, in the short term we may want to consider the extent to which there is fidelity with the particular approach and the scale of reach. This may differ as the implementation journey continues. 

• Select appropriate SYSTEMS and STRUCTURES:

Schools are data rich and can use a range of methods and approaches to capture data linked to our implementation outcomes; this needs to be decided from the outset. We need to consider how monitoring of this particular approach fits with broader SYSTEMS to reduce the burden on leaders, implementation teams, teachers and pupils. We need to make decision about when will this happen? By who? With who? 

• Provide feedback loops

The data that we collect is only useful for supporting further improvements if we do something with it. Who is responsible for collecting and summarising the data in an accessible way? Often we make sense of this through combining it with multiple perspectives, when are our checkpoints for discussion of the latest findings? We need to make decisions about how we will share findings with staff and stakeholders to build trust and UNITE further around a culture of continuous improvement. At each point we must REFLECT on the data captured and the subsequent decisions that we should make.

Focusing on implementation outcomes ensures that we are not only striving for final results but are also optimising the processes that lead to those results. By designing a monitoring system that captures a range of outcomes, we can create a dynamic, data-rich approach to continuous improvement. When implementation is monitored effectively, it builds the foundation for sustainable, impactful change that benefits students, staff, and the broader school community.

References

EEF Guide to Effective Implementation