Research School Network: Implementation Outcomes: Stepping Stones to Effective Implementation A review of the implementation outcomes identified in recommendation 3 of the new EEF Implementation Guidance Report.

The EEF’s recently updated ​“Guide to Implementation” has definitely been the talk of the town, with much of the narrative focusing on the social aspect of effective implementation and the required behaviors and structures (Recommendations 1 and 2) that feature so much more prominently versus its predecessor.

This focus is not misplaced as ensuring the appropriate behaviours and structures are in place is fundamental to driving effective implementation. However, I feel the adapted focus on implementation outcomes, part of Recommendation 3 deserves similar attention.

When implementing any new approach, we can classify are desired outcomes into two types:

• Final Outcomes – these specify the overall goals of change (i.e. improved pupil attainment or attendance)
• Implementation Outcomes – these indicate whether implementation is being done well and are the visible actions/​changes in behaviour that are indicative of an approach taking hold.

It may help to think of ​“Implementation Outcomes” as the stepping stones that allow you to reach the ​“Final Outcomes”. The guidance report identifies 7 main implementation outcomes, as shown in the image below. When planning for implementation it is important to consider how schools can support these implementation outcomes in both the short and long term.

Buy-in and Acceptability:

These two implementation outcomes go hand in hand. Acceptance refers to the extent to which an intervention is agreeable and palatable to the stakeholders involved in its implementation. The degree of acceptance naturally influences the degree of buy-in, and itself can be influenced by a number of factors, including past experiences of implementation and staff training. Acceptance can be positively influenced by a number of factors, including:

• Ensuring sufficient time for planning
• Consideration of the impact on staff workload of an intervention
• Staff being included in the decision making (links back to the ​“Engage” element of recommendation 1)
• Uniting (recommendation 1) on a shared need, strategy and understanding

Feasibility is also intrinsically linked to acceptability. Even if an approach appears promising, leaders still need to decide whether or not it can be delivered in their setting given other constraints. Leaders need to ask themselves if they are in a position to implement this change, and if not, are there any barriers they can remove to improve their position? Doing so will ensure higher acceptability of any given intervention.

Buy-in is also significantly influenced by professional development, with 3 of the 14 mechanisms from the EEF’s Guide to Effective Professional Development focusing on mechanisms intended to motivate teachers, including setting and agreeing on goals, presenting information from a credible source and providing affirmation/​reinforcement after progress. Positive buy-in and acceptance will usually lead to high levels of initial adoption of the new approach from which implementation teams can aim for reach and fidelity.

Fidelity and Reach

The degree to which staff use an approach as intended (fidelity) can be surprisingly difficult to ensure. We can often get surface level engagement and agreement with a proposal, but once teachers are back in their own four walls old habits can be hard to break and lethal mutations can take hold. Ensuring fidelity requires leaders to share a clear description of what they want to see happening – the ​“active ingredients” (essential principles and practices that relate to the change) should be explicitly shared with staff and appropriate training provided to support staff in enacting these. Staff that have a strong understanding of why and how they should be doing something are more likely to enact this as desired. Once these initial steps have been taken fidelity and reach rely on a usable and valued monitoring system. Monitoring can often be an after thought of implementation, and can be perceived negatively or with suspicion by some staff. Initially we should set out to capture the degree to which core components are being implemented, how people’s skills are developing and any adaptions that are made. Monitoring systems need to fit with school routines and systems, so that they are achievable amongst other work demands. Where possible integrating monitoring into existing structures can help facilitate effective monitoring and therefore support with fidelity and reach.

Sustainability

Teacher habits are hard to break and therefore any new initiatives, however well intended, supported and delivered by motivated staff will face challenges. To ensure a new approach becomes part of normal practice, it is vital that it is kept live. Too often new approaches have a wave of input and subsequent take up (normally following whole school INSET etc.), but then slowly fade into the background as we assume they become embedded across the school. Regularly re-visiting the desired change, reminding staff of the active ingredients and the reason for the change will help sustain implementation, as does providing staff an opportunity to discuss their successes and challenges.

Furthermore, we should use our monitoring systems to identify and praise staff that are implementing (or attempting to implement) the new approach with high fidelity. Subsequently monitoring cannot stop – it is essential to avoiding assumptions about the effectiveness of implementation and therefore the degree to which new approaches have become part of the every day fabric.

High quality final outcomes are naturally desirable, but it can be easy to jump straight to these without considering the stepping stones (implementation outcomes) we will need to embed to get there.