Research School Network: Top Ten Tips for Researchers Working with Schools Top tips following the EEF Funded RCT for ​‘Helping Handwriting Shine’

Charlotte Clowes, Principal at The Wilmslow Academy, has worked with Dr Emily Williams and Jo Atkinson (Research Fellows at the University of Leeds) and Professor Roisin Corcoran (University of Nottingham) as part of the EEF Funded RCT for ​‘Helping Handwriting Shine’ and they have written their Top 10 Tips for Researchers Working with Schools, which are shared with you here.

Top 10 Tips for Researchers Working with Schools

Dr Emily Williams and Jo Atkinson – Research Fellows at the University of Leeds – lead the intervention delivery arm of the EEF-funded RCT, ​‘Helping Handwriting SHINE’. Their experiences in delivering a school-based RCT are shared here in the form of tips. These tips have been developed in collaboration with Professor Roisin Corcoran from the University of Nottingham, leader of numerous randomised controlled trials in educational settings across the world, and Charlotte Clowes, Principal of Wilmslow Academy, who acted as an independent consultant throughout the Handwriting RCT.

Randomised controlled trials make up 80% of the evaluations funded by the Education Endowment Foundation[1], but can be difficult to run in schools. Last year, Dr Anneka Dawson and EEF colleagues[2]
proposed solutions to five challenges inherent in educational RCTs. These insights proved of key importance, so we thought it useful to build on their work, focusing specifically on their second challenge; recruiting and retaining schools in trials. Our experiences in building mutual trust between schools and researchers comes from working on a teacher-led intervention, with an independent evaluator. However, we believe our ten recommendations will be useful to any researchers interested in optimising their working relationships with schools when working on the delivery arm of an educational RCT.

1. Consult with service/​intervention users
Consider hiring a member of Senior Leadership from a school not recruited into your RCT as a consultant. The ​‘Helping Handwriting SHINE’ (HHS) steering group included a Primary School Principal (Charlotte Clowes) whose input into the training materials was invaluable. When training teaching staff, commenting on whether the intervention has been implemented in a school previously can help them judge if the intervention is likely feasible in their workplace. Our training, for example, included short videos of (RCT-independent) teachers using the HHS approach with their students. Make the most of all opportunities to hear from students about their experience and opinion of the intervention.

2. Pitch training and guidance in the right style
When creating training materials, strike a balance between writing academically and for a lay audience, in terms of the level of detail and tone. Avoiding both condescension and jargon is key. Resist the urge to use training as an opportunity to disseminate as much recent research as possible. Your audience are experts in education and the classroom environment but navigating a research base may be new to them, so stick primarily to the background and basics of the intervention. You can always have some EEF guidance reports on hand in case of further interest.

3. Maximise the professional development events
Structure training events wisely to get to know the teaching staff, their schools, their areas, interests and issues, and allow for networking opportunities.
Include experiential learning activities (e.g., multiple choice quizzes, practicals, problem-solving, or team-based learning) to make the session as enjoyable and useful as possible. Meanwhile, remember that training must stay relevant and focused. It’s a tricky balance, but the bottom line is making the most of the time the teaching staff are offering, and ensuring they feel they have had the best possible ​‘value’ for their investment of time.

It’s also good to remember that these events are as much for you as for the teaching staff. Arrange a section where your attendees trial-run the initiative, perhaps treating others on their table as pupils. This will allow you to observe the teaching staff practising the intervention while you are on-hand with tips and potential corrections. Have a thorough evaluation process at the end, with time built in for the teaching staff to complete an evaluation form.

4. Ask the evaluators to attend and present at training events
Ask a representative from the evaluation team to attend in a formal capacity, including presenting any evaluation-related information to the teaching staff and stakeholders, and being on hand to answer any questions directly. The evaluation team may ask to attend a sample of training events to monitor the consistency of delivery and observe the engagement of the teaching staff. This suggestion will help the teaching staff to see the differences between the independent groups and make it easier for them to know who to contact with a given query. This would remedy Dawson and colleagues’ observation that ​“When developers, evaluators and separate recruiters are involved in a project, there is potential for schools to become confused about who they need to work with and when”.

5. Develop seamless signposting
Arrange a formal introduction with the members of the evaluation team at the start of the project, even if just through emails or a conference call.
Find out who will be managing the school-directed inbox as you will be forwarding questions from schools to them at times, particularly if Tip #4 doesn’t work out as you’d hoped! Provide teaching staff with the name, phone number, email information and address for the contact person responsible for leading the evaluation.

In particular, know who you should contact if you receive an evaluation-related question at the training events and a representative from the evaluation team is not present. Have their phone number handy so that you can call during a break and provide the answer as soon as possible. It’s understandable that developers won’t know the answers to all such questions, but being able to have someone to point a school to or a quick route to finding an answer really helps build trust with the schools.

6. Support the evaluation team
Remind and encourage teachers to complete any fidelity logs or focus groups as these aren’t ​‘optional extras’. School-based intervention programmes are not always implemented as designed. When implementing complex interventions in the ​‘real world’, variability both within and between schools is to be expected, and thus the independent evaluator will usually design and conduct both an impact and process evaluation. The study protocol will detail the approach that will be used to capture fidelity of implementation, including the logic model, the key components that will be assessed, and when each will be measured. High-quality impact and implementation studies are essential to provide information about why, when, and how interventions work. To answer all of these questions it is essential to support teachers in providing information on implementation. Without this information, the evaluation team will not get an accurate picture of the implementation, which is as important to understand as the summary of the impact of the intervention.

7. Have a joint team email address
If you’re working in a team, ask your IT department for a joint email address for the group to streamline communications with schools. Offering separate email addresses for each team member often burdens the inbox of the first-listed contact and can increase response times to teaching staff queries. You can add the joint email address to your email managing software, allowing you to see both your personal and project inboxes at a glance.

Avoid setting a forwarding rule from the joint account to your separate inboxes; always send emails to teaching staff from this joint account so that all team members can access the emails that have been sent. For some team members with unique roles it may be best to list the individual email addresses, but if you do this make clear to schools what each person is best suited to help with.

8. Mail Merge is your friend
Take advantage of software programmes that make it easier to manage communications. For example, Microsoft Word’s Mail Merge function (usually found under the ​‘Mailings’ tab) will allow you to type out an email (or letter) and insert certain fields (e.g. name, year group) from a spreadsheet or database such as Excel. Personalising emails can help to build your relationship with the teaching staff and increase the chance of a response. You can also add other mail merge fields, such as, ​“It was nice to meet you on your training day on [date]” or use some ​“IF” statements to include a few different sentences depending on year group or whether they are a member of Senior Leadership. This means that you don’t waste their time reading information that’s not tailored to them, and minimises the chance your recipients switch off when reading.

Sending emails through the Mail Merge function also avoids the possible data protection risk of pasting your recipient’s emails into the ​‘CC: box’ instead of the ​‘BCC: box’, if you were sending a generic batch email. Instead, each recipient will receive the email as it if was sent solely to them. You can find an introduction to Mail Merge here.

9. Consider alternative lines of communication
Diversify your means of communicating with schools for the best chance of providing easy access to information and sustaining interest. While email will be the standard form of communication, it is sometimes helpful to use alternative (and more informal) ways to maintain engagement with teaching staff. If you choose to use social media, then have this set this up ahead of time ready to provide teaching staff with details at the training. Sticking to platforms already used by the teaching staff (e.g. Twitter and Facebook) may achieve the best outcomes, rather than asking teachers to join something unfamiliar that they may be less likely to interact with (e.g. Slack, Groups.io, or Reddit). For example, Professor Roisin Corcoran’s team for the EEF-funded ​‘Glasses in Classes’ project at the University of Nottingham send out regular newsletters to stakeholders using Microsoft Sway.

Whatever the platform, if you’re communicating about specific aspects of the intervention, keep this private. You could use a ​‘protected’ Twitter account where your tweets are only visible to the people who you have approved in advance and therefore not visible to any schools in the control group or people outside of the trial. If you would like to encourage discussion between teaching staff of different schools, perhaps a ​‘secret’
Facebook group would be appropriate. You may be unlikely to reach even half of the teaching staff with social media, so ensure your updates are engaging yet non-essential.

10. Know when to quit (contacting) and help track attrition
Create a clear protocol for engaging with schools who are not responding and ensure the protocol has a stopping rule. It’s unfortunate, yet almost inevitable, that a small number of schools will drop out of an intervention. However, it can be difficult to determine whether a lack of response to emails is due to a member of teaching staff simply lacking time to respond, or disengagement from the project. As drop out can affect the integrity of the trial it is important to keep the evaluation team posted on any suspected disengagement.

Where a response from teaching staff is required, e.g. when booking on to a training event, decide in advance on the process for reminders or past-deadline prompts. You could include a small line at the bottom of the reminder email stating that you will attempt to make contact (by email or phone) a few days after the deadline, but that if this is unsuccessful it will be brought to the attention of the evaluation team.

Conclusion
When aiming for positive collaborative relationships with schools, key issues appear to be; recognising teaching staff’s role as co-experts, respecting their time, and being as approachable as possible. While some of these tips may seem trivial (e.g. personalising email greetings) any sensible action that increases engagement with, or fidelity to, the intervention is worthwhile in an educational RCT. The most effective approaches don’t just aim to get ​‘buy-in’ from stakeholders, they also include high-quality professional development. Social and Emotional Learning[3]
interventions, for example, involve approaches that go well beyond classroom teaching and learning to promote positive school climate, ethos, and culture. Thus, it is crucial to keep the upskilling opportunities for teaching staff at the front of your mind when designing the training events.

We have thoroughly enjoyed the experience of working with teachers and teaching assistants through the course of the RCT, and have been deeply impressed by the work we have seen and heard about in Primary Schools. Without fail, we have met inspirational people at all of our training events. We hope these recommendations will continue to be discussed and developed with the aim of best supporting teachers and researchers in their practice.


[1] Nevill, C. (2019, March 13). How do we make EEF trials as informative as possible? [Web log post]. Retrieved July 22, 2019, from https://educationendowmentfoundation.org.uk/news/eef-blog-how-do-we-make-eef-trials-as-informative-as-possible/

^[2] Dawson, A., Yeomans, E., & Brown, E. R. (2018). Methodological challenges in education RCTs: reflections from England’s Education Endowment Foundation. Educational Research, 60(3), 292 – 310.

^[3] Corcoran, R. P., Cheung, A. C., Kim, E., & Xie, C. (2018). Effective universal school-based social and emotional learning programs for improving academic achievement: a systematic review and meta-analysis of 50 years of research. Educational Research Review, 25, 56 – 72.