Research School Network: How to plan an RCT

How to plan an RCT


Have you ever sat and listened to a new teaching technique and thought ​‘well that won’t work for my pupils’? Planning and engaging in a Randomised Control Trial (RCT) will give you an evidence-based answer to that very issue.

As a scientist, and science teacher, the scientific method is very important to me. Throughout University, I worked on the premise that quantitative data is the only type of data that matters. Coming into teaching was quite a culture shock, however evidence-based practice allows tried and tested methods to be adopted in the classroom.

How do we do it? Firstly, start with the issue at hand- what is the problem you are trying to solve: Is there a new innovation you want to try? Is there a current method you want to check the effectiveness of? Once you have decided the approach that you wish to try, the next step is to design the RCT.

RCTs are research tools that can tell us whether one approach works better than the other, by randomly allocating participants to two or more groups. One group carries on doing what they would usually do- the control group. The other group tries something new, or something already established you want to test. All the participants are then tested to find out which group has improved the most.

When contemplating your RCT it is important to think about what data you will generate, there are two types: quantitative and qualitative. Quantitative data can be counted, measured and expressed using numbers. Qualitative data is descriptive and conceptual. A good source of quantitative data is test results from either existing tests or new tests created for the trial. For the quantitative data in pedagogical RCTs, questionnaires and interviews are generally used.

Diagram 1 shows an example of a ​‘between subject’ trial design where you’re finding out the difference between two groups of participants. There are several ways to organise how this happens during your trial.

If the effect of your intervention can ​‘wear off’ then you can use a ​‘within subject’ design (see diagram 2). In this case, you are finding out the change within each participant. Everyone is tested after doing what they would usually do and after trying the new intervention. This has the advantage that all participants try out the new intervention and so there is no variation between the intervention and control groups.

An important note when talking about RCTs is the difference between cause and correlation. As humans, we often assume a link between two things that happen at the same time without any evidence for it. For example, I changed my display boards to yellow this year and the learners performed better than ever. Therefore, all classrooms should have yellow displays. This demonstrates a correlation
but not a cause.

Many times, we have heard ​“correlation does not cause causation” or ​“correlation does not imply causation” or ​“correlation is not causation”. But what does this actually mean? Correlation is a statistical technique, which tells us how strongly the pair of variables are linearly related and change together. It does not tell us the ​‘why and how’ behind the relationship, it just says the relationship exists. For example, the correlation between ice cream sales and sunglasses sold. As the sales of ice creams increases, so do the sales of sunglasses. This does not mean that we need sunglasses to eat ice cream.

Causation takes a step further than correlation. It says any change in the value of one variable will cause a change in the value of another variable, which means one variable makes a change in the other variable. It is also referred as cause and effect e.g. When a person is exercising then the number of calories used increases up every minute. The former is causing the latter to happen. RCTs allow us to define a causal probability, rather than just correlation.

These are the basic factors to consider when planning the use of RCTs in the classroom. The larger the sample, the more accurate your findings will be. Although a large sample can be a challenge in a school and comes with a larger workload. RCTs are a valuable contribution to the wider movement of evidence-based practice and improving your skills as an effective practitioner.

Chris Boyce
Research Lead (Science)
@CBoyceEduRsrch