Research School Network: A walk through the forest (plot) An exciting way to analyse and understand data

This week, the latest Landsat satellite will be launched into Earth’s orbit. From this vantage point – and equipped with the latest sensors, it will monitor the growth of cities, the spread of farming and the evolving outlines of coasts, forests, and glaciers.

Likewise, the EEF’s Teaching and Learning Toolkit has been upgraded to provide sharper insights from its vantage point high above the landscape of education research. Forest plots are an excellent new feature, and allow us to get closer to the evidence than ever before.


What is a forest plot?
A forest plot is a diagram. At a glance, forest plots allow us to go beyond the headline and begin to make sense of the different impacts found in a range of studies. Forest plots look chaotic at first, but they are actually simpler and more elegant than they first appear.

Reading a forest plot
To read a forest plot, you need to understand the main features. As an example, let’s look at the 19 studies in secondary schools that involved extending the school day.

The names: Down the left hand side are the names of the 19 studies in our forest plot with the year they were published in brackets.

The position of the squares: If you look along from each of the names, you will see a corresponding square and a vertical red line. Squares to left of the red line indicate a negative effect and those to the right a positive effect. The further a study is from the red line the bigger the effect size, so Smealie (1997) found the most extreme negative effect.

The size of the squares: The size of the each square tells us how much that study influences the overall average. The smallest study in our forest plot, Molina (2008), involved just 23 pupils, while Styles (2014) included 577 pupils. We want the larger studies to ​‘count’ for more in our overall average, so these studies are given more ​‘weight’ and represented by larger squares. The weight given to each study is a bit complicated, but thinking of it as a measure of the number of participations is a fair approximation.

The lines coming out of each square: Finally, each of the squares has lines coming out of it. It is these lines – which apparently look like trees – which gives the forest plot its name. These lines are known as confidence intervals, and represent the uncertainty around each study. We can imagine that the ​‘true’ finding of each study – the square – could be anywhere along the lines.

A walk in the forest
Now that we know how to read forest plots, we can start to explore the studies. One interesting way to do this is to sort the studies by different criteria to give us different perspectives. For instance, below we can see the studies organised by the year they were published.

What trends do you notice? What further questions might you want to explore? A common observation is that more recent studies tend to be of higher quality so the squares tend to be larger and the confidence intervals smaller. We might also wonder if the recent studies are more applicable than some of the older studies.

Below we can see the same 19 studies, but this time organised by the confidence intervals. Again, what do you notice? What further questions might you want to explore?

Finally, you can arrange the studies based on their effect sizes. Again, what do you notice? What further questions might you want to explore?

Just like a real forest, learning more about how to understand the forest plots allows us to appreciate our walk in the forest on a new, more profound level. Why not set aside some time to wander through the EEF’s forest plots by clicking on the technical appendix of each toolkit strand? I found the teaching assistant interventions and phonics strands particularly interesting.