A network for students interested in evidence-based health care

Tutorial: How to read a forest plot

Posted on July 11, 2016 by Nathan Cantley

Tutorials and Fundamentals

As we get towards the top of the evidence-based medicine pyramid, strange looking graphs sometimes emerge. The forest plot is a key way researchers can summarise data from multiple papers in a single image. [If you have difficulty reading the text in any of the figures, clicking on the image will enlarge it].  

Figure 1. An example of a forest plot. Image adapted from Table 4 Roberts et al. (2006). [1]

As students, we sometimes think graphs like the one in figure 1 are a bit hard to interpret. Never fear- the following tutorial should give you a step by step way to interpret any forest plot!

Tutorial information

Learning Objectives.
By the end of this tutorial you should be able to:

  1. Understand what forest plots are used for
  2. Understand how to read a forest plot and what the results displayed mean both at the level of individual studies and the averaged result
  3. Understand why forest plots look different depending on the statistics being analysed
  4. Understand the importance of heterogeneity within forest plots and how it affects interpretation

Time to complete tutorial: 20-30 minutes
Example questions: Yes
References all open access: Yes

So why make a forest plot in the first place?

Trying to look at lots and lots of different papers that ask the same question can be difficult. This is especially true if the papers analysed come to different conclusions and have different statistics either in favour or against an association.

What a forest plot does, is take all the relevant studies asking the same question, identifies a common statistic in said papers and displays them on a single set of axis. Doing this allows you to compare directly what the studies show and the quality of that result all in one place. 

Analysing the forest plot: the basics

Part 1: The axis.

Throughout this tutorial we will take Figure 1 (shown above), a forest plot from a Cochrane systematic review, as an example. As we go through the tutorial we will build Figure 1 up from first principles. I will reveal the significance of this particular forest plot at the end of the blog!

Figure 2. Let’s take this from the beginning.

What you see to the left is the basic set of axes that forest plots employ. The horizontal axis usually represents the statistic the studies being profiled show. This could either be a ‘relative’ statistic like an odds ratio (OR) or a relative risk (RR). Or the statistics being used might be an ‘absolute’ one such as Absolute Risk Reduction (ARR) or Standardised Mean Difference (SMD). Knowing the difference between relative and absolute statistics is important because it affects which number sits at the vertical line.

The vertical line is known as the “line of null effect.” This line is placed at the value where (as the title suggests) there is no association between an exposure and outcome or no difference between two interventions. If you remember from your statistics classes, relative statistics like OR or RR have a null effect value of 1. For absolute statistics like Absolute Risk or ARR or SMD, the null difference value is 0. Hence why the value at the line of no effect is relevant to the statistic being used. If you want a refresher about relative and absolute statistics why not check up on this S4BE blog.

Figure 3. Let’s put some lines on these axis.

Part 2: The study lines. 

Ok, so now that we understand the axes of the forest plot, let’s put some values on. Each horizontal line put onto a forest plot represents a separate study being analysed. In Figure 3, three studies are represented. Each study ‘result’ has two components to it:

  1. A point estimate of the study result represented by a black box. This black box also gives a representation of the size of the study. The bigger the box, the more participants in the study.
  2. A horizontal line representing the 95% confidence intervals of the study result, with each end of the line representing the boundaries of the confidence interval.

What each side of the null effect line represents (i.e. if it favours the control or the intervention) is also important when looking at the individual studies. This will be different depending on what question you are asking in your studies. For example if you are looking at the risk between an exposure and an outcome, what each side of the vertical line represents will be different to an instance where you are comparing an intervention with a control. Helpfully, for most forest plots published today, the authors helpfully mark what each side of the line represents. If it isn’t marked, remember to always go back to your first principles of the statistic you are using.

The horizontal line and whether it crosses the “line of null effect” is particularly important to take note of for each study. If you remember, the incredibly basic definition of the 95% confidence interval is: “The range of values within which you can be 95% certain the true value lies.” If the horizontal line crosses the line of null effect what that is effectively saying is that the null value lies within your confidence interval and hence could be the true value. If I were breaking this down to its most simplest explanation: “any study line which crosses the line of null effect does not illustrate a statistically significant result.”

There is one more component to the line which is useful to take note of. Whilst it is not guaranteed, as a rule of thumb the studies with a greater number of participants or patients typically have a narrower confidence interval and hence a smaller horizontal line. So in basic terms:

  1. The bigger the study, the smaller the horizontal line and bigger the black box representing the point estimate. This can mean it is less likely those studies will cross the line of null effect. Why? Because your 95% confidence intervals should have a much smaller range.
  2. The smaller the study, the wider the horizontal line and smaller the black box representing the point estimate. This means it is more likely those studies will cross the line of null effect (because your 95% confidence intervals will be much bigger).

Now you have read through the above description, have a look at Study (A) and Study (B) on Figure 3. Have a go at interpreting what each study is telling you before looking to the bottom of the webpage for the answer.

Part 3: Combining all the studies of the forest plot.

Figure 4. The Diamond of the bunch.

Figure 4 adds two more studies at the top (have a go at interpreting them) as well as a diamond. Now, the diamond is probably the most important thing you will see on a forest plot.

The diamond represents the point estimate and confidence intervals when you combine and average all the individual studies together. If you drew a vertical line through the vertical points of the diamond, that represents the point estimate of the averaged studies. The horizontal points of the diamond represent the 95% confidence interval of this combined point estimate. If you remember what I said in part 2 about the size of the confidence intervals- because this combined value effectively groups all the participants from all the individual studies, the CI range for this result should be the smallest on the forest plot (and in 99% of cases it normally is).

The rules about crossing the line of null effect are still true here: if the horizontal tips of the diamond cross the vertical line, the combined result is potentially not statistically significant. Why? If you remember, if the 95% confidence interval contains the null value, you cannot be certain that the null value isn’t the true value.

Part 4: Bringing it all together.

So we have talked about a number of the elements of the forest plot itself. Let’s have a bit of a look at all the “bumf” so to speak that sits around the forest plot on the graph. Let’s go back to our original image. Figures 5-8 highlight different elements on the forest plot that are worth noting.

Figure 5. There is a wealth of information aside from the plot itself.

In Figure 5- to the far left of the forest plot is the name of the lead author for each individual study as well as the year of publication.

To the immediate left of the forest plot, are two columns of numbers- highlighted in Figure 6. Each column of numbers has two numbers separated by a ‘/’. If you look at the heading of each column you will see the numbers are organised as “n/N.” What does this mean? Put simply, ‘n’ denotes the number of patients or individuals which had the event/outcome in that particular group whilst ‘N’ denotes the total number of people in that group.

Figure 6. Makes sense when you compare the numbers to the graph.

So in our example paper we have two columns of numbers. The first column is for the group that received the treatment (n= number of treated people who had outcome, N= total number of people in study who got treatment). Whereas the second column is for the group that got the control (n= number of people in the control group who had outcome, N= total number of people who were in the control group).

Figure 7. Sometimes it is just simpler to compare the numbers…

Figure 7 is on the other side of the forest plot. The far right column basically gives you the forest plot as numbers (both the point estimate and the 95% confidence interval in brackets). Some people might find it easier just to look straight to these numbers instead of looking at the plot and trying to interpret it.

Figure 8. If it’s in bold- probably a reason for it!

Figure 8- This simply highlights the statistics associated with the diamond on the forest plot. The ‘subtotal’ is what it says on the tin. It tells you the total number of participants in the treatment and control groups across all the individual studies as well as the averaged statistic and 95% confidence interval. It’s useful to look at this line of numbers and the diamond when drawing conclusions for your meta-analysis/systematic review.

Part 5: Heterogeneity of the papers.

The final part of analysing a forest plot is about looking at something called ‘heterogeneity.’ Ideally, if different trials are testing the same thing, the effects of the intervention/exposure should be consistent across all the studies. Unfortunately, this is rarely the case. Many things can affect the results of a trial, such as researcher bias or problems with data collection [2].

So, in addition to analysing the study results, systematic reviews or meta-analyses are designed to ask a question. If these studies are all testing the same intervention, why don’t they get the same results? Are the differences caused by chance, or is there something else involved? If it is chance, then we have nothing to worry about. If the differences are not the result of chance, then we need to be cautious in how we interpret the results. To make it easy to assess the consistency of the papers analysed, a statistic called I2 is used (‘i-sqaured’).

Figure 9. I spy with my little eye…something beginning with h…

As Figure 9 shows, the statistics related to heterogeneity are usually at the bottom of the chart. The rule of thumb is that you want the I2 to be less than 50%. Anything higher than that and the papers could be inconsistent due to some reason other than chance (which is bad!). For our example, thankfully, the I2 is 38%- not perfect but still within our target range. You will notice there are other statistics there like Chi2 and z. For the purposes of this tutorial, the I2 is the most useful in interpreting a forest plot.

Summary time

Hopefully, going through a forest plot from start to finish has helped you understand what makes forest plots tick. I would really encourage you to scroll to the top and look at Figure 1 which doesn’t have any annotations on it and work through the 5 steps we have gone through.

To summarise what we have covered:

  1. Each horizontal line on a forest plot represents an individual study with the result plotted as a box and the 95% confidence interval of the result displayed as the line.
  2. The implication of each study falling on one side of the vertical line or the other depends on the statistic being used.
  3. If the individual study crosses the vertical line, it means the null value lies within the 95% confidence interval. This implies the study result is in fact the null value and therefore the study did not observe a statistically significant difference between the treatment and control groups.
  4. The diamond at the bottom of the forest plot shows the result when all the individual studies are combined together and averaged. The horizontal points of the diamond are the limits of the 95% confidence intervals and are subject to the same interpretation as any of the other individual studies on the plot.
  5. The I2 statistic gives you an idea of the heterogeneity of the studies, i.e. how consistent they are. If the I2 value is >50% it might mean the studies are inconsistent due to a reason other than chance. This might make the conclusions you draw from the forest plot questionable.

Figure 10. Another forest plot from the same paper to work through. Image from Roberts et al. (2006) [1]

Make sense? I hope so! If you want another example to work through, have a look at Figure 10- a forest plot from the paper Figure 1 is from. What is the significance of the Cochrane Review I hinted at the start you might wonder?

And finally…Cochrane logo

Well, if you have ever seen the Cochrane Collaboration logo, you might recognise the horizontal lines and diamond.

That is because the Cochrane Collaboration logo is in fact the representation of a forest plot. The forest plot in the Cochrane Collaboration logo is from one of the first systematic reviews ever published [3]. The paper, published in 1991, showed that giving steroids to mothers whose babies were due to be born prematurely, reduced the complications of prematurity. The Cochrane Review which Figure 1 and 10 comes from is actually an update of that original review that went on to form the Cochrane Collaboration logo.

References:

[1] Roberts D, Dalziel SR. Antenatal corticosteroids for accelerating fetal lung maturation for women at risk of preterm birth. Cochrane Database of Systematic Reviews 2006, Issue 3. Art. No.: CD004454.
[2] “How to analyse the forest plot: Assess the Heterogeneity amongst the studies.” Centre for Evidence Based Intervention, University of Oxford. URL: http://www.cebi.ox.ac.uk/for-practitioners/what-is-good-evidence/how-to-interpret-the-sample-forest-plot.html#c56 Accessed: June 2016
[3] “Our logo: Cochrane Collaboration.” The Cochrane Collaboration, 2016. URL: http://www.cochrane.org/about-us/our-logo. Accessed: June 2016

Part 2 ANSWERS: DONT LOOK UNTIL YOU HAVE HAD A GO!

So, how did that go? What you should hopefully have found is:

  • For study A- it has quite a wide horizontal line that crosses the line of null effect with a black box that is to the right of the vertical line. This most likely demonstrates a small scale study with few participants where the point estimate result favours the control used in the study. But because the study contains the null value in its 95% confidence interval, it is likely to have a p value >0.05 and not be statistically significant.
  • For study B- it has quite a narrow line that does not cross the line of null effect with a black box that is to the left of the vertical line and is bigger than study A. This most likely demonstrates a larger study with more participants compared to study A, where the result favours the intervention used. Because the 95% confidence interval does not contain the null value, this study is likely to have a p value <0.05 and hence the differences observed in the study can be regarded as statistically significant.

 

The featured image in this blog has been retrieved from: en.wikipedia.org and is free to re-use.

Tags:

Nathan Cantley

Nathan Cantley

I am graduating in July 2016 in Medicine at Queen's University Belfast after previously gaining a Masters in Translational Medicine in December 2014. My interests include academia and global health. View more posts from Nathan

Leave a Reply

Your email address will not be published. Required fields are marked *

21 Comments on Tutorial: How to read a forest plot

  • David Colquhoun

    I think that this nice account might be improved by mention of what really matters, the false positive rate. See, for example, http://rsos.royalsocietypublishing.org/content/1/3/140216

    Concentrating on P < 0.05 is probably a substantial part of the reason why so many false positives get accepted.

    July 13, 2016 at 5:12 pm
    Reply to David
  • Fishiwe Chiyaka

    this has been a very useful account for a beginner.

    February 10, 2017 at 8:39 am
    Reply to Fishiwe
  • KB

    Hi Nathan! Just loved this page. The trivia related to forest plot and Cochrane logo is awesome!
    Thanks for these helpful tips.
    Good luck!

    February 24, 2017 at 1:01 pm
    Reply to KB
  • Leo Chukwuali

    Clear and concise. Thank you.

    March 2, 2017 at 5:47 am
    Reply to Leo
  • Suganya

    Hey i am doing MD in preventive and social medicine…..always had problem in forest plot interpretation….thanks alot….dats really very wonderful explanation…..

    March 4, 2017 at 4:54 am
    Reply to Suganya
  • Poonam

    Very lucid.
    Explained the concepts so well.

    March 4, 2017 at 5:55 pm
    Reply to Poonam
  • Najamuddin

    Easy to understand the concept. Really I like it.

    March 5, 2017 at 8:46 am
    Reply to Najamuddin
  • Elizabeth Aviles

    It is so good that my professors direct all their advanced practice nursing students to your site so that we can get a better understanding of Forest Plots. You did great! Congratulations on your graduation!

    August 10, 2017 at 5:40 am
    Reply to Elizabeth
  • Milica Pavlović

    Thank you so much for this interpretation. I’m preparing an exam (Clinical Pharmacy) so it’s very helpful :) I’m not sure I understood…how to interpret when the diamond doesn’t cross but is attached to the vertical line?

    August 17, 2017 at 7:03 pm
    Reply to Milica
    • Selena Ryan-Vig
      Selena Ryan-Vig

      Hi Milica,

      If the diamond touches or crosses the vertical line, the combined result is potentially not statistically significant. This is because if the 95% confidence interval contains the null value, you cannot be certain that the null value isn’t the true value.

      August 18, 2017 at 9:08 am
      Reply to Selena
  • Mike

    Great work, very detailed and simplified.

    November 20, 2017 at 6:02 pm
    Reply to Mike
  • Jasmine Aziz

    Hi Nathan! thanks for the explanation! I was wondering though, how do you assign weights to individual studies as done in Figure 10?

    November 22, 2017 at 10:05 pm
    Reply to Jasmine
  • Bahaa Eldin Borayek Mohamed Borayek

    Pt with PMB was referred to either quick one stop or general gynae clinic. Time was compared from referral to ?investigation.
    I’m appreachating your efforts and the simplest nice expanation that cope with all the questaire of any student or researcher.
    Thanks.
    Wpuld you mind please explain the type of study that compatible with the question up.
    Thanks again.

    January 3, 2018 at 4:19 am
    Reply to Bahaa
  • ac servicing sharjah

    I like looking through a post that will make people think.

    Also, many thanks for allowing for me to comment!

    March 8, 2018 at 3:00 am
    Reply to ac
  • Yves Homsy

    Great introduction to forest plots! What does WMD (weighted mean difference) mean?
    What does it mean when the values are preceded by a minus sign?
    Kindly post copy of your answer to my email address.
    Thanks.

    March 9, 2018 at 4:14 am
    Reply to Yves
  • bakman

    really amazing stuff :) thank you so so much!

    March 10, 2018 at 6:35 am
    Reply to bakman
  • marwa bsyouni

    it is perfect

    March 14, 2018 at 8:41 am
    Reply to marwa
  • ezekiel Luoga

    Thanks for the training now I a happy and I can interpret the forest plot.

    April 25, 2018 at 4:21 pm
    Reply to ezekiel
  • Lia A

    Thank you so much for this insightful tutorial, it really covers a lot, and well.

    May 5, 2018 at 5:26 pm
    Reply to Lia
  • NN

    Nathan, thank you so very much for this explanation. Very useful. I’m doing my PhD and have just finished a meta-analysis.

    August 22, 2018 at 5:49 pm
    Reply to NN
  • rigzin

    very comprehensive way to understand forest plot ..thank you

    September 4, 2018 at 12:39 pm
    Reply to rigzin

Subscribe to our newsletter

You will receive our monthly newsletter and free access to Trip Premium.