Critical Appraisal: A Checklist
Posted on 6th September 2016 by Robert Will
Critical appraisal of scientific literature is a necessary skill for healthcare students. Students can be overwhelmed by the vastness of search results. Database searching is a skill in itself, but will not be covered in this blog. This blog assumes that you have found a relevant journal article to answer a clinical question. After selecting an article, you must be able to sit with the article and critically appraise it. Critical appraisal of a journal article is a literary and scientific systematic dissection in an attempt to assign merit to the conclusions of an article. Ideally, an article will be able to undergo scrutiny and retain its findings as valid.
The specific questions used to assess validity change slightly with different study designs and article types. However, in an attempt to provide a generalized checklist, no specific subtype of article has been chosen. Rather, the 20 questions below should be used as a quick reference to appraise any journal article. The first four checklist questions should be answered “Yes.” If any of the four questions are answered “no,” then you should return to your search and attempt to find an article that will meet these criteria.
Critical appraisal of…the Introduction
- Does the article attempt to answer the same question as your clinical question?
- Is the article recently published (within 5 years) or is it seminal (i.e. an earlier article but which has strongly influenced later developments)?
- Is the journal peer-reviewed?
- Do the authors present a hypothesis?
Critical appraisal of…the Methods
- Is the study design valid for your question?
- Are both inclusion and exclusion criteria described?
- Is there an attempt to limit bias in the selection of participant groups?
- Are there methodological protocols (i.e. blinding) used to limit other possible bias?
- Do the research methods limit the influence of confounding variables?
- Are the outcome measures valid for the health condition you are researching?
Critical appraisal of…the Results
- Is there a table that describes the subjects’ demographics?
- Are the baseline demographics between groups similar?
- Are the subjects generalizable to your patient?
- Are the statistical tests appropriate for the study design and clinical question?
- Are the results presented within the paper?
- Are the results statistically significant and how large is the difference between groups?
- Is there evidence of significance fishing (i.e. changing statistical tests to ensure significance)?
Critical appraisal of…the Discussion/Conclusion
- Do the authors attempt to contextualise non-significant data in an attempt to portray significance? (e.g. talking about findings which had a trend towards significance as if they were significant).
- Do the authors acknowledge limitations in the article?
- Are there any conflicts of interests noted?
This is by no means a comprehensive checklist of how to critically appraise a scientific journal article. However, by answering the previous 20 questions based on a detailed reading of an article, you can appraise most articles for their merit, and thus determine whether the results are valid. I have attempted to list the questions based on the sections most commonly present in a journal article, starting at the introduction and progressing to the conclusion. I believe some of these items are weighted heavier than others (i.e. methodological questions vs journal reputation). However, without taking this list through rigorous testing, I cannot assign a weight to them. Maybe one day, you will be able to critically appraise my future paper: How Online Checklists Influence Healthcare Students’ Ability to Critically Appraise Journal Articles.
Feature Image by Arek Socha from Pixabay
No Comments on Critical Appraisal: A Checklist
Hi Ella, I have found a checklist here for before and after study design: https://www.nhlbi.nih.gov/health-topics/study-quality-assessment-tools and you may also find a checklist from this blog, which has a huge number of tools listed: https://s4be.cochrane.org/blog/2018/01/12/appraising-the-appraisal/
15th February 2023 at 10:28 amWhat kind of critical appraisal tool can be used for before and after study design article?
12th February 2023 at 8:49 pmThanks
Hello,
9th July 2021 at 1:43 pmI am currently writing a book chapter on critical appraisal skills. This chapter is limited to 1000 words so your simple 20 questions framework would be the perfect format to cite within this text. May I please have your permission to use your checklist with full acknowledgement given to you as author?
Many thanks
Thank you Robert, I came across your checklist via the Royal College of Surgeons of England website; https://www.rcseng.ac.uk/library-and-publications/library/blog/dissecting-the-literature-the-importance-of-critical-appraisal/.
13th April 2021 at 11:13 amI really liked it and I have made reference to it for our students.
I really appreciate your checklist and it is still current, thank you.
Hi Kirsten. Thank you so much for letting us know that Robert’s checklist has been used in that article – that’s so good to see. If any of your students have any comments about the blog, then do let us know. If you also note any topics that you would like to see on the website, then we can add this to the list of suggested blogs for students to write about. Thank you again. Emma.
13th April 2021 at 2:29 pmi am really happy with it. thank you very much
1st January 2021 at 8:40 amA really useful guide for helping you ask questions about the studies you are reviewing
16th October 2020 at 7:55 amBRAVO
Dr.Suryanujella,
Thank you for the comment. I’m glad you find it helpful.
Feel free to use the checklist. S4BE asks that you cite the page when you use it.
-Robert
9th May 2019 at 4:19 pmI have read your article and found it very useful , crisp with all relevant information.I would like to use it in my presentation with your permission
8th May 2019 at 4:13 pmThat’s great thank you very much. I will definitely give that a go.
30th December 2018 at 2:27 pmBijou,
I find the MEAL writing approach very versatile. You can use it to plan the entire paper and each paragraph within the paper. There are a lot of helpful MEAL resources online. But understanding the acronym can get you started.
M-Main Idea
(What are you arguing?)
E-Evidence
(What does the literature say?)
A-Analysis
(Why does the literature matter to your argument?)
L-Link
(Transition to next paragraph or section)
I hope that is somewhat helpful.
27th December 2018 at 5:11 pm-Robert
Bijou,
I find the MEAL writing approach very versatile. You can use it to plan the entire paper and each paragraph within the paper. There are a lot of helpful MEAL resources online. But understanding the acronym can get you started.
M-Main Idea
(What are you arguing?)
E-Evidence
(What does the literature say?)
A-Analysis
(Why does the literature matter to your argument?)
L-Link
(Transition to next paragraph or section)
I hope that is somewhat helpful.
27th December 2018 at 5:00 pm-Robert
Hi, I am a university student at Portsmouth University, UK.
26th December 2018 at 5:44 pmI understand the premise of a critical appraisal however I am unsure how to structure an essay critically appraising a paper. Do you have any pointers to help me get started?
Seif,
Thank you. I’m glad that you find this helpful.
-Robert
17th December 2018 at 5:37 pmVery informative & to the point for all medical students
14th December 2018 at 10:41 amHow can I know what is the name of this checklist or tool?
16th November 2018 at 7:24 pmThis is a checklist that the author, Robert Will, has designed himself.
19th November 2018 at 9:42 amVictoria,
Thank you for asking. I am glad you found it helpful. As Emma said, please cite the source when you use it.
-Robert
3rd October 2018 at 4:12 pmGreetings Robert,
3rd October 2018 at 1:29 pmI am a postgraduate student at QMUL in the UK and I have just read this comprehensive critical appraisal checklist of your. I really appreciate you. if I may ask, can I have it downloaded?
Please feel free to use the information from this blog – if you could please cite the source then that would be much appreciated.
3rd October 2018 at 2:02 pmRobert
6th September 2016 at 2:23 pmThank you for your comptrehensive account of critical appraisal. I have just completed a teaching module on critical appraisal as part of a four module Evidence Based Medicine programme for undergraduate Meducal students at RCSI Perdana medical school in Malaysia. If you are agreeable I would like to cite it as a reference in our module.
Anthony,
19th September 2016 at 2:23 pmPlease feel free to cite my checklist. Thank you for asking. I hope that your students find it helpful.
They should also browse around S4BE. There are numerous other helpful articles on this site.