A network for students interested in evidence-based health care

Appraising the appraisal

Posted on 12th January 2018 by

Learning Resources
Coloured pencils in a pot

Critical appraisal tools (CATs) are commonly used by students and researchers alike, as a way of judging a study’s quality. They are useful because of their structured approach, normally using categorical questions (yes/no) in standardised headings, which means that key components of study assessment are not missed e.g. internal and external validity.

The rise of FOAMed (Free Open Access Medical Education) means that learning about critical appraisal is easier than ever. Whether it’s modules or checklists, there are widespread resources all over the internet.

Somewhat ironically, an important thing to consider when using critical appraisal tools is that we need to ‘appraise’ them too. A review of CATs in 2011 showed that a comprehensive explanation of how a CAT is developed was found in only 11% of tools [1]. Similarly, only 23% and 75% of CATs had been tested for reliability and validity, respectively [1]. Whilst some questions in checklists are self-explanatory and straight-forward, there are others that are more difficult, for example ‘How precise were the results?’. Whilst many people would immediately look for the range of results and 95% confidence intervals, those with less experience in critical appraisal may need more guidance. A systematic review in 2004 showed that guidelines regarding administration were provided in only 43% of 121 published CATs [2].

Many studies have commented on the need for cross-communication between evidence-based medicine groups in order to reach a consensus and develop a suitable tool/s for critical appraisal that are valid, reliable and easily administered [2-4].

Until then, Students 4 Best Evidence have developed a non-exhaustive list of critical appraisal tools available on the internet from all over the world and we’d like you to contribute. There is no gold standard critical appraisal tool.

Are there ones that aren’t listed here? Which ones have you found useful? Let us know and we’ll update the list.  (Links to all of the CATs are at the bottom of the blog)

 

https://jbi.global/critical-appraisal-tools

http://amstar.ca/Amstar_Checklist.php

http://thesgem.com/2014/03/make-it-so-beem-appraisal-tools/

http://www.cebm.net/critical-appraisal/

http://www.casp-uk.net/casp-tools-checklists

https://www.gla.ac.uk/researchinstitutes/healthwellbeing/research/generalpractice/ebp/checklists/

http://clinicalevidence.bmj.com/x/set/static/ebm/toolbox/665062.html

http://www.sign.ac.uk/checklists-and-notes.html

www.equator-network.org/about-us

http://bestbets.org/links/BET-CA-worksheets.php

http://www.cardiff.ac.uk/specialist-unit-for-review-evidence/resources/critical-appraisal-checklists

http://www.dartmouth.edu/~library/biomed/guides/research/ebm-resources-materials.html

References

  1. Crowe, M. and L. Sheppard, A review of critical appraisal tools show they lack rigor: alternative tool structure is proposed. Journal of clinical epidemiology, 2011. 64(1): p. 79-89.
  2. Katrak, P., et al., A systematic review of the content of critical appraisal tools. BMC medical research methodology, 2004. 4(1): p. 22.
  3. Quigley, J., et al., Critical Appraisal Of Real World Evidence: Review Of Recommended And Commonly Used Tools. Value in Health, 2015. 18(7): p. A684.
  4. Glenny, A.-M., No ‘gold standard’ critical appraisal tool for allied health research. Evidence-based dentistry, 2005. 6(4): p. 100-101.

Tags:

Dennis Neuen

Dennis Neuen is a final year medical student from the University of Notre Dame Sydney in Australia. He is interested in rheumatology and evidence-based medicine. He previously studied jazz piano and history at the Australian National University. Dennis recently spent a month at Cochrane UK as part of his medical elective. View more posts from Dennis

Leave a Reply

Your email address will not be published. Required fields are marked *

Subscribe to our newsletter

You will receive our monthly newsletter and free access to Trip Premium.