Date Last Updated: 24/02/06

Levels of Evidence

When developing the evidence reports, studies of various designs are assessed using a grading system developed by NZGG (see Table 1).

There are many different evidence grading systems in international use. These systems are used to show how well a study was able to answer a question and how reliable or uncertain that answer is. Both study design and quality are considered in this process.

This CAM system has been devised to be compatible with the other grading systems used by NZGG and also to map to other international systems (see Table 2).

Due to the emerging nature of the evidence for complementary and alternative therapies, many studies are non-randomised or uncontrolled. Often no Level 1 or 2 evidence is available. Sometimes Level 1 or 2 studies cannot be carried out because it would involve a safety risk for participants. Sometimes it would be too difficult to carry out a Level 1 or 2 study large enough to measure rare effects. In these instances evidence is based on lower level studies.

Lower level evidence is subdivided into Levels 3 and 4. This serves to illustrate a progression that may occur when investigating CAM from Level 4 through Levels 3 and 2 to Level 1 evidence.

Although possible harms and adverse events are important aspects of any CAM, they are often only reported from lower level studies. Higher level evidence is not often available for the reasons stated above.

A range of expert opinion also exists. In other grading systems, this is usually included in a fifth level. This level of evidence has not been reviewed for these reports.

Table 1: Levels of evidence used in Evidence Reports for www.cam.org.nz.

Level of evidence		Where the evidence comes from
1	Evidence with a high degree of reliability	Studies that use well-tested methods to make comparisons in a fair way and where the results leave very little room for uncertainty.   Trial design: usually Level 1 studies are systematic reviews or large, high-quality randomised controlled studies.
2	Evidence with reliability but open to debate	Studies that use well-tested methods to make comparisons in a fair way but where the results leave room for uncertainty (for example, due to the size of the study, losses to follow-up or the method used for selecting groups for comparison).   Trial design: usually Level 2 studies are systematic reviews without consistent findings, small randomised controlled trials, randomised controlled trials in which large numbers of participants are lost to follow-up, or cohort studies.
3	Some evidence without a high degree of reliability	Studies where the results are doubtful because the study design does not guarantee that fair comparisons can be made.   Trial design: usually Level 3 studies are systematic reviews of case-control studies or individual case-control studies.
4	Some evidence but based on studies without comparable groups.	Studies where there is a high probability that results are due to chance (for example because there is no comparison group or because the groups compared were different at the outset of the study).   Trial design: usually cohort or case-control studies where the groups were not really comparable, or case-series studies.

 Table 2: Systems for grading the quality of individual studies

NZGG* CAM	NZGG /GATE†	SIGN‡	GRADE§	USPTF**	Oxford CEBM††	NHMRC 2000‡‡	2000 CCS CONSENSUS§§
Level 1	Good / +	++	High	Good	Level 1 abc	Level I	Level I
Level 2	Fair / ~	+	Moderate	Fair	Level 2 abc	Level II	Level II
Level 3 and 4	Poor / -	-	Low (very low)	Poor	Level 3 ab, and 4	Level III (1, 2, 3) and IV	Level III,IV and V

    *  New Zealand Guidelines Group
^†  Graphic Appraisal Tool for Epidemiology
^‡  Scottish Intercollegiate Guidelines Network
^§  Grading of Recommendations Assessment Development and Evaluation
** US Preventable Services Task Force
^†† Oxford Centre for Evidence-based Medicine
^‡‡ National Health and Medical Research Council
^§§ Canadian Cardiovascular Society 2000 Consensus

The levels of evidence are used in the Plain Language Summaries for specific questions. See Figure 1 for an example of how the levels of evidence are highlighted in the key messages section of the Plain Language Summaries. The definitions of the levels are in a key below the key messages, as well as highlighted in a scroll over function for each number to the right of the key message.

The key messages do not provide recommendations. There are methods for considering the evidence from multiple studies that address a specific question, incorporating trial design and for weighing competing factors in forming a recommendation. For more information on study design and for guidance on balancing the benefits and harms of an intervention the reader is referred to the following websites.

·      http://www.nzgg.org.nz

·      http://www.health.auckland.ac.nz/population-health/epidemiology-biostats/epiq/

·      http://www.cebm.net/levels_of_evidence.asp

·      http://www.ahrq.gov/clinic/ajpmsuppl/harris3.htm#table7

·      http://www.gradeworkinggroup.org/links.htm

·      http://www.sign.ac.uk/methodology/index.html

·      http://gacguidelines.ca/article.pl?sid=03/01/29/1642226&mode=thread

 

 

 

DISCLAIMER: This website does not provide specific medical advice and the information provided should not be used as a substitute for seeking medical advice from a registered health practitioner. Full disclaimer . . .