.Evaluating the relevance of a meta-analysis
Now that we have reviewed the steps in a meta-analysis, we can develop a formal approach to their critical appraisal. As with other kinds of studies, there are three basic steps:
- Would the results change my practice if valid?
- Are the outcomes important to my patients?
- Are the results valid?
A study which meets all three of these criteria is a POEM (Patient-Oriented Evidence that Matters). The first two criteria have to do with relevance, while the third is concerned with validity.
1. Would the results change my practice if valid?
This question must be answered by the individual clinician. A meta-analysis which confirms existing practice need not be subjected to rigorous evaluation, since they results don't change what you are already doing. On the other hand, if the results would change what you do for your patients, it is incumbent on you to take the next step.
2. Are the outcomes important to my patients?
Most meta-analyses use patient-oriented outcomes such as morbidity, mortality, and symptoms. However, if a study uses surrogate or intermediate outcomes such as FEV1 or hemoglobinA1C, the results should be interpreted cautiously. Proponents of the use of POEMs would argue that if a meta-analysis does not use patient-oriented outcomes, you don't have to proceed to the next step.
3. Are the results valid?
If a study would change practice, and uses patient-oriented outcomes, you are obligated to evaluate its validity. Key criteria include:
- Did the authors ask a focused clinical question?
- Were the criteria used to select articles for inclusion appropriate?
- Is it unlikely that important, relevant studies were missed?
- Was the validity of the included studies appraised (study quality)?
- Were assessments of studies reproducible (data abstraction)?
- Were the results similar from study to study (homogeneity)?
In the next section, we will go into more detail on the evaluation of validity.
For more on this approach to evaluating relevance, see the section on POEMs and DOEs .