Laupacis and colleagues of the Evidence-Based Medicine Working Group have described several important questions regarding validity of an article about prognosis. They are paraphrased below:
1. Were the study subjects representative of patients with the disease in question?
Many studies are done in referral settings, where the sickest patients end up. The study may describe the prognosis of more severely ill patients, rather than the patients typically seen in the primary care setting.2. Were the patients at a similar point in the course of their disease?
In a study about breast cancer prognosis, patients should be at a similar stage in the disease. Otherwise, it is difficult to interpret the results.3. Were the outcomes well-defined, and were the people recording the outcomes blinded to the prognostic factors?
Mortality is a clearly defined outcome. Some studies, though, use more subjective outcomes. For example, a study of the decline in patients with Alzheimer's may have institutionalization as a primary outcome . Clearly, the decision to institutionalize is social and behavioral as well as medical.4. Were patients followed long enough for outcomes to occur?
It is important that patients are followed long enough for the outcome in question to occur. Five years is a minimum for most chronic illnesses, and ten years is preferable.5. Was the dropout rate excessive?
If too many patients are lost to follow-up, a serious bias may be introduced. Patients lost to follow-up may differ significantly in terms of severity and outcome of their disease. For a study to be valid, at least 70% to 80% of patients should be followed for the duration of the study.6. Did the authors adjust for differences between groups?
Because prognostic trials are usually cohort or case-control studies, and patients are not randomized to various prognostic factors, it is important that the researchers adjust for differences between groups. Common confounders include age, stage of disease, and comorbidities.
Exercise
You have a 64 year old patient with new onset congestive heart failure, NHYA Class I. He asks you what he can expect as his prognosis. You remember the article by Ho and colleagues, which looks at the prognosis of patients with congestive heart failure (CHF) from the Framingham Heart Study, and you promise to consult it before his next visit.
Before evaluating the validity, let's consider the relevance. Do the patients in this study look like our gentleman? On page 107, under "Study Population", we learn that over 5000 residents of Framingham aged 28 to 62 years were enrolled in 1948, and in 1971 5000 of their offspring were enrolled in the Framingham Offspring Study. Patients with CHF at the beginning of the study were excluded from the analysis. These patients were then followed longitudinally with regular follow-up. Are these patients generally similar to our patient?
Now, let's talk about the validity. Is this a:
This study design is a good choice for a study about prognosis, so we'll continue with our evaluation of validity. We will consider each of the criteria described above in turn:
1. Were the study subjects representative of patients with the disease in question?
2. Were the patients at a similar point in the course of their disease?
3. Were the outcomes well-defined, and were the people recording the outcomes blinded to the prognostic factors?
4. Were patients followed long enough for outcomes to occur?
5. Was the dropout rate excessive?
6. Did the authors adjust for differences between groups?
Congratulations! You've just examined this article, and found it to be a valid study of prognosis among patients with CHF. In the next section, we'll discuss how to interpret survival curves, the most common way that researchers present survival data.