To be useful, medical information should be relevant to everyday practice, correct (valid) and easy to obtain. Slawson and Shaughnessy describe a formula which relates these three factors in a "Usefulness equation":
Let's consider each of the elements of this equation. Validity is the hardest part of information mastery, and it is the element on which the Evidence-Based Medicine Working Group at McMasters University and other pioneers of evidence-based medicine have focused.(Sackett, 1997) Their work has focused on evaluting the validity of original research articles in the primary medical literature. However, physicians seek and get information from a wide variety of sources, not just journal articles. In fact, journal articles are used quite infrequently to answer clinical questions. Much more commonly, we ask colleagues, talk to pharmaceutical representatives, read review articles, and consult textbooks and other references. In addition, validity assessment is an acquired skill that can be fairly time-consuming. While we will teach you those skills in this course, it may not be realistic or even appropriate to expect all physicians to become experts at validity assessment. An alternate approach is for physicians to identify YODAs (Your Own Data Analyzer) in their communities, and high-quality evidence-based resources (such as the Journal of Family Practice POEMs feature or the Cochrane Database) to do validity assessments for them.
Relevance is based on the frequency that we are exposed to the clinical question in our practice and the type of evidence presented. Medical information can be categorized as either disease-oriented (e.g. pathophysiology, pharmacology, etiology) or patient-oriented (e.g. symptoms, mortality, days in the hospital, cost). Relevant information is therefore patient-oriented evidence, and it focuses on medical problems common to our practice.
One of the basic laws of human behavior is that we want to do the least amount of work we can get away with. That applies to searching for the answers to clinical questions. It is therefore important when considering the usefulness of medical information to balance the relevance and validity with the work needed. The most useful information is therefore very relevant to our practice, has high validity, and doesn't take much work to access. Let's consider apply this equation to some common sources of medical information:
My partner
My partner is likely to see the same patients as I do, so his or her relevance is good. Work is low; all I have to do is catch them in the hallway. Validity is likely to be variable, though, depending on their training in evidence-based medicine and their ability to critically appraise their own practice.
A pharmaceutical representative
While the work is usually low (too low, some might argue!), relevance and validity are questionable. Pharmaceutical reps often present disease-oriented evidence, and they may be more comfortable with a specialty rather than primary care perspective.
An article in the latest New England Journal of Medicine
This source of information generally has pretty good validity. However, the results may have been obtained in a carefully selected sample of patients in a tertiary care setting, and may not have much relevance for our primary care practice. In addition, the work needed to find and read such an article, and assess its validity, is considerable.
Richard Smith, editor of the British Medical Journal, has created a table which classifies different sources of medical information using the "Usefulness equation" (Smith, 1996):
Information source |
Relevance |
Validity |
Work |
Usefulness |
| Evidence-based textbook | High |
High |
Low |
High |
| Systematic review (evidence-based) | High |
High |
Low |
High |
Portable summary of
systematic reviews |
High |
High |
Low |
High |
| POEMs based resources: Journal of Family
Practice POEMs feature and Evidence-Based Practice newsletter |
High | High | Low | High |
| Internet in 10 years | High |
High |
Low |
High |
| Drug reference book (PDR) | High |
Mod |
Low |
High-mod |
| ACP Journal Club,
Evidence-Based Medicine |
Moderate |
High |
Low |
High-mod |
| Colleagues | High |
Mod |
Low |
High-mod |
| Practice guidelines (evidence-based) | Mod |
High |
Low |
High-mod |
| Cochrane Database of Systematic reviews | Mod-High |
High |
Mod-High |
High-mod |
| Standard textbook | High |
Low |
Low |
Mod |
| Standard journal review | High |
Mod |
Low |
Mod |
| Free medical newspapers | High |
Low |
Low |
Mod |
| CME lectures | Mod |
Mod |
Low |
Mod |
| CME small groups | High |
Mod |
Mod |
Mod |
| Consensus statements | Mod |
Mod |
Low |
Mod |
| Practice guidelines (consensus) | Mod |
Mod |
Low |
Mod |
| Online searching | Mod |
High |
High |
Mod |
| Journal articles | Low |
High |
High |
Low |
| Drug advertising | Mod |
Low |
Low |
Low |
| Drug company representatives | High |
Low |
Low |
Low |
| Mass media | Low |
Low |
Low |
Low |
| Internet now | Low |
Low |
High |
Low |
Note that some of the best sources of information have yet to be developed (the Internet in 10 years and a regularly updated evidence-based textbook), and some of the most widely available (journal articles, drug advertising, and drug company representatives) are the least useful. Such is the paradox of current medical information management!
By becoming a "Medical Information Master", you will learn to emphasize sources of information that are high in relevance and validity, and low in work. It doesn't mean working harder - it means working smarter, and making the best use of your limited time to get the information that matters for you and your patients.
Knowing where to look to answer clinical questions is an important skill. A somewhat different set of skills is needed to keep up with the literature. While "foraging" every month in the literature, you should look for POEMs and avoid DOEs. Don't know what that means? In the next section we'll teach you the difference between POEMs and DOEs, and how they lead the way toward information mastery.