It would be hard to find anyone or any group today making health policy recommendations who doesn't claim their conclusions are "evidence-based." But since recommendations vary considerably in some areas, such as whether there is a health benefit to reducing dietary sodium/salt, it should be important to all of us exactly how "evidence-based medicine" (EBM) works and how it fits into the overall process of producing health care or health policy decisions.

An interview in the June 19 issue of Health Affairs magazine illuminates the limits of EBM and the shadowy abuses of the term.

Junkfoodscience.com has taken the American Heart Association to task for the poor quality of its evidentiary review to support its lifestyle recommendations for women. Writing of the American Heart Association's Evidence-based Guidelines for Cardiovascular Disease Prevention in Women released in May, Sandy Szwarc:

Not one observational study was able to credibly support the AHA heart healthy eating recommendations for women to prevent heart disease or premature death. The only observational study specifically looking at Healthy Eating in accordance with our government's dietary guidelines found no benefit. And finally, the strongest evidence - an actual clinical trial of the heart healthy diet on the primary prevention of heart disease in women, that went on for more than 8 years - found it had no effect on heart disease. Reviews of clinical trials conducted on heart healthy programs to date have found them of doubtful effectiveness, with no effect on mortality. Our beliefs in healthy eating have gone far beyond well-founded advice to eat normally and enjoy a variety of foods in order to prevent deficiencies, fuel our bodies, and for pleasure; to beliefs in special powers of foods as medicines or poisons.

This review looked at the evidence being used to support "evidence-based" recommendations for a heart healthy diet. When we hear the term "evidence-based," most of us probably had a very different picture in our minds.

While the AHA calls for rigorous public policies to implement its preventive guidelines population-wide in order to "combat the pandemic of heart disease in women," how many politicians and healthcare professionals will have taken the time to look at the evidence behind these recommendations? But we will have, and can make a more informed choice about what we want to eat.

This article is the Szwarc's second. The first , published May 2, pointed out:

A major medical paper on primary heart disease prevention admitted that cardiovascular disease risk factors have proven useless for predicting heart disease among our population and that reducing risks factors doesn't translate into reduced clinical disease or fewer premature deaths.

Sounds like our complaints about self-proclaimed "evidence-based" labels need a truth-in-labeling watchdog.

The American Medical Association House of Delegates will gather later this month in Chicago, where it will consider a vast array of resolutions directing their lobbyists in Washington as well as advising their physician members throughout the country.

Resolution 611 deals with "Evidence-Based Policy Development" and would require

RESOLVED, That our American Medical Association House of Delegates resolutions should include, whenever possible or applicable, appropriate reference citations to facilitate independent review by delegates prior to policy development. (Directive to Take Action)

Great idea -- as far as it goes. But evidence-based public health is more than a platitude . Two problems: timing and content.

We can hope that the controversy generated when AMA approved an anti-salt resolution last year might be the genesis of this year's resolution to get serious about basing resolutions on evidence. Evidence-based considerations were missing-in-action last year. Opinion trumped facts. So the timing's bad; it puts the cart before the horse.

The content is also suspect. It's a tepid gesture that will allow any proposer to claim his or her resolution is "evidence-based" merely by including literature citations. That's even worse than what has been passed-off as "evidence-based" policy recommendations by the experts on the Dietary Guidelines Advisory Committee (DGAC). We had recommended the DGAC model their analysis on international "evidence-based" standards. They didn't. Just as everyone wants to be felt to be fair, honest, to say nothing of attractive, witty, etc., self-proclaiming these virtues can often be wildly misleading.

What is "evidence-based" health policy?

Let's give credit where it's due. Back in 1972 A.L. "Archie" Cochrane penned his pathbreaking book, Effectiveness and Efficiency: Random Reflections on Health Services. This is the original textbook on "evidence-based medicine", a term familiar to most doctors and other healthcare professionals today. Cochrane's classic text has had a profound influence on the practice of medicine and on the evaluation of medical interventions. He was the first to set out clearly the vital importance of randomized controlled trials (RCTs) in assessing the effectiveness of treatments, and his work led directly to the setting-up of the Cochrane Collaboration , now a world-wide endeavor dedicated to tracking down, evaluating and synthesizing RCTs in all areas of medicine .

In my reading on the subject, for a recommendation to be evidence-based it should be developed using a structured and rigorous methodology based on a pre-determined set of criteria for grading the strength of any proposed recommendation. Consideration would be limited, most likely, to evidence found in randomized controlled trials and comparative controlled trials identified and synthesized using methods defined by the Cochrane Collaboration. An expert panel should first define evidence of outcomes important to individual subjects and, in aggregate, to the population, by determining the required strength of the evidence considered valid for inclusion in the review. Then, in blinded fashion, the evidence should then be graded against those criteria. The Cochrane methodology is preferred as a means to minimize bias since it outlines an objective and systematic approach to literature search, study selection, data extraction and data synthesis. Anything short of this dilutes the meaning of "evidence-based." To call the AMA approach "evidence-based" is whistling past the park.

Cochrane's home base, Oxford University, hosts a Centre for Evidence-based Medicine which answers the question "What is EBM? by extracting from an editorial in the British Medical Journal back in 1996:

Evidence-Based Medicine, whose philosophical origins extend back to mid-19th century Paris and earlier, remains a hot topic for clinicians, public health practitioners, purchasers, planners, and the public. There are now frequent workshops in how to practice and teach it; undergraduate and post-graduate training programmes are incorporating it (or pondering how to do so); British centres for evidence-based practice have been established or planned in adult medicine, child health, surgery, pathology, pharmacotherapy, nursing, general practice, and dentistry; the Cochrane Collaboration and the York Centre for Review and Dissemination in York are providing systematic reviews of the effects of health care; new evidence-based practice journals are being launched; and it has become a common topic in the lay media. But enthusiasm has been mixed with some negative reaction. Criticism has ranged from evidence-based medicine being old-hat to it being a dangerous innovation, perpetrated by the arrogant to serve cost-cutters and suppress clinical freedom. As evidence-based medicine continues to evolve and adapt, now is a useful time to refine the discussion of what it is and what it is not.

Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence-based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research. By individual clinical expertise we mean the proficiency and judgement that individual clinicians acquire through clinical experience and clinical practice. Increased expertise is reflected in many ways, but especially in more effective and efficient diagnosis and in the more thoughtful identification and compassionate use of individual patients' predicaments, rights, and preferences in making clinical decisi ons about their care. By best available external clinical evidence we mean clinically relevant research, often from the basic sciences of medicine, but especially from patient centred clinical research into the accuracy and precision of diagnostic tests (including the clinical examination), the power of prognostic markers, and the efficacy and safety of therapeutic, rehabilitative, and preventive regimens. External clinical evidence both invalidates previously accepted diagnostic tests and treatments and replaces them with new ones that are more powerful, more accurate, more efficacious, and safer.

Good doctors use both individual clinical expertise and the best available external evidence, and neither alone is enough. Without clinical expertise, practice risks becoming tyrannised by evidence, for even excellent external evidence may be inapplicabl e to or inappropriate for an individual patient. Without current best evidence, practice risks becoming rapidly out of date, to the detriment of patients.

This description of what evidence-based medicine is helps clarify what evidence-based medicine is not. Evidence-based medicine is neither old-hat nor impossible to practice. The argument that everyone already is doing it falls before evidence of striking variations in both the integration of patient values into our clinical behaviour and in the rates with which clinicians provide interventions to their patients. The difficulties that clinicians face in keeping abreast of all the medical advances reported in primary journals are obvious from a comparison of the time required for reading (for general medicine, enough to examine 19 articles per day, 365 days pe r year) with the time available (well under an hour per week by British medical consultants, even on self-reports.

The argument that evidence-based medicine can be conducted only from ivory towers and armchairs is refuted by audits in the front lines of clinical care where at least some inpatient clinical teams in general medicine, psych iatry (JR Geddes, et al, Royal College of Psychiatrists winter meeting, January 1996), and surgery (P McCulloch, personal communication) have provided evidence-based care to the vast majority of their patients. Such studies show that busy clinicians who devote their scarce reading time to selective, efficient, patient-driven searching, appraisal and incorporation of the best available evidence can practice evidence-based medicine.

Evidence-based medicine is not "cook-book" medicine. Because it requires a bottom-up approach that integrates the best external evidence with individual clinical expertise and patient-choice, it cannot result in slavish, cook-book approaches to individual patient care. External clinical evidence can inform, but can never replace, individual clinical expertise, and it is this expertise that decides whether the external evidence applies to the individual patient at all and, if so, how it should be integrated into a clinical decision. Similarly, any external guideline must be integrated with individual clinical expertise in deciding whether and how it matches the patient's clinical state, predicament, and preferences, and thus whether it should be applied. Clinicians who fear top-down cook-books will find the advocates of evidence-based medicine joining them at the barricades.

Evidence-based medicine is not cost-cutting medicine. Some fear that evidence-based medicine will be hijacked by purchasers and managers to cut the costs of health care. This would not only be a misuse of evidence-based medicine but suggests a fundamental misunderstanding of its financial consequences. Doctors practising evidence-based medicine will identify and apply the most efficacious interventions to maximise the quality and quantity of life for individual patients; this may raise rather than lower the cost of their care.

Evidence-based medicine is not restricted to randomised trials and meta-analyses. It involves tracking down the best external evidence with which to answer our clinical questions. To find out about the accuracy of a diagnostic test, we need to find proper cross-sectional studies of patients clinically suspected of harbouring the relevant disorder, not a randomised trial. For a question about prognosis, we need proper follow-up studies of patients assembled at a uniform, early point in the clinical course of their disease. And sometimes the evidence we need will come from the basic sciences such as genetics or immunology. It is when asking questions about therapy that we should try to avoid the non-experimental approaches, since these routinely lead to false-positive conclusions about efficacy. Because the randomised trial, and especially the systematic review of several randomised trials, is so much more likely to inform us and so much less likely to mislead us, it has become the "gold standard" for judging whether a treatment does more good than harm. However, some questions about therapy do not require randomised trials (successful interventions for otherwise fatal conditions) or cannot wait for the trials to be conducted. And if no randomised trial has been carried out for our patient's predicament, we follow the trail to the next best external evidence and work from there.

Despite its ancient origins, evidence-based medicine remains a relatively young discipline whose positive impacts are just beginning to be validated, and it will continue to evolve. This evolution will be enhanced as seve ral undergraduate, post-graduate, and continuing medical education programmes adopt and adapt it to their learners' needs. These programmes, and their evaluation, will provide further information and understanding about what evidence-based medicine is, and what it is not.

Authors:

David L. Sackett, Professor, NHS Research and Development Centre for Evidence-Based Medicine, Oxford. William M. C. Rosenberg, Clinical Tutor in Medicine, Nuffield Department of Clinical Medicine, Oxford. J. A. Muir Gray, Director of Research and Development, Anglia and Oxford Regional Health Auhtority, Milton Keynes R. Brian Haynes, Professor of Medicine and Clinical Epidemiology, McMaster University Hamilton, Canada W. Scott Richardson, Rochester, USA

The AMA does not welcome "outside" comments (hence this blog), but AMA members may wish to share their thoughts on how their association may be over-reaching by claiming an evidence-based procedure. The contact is Susan L. Hubbell, MD, chair, Reference Committee F, and, again, the proposal is Resolution 611 .

So says Elizabeth Bromstein writing in NOW Magazine (Toronto). "I don't think anybody ever had a heart attack from drinking too much mineral water or vegetable juice," she says, noting that while some think we ingest too much salt, "others say the warnings are way out of whack." So she did her own research and reported:

What the experts say

"Around 1900 we had an average sodium intake of 200 mg a day, and now we have an average intake of 5,000 mg. Sodium has to work with potassium and magnesium, and while our salt intake has increased, our potassium and magnesium intake has gone down. The increase is mostly due to processed foods, but if you eat a diet low in processed foods and high in fruits and vegetables, your salt level is probably fine. Your potassium and magnesium are probably fine as well, and you can add table salt to foods."

AILEEN BURFORD MASON, immunologist and nutritional consultant, Toronto

"In Canada, 25 per cent of people have hypertension, and one in three would not have it if his or her sodium intake were lower. Processors and restaurants are responsible for 80 per cent of our salt intake. We add 10 per cent ourselves, and 10 per cent occurs naturally in food. If we asked restaurants to cut the amount, we'd see a dramatic improvement. Hypertension causes two-thirds of strokes, one-half of all cases of heart failure and one-quarter of all cases of kidney failure and heart attack. Hypertension is one of the major drivers of dementia. There are more effective ways of reducing hypertension than reducing salt, like increasing physical activity, losing weight, getting enough soluble fibre and eating low-fat dairy products."

NORMAN CAMPBELL, professor of medicine, Libin Cardiovascular Institute, University of Calgary

"We know salt is associated with blood pressure, so we concluded that anything we could do to reduce blood pressure would achieve the same risk profile of lower-risk populations, but that hasn't turned out to be true. Some studies even suggest that there may be an increased risk for cardiovascular disease with a reduced salt diet, since it increases insulin resistance and can affect plasma renin activity. Until 12 years ago, we did not look at the net effects [of reducing salt] and only looked at blood pressure. Even then, about a third of the population responded [positively] to salt restriction. What we need is a five-year controlled intervention trial."

RICHARD L. HANNEMAN, president, Salt Institute, Alexandria, Virginia

"Salt is one of those essential elements we need in moderation. There are people who are salt-sensitive. Your intake should depend on your sensitivity level. Salt helps stimulate the kidneys, helps promote fluid metabolism and has a moistening effect. A little bit [taken internally] is good if your skin is very dry. It also has a mild detoxifying effect. In Chinese medicine it is also known as a s oftener for hardened lymph nodes, glands or muscles. It gently promotes bowel regularity. Most importantly, it needs to be kept in balance with potassium. Aside from causing hypertension, too much salt can interfere with calcium absorption and lead to poor bone health. If you have PMS and bloating, it's important to reduce your salt intake as well."

DU LA, naturopath, Toronto

Not to pick a quarrel with experts (especially since Ms. Bromstein kindly denotes me one), but Ms. Mason is just flat-out wrong about salt intakes increasing from 200 mg/day to 5,000 mg/day over the past century. In fact, sodium intakes are virtually the same at about 3,500 mg/day. Dr. Campbell employs the classically flawed extrapolation of blood pressure to heath outcomes; a model rejected by direct health outcomes studies. And the solution to the "bloating" probem identified by Mr. Du is to drink more water, not consume less salt.

That said, the experts also made some important points: Ms. Mason points out any health problem owes to the imbalance of sodium on one side and potassium and magnesium on the other, advising to eat more fruits and vegetables so that your "potassium and magnesiium are probably fine as well and you can add table salt to foods." Balance and moderation: good advice. Though limiting his comments to blood pressure and not health outcomes, Dr. Campbell concedes the same point: "There are more effective ways of reducing hypertension than reducing salt, like increasing physical activity, losing weight, getting enough soluble fibre and eating low-fat dairy products." And Mr. Du agrees: "Salt is one of those essential elements we need in moderation. ... Most importantly, it needs to be kept in balance with potassium."

Thank you, Ms. Bromstein for your reasoned advice. For more on the Salt Institute perspective see our summary and reference citations .