« May 2007 | Main | July 2007 »

June 30, 2007

Will politically-correct search engine censorship stifle medical science?

One reason MSM journalists are losing their audience is the open secret that much of what is reported as "news" comes pre-packaged in canned stories from various advocacy groups and advertisers. Add that to the herd instinct that produces "PC"-slanted reporting and it's not difficult to poke holes in what we read in the newspapers or see on TV.

Cyber-journalism has the opposite problem. Rather than lemming-like PC stories based on pooled source information like wire services, bloggers and Web authors come in all shapes and sizes of quality and credibility. Who to believe?

When it comes to making sense of the chaos of Internet health reporting, Google is trying to intervene -- to make money and, they hope, to help seekers of quality information pertaining to their precious personal health. Query: will this be another case where the surgery is successful, but the patient dies? Will the choices of the Google censors preserve the essence of scientific inquiry where competing ideas are bombarded with data that either confirms or destroys them? Or will the desire to "help" consumers understand the meaning of medical scholarship excise aberrant findings, leaving only politically-correct interpretations?

We share the concerns of Sandra Szwarc in Junkfoodscience.com:

Search engines have inordinate abilities to censor information by simply making it invisible to searches. It is not uncommon for key documents and papers that don’t support government initiatives or special interest agendas to be buried and take extraordinary effort to hunt down, or to disappear from the internet altogether, something anyone who’s been researching for any length of time quickly discovers...

Google has just announced that it has created a “Google Health Advisory Council.” ... Says Google: "We want to help users make more empowered and informed healthcare decisions, and have been steadily developing our ability to make our search results more medically relevant and more helpful to users."

Screening out “irrelevant” and “unhelpful” information? That sounds like a euphemism for censorship. Look at every name on their new prestigious advisory panel and the interests they represent. Most will be familiar to Junkfood Science readers, as we've examined the soundness of their consumer information — RAND Corp., the Cleveland Clinic, the AMA, Robert Wood Johnson Foundation, AARP, Kaiser Foundation Health Plan, Inc., California HealthCare Foundation, and others.

Despite all of the flaws and utter garbage on the net, it’s still been the primary way for most people to break through the media groupthink to learn other viewpoints and sounder information. This development could be the beginnings of the world's biggest internet information firewall.

Substituting a Google censor for the rigor of true "evidence-based" analysis would be a clear step backwards. Search engines seeking access to the Chinese market, reportedly, have agreed to censor their search results. That's unfortunate for 1.3 billion Chinese and a disturbing parallel to the new Google Health Advisory Council. What we need in public health policy is more transparency and solid information, not greater opaqueness and opinion. As we blogged recently, the quality of the process is of paramount importance. We need to be able to "lift the hood and kick the tires" of new medical studies, not have a secret censor decide for us what's relevant and what's not.


June 23, 2007

Medical evidence: why process counts in health policy decisions

It would be hard to find anyone or any group today making health policy recommendations who doesn't claim their conclusions are "evidence-based." But since recommendations vary considerably in some areas, such as whether there is a health benefit to reducing dietary sodium/salt, it should be important to all of us exactly how "evidence-based medicine" (EBM) works and how it fits into the overall process of producing health care or health policy decisions.

An interview in the June 19 issue of Health Affairs magazine illuminates the limits of EBM and the shadowy abuses of the term.

The interview is a conversation between Sean Tunis who has contributed his insights working in the federal Office of Technology Assessment and the Centers for Medicare and Medicaid Services who has now founded and directs the Center for Medical Technology Policy in San Francisco and David Eddy, founder and medical director of Archimedes Inc. in Aspen, Colorado.

“Many people think that EBM is a more or less automated process that delivers a final answer. It does not,” Eddy explains. Rather, EBM is the first step in a multiple-step process. After EBM determines what the evidence is, the process requires someone to apply this information about health outcomes of the options in making preferences and value judgments leading to a decision or policy. Claiming the output is evidence-based, even if true, doesn't tell the whole story.

Eddy continues: "the main contribution of EBM is to improve the first step. It establishes principles and methods for how that step should be taken. That might seem like a relatively small contribution, but it’s huge." He points out that, prior to EBM, an authority figure or reference to the consensus of practicing physicians would be the accepted basis for decisions.

If evidence was cited at all, it would likely have been a few papers cherry-picked to support a particular point of view. So the first contribution of EBM is to change the anchor for the decision from the beliefs of experts to evidence of effectiveness. If we stopped there, that would be very big. But there are several other benefits. One is the process for creating guidelines and related policies. In the recent past, a guideline would have been set by convening a group of experts in a room and asking them what they thought the appropriate practice was. The job might be completed in a day, and the guideline issued as a one-page statement. ... There would be no systematic search of the literature, no explicit description of the evidence, no statistical analysis of clinical trials, no formal synthesis of the evidence, no written rationale, or anything like that. Today that personal, informal, incomplete, undocumented, and easily biased approach has been replaced by a much more formal, systematic, explicit, informative, and reproducible approach. ... (B)y forcing a systematic evaluation of the evidence, it shines a spotlight on the quality of the evidence. Prior to EBM, most people, including physicians, simply assumed that if something was in common practice, there must be good evidence for it. It wasn’t until we began to explicitly focus on evidence—to demand to see it—that we learned that there were huge gaps in the evidence. This revelation has had at least two beneficial effects. First, it informs everyone about the lack of evidence, so people can be more realistic in their expectations and more aware of the uncertainty in their decisions. Second, it stimulates the search for better evidence and participation in clinical trials.

Moreover, Eddy declares, EBM creates an "evidence threshold" for policy.

While EBM does not tell us whether the benefits of a treatment outweigh its risks and costs, it does tell us that nothing should be affirmatively promoted unless there is good evidence of at least some benefit. A simple belief of effectiveness is not good enough.

Tunis agreed and added:

It helps highlight those aspects of decision making where differences in the decision are not primarily disagreements about evidence, but rather reflect differences of values, preferences, and perspective. The implication of this is that further improvements in evidence-based decision making will probably require greater awareness when differences among stakeholders are primarily nonscientific issues and better approaches to arriving at acceptable decisions despite those value differences.

A primary "value judgment" explored in the interview is the absence from EBM of any consideration of cost -- certainly something taxpayers might wish was intrinsic to the Medicare and Medicaid programs, but which Tunis laments in its absence.

And I think that all of this is consistent with what you have said about how costs might differentially influence the judgments of different stakeholders. My view is that Medicare decisionmakers didn’t consider costs any more or less than the members of the ACC (American College of Cardiology)/ AHA (American Heart Association) guideline committee did, but I think that our implicit level of concern about costs was different and perhaps influenced the analysis of evidence and conclusions drawn by each organization.

The entire interview is worth a read, but the take-away message is that EBM is an explicit, disciplined process, but it only describes the evidence-base. From there, policy advocates apply their various perspectives and values in advocating policy. Let's get the process straightened out. Let's agree on the evidence base. Then, perhaps, we can move beyond the current situation on salt and health where advocates of salt reduction point loudly to evidence pertaining to an intermediate variable (blood pressure) while skeptics in medical schools and industry keep saying: not so fast in moving towards "implementation" -- let's get the evidence sorted out properly as the first step.

This kind of article should be required reading for the federal Dietary Guidelines Advisory Committee.

June 18, 2007

American Heart Assn guidelines lambasted for poor science base

Junkfoodscience.com has taken the American Heart Association to task for the poor quality of its evidentiary review to support its lifestyle recommendations for women. Writing of the American Heart Association's Evidence-based Guidelines for Cardiovascular Disease Prevention in Women released in May, Sandy Szwarc:

Not one observational study was able to credibly support the AHA heart healthy eating recommendations for women to prevent heart disease or premature death. The only observational study specifically looking at Healthy Eating in accordance with our government’s dietary guidelines found no benefit. And finally, the strongest evidence — an actual clinical trial of the heart healthy diet on the primary prevention of heart disease in women, that went on for more than 8 years — found it had no effect on heart disease. Reviews of clinical trials conducted on heart healthy programs to date have found them of doubtful effectiveness, with no effect on mortality. Our beliefs in healthy eating have gone far beyond well-founded advice to eat normally and enjoy a variety of foods in order to prevent deficiencies, fuel our bodies, and for pleasure; to beliefs in special powers of foods as medicines or poisons.

This review looked at the evidence being used to support “evidence-based” recommendations for a heart healthy diet. When we hear the term “evidence-based,” most of us probably had a very different picture in our minds.

While the AHA calls for rigorous public policies to implement its preventive guidelines population-wide in order to “combat the pandemic of heart disease in women,” how many politicians and healthcare professionals will have taken the time to look at the evidence behind these recommendations? But we will have, and can make a more informed choice about what we want to eat.

This article is the Szwarc's second. The first, published May 2, pointed out:

A major medical paper on primary heart disease prevention admitted that cardiovascular disease risk factors have proven useless for predicting heart disease among our population and that reducing risks factors doesn’t translate into reduced clinical disease or fewer premature deaths.

Sounds like our complaints about self-proclaimed "evidence-based" labels need a truth-in-labeling watchdog.

June 16, 2007

The cart and the horse

The American Medical Association House of Delegates will gather later this month in Chicago, where it will consider a vast array of resolutions directing their lobbyists in Washington as well as advising their physician members throughout the country.

Resolution 611 deals with "Evidence-Based Policy Development" and would require

RESOLVED, That our American Medical Association House of Delegates resolutions should include, whenever possible or applicable, appropriate reference citations to facilitate independent review by delegates prior to policy development. (Directive to Take Action)

Great idea -- as far as it goes. But evidence-based public health is more than a platitude. Two problems: timing and content.

We can hope that the controversy generated when AMA approved an anti-salt resolution last year might be the genesis of this year's resolution to get serious about basing resolutions on evidence. Evidence-based considerations were missing-in-action last year. Opinion trumped facts. So the timing's bad; it puts the cart before the horse.

The content is also suspect. It's a tepid gesture that will allow any proposer to claim his or her resolution is "evidence-based" merely by including literature citations. That's even worse than what has been passed-off as "evidence-based" policy recommendations by the experts on the Dietary Guidelines Advisory Committee (DGAC). We had recommended the DGAC model their analysis on international "evidence-based" standards. They didn't. Just as everyone wants to be felt to be fair, honest, to say nothing of attractive, witty, etc., self-proclaiming these virtues can often be wildly misleading.

What is "evidence-based" health policy?

Let's give credit where it's due. Back in 1972 A.L. "Archie" Cochrane penned his pathbreaking book, Effectiveness and Efficiency: Random Reflections on Health Services. This is the original textbook on "evidence-based medicine", a term familiar to most doctors and other healthcare professionals today. Cochrane's classic text has had a profound influence on the practice of medicine and on the evaluation of medical interventions. He was the first to set out clearly the vital importance of randomized controlled trials (RCTs) in assessing the effectiveness of treatments, and his work led directly to the setting-up of the Cochrane Collaboration, now a world-wide endeavor dedicated to tracking down, evaluating and synthesizing RCTs in all areas of medicine.

In my reading on the subject, for a recommendation to be evidence-based it should be developed using a structured and rigorous methodology based on a pre-determined set of criteria for grading the strength of any proposed recommendation. Consideration would be limited, most likely, to evidence found in randomized controlled trials and comparative controlled trials identified and synthesized using methods defined by the Cochrane Collaboration. An expert panel should first define evidence of outcomes important to individual subjects and, in aggregate, to the population, by determining the required strength of the evidence considered valid for inclusion in the review. Then, in blinded fashion, the evidence should then be graded against those criteria. The Cochrane methodology is preferred as a means to minimize bias since it outlines an objective and systematic approach to literature search, study selection, data extraction and data synthesis. Anything short of this dilutes the meaning of "evidence-based." To call the AMA approach "evidence-based" is whistling past the park.

Cochrane's home base, Oxford University, hosts a Centre for Evidence-based Medicine which answers the question "What is EBM? by extracting from an editorial in the British Medical Journal back in 1996:

Evidence-Based Medicine, whose philosophical origins extend back to mid-19th century Paris and earlier, remains a hot topic for clinicians, public health practitioners, purchasers, planners, and the public. There are now frequent workshops in how to practice and teach it; undergraduate and post-graduate training programmes are incorporating it (or pondering how to do so); British centres for evidence-based practice have been established or planned in adult medicine, child health, surgery, pathology, pharmacotherapy, nursing, general practice, and dentistry; the Cochrane Collaboration and the York Centre for Review and Dissemination in York are providing systematic reviews of the effects of health care; new evidence-based practice journals are being launched; and it has become a common topic in the lay media. But enthusiasm has been mixed with some negative reaction. Criticism has ranged from evidence-based medicine being old-hat to it being a dangerous innovation, perpetrated by the arrogant to serve cost-cutters and suppress clinical freedom. As evidence-based medicine continues to evolve and adapt, now is a useful time to refine the discussion of what it is and what it is not.

Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence-based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research. By individual clinical expertise we mean the proficiency and judgement that individual clinicians acquire through clinical experience and clinical practice. Increased expertise is reflected in many ways, but especially in more effective and efficient diagnosis and in the more thoughtful identification and compassionate use of individual patients' predicaments, rights, and preferences in making clinical decisi ons about their care. By best available external clinical evidence we mean clinically relevant research, often from the basic sciences of medicine, but especially from patient centred clinical research into the accuracy and precision of diagnostic tests (including the clinical examination), the power of prognostic markers, and the efficacy and safety of therapeutic, rehabilitative, and preventive regimens. External clinical evidence both invalidates previously accepted diagnostic tests and treatments and replaces them with new ones that are more powerful, more accurate, more efficacious, and safer.

Good doctors use both individual clinical expertise and the best available external evidence, and neither alone is enough. Without clinical expertise, practice risks becoming tyrannised by evidence, for even excellent external evidence may be inapplicabl e to or inappropriate for an individual patient. Without current best evidence, practice risks becoming rapidly out of date, to the detriment of patients.

This description of what evidence-based medicine is helps clarify what evidence-based medicine is not. Evidence-based medicine is neither old-hat nor impossible to practice. The argument that everyone already is doing it falls before evidence of striking variations in both the integration of patient values into our clinical behaviour and in the rates with which clinicians provide interventions to their patients. The difficulties that clinicians face in keeping abreast of all the medical advances reported in primary journals are obvious from a comparison of the time required for reading (for general medicine, enough to examine 19 articles per day, 365 days pe r year) with the time available (well under an hour per week by British medical consultants, even on self-reports.

The argument that evidence-based medicine can be conducted only from ivory towers and armchairs is refuted by audits in the front lines of clinical care where at least some inpatient clinical teams in general medicine, psych iatry (JR Geddes, et al, Royal College of Psychiatrists winter meeting, January 1996), and surgery (P McCulloch, personal communication) have provided evidence-based care to the vast majority of their patients. Such studies show that busy clinicians who devote their scarce reading time to selective, efficient, patient-driven searching, appraisal and incorporation of the best available evidence can practice evidence-based medicine.

Evidence-based medicine is not "cook-book" medicine. Because it requires a bottom-up approach that integrates the best external evidence with individual clinical expertise and patient-choice, it cannot result in slavish, cook-book approaches to individual patient care. External clinical evidence can inform, but can never replace, individual clinical expertise, and it is this expertise that decides whether the external evidence applies to the individual patient at all and, if so, how it should be integrated into a clinical decision. Similarly, any external guideline must be integrated with individual clinical expertise in deciding whether and how it matches the patient's clinical state, predicament, and preferences, and thus whether it should be applied. Clinicians who fear top-down cook-books will find the advocates of evidence-based medicine joining them at the barricades.

Evidence-based medicine is not cost-cutting medicine. Some fear that evidence-based medicine will be hijacked by purchasers and managers to cut the costs of health care. This would not only be a misuse of evidence-based medicine but suggests a fundamental misunderstanding of its financial consequences. Doctors practising evidence-based medicine will identify and apply the most efficacious interventions to maximise the quality and quantity of life for individual patients; this may raise rather than lower the cost of their care.

Evidence-based medicine is not restricted to randomised trials and meta-analyses. It involves tracking down the best external evidence with which to answer our clinical questions. To find out about the accuracy of a diagnostic test, we need to find proper cross-sectional studies of patients clinically suspected of harbouring the relevant disorder, not a randomised trial. For a question about prognosis, we need proper follow-up studies of patients assembled at a uniform, early point in the clinical course of their disease. And sometimes the evidence we need will come from the basic sciences such as genetics or immunology. It is when asking questions about therapy that we should try to avoid the non-experimental approaches, since these routinely lead to false-positive conclusions about efficacy. Because the randomised trial, and especially the systematic review of several randomised trials, is so much more likely to inform us and so much less likely to mislead us, it has become the “gold standard” for judging whether a treatment does more good than harm. However, some questions about therapy do not require randomised trials (successful interventions for otherwise fatal conditions) or cannot wait for the trials to be conducted. And if no randomised trial has been carried out for our patient’s predicament, we follow the trail to the next best external evidence and work from there.

Despite its ancient origins, evidence-based medicine remains a relatively young discipline whose positive impacts are just beginning to be validated, and it will continue to evolve. This evolution will be enhanced as seve ral undergraduate, post-graduate, and continuing medical education programmes adopt and adapt it to their learners’ needs. These programmes, and their evaluation, will provide further information and understanding about what evidence-based medicine is, and what it is not.

Authors:

David L. Sackett, Professor, NHS Research and Development Centre for Evidence-Based Medicine, Oxford.
William M. C. Rosenberg, Clinical Tutor in Medicine, Nuffield Department of Clinical Medicine, Oxford.
J. A. Muir Gray, Director of Research and Development, Anglia and Oxford Regional Health Auhtority, Milton Keynes
R. Brian Haynes, Professor of Medicine and Clinical Epidemiology, McMaster University Hamilton, Canada
W. Scott Richardson, Rochester, USA

The AMA does not welcome "outside" comments (hence this blog), but AMA members may wish to share their thoughts on how their association may be over-reaching by claiming an evidence-based procedure. The contact is Susan L. Hubbell, MD, chair, Reference Committee F, and, again, the proposal is Resolution 611.

June 14, 2007

"Salt paranoia seems a little silly"

So says Elizabeth Bromstein writing in NOW Magazine (Toronto). "I don't think anybody ever had a heart attack from drinking too much mineral water or vegetable juice," she says, noting that while some think we ingest too much salt, "others say the warnings are way out of whack." So she did her own research and reported:

What the experts say

"Around 1900 we had an average sodium intake of 200 mg a day, and now we have an average intake of 5,000 mg. Sodium has to work with potassium and magnesium, and while our salt intake has increased, our potassium and magnesium intake has gone down. The increase is mostly due to processed foods, but if you eat a diet low in processed foods and high in fruits and vegetables, your salt level is probably fine. Your potassium and magnesium are probably fine as well, and you can add table salt to foods."

AILEEN BURFORD MASON, immunologist and nutritional consultant, Toronto

"In Canada, 25 per cent of people have hypertension, and one in three would not have it if his or her sodium intake were lower. Processors and restaurants are responsible for 80 per cent of our salt intake. We add 10 per cent ourselves, and 10 per cent occurs naturally in food. If we asked restaurants to cut the amount , we'd see a dramatic improvement. Hypertension causes two-thirds of strokes, one-half of all cases of heart failure and one-quarter of all cases of kidney failure and heart attack. Hypertension is one of the major drivers of dementia. There are more effective ways of reducing hypertension than reducing salt, like increasing physical activity, losing weight, getting enough soluble fibre and eating low-fat dairy products."

NORMAN CAMPBELL, professor of medicine, Libin Cardiovascular Institute, University of Calgary

"We know salt is associated with blood pressure, so we concluded that anything we could do to reduce blood pressure would achieve the same risk profile of lower-risk populations, but that hasn't turned out to be true. Some studies even suggest that there may be an increased risk for cardiovascular disease with a reduced salt diet, since it increases insulin resistance and can affect plasma renin activity. Until 12 years ago, we did not look at the net effects [of reducing salt] and only looked at blood pressure. Even then, about a third of the population responded [positively] to salt restriction. What we need is a five-year controlled intervention trial."

RICHARD L. HANNEMAN, president, Salt Institute, Alexandria, Virginia

"Salt is one of those essential elements we need in moderation. There are people who are salt-sensitive. Your intake should depend on your sensitivity level. Salt helps stimulate the kidneys, helps promote fluid metabolism and has a moistening effect. A little bit [taken internally] is good if your skin is very dry. It also has a mild detoxifying effect. In Chinese medicine it is also known as a s oftener for hardened lymph nodes, glands or muscles. It gently promotes bowel regularity. Most importantly, it needs to be kept in balance with potassium. Aside from causing hypertension, too much salt can interfere with calcium absorption and lead to poor bone health. If you have PMS and bloating, it's important to reduce your salt intake as well."

DU LA, naturopath, Toronto

Not to pick a quarrel with experts (especially since Ms. Bromstein kindly denotes me one), but Ms. Mason is just flat-out wrong about salt intakes increasing from 200 mg/day to 5,000 mg/day over the past century. In fact, sodium intakes are virtually the same at about 3,500 mg/day. Dr. Campbell employs the classically flawed extrapolation of blood pressure to heath outcomes; a model rejected by direct health outcomes studies. And the solution to the "bloating" probem identified by Mr. Du is to drink more water, not consume less salt.

That said, the experts also made some important points: Ms. Mason points out any health problem owes to the imbalance of sodium on one side and potassium and magnesium on the other, advising to eat more fruits and vegetables so that your "potassium and magnesiium are probably fine as well and you can add table salt to foods." Balance and moderation: good advice. Though limiting his comments to blood pressure and not health outcomes, Dr. Campbell concedes the same point: "There are more effective ways of reducing hypertension than reducing salt, like increasing physical activity, losing weight, getting enough soluble fibre and eating low-fat dairy products." And Mr. Du agrees: "Salt is one of those essential elements we need in moderation. ... Most importantly, it needs to be kept in balance with potassium."

Thank you, Ms. Bromstein for your reasoned advice. For more on the Salt Institute perspective see our summary and reference citations.

June 05, 2007

SI endorses strict food label health claims

The European Food Safety Authority (EFSA) has proposed guidelines for the evidence it will require to authorize food label health claims. The Salt Institute has written in support of the proposal, specifically "strongly urging" that the final guidelines retain three "core principles" in the draft proposal: first, that applications employ a strong "evidence- based" objective review of the medical literature; second, that claims be allowed only for the general population, not sensitive sub-groups; and third, that the claimed benefit be a "human health outcome" and not an intermediate risk factor such as blood pressure.

The Institute "saluted" EFSA for its pledge to base health claims on a systematic review of the evidence and urged the draft be further strengthened "to embrace explicitly the systematic review procedures of the Cochrane Collaboration whereby all evidence meeting pre-established quality-related standards is required to be included in the analysis."

The Institute also commended the draft's "explicit recognition that health claims be made only for foods shown to produce improved human 'health outcomes'. Too often have health claims been authorized for health conditions which are only intermediate variables affecting health outcomes. Hard outcomes like mortality or, at least, incidence of a medical event such as a stroke or heart attack, are the proper target, not such intermediate 'risk factors' as insulin resistance, blood pressure, plasma renin activity or other variables affecting the final health outcome (e.g. a heart attack)."

The proposal should be amended to address two concerns, advised the Institute. "The final Guidelines should not permit health claims for individual risk factors for a disease; they should be authorized when evidence shows that the net health outcome effect is positive," the Institute explained. "An intervention often affects multiple risk factors and the same intervention can increase some risks even while reducing others. It is the net effect of the intervention that should be the subject of the health claim." The Institute also argued against any "grandfathering" of previous health claims: "Advent of these new rules should supersede these older claims since EFSA's newer, higher standard should be pre-emptive."

Comments will be accepted until June 17.