Nutritional Epidemiology: More Harm Than Good?

When a New York Times headline reads: More Red Meat, More Mortality, what are we to believe?

When the lede reads: “Eating red meat is associated with a sharply increased risk of death from cancer and heart disease, according to a new study, and the more of it you eat, the greater the risk,” what are we to think?

We have all heard about how eggs, coffee, wine, and cocoa will kill us --- only to read the paper 6 months later and find that these foods will actually help us live longer --- and the cycle continues. How can this be?

“[H]ow should we respond the next time we’re asked to believe that an association implies a cause and effect, that some medication or some facet of our diet or lifestyle is either killing us or making us healthier?,” wrote Gary Taubes in a 2007 New York Times magazine cover story.

We should doubt it.

But how many of us do? The vast majority of Americans will see the myriad news outlets reporting the latest research from the Harvard Public School of Health in the Archives of Internal Medicine, which concluded “Red meat consumption is associated with an increased risk of total, [cardiovascular disease], and cancer mortality,” and they will justifiably believe that they can significantly improve their health by avoiding any “red” animal meat.

At its most basic level, we should not be questioning whether meat and eggs will help us live longer after reading these types of headlines, we should be questioning the research itself.

To put it succinctly, Taubes refers to these types of epidemiological studies as “conventional wisdom-confirmation machines.”

A very large and mischievous issue with observational studies on health is the healthy-user bias. We all know people who conduct their daily lives in an apparent quest to live forever: they brush their teeth five times a day, they meditate, they practice yoga, they eat only grass-fed, organic, free-range, hormone-free, additive-free, wild caught, preservative-free, nitrate-free, nitrite-free, fructose-free, fortified, they don’t smoke, they contribute to charity, they have 0.75 glasses of wine a night, they have 0.25 ounces of dark chocolate a day, they exercise daily, they take an aspirin daily, they count their calories, they always have breakfast, they eat 10 servings of fruits and vegetables every day, they eat only unrefined foods, they watch their intake of grilled meat, they listen to classical music, they attend weekly health seminars, they get 9.5 hours of sleep every night --- and this might be the tip of the iceberg.

When a new health finding comes along, the aforementioned health-conscious people above are more likely to read the health finding and implement it into their regimen. Let’s say it’s 500 IU of Vitamin E. These people now regularly take a capsule daily, where the rest of the public isn’t aware of the new finding, or they don’t put much stock in the news, or they just don’t care.

If you’re a researcher and you begin tracking people who take Vitmain E compared to those that don’t, you have all of the other variables above to contend with and tease apart. You can’t do it. But the researchers at the Harvard Public School of Health apparently think you can.

Another confounder -- a variable not accounted for that can skew the results -- is the bias of compliance. Many people might try to adhere to the regimen of our yoga-practicing, nitrate-free, classical-music-listening friend, but they just can’t comply. The people who can follow the supposed path to health are inherently different from the people who cannot, and this cannot be accounted for in nutritional epidemiology adequately.

Here is Taubes explaining the science of epidemiology, and a kind of study known as a prospective cohort study, “of which the Nurses’ Health Study is among the most renowned.”

“In these studies, the investigators monitor disease rates and lifestyle factors (diet, physical activity, prescription drug use, exposure to pollutants, etc.) in or between large populations (the 122,000 nurses of the Nurses’ study, for example). They then try to infer conclusions — i.e., hypotheses — about what caused the disease variations observed. Because these studies can generate an enormous number of speculations about the causes or prevention of chronic diseases, they provide the fodder for much of the health news that appears in the media — from the potential benefits of fish oil, fruits and vegetables to the supposed dangers of sedentary lives, trans fats and electromagnetic fields. Because these studies often provide the only available evidence outside the laboratory on critical issues of our well-being, they have come to play a significant role in generating public-health recommendations as well.”

“Observational studies like the Nurses’ Health Study can come up with the right hypothesis of causality about as often as a stopped clock gives you the right time. It’s bound to happen on occasion, but there’s no way to tell when that is without doing experiments to test all your competing hypotheses. And what makes this all so frustrating is that the Harvard people don’t see the need to look for alternative explanations of the data — for all the possible confounders — and to test them rigorously, which means they don’t actually see the need to do real science.”

“[E]very time in the past [the researchers from the Harvard Public School of Health using the Nurses’ Health Study] had claimed that an association observed in their observational trials was a causal relationship, and that causal relationship had then been tested in experiment, the experiment had failed to confirm the causal interpretation — i.e., the folks from Harvard got it wrong. Not most times, but every time. No exception. Their batting average circa 2007, at least, was .000.”

In the latest trip up to bat, the researchers are telling us that  “the subjects who ate the most meat (the top quintile) had a 20 percent greater risk of dying over the course of the study than the subjects who ate the least meat (the bottom quintile).”

However, obesity researcher Zoe Harcombe, pointed out on her blog, that the data can be interpreted quite differently after closer inspection:

“The raw data actually shows deaths rates falling with increased meat consumption up to the third or fourth quintile – and this is before all the other variables have been allowed for. This would suggest that meat consumption has a protective effect while weight, alcohol, calorie intake, lack of exercise and so on are all taking their toll.”

The type of research that nutritional epidemiology delivers could be considered pseudoscience, at best. The list of problems with the study itself, and the research in general, are too many to list, however, here is a brief compilation from various sources.

Colin E. Champ:

Strike 1: Lumping many food-types together, even though in practice they are very different.

Strike 2: Basing an entire study on food frequency questionnaires and self-reporting, both recording methods fraught with error and bias.

Strike 3: Using a poor method of dietary recording and assessment to begin with, and adding insult to injury by further inadequate methods of data collection.

Strike 4: Finding out there may be several other factors in the these groups that can lead to your findings and then using the magic wand of “controlling” for these variables to make them disappear.

Zoe Harcombe:

There are numerous key problems with this study – I’ll share seven:

1) This study can at best suggest an observed relationship, or association. To make allegations about causation and risk is ignorant and erroneous.

2) The numbers are very small. The overall risk of dying was not even one person in a hundred over a 28 year study. If the death rate is very small, a possible slightly higher death rate in certain circumstances is still very small. It does not warrant a scare-tactic, 13% greater risk of dying headline – this is ‘science’ at its worst.

3) Several other critical variables showed correlation with death rates – lack of activity, low cholesterol, BMI, smoking, diabetes, calorie intake and alcohol intake. These have not been excluded to isolate meat consumption alone. The raw data actually shows deaths rates falling with increased meat consumption up to the third or fourth quintile – and this is before all the other variables have been allowed for. This would suggest that meat consumption has a protective effect while weight, alcohol, calorie intake, lack of exercise and so on are all taking their toll.

4) Several other critical variables were not measured, which would logically correlate with certain meat consumption. Unprocessed meat inexplicably included sandwiches, curries, hamburgers (which come in buns) – has the correlation with bread, margarine, white rice, egg fried rice, poppadoms, burger buns, ketchup, relish or even fizzy drinks been correlated with the death rates? Indeed, Frank Hu, one of the authors of this meat study, is also quoted in today’s paper saying that one soft drink a day raises the risk of heart attacks.  It doesn’t of course – it is association at best, just as the meat article is – but one does wonder if that harmful soft drink was the one that just happened to be consumed with the hamburger or the bacon, lettuce and tomato sandwich ‘meal deal’?!

5) Hamburgers and pork sandwiches or lamb curries have been included as unprocessed meat. This is not a study of what real food devotees would consider unprocessed meat therefore. May I suggest that a study of consumers of grass fed ruminants would not deliver the desired headline? The lamb and beef grazing in the fields around me in Wales could not be further in health benefits from the hamburgers in buns and hot dogs in white rolls in fast food America.

6) We are all going to die. We have 100% risk of it in fact. We are not going to increase this risk by 13% or 20% if we have a hamburger and certainly not if we have a grass fed nutrient rich steak. This is headline grabbing egotistical academics doing their worst.

7) As I always consider conflict of interest, it would be remiss of me to end without noting that one of the authors (if not more) is known to be vegetarian and speaks at vegetarian conferences and the invited ‘peer’ review of the article has been done by none other than the man who claims the credit for having turned ex-President Clinton into a vegan – Dean Ornish.

Robb Wolf:

These retrospective cohort “studies” are a waste of time and honestly, it’s how the dominant paradigm fights to maintain control of the conversation. They generate, cheap, easily manipulated tripe in which the data can be bent to meet the desired conclusion.

1-Nutrition data was collected via Food Frequency Questionnaires.  Yes, folks just had to remember what they thought they ate.

2-Confounders galore. The higher meat consumption group tended to be overweight, smoked and was less active. Apparently they did not get a Paleo cohort in that mix?

3-Correlation does not equal causation.

Chris Masterjohn:

[Writing about a previous study, done in the same fashion, with the same message.]

The end result is that this study constitutes an observation, and cannot be used to support a hypothesis of any kind. Hypotheses are ideas developed to try to explain observations. You cannot test a hypothesis by making more observations. It is not impossible to test a hypothesis about diet over the long-term, and indeed trials have done this in the past, usually quietly swept under the rug because the establishment didn't like their results. Compliance with low-carb diets will never be perfect, but it will probably be better than the 1.4% accuracy with which food frequency questionnaires predict hamburger intake. Logical fallacies cannot substitute for the scientific method just because the scientific method seems difficult or even infeasible.

Denise Minger:

Although the wildfire-esque media coverage of this study is enough to make any omnivore feel like punching Al Gore for ever inventing the internet, it’s actually a great opportunity to test our critical thinking skills and explore the unending deficiencies of observational studies—including the self-reported data they’re often built from. We might not emerge with any newfound health guidance after breaking down bad science, but it’s always nice to have a better understanding of what the tumultuous world of research is really saying.

Back to the mainstream media:

“When you have these numbers in front of you, it’s pretty staggering,” said the study’s lead author, Dr. Frank B. Hu, a professor of medicine at Harvard.

What’s most staggering is Dr. Hu putting this kind of stock in his research. When the overall risk of dying is reportedly less than 1-in-100 over a 28-year observational study in which causation necessarily cannot be implied, and the study itself is rife with confounders and biases, it’s downright embarrassing that we consider this relevant to report.

While long-term experimental research on humans is difficult to perform and is potentially prohibitively expensive, it does not absolve the researchers from conducting pseudoscience and implying it to be scientific and definitive.

The links below provide further illumination into just how sad the state of affairs are when it comes to Public Health and the advice (along with the “evidence” behind it) provided to the public.

Do We Really Know What makes Us Healthy? | Gary Taubes | New York Times Magazine

Science, Pseudoscience, Nutritional Epidemiology, and Meat | Gary Taubes

Red Meat & Mortality & The Usual Bad Science | Zoe Harcombe

Will Eating Red Meat Kill You? | Mark Sisson

Observational Studies | Michael R. Eades, MD

Is Red Meat Healthy? | Robb Wolf

Red Meat Kills? It’s Flagrant Bias That’s Killin Me... | John Briffa, MD

Always Be Skeptical Of Nutrition Headlines: Or, What “Red Meat Consumption and Mortality” (Pan et.al.) Really Tells Us | J Stanton

Red Meat Does Not Raise Risk of Cancer, Heart Disease | Cameron English

New Study: Driving & Watching Red Meat Can Kill You | Richard Nikoley

Red Meat Study: Here We Go Again | Constantly Varied

Red Meat Consumption and Mortality | Colin E. Champ, M.D.

New [2010] Study Shows that Lying About Your Hamburger Intake Prevents Disease and Death When You Eat a Low-Carb Diet High in Carbohydrates | Chris Masterjohn, PhD

Red Meat Is Still Not Bad For You | Chris Kresser

*Keep an eye on Dr. Peter Attia’s blog for a post next week regarding the latest research and observational epidemiology, and I suspect, its limits.

Bob Kaplan holds advance degrees in exercise physiology and business, an undergraduate degree in nutrition, is a nationally certified personal trainer, and owns six Get In Shape For Women locations in Bedford, Wayland, Wellesley, Westford, Weston, and Winchester. He is currently the manager in the Wayland studio.

For more information about Kaplan's services at Get in Shape For Women in Winchester, please call 781-729-8100 or visit at 564 Main Street, Winchester, MA  01890, or online at www.getinshapeforwomen.com for a free week trial.