Unhealthy Data? US Dietary Guidelines Criticized

We've all heard it: Eat more vegetables, drink more water and try to cut back on red meats and processed foods. And maybe that's all sound advice, but the actual data supporting some U.S. dietary recommendations border on fantasy, according to a provocative article published last week in the journal Mayo Clinic Proceedings.

The article's claims have put some scientists on the defensive, who say the article's conclusions are overblown.

At issue is the fact that U.S. dietary guidelines are based heavily on data from self-reported questionnaires, on which ordinary people reported the types of food and drinks they consume day-to-day.

Latest Videos From

But ordinary people can forget, guess or even lie their way through the questionnaires. How many servings of vegetables did I have last Friday? Wait, what's a serving again? How many doughnuts do I consume weekly? Hmmm, maybe I should say just one instead of five. How many ounces are in a cup?

You get the idea.

For the past five decades, such questionnaires have been at the heart of a program called the National Health and Nutrition Examination Survey (NHANES), which is run by researchers at the Centers for Disease Control and Prevention. Data from these questionnaires are used extensively, every day, for myriad health studies that sometimes produce findings that seem to contradict each other: Coffee is good for you; coffee is bad for you? Much of this advice goes back to NHANES and the self-reported questionnaires.

Researchers usually note their reliance on self-reported questionnaires as a limitation when they publish their results. And most researchers use statistical tools to address these limitations. One reason for the use of questionnaires is that alternatives — such as conducting a study that monitors the food and drink consumption of a large group of people, for months, under personal supervision — would be too expensive to do.

But now, scientists led by Edward Archer, an obesity researcher at the Nutrition Obesity Research Center at the University Alabama at Birmingham, are asking which is worse: inaccurate data or no data at all?

The NHANES data are "pseudoscientific and inadmissible in scientific research," Archer told Live Science. "To attempt to base our public health guidelines on anecdotes, and pretend it is science, is fraudulent."

Archer and his colleagues claim that the majority of the data from NHANES and another survey from the USDA called "What We Eat In America" are physiologically implausible — often representing unrealistic frequencies or amounts. Therefore, the surveys do not provide not valid estimates of people's actual food and beverage consumption, the researchers say. [7 Food You Can Overdose On]

The group also claims that measurement protocols are similar to the methods used to deliberately create false memories in psychological studies For example, the questionnaire might provide choices of related breakfast foods that may make the respondent say they had bacon with their eggs even when they didn't. As a result of basing studies on such flawed data, mainstream diet advice about what to eat and what not to eat has evolved in ways that have confused the public and done little to curb the epidemics of obesity and Type 2 diabetes, the researchers claim.

The net outcome of all of this has been that people are "ignoring the actual causes of chronic, non-communicable diseases," Archer said.

Archer said the primary cause of obesity is not nutrition but rather inactivity. In a study published last year, Archer used activity monitors and urine analysis, instead of questionnaires, to demonstrate the hand-and-glove relationship between inactivity and weight gain.

Other researchers, however, view the NHANES and USDA data as rich and informative, albeit imperfect.

In an accompanying editorial to the Archer-led article, researchers Brenda Davy and Paul Estabrooks of Virginia Tech in Blacksburg said that memory-based diet assessment methods, such as the questionnaires used in NHANES, have revealed important associations between diet and health outcomes, for example, the connection between heart disease and fat intake, as well as between weight gain and consumption of sugary drinks.

"To argue that these data represent a waste of resources… [is] an impediment to scientific progress in obesity and nutrition research," Davy and Estabrooks concluded.

James Hébert, a professor of epidemiology and biostatistics at University of South Carolina's Arnold School of Public Health, told Live Science that researchers have developed methods to quantify the measurement errors that can come with using questionnaires, and mitigate their effects. 

In an article published in 2014 in the journal Advances in Nutrition, Hébert and his colleagues addressed misunderstandings of the NHANES data and also discussed improvements, such as increasing the number of 24-hour dietary recall interviews that are done during the survey to improve its accuracy, an innovation now incorporated into the NHANES protocol.

For the "What We Eat In America" survey, the USDA employed a technique called the Automated Multiple-Pass Method (AMPM), a computerized method for collecting 24-hour dietary recalls either in person or by telephone. Studies on the AMPM method itself have demonstrated its accuracy.

"There are a lot of smart people thinking and working on improving dietary assessment methodology," Hébert said, adding that data collection of any sort will have errors.

Also, scientists continue to develop inexpensive and noninvasive methods to test blood and urine for chemical byproducts, or biomarkers, of the food and beverages a study participant consumes, and these could be a useful complement to dietary recall data, Davy and Estabrooks told Live Science.

Follow Christopher Wanjek @wanjek for daily tweets on health and science with a humorous edge. Wanjek is the author of "Food at Work" and "Bad Medicine." His column, Bad Medicine, appears regularly on Live Science.