Problems with Meta Studies
Meta analysis is the process of combining the findings of several prior studies to “increase statistical power, provide quantitative summary estimates, and identify data gaps and biases,” according to Dr Neal D Barnard, Dr Walter C Willett and Eric Ding writing about the misuse of meta-analysis in nutrition research in The Journal of the American Medical Association published on 18th September. When applied to studies conducted with similar populations and methods, meta-analyses can be useful. However, many published meta-analyses have combined the findings of studies that differ in important ways, prompting (Hans) Eysenck to complain that they have mixed apples and oranges—and sometimes “apples, lice, and killer whales”—yielding meaningless conclusions.
This is truer with nutritional studies. “Populations range widely in their dietary habits, and the consumption of most foods (e.g., vegetables) and nutrients (e.g., sodium) is variable and difficult to quantify,” write the authors. “Combining results may require contacting the original investigators for participant-level data, which may have been produced using dissimilar dietary assessment techniques.” If corrections for variations are not done, the conclusions would be wrong and the media will end up blowing up something that is false.
A study involving the Malmö Diet and cancer had no groups at the lower end of saturated fat intake, which ranged from 13% to more than 22% for the lowest to the highest quintiles, and no significant association between saturated fat intake and risk of cardiovascular events. The authors cautioned, “Only 1.2 percent of the present study population actually followed national Swedish recommendations (less than 10 energy percent) on saturated fat intake” and, hence, the lack of correlation between fat and heart disease should not be seen as conclusive. “Nevertheless, the study was given substantial weight in the meta-analysis, which concluded that available evidence did not support limiting saturated fat, a conclusion repeated in a New York Times commentary proclaiming “Butter is Back” and a Time magazine cover declaring “Eat Butter”.
US Dietary Guidelines Unscientific?
The US National Academies of Sciences, Engineering, and Medicine (NASEM) released a report “Redesigning the Process for Establishing the Dietary Guidelines for Americans” which different sections of the US researchers have interpreted differently. The Nutrition Coalition (NC) highlighted the report’s comments on dietary guidelines for americans (DGA). NC says that the current DGA process for reviewing the science falls short of meeting the “best practices for conducting systematic reviews,” and that methodological approaches and scientific rigour for evaluating the scientific evidence needs to be strengthened. The report states, “To develop a trustworthy DGA, the process needs to be redesigned.” NC argues that this lack of a firm scientific foundation for the DGA is emphasised throughout the report.
NASEM’s report mentions: “The adoption and widespread translation of the DGA requires that they be universally viewed as valid, evidence-based, and free of bias and conflicts of interest to the extent possible. This has not routinely been the case.” At another place: “The methodological approaches to evaluating the scientific evidence require increased rigor to better meet current standards of practice.” Also, “The process to update the DGA should be comprehensively redesigned to allow it to adapt to changes in needs, evidence, and strategic priorities.” Jeff Volek, professor, department of human sciences, The Ohio State University, pointed out: “For years, we’ve been told that the Dietary Guidelines are the gold standard and that if Americans are obese and diabetic, it must be their fault. This report confirms that this is not the case.” Sarah Hallberg, executive director of the nutrition coalition, says: “I find my patients get healthier—lose weight and even reverse their diabetes—by doing what the current science says, which is the complete opposite of what the Guidelines tell them. It’s obvious to me, as a practitioner, that these Guidelines do not reflect the best and most current science.”
