Thoughts on Meta-Analyses

 "Meta-analysis is to analysis, what metaphysics is to physics."—Richard D. Feinman

Published at Substack.

"Thoughts on Meta-Analyses"

What Causes Heart Disease, LDL or Seed Oils?

Hat tip to Dave Feldman for calling my attention to this podcast:

1/ I finished listening to the #SigmaNutrition podcast featuring Professor Chris Packard. This quote is worth it's own thread re #LMHRstudy

Alan asked the Prof about someone extolling an LDL of 13 mmol/L (~500 mg/dL) while having a CAC of 0, and this was his reply: pic.twitter.com/6oKgKpJ5JZ

— Dave Feldman (@DaveKeto) March 31, 2021

Which sounded interesting, so I downloaded it to listen to on my morning run.

Nevertheless he says something false in the first five minutes of the podcast. That didn't take long.

Why arguments from authority are fallacious. https://t.co/zAgEUvG2hQ

— Tucker Goodrich (@TuckerGoodrich) April 2, 2021

Given the comment by the interviewer, I had been hoping to get farther than a few minutes into my run before having to stop and tweet!

So to be correct, what he says (LDL is the "genetic", "primary cause" of CVD) is what got me. This can only be said by relying on a subset of the science, not, as the interviewer says, "the science".

But let's let Professor Packard put it in his own words (all transcriptions are mine, as are any errors therein. These are all taken from the tweet thread starting with my retweet above.).

He's discussing LDL, and a sub-type of LDL, Lp(a), which is a sub-type of oxidized LDL (oxLDL). He mentions that Lp(a) can be reduced by niacin (nicotinic acid), but that:

"...was put aside because of the THRIVE trial..." 

Huh, so what's the THRIVE trial?

"...participants who were assigned to extended-release niacin–laropiprant had an LDL cholesterol level that was an average of 10 mg per deciliter (0.25 mmol per liter as measured in the central laboratory) lower..."

So if you think LDL causes CVD, then a reduction of LDL must be a good thing, right? Unfortunately, the THRIVE trial was a "failed" trial, one which failed to show a benefit:

"...Among participants with atherosclerotic vascular disease, the addition of extended-release niacin–laropiprant to statin-based LDL cholesterol–lowering therapy did not significantly reduce the risk of major vascular events..."

So LDL is a "genetic" "primary cause" except when it has no effect whatsoever, apparently.

"Effects of Extended-Release Niacin with Laropiprant in High-Risk Patients"

Niacin also reduced Lp(a), among other seemingly beneficial things, "...lowering triglyceride and lipoprotein(a) levels and blood pressure." But all was for naught.

It's an odd cause, in my simple engineering view, that is only occasionally causal, as this is not the first time that reducing LDL has failed to show a benefit for CVD.

Next we come to this:

"...the classic exception is the guy in the [NEJM] who ate 40 eggs a day... and never got any problems with his LDL cholesterol... "

Seriously? We're back to eggs causing heart disease? This is the expert? This was considered debunked in the 1950s, for heaven's sake.

"From the animal experiments alone the most reasonable conclusion would be that the cholesterol content of human diets is unimportant in human atherosclerosis." — Ancel Keys, 1952https://t.co/fmMF82TiRT

— Tucker Goodrich (@TuckerGoodrich) May 9, 2018

Even the Dietary Guidelines dropped the recommendation against dietary cholesterol for lack of evidence.

"Normal Plasma Cholesterol in an 88-Year-Old Man Who Eats 25 Eggs a Day"

So ponder this: 

"LDL cholesterol is a marker of nutrition. So, that if you have a very poor diet, or you have existing disease, then you'll have a low LDL cholesterol."

Well, I agree with Prof. Packard! In fact, one of my major arguments for avoiding seed oils is the fact that they lower cholesterol, which seems to correlate with their negative effects on heart disease. 

So one would logically conclude a diet change that raised LDL would be good, and one that lowered it (like seed oils) would be bad, right?

He continues: 

"With the canakinumab trial we had a proof of concept, and absolutely correct that Paul [Ridker]... showed that lowering inflammation gave rise to a benefit [in CVD] without a reduction in LDL. And that does suggest that if you target the correct pathway, the correct inflammatory pathway, which is the IL-1β / IL-6 pathway... you end up probably stopping LDL having it's deleterious effects."

We won't get too into the details there, suffice to say that drug stops a cytokine from being produced, and this has a benefit for CVD that does not depend on LDL reduction. 

Now how could we do that to prevent people from getting CVD in the first place?

"The induction of [IL-1β] release from human monocyte-derived macrophages by 9-HODE and cholesteryl-9-HODE suggests a role for modified LDL, and its associated [linoleic acid] oxidation products, in vascular smooth muscle cell proliferation."

That's from 1992, which is shortly after it was shown that LDL alone did not induce the first step of atherosclerosis, it had to be "modified". The modification, it turned out, was the inclusion of linoleic acid from dietary seed oils, which would then become oxidized, and thus pathological, inducing the first step of atherosclerosis—and the rest, IMHO, and as suggested by the canakinumab trial.

"Induction of interleukin 1 beta expression from human peripheral blood monocyte-derived macrophages by 9-hydroxyoctadecadienoic acid."

So after assuring us that LDL is causal, Prof. Packard goes on to demonstrate, from the science, that it is not.

Yes, stop eating the precursor that causes that harmful inflammation, linoleic acid. 

Fix your poor diet: eat no seed oils. 

Don't worry about LDL.

Here's that podcast.

Listen to Prof. Packard explain (unwittingly) how a low seed oil diet could allow you to stop worrying about your LDL.

---

Effects of Extended-Release Niacin with Laropiprant in High-Risk Patients. (2014). New England Journal of Medicine, 371(3), 203–212. https://doi.org/10.1056/NEJMoa1300955

Kern, F. (1991). Normal Plasma Cholesterol in an 88-Year-Old Man Who Eats 25 Eggs a Day. New England Journal of Medicine, 324(13), 896–899. https://doi.org/10.1056/NEJM199103283241306

Ku, G., Thomas, C. E., Akeson, A. L., & Jackson, R. L. (1992). Induction of interleukin 1 beta expression from human peripheral blood monocyte-derived macrophages by 9-hydroxyoctadecadienoic acid. Journal of Biological Chemistry, 267(20), 14183–14188. https://doi.org/10.1016/S0021-9258(19)49695-6

Ridker, P. M., Everett, B. M., Thuren, T., MacFadyen, J. G., Chang, W. H., Ballantyne, C., Fonseca, F., Nicolau, J., Koenig, W., Anker, S. D., Kastelein, J. J. P., Cornel, J. H., Pais, P., Pella, D., Genest, J., Cifkova, R., Lorenzatti, A., Forster, T., Kobalava, Z., … Glynn, R. J. (2017). Antiinflammatory Therapy with Canakinumab for Atherosclerotic Disease. New England Journal of Medicine, 377(12), 1119–1131. https://doi.org/10.1056/NEJMoa1707914

Witztum, J. L., & Steinberg, D. (1991). Role of oxidized low density lipoprotein in atherogenesis. Journal of Clinical Investigation, 88(6), 1785–1792. https://doi.org/10.1172/JCI115499

Book Review: "Pure, White and Deadly" by John Yudkin

Dr. John Yudkin (1910 - 1995) was a British scientist and medical doctor who founded a prestigious Department of Nutrition at Queen Elizabeth College in London. He was in fact a pioneer in the field, founding the first Bachelor of Science degree program in nutrition in a European university.

Yudkin became interested in the correlation he noticed between sugar (sucrose, properly) consumption and coronary thrombosis, the formation of a clot in the heart. Yudkin had already become a best-selling proponent of a low-carbohydrate diet for losing weight, and was therefore familiar with the health problems associated with the excess consumption of carbohydrates. Obesity, diabetes, and heart disease often come as a package, and in the 1960s Yudkin came to the conclusion that there was sufficient epidemiological evidence to link sugar consumption to not only obesity but to the new epidemic of coronary thrombosis.

 Epidemiology is not, as Yudkin is careful to make clear with many amusing examples, sufficient to prove a cause-and-effect relationship, so he commenced to perform many experiments over the following decades in an attempt to discern a real relationship. He then proceeded to explore other harmful effects of excess sugar in the diet, some of which are quite surprising. He goes into detail in the chapter "Eat sugar and see what happens", and among other effects the cholesterol profile took a dramatic turn for the worse, insulin rose, glucose tolerance declined, and blood pressure rose. Additionally, in support of his theory that sugar might be related to thrombosis, the platelets of humans on a high-sugar diet in the blood became more sticky (in subjects eating the amount that he'd found was typical of consumption for those who'd already suffered from thrombosis). The platelets behaved just like those in people who already had atherosclerosis, in fact.

He also makes clear that all carbohydrates are not created equal, as he found that sugar (50% glucose and 50% fructose) has far worse effects on the body than starch (100% glucose), as some of the feeding studies he and others conducted compared the effects of both.

After making a very strong case for the fundamentally damaging effects of the over-consumption of sugar, in the chapter "A host of diseases" he lists those which are reasonably thought to be caused, at least in part, by sugar consumption:

• Damage to the eyes
• Damage to the teeth
• Damage to the skin
• Damage to the joints
• Is there a link between sugar and cancer?
• Sugar and drug action
• Sugar and protein
• Sugar and indigestion

He also details the evidence that a high sugar diet leads to a substantial reduction in lifespan, on the order of from 71 to 50 years, comparing rat lifespans to human.

In an earlier chapter he'd already detailed the astounding increase—25-fold!—in sugar consumption in the wealthy countries over the previous 200 years as sugar went from being a delicacy stored in locked boxes to an ever-present food additive. It's not unusual to see children getting 50% of their daily calories from sugar in one form or another.

So it's not hard to see how it's perfectly reasonable to connect the dramatic increase in morbidity, the chronic diseases of affluence that most of us now suffer from, to this huge change in the human diet.

But Yudkin is careful throughout to explain the limits of his knowledge and experiments, and to include the contrary viewpoint, noting that many don't agree with his conclusions.  This is, of course, the mark of a true scientist: knowledge is limited, and often imperfect, and any who claim to have perfect knowledge are often frauds.

Paleo diet fans like myself will be pleased to see Yudkin explaining that a low-carb diet like those of our ancient ancestors is likely the best option:

"In nutritional terms, the diet of prehistoric human beings and their ancestors during perhaps two million year or more was rich in protein, moderately rich in fat, and usually poor in carbohydrate....

"...The effect of the Neolithic revolution was thus to alter the components of the diet so that it was now rich in carbohydrate and poor both in protein and fat.  The carbohydrate was overwhelmingly starch, with sugars supplied only to a small extent as before by wild fruits and vegetables."

Pure, White, and Deadly was originally published in 1977.  The currently available version published in 2012 is a reprint of the 1986 revision.  As such, it's a bit out-of-date on certain topics.  Yudkin nevertheless uses his evolutionary perspective to good effect:

"It may turn out in the end that people would reduce their chances of getting heart attacks if they took large quantities of polyunsaturated fats such as those found in maize (corn) oil or in sunflower-seed oil, or in the special sorts of margarines made with these oils [omega-6 fats, specifically linoleic acid].  I have to say however that I think it very unlikely that this will happen.  I believe that the best diet for the human species is one made up as far as possible of the foods that were available in our hunting and food-gathering days.  The oils rich in polyunsaturates have been available only because of recent advances in agriculture and in the even more recent elaborate industrial techniques of extracting and refining oils; the complex chemical processing of these and other oils [hydrogenating] to produce margarine removes this product even further from the sorts of foods available to humanity during millions of years of evolution."

The last chapter, "Attack is the best defense", is an account of the many attempts to silence Yudkin and his research, most at the request of the of the sugar industry, but some by other scientists such as Ancel Keyes.  Cancelling conferences, suppression of speeches he did give at conferences, refusal to fund research that might prove contrary to commercial interests, propaganda, and flat libel were among the tools used to silence his message.  This is perhaps the most disheartening part of the book, as the sugar and food companies seem to have a great deal in common with the tobacco companies, despite a few ethical individuals with open minds.  They're not the worst villains that Yudkin details, however.  One would have to reserve that title for the parents who feed their children foods that the parents know to be harmful, as almost all parents do know that sugary foods are not the best options.  The child who ate 65% of his calories as sugar and promptly rotted his teeth out was probably not buying all his food himself, and Yudkin mentions the large number of people in Britain who are entirely without teeth, even at a young age.

What comes across in this book is that Yudkin is clearly an intelligent and eminently fair man.  His message is one that those concerned with the health of themselves and their children would be well-advised to adopt.

Personally, I suffered from countless cavities and had eight teeth pulled in my sugar-eating days.  At the age of 18, I decided to finally listen to my dentist and largely eliminated sugar from my diet.  I went 20 years without another cavity, until my company installed a mint bowl in the lobby.  I got one cavity, where I held the mints against my tooth while sucking on them, and have since reverted to my largely sugar-free ways.  Like Yudkin, I'll indulge in the occasional treat, but sugar's no longer a staple.  I wish I'd read this book back before I'd ruined my eyes, as that and missing teeth are the two effects of sugar that don't seem to regress on a sugar-free diet.  The steady increase in my myopia did cease, however, and I've had the same prescription for over 25 years.

The introduction to the 2012 edition is written by Dr. Robert Lustig, a specialist in childhood obesity.  Below is his hour-long lecture, "Sugar, the Bitter Truth", in which he expands on some of the topics of Yudkin's book.

"I stumbled upon Dr. Yudkin quite by accident in 2008.  I was in Adelaide, Australia, giving a talk at the Australian Association of Clinical Biochemists on my research into the role of sugar in the pathogenesis of metabolic syndrome.  Dr. Leslie Bennett said to me, 'Surely you've read Yudkin,' and I admitted I hadn't.  When I got home, I looked for Pure, White, and Deadly, and couldn't find it in our UCSF library of in any bookstore in San Francisco.  Eventually I got it by interlibrary loan.  I opened the book, and it opened my eyes.  I already knew from my own work that sugar at our current  rate of consumption is a medical disaster.  But to learn that Yudkin foresaw what a problem sugar was thirty-six years earlier, and at a much lower dose ... was a true revelation.  Indeed, I was a Yudkin disciple and I hadn't even realized it.

Yudkin's unpopular work had been buried, as it threatened nutrition scientists' major source of funding: the packaged-food industries. Science deals harshly with its heretics, not matter how correct they are.

Is Science Broken? Part 10: Observational Studies About Nutrition

I should just farm this series out to Derek Lowe, who is after all a practicing scientist.
But here we go, from his blog:

"If you go back and look at the instances where observational effects in nutritional studies have been tested by randomized, controlled trials, the track record is not good. In fact, it's so horrendous that the authors state baldly that 'There is now enough evidence to say what many have long thought: that any claim coming from an observational study is most likely to be wrong.'"

Science has spoken: we can now stop paying attention to these nonsensical studies.
But do read the whole thing, as there is also a recommendation on how to fix it. I think it's unlikely to ever be fixed, however, as the real "product" of this process is employment for academics, not actionable science.
If they employed real process control and measurement, someone might figure out that the whole thing is a waste of time and money. And then those academics would be out of work.

Is Science Broken? Part 9: Yes

Really no other way to summarize this post at In the Pipeline:

"Sydney Brenner on the State of Science":

...here's an outspoken interview with Sydney Brenner, who's never been the sort of person to keep his opinions bottled up inside him. Here, for example, are his views on graduate school in the US:

Today the Americans have developed a new culture in science based on the slavery of graduate students. Now graduate students of American institutions are afraid. He just performs. He’s got to perform. The post-doc is an indentured labourer. We now have labs that don’t work in the same way as the early labs where people were independent, where they could have their own ideas and could pursue them.

The most important thing today is for young people to take responsibility, to actually know how to formulate an idea and how to work on it. Not to buy into the so-called apprenticeship. I think you can only foster that by having sort of deviant studies. That is, you go on and do something really different. Then I think you will be able to foster it.

But today there is no way to do this without money. That’s the difficulty. In order to do science you have to have it supported. The supporters now, the bureaucrats of science, do not wish to take any risks. So in order to get it supported, they want to know from the start that it will work. This means you have to have preliminary information, which means that you are bound to follow the straight and narrow.

And:

Here are Brenner's mild, temperate views on the peer-review system and its intersection with academic publishing:

. . .I don’t believe in peer review because I think it’s very distorted and as I’ve said, it’s simply a regression to the mean.

I think peer review is hindering science. In fact, I think it has become a completely corrupt system. It’s corrupt in many ways, in that scientists and academics have handed over to the editors of these journals the ability to make judgment on science and scientists. There are universities in America, and I’ve heard from many committees, that we won’t consider people’s publications in low impact factor journals.

Now I mean, people are trying to do something, but I think it’s not publish or perish, it’s publish in the okay places [or perish]. And this has assembled a most ridiculous group of people. I wrote a column for many years in the nineties, in a journal called Current Biology. In one article, “Hard Cases”, I campaigned against this [culture] because I think it is not only bad, it’s corrupt. In other words it puts the judgment in the hands of people who really have no reason to exercise judgment at all. And that’s all been done in the aid of commerce, because they are now giant organisations making money out of it.

I don't find a lot to disagree with there, either.

The point of academic Science is to provide jobs for academics. Any "science" that occurs is a happy coincidence, in large part.

Is Science Broken? Part 8: "Science’s Sokal moment"

An interesting experiment:

"John Bohannon, a biologist at Harvard with a side gig as a science journalist, wrote his own Sokalesque paper describing how a chemical extracted from lichen apparently slowed the growth of cancer cells. He then submitted the study, under a made-up name from a fictitious academic institution, to 304 peer-reviewed journals around the world."

But it doesn't really demonstrate that something is wrong with institutional Science, per se; unlike the other examples in this series.

"The publications Dr Bohannon selected for his sting operation were all open-access journals. These make papers available free, and cover their costs by charging authors a fee (typically $1,000-2,000). Policymakers have been keen on such periodicals of late. Since taxpayers already sponsor most academic research, the thinking goes, providing free access to its fruits does not seem unreasonable. But critics of the open-access model have long warned that making authors rather than readers their client risks skewing publishers’ incentives towards tolerating shoddy science."

Bohannan demonstrates that most of the "open-access" journals shouldn't exist, and that the policymakers, as usual, are doing more harm than good. I agree that science paid for by the public should be freely available to the public, but too many of these open-access journals seem to be simply viewing this as a gold-rush-style business opportunity; quality be damned.

Is Science Broken? Part 7: "Current Hydration Guidelines are Erroneous..."

"Current hydration guidelines are erroneous: dehydration does not impair exercise performance in the heat."

But they're profitable. And that's what matters, isn't it?

Waterlogged covers this topic incredibly well.

Via Prof. Noakes on Twitter, naturally:

New research suggests current guidelines about hydration during exercise are wrong http://t.co/h8Ml3mx99y #waterlogged /cc @ProfTimNoakes

— Tom H (@TrailDragon) September 21, 2013

Is Science Broken? Part 6: Most Medical "Advances" Are A Waste Of Time And Money

From Dr. Eades on Twitter:

"After 65 yrs of [Random Controlled Trial]s, ineffective, harmful, expensive med practices are being introduced more freq now than ever."

He links to "How Many Contemporary Medical Practices Are Worse Than Doing Nothing or Doing Less?" [PDF] which opens:

"How many contemporary medical practices are not any better than or are worse than doing nothing or doing something else that is simpler or less expensive? This is an important question, given the negative repercussions for patients and the health care system of continuing to endorse futile, inefficient, expensive, or harmful interventions, tests, or management strategies. In this issue of Mayo Clinic Proceedings, Prasad et al describe the frequency and spectrum of medical reversals determined from a review of all the articles published over a decade (2001-2010) in New England Journal of Medicine (NEJM). Their work extends a previous effort that had focused on data from a single year and had suggested that almost half of the established medical practices that are tested are found to be no better than a less expensive, simpler, or easier therapy or approach. The results from the current larger sample of articles are consistent with the earlier estimates: 27% of the original articles relevant to medical practices published in NEJM over this decade pertained to testing established practices. Among them, reversal and reaffirmation studies were approximately equally common (40.2% vs 38%). About two-thirds of the medical reversals were recommended on the basis of randomized trials. Even though no effort was made to evaluate systematically all evidence on the same topic (eg, meta-analyses including all studies published before and after the specific NEJM articles), the proportion of medical reversals seems alarmingly high. At aminimum, it poses major questions about the validity and clinical utility of a sizeable portion of everyday medical care....

"...Despite better laboratory science, fascinating technology, and theoretically mature designs after 65 years of randomized trials, ineffective, harmful, expensive medical practices are being introduced more frequently now than at any other time in the history of medicine. Under the current mode of evidence collection, most of these new practices may never be challenged."

Emphasis mine. Read the whole thing.

Doctors are notoriously prone to jumping on the latest treatment bandwagon without waiting for valid scientific confirmation of treatment, and without following up to see if the treatment is found to be ineffective, or fraudulent.

Of course patients assume that they do. The logical course to take, therefore, since new treatments are untrustworthy, is not to take any. Take treatments that have been around for a few generations, ideally, since the effectiveness and the side-effects should be well-known, and you can double-check what your doctor is telling you to confirm that he knows what he's talking about.

If it's a life-or-death situation and a you want to take a flyer on a new treatment, fine; but be aware that that's what you're doing. For most medicines and most situations, that's not the correct approach.

(See my series Is Science Broken? and You Are The Long-Term Test for more, if you have the stomach.)

Is Science Broken? Part 4: Studies Often Aren't Reproducable

“Towards Better Papers, With Real Results in Them.”

What a radical notion.

“This has to be a good thing. From the latest issue of Nature comes news of an initiative to generate more reproducible papers:

“From next month, Nature and the Nature research journals will introduce editorial measures to address the problem by improving the consistency and quality of reporting in life-sciences articles. To ease the interpretation and improve the reliability of published results we will more systematically ensure that key methodological details are reported, and we will give more space to methods sections. We will examine statistics more closely and encourage authors to be transparent, for example by including their raw data. . .

“…I hope that Science, the Cell journals at Elsevier, and other other leading outlets for such results will follow through with something similar….”

Indeed.  Science has slipped pretty badly, as the need for an initiative like this demonstrates.  Reproducibility, not “statistical significance”, is the sine qua non of Science, after all. 

At least some are trying to fix the problem.

Is Science Broken? Part 5: Retractions Of Fraudulent Work Are Near-Impossible

Incredible.

They have the guy on film tampering with lab samples at night, and that's not enough to get a retraction in Nature, supposedly the leading scientific journal. Nineteen years later, a letter retracting the original study gets published.

"The Nature paper has been cited 255 times, according to Thomson Scientific’s Web of Knowledge."

Careerism is the one universal scientific principle, apparently.

From one of the comments:

"You know what would make things like less a lot less painful? If you could simply publish negative results without all this ‘reluctance’ from journals to accept them. Honestly, if you were to take a survey of how many scientists out there have been unable to replicate experiments in high profile journals, and just ended up keeping that data in a file drawer and thinking to themselves ‘I’ve proven to myself that the findings of paper X are incorrect’, I bet it would be a disturbingly high number. We desperately need to change this!"

That would be bad for people's careers...

Is Science Broken? Part 3: Incentives Favor Fraud.

Has modern science become dysfunctional?:

"More concerning, say the editors, is that this trend may be a symptom of a growing dysfunction in the biomedical sciences, one that needs to be addressed soon. At the heart of the problem is an economic incentive system fueling a hypercompetitive environment that is fostering poor scientific practices, including frank misconduct."

It's a logical product of the end of the idea of honor.

Previous posts. Thanks to John Durant.

Is Science Broken? Part 0: Most Medical Research "Is Misleading, Exaggerated, And Often Flat-Out Wrong."

This is the article that drew my attention to the wholesale nature of the problem:

"Lies, Damned Lies, and Medical Science"

"...Indeed, given the breadth of the potential problems raised at the meeting, can any medical-research studies be trusted?

"That question has been central to Ioannidis’s career. He’s what’s known as a meta-researcher, and he’s become one of the world’s foremost experts on the credibility of medical research. He and his team have shown, again and again, and in many different ways, that much of what biomedical researchers conclude in published studies—conclusions that doctors keep in mind when they prescribe antibiotics or blood-pressure medication, or when they advise us to consume more fiber or less meat, or when they recommend surgery for heart disease or back pain—is misleading, exaggerated, and often flat-out wrong. He charges that as much as 90 percent of the published medical information that doctors rely on is flawed. His work has been widely accepted by the medical community; it has been published in the field’s top journals, where it is heavily cited; and he is a big draw at conferences. Given this exposure, and the fact that his work broadly targets everyone else’s work in medicine, as well as everything that physicians do and all the health advice we get, Ioannidis may be one of the most influential scientists alive. Yet for all his influence, he worries that the field of medical research is so pervasively flawed, and so riddled with conflicts of interest, that it might be chronically resistant to change—or even to publicly admitting that there’s a problem."

Even though this is not new, I wanted to have a link to it on this blog, just for reference. (Emphasis mine.)

Part 1, Part 2.

Is Science Broken? Part 2: Is Most Academic Research Unreliable?

The answer appears to be "Yes", at least in the academic setting:

"Bayer halts nearly two-thirds of its target-validation projects because in-house experimental findings fail to match up with published literature claims, finds a first-of-a-kind analysis on data irreproducibility.

"An unspoken industry rule alleges that at least 50% of published studies from academic laboratories cannot be repeated in an industrial setting, wrote venture capitalist Bruce Booth in a recent blog post. A first-of-a-kind analysis of Bayer’s internal efforts to validate ‘new drug target’ claims now not only supports this view but suggests that 50% may be an underestimate; the company’s in-house experimental data do not match literature claims in 65% of target-validation projects, leading to project discontinuation.

"“People take for granted what they see published,” says John Ioannidis, an expert on data reproducibility at Stanford University School of Medicine in California, USA. “But this and other studies are raising deep questions about whether we can really believe the literature, or whether we have to go back and do everything on our own.”"

So the academics are just faking stuff wholesale to advance their careers?  This is pretty shocking, but not surprising.  Stuff like this has been going on in the liberal arts, which I studied, to the point that I'd argue pursuing a liberal arts degree in 99% of schools is probably a waste of time.

From the comments in one of the links from the page above:

"This article highlights an essential difference between how success is defined in academia vs. industry. Academics make their names with high profile papers in journals like Nature, whose selection process favors the new and exciting 'big ideas' from individuals deemed to be rising stars by their peers. In general, unless overt fraud can be demonstrated, it makes no difference to the individual's career if their high profile papers are later found to be incorrect. However, in industry, the bottom line for success is whether the research will lead to a new product, so there is little incentive for researchers to overhype their data since they will be accountable if it is not reproducible."

That about nails it.

Part 1.

Is Science Broken? Part 1: Science Doesn't Produce Reliable Results.

I hope not, but it's clearly got a problem (subscription-only link):

"It was the kind of study that made doctors around the world sit up and take notice: Two popular high-blood-pressure drugs were found to be much better in combination than either alone.
""There was a 'wow' reaction," recalls Franz Messerli, a New York doctor who, like many others, changed his prescription habits after the 2003 report.

"Unfortunately, it wasn't true. Six and a half years later, the prestigious medical journal the Lancet retracted the paper, citing "serious concerns" about the findings.

"The damage was done. Doctors by then had given the drug combination to well over 100,000 patients. Instead of protecting them from kidney problems, as the study said the drug combo could do, it left them more vulnerable to potentially life-threatening side effects, later studies showed. Today, "tens of thousands" of patients are still on the dual therapy, according to research firm SDI.

"...Just 22 retraction notices appeared in 2001, but 139 in 2006 and 339 last year. Through seven months of this year, there have been 210, according to Thomson Reuters Web of Science, an index of 11,600 peer-reviewed journals world-wide...."

Sounds like any new therapy or finding should be approached with great skepticism.  And not implemented until confirmed.

For myself, I like to take drugs that have been around for a couple of human generations.  Hopefully the effects are at least all known by that point, since long-term, multi-generational human tests are simply not possible.

Just remember, you are the long-term test.

Part 2.