Friday, April 02, 2010

Not even bad science

Once in a while, one comes across a published study which is so bad that one cannot even consider it to be bad science, rather it's so completely wrong, that it has nothing to do with science at all.

I recently came across one such study.

For some strange reason, I had clicked on a homeopathy hashtag in Twitter, showing me the latest tweets about homeopathy. This is not something I recommend, as the stupidity is out in force there. Anyway, one of the most recent tweets at the time referred to a study demonstrating that homeopathy was as efficient as anti-depressants. Unsurprisingly, I took a closer look at the study.

The study is this one: Homeopathic Individualized Q-potencies versus Fluoxetine for Moderate to Severe Depression: Double-blind, Randomized Non-inferiority Trial by U. C. Adler et al. (read it at your own risk)

My last post was triggered by reading the study for reasons I will come into later, but let me just say for now that whoever was in charge of that study was obviously a true believer of homeopathy, and had very little grasps of science and evaluating results.

What makes me say that? Well, let's tackle the claim about the person in charge being a true believer and not understanding science. Passages like this one, should clearly demonstrates why I say that.

Hahnemann's dynamization gained support of physics: thermoluminescence emitted by ‘ultra-high dilutions’ (dynamizations) of lithium chloride and sodium chloride was specific of the salts initially dissolved, despite their dilution beyond the Avogadro number (11).

I think I can safely say that no physicist would agree that Hahnemann's idea of dynamization has gained the support of physics - as a matter of fact, physicists would call Hahnemann's claims pure nonsense.

Don't believe me? Well, this is what Hahnemann had to say on the subject of dynamization:

This remarkable transformation of the properties of natural bodies through the mechanical action of trituration and succussion on their particles (while these particles are diffused in an inert dry or liquid substance) develops the latent dynamic powers previously imperceptible and as it were lying hidden asleep in them. These powers electively affect the vital principle of animal life. This process is called dynamization or potentization (development of medicinal power), and it creates what we call dynamizations or potencies of different degrees.

In other words, this is the claim that diluting the substance makes it more potent.

The claim that physics supports this is based upon one article, Thermoluminescence ofultra-high dilutions of lithium chloride and sodium chloride (.pdf) by Louis Rey, which hasn't been replicated, and which certainly doesn't support Hahnemann's claims about the substance becoming more potent. There are also several people who points out problems with Rey's study.

So, all in all, a good scientist would most certainly not make claims like "Hahnemann's dynamization gained support of physics", since there is only one, non-replicated study which might support this claim - in terms of science, this amounts to an unsupported claim.

Still, even though the people in charge of the study are true believers, and don't really understand science, it doesn't mean that the study can't be useful - after all, if done properly, the results should speak for themselves. So, let's return to the study.

Basically, the study was conducted by assigning patients into two groups - one group which would receive Fluoxetine and a placebo, and one group which would receive the homeopathic remedy and a placebo.

I think you all can see the issue there. Since homeopathy is placebo, what this study is doing, is comparing Fluoxetine to placebo. Of course, the people conducting the study doesn't see it this way, but as I wrote in my last post: Until homeopaths can explain how homeopathy works in terms which doesn't mean that everything we know about chemistry, physics, and physiology is wrong, then we can safely reject their claims.

But let's for the sake of the argument accept that we are comparing two different types of remedies for depression, and look at what the study concludes.

This sample consisted of patients with moderate to severe depression, because their mean MADRS depression scores were close to the 31 score cut-off for moderate and severe depression (28). Initially, 284 subjects were screened, 105 of them met the inclusion criteria, 14 out of them did not attend the first appointment, 91 were randomized and 55 completed the 8-week trial. A detailed flow chart of subject progress through the study is shown in Fig. 1.

So, out of 91 people, only 55 completed the 8-week trial. That's a drop-out rate of ~40%. That's a quite significant number, and would impact the reliability of the study. Let's see how they go into this later.

There were no significant differences between the proportions of excluded and lost for follow-up patients in the two groups (P = 0.99), though there was a trend toward greater treatment interruption for adverse effects in the fluoxetine group, as can be seen in Table 1.

Well, true, there was no significant difference between the proportions, but there were quite different reasons why people were excluded.

As they state, there was "a trend toward greater treatment interruption for adverse effects in the fluoxetine group", hardly surprising since they were given actual medicine, rather than placebo, which has side-effects. What's more surprising is that there were actually 3 people excluded from the placebo homeopathy group for adverse effects. Since there are no active ingredients in the homeopathic remedies, this must either be due to an negative placebo effect (called the nocebo effect), or due to a misdiagnosis of e.g. clinical worsening.

What they did leave out was the fact that there is a significant difference in the clinical worsening in the two groups. Among people receiving the medicine, there was one person excluded because of worsening (approximately 2.3% of the cohort), while among the receivers of homeopathic remedies, there were five who were excluded for this reason (more than 10% of the cohort, 10.4% to be more precise). Again, this is entirely in line with our knowledge of which group is receiving actual medicine, and it certainly is a significant enough difference for it to be taken into consideration when writing about the results, yet this was not done.

The rest of the result section goes on to analyzing the results of the people who made it through the 8 week period, yet doesn't address neither the high drop-out rate nor the differences in the reasons for exclusions.

Going to the discussion section, there are a couple of things which jumps out, particularly this paragraph:

A placebo-arm was not included in the present study because it was not authorized by the National Ethic Council. Although placebo interventions are associated with mean response or remission rates of ~35% (37,38), a placebo effect cannot be ruled out, since the homeopathic Q-potencies were compared with an antidepressant and ‘it is becoming more and more difficult to prove that antidepressants—even well-established antidepressants—actually work better than placebo in clinical trials’ (39). Nevertheless, it also has to be taken into consideration that the antidepressant-placebo difference seems to be smaller in the trials aiming at mild to moderate depression (40,41) and the present sample consisted of patients suffering from moderate to severe depression. Placebo-controlled studies would be recommendable to clarify these findings.

The first part of the paragraph is what caused the ranting in my last post. I presume that the National Ethic Council didn't allow a placebo-arm in the study, because it considered it unethical to submit people suffering from depression to placebo, yet by allowing this homeopathy study to go ahead, it submitted the very same type of people to something which we know is placebo. What the hell is wrong with these people? Are they really so gullible that they don't realize this?

And when reading the rest of the paragraph, it becomes clear that the people who conducted the study is aware that they were comparing homeopathic remedies to something which hasn't been shown to be any better than placebo. In other words, they are comparing one type of placebo, homeopathy, to something which might very well be another type of placebo, or which at the very least seems to have a very limited effect on top of the placebo effect. And we are supposed to be impressed that homeopathy is as effective?

Labels: , , ,


Anonymous Anonymous said...

If you thought that was bad, feast your eyes on this: Entanglement model of homeopathy as an example of generalized entanglement predicted by weak quantum theory

If you have institutional access to the journal through something like ISI and want to know what it feels like to laugh and groan at the same time... read away.

April 03, 2010 6:09 PM  
Anonymous Jim said...

Why did you read the 'study'? Why did I attend a couple of sessions of a Creationist 'seminar'? aarghh...

It's a dirty job, but somebody has to do it.

Thanks for your sharp dissection of the study. How do we get non-scientific folks (the victims) to understand this?

April 07, 2010 8:46 AM  
Anonymous Dr. Nancy Malik said...

130+ studies in support of homeopathy medicine published in 51 peer-reviewed international journals

Medicines for specific disease conditions, Ultra-molecular dilutions, Structure & Memory of Water, Animal Studies, Plant Studies

October 03, 2010 6:48 PM  
Anonymous Mack said...

I think there is a considerable difference between "Homeopathy" and "Placebo"...
I know next to nothing about homeopathy, so I looked it up: "Homeopathy is a system of medicine which involves treating the individual with highly diluted substances, given mainly in tablet form, with the aim of triggering the body’s natural system of healing"

Now that I know the definition it actually makes really good scientific sense... from a "anecdotal" point of view.

The fact is... many poisons becomes treatments at very low concentrations. From a Common Sense viewpoint, it's pretty clear that the concept of Homeopathy is mostly ridiculous. BUT SOMETIMES it will work. And that is the most important thing... it actually works sometimes.

Now back to this study... and "placebo" vs Homeopathic Remedy... obviously an ultralow concentration of a poison... can still have an effect. Therefore, its scientifically unsound to argue that a homeopathic remedy is "placebo" simply because it is homeopathic.

Let's look at the actual study... they are using Fluoxetine.
SSRI's are known to take quite a while before they even work.
6 wks before ANY effect is seen vs placebo in many studies.

So my problem with this study (and this is just in very brief cursory review) is that SSRI's in general are not much better than placebo to begin with, and certainly have a very low response at 6 weeks. 8 weeks is not much better.

So in reality...
You should be happy.

Because this study has provided evidence, that:
1. This particular Homeopathic remedy is "No better" than something that does not work too well in the short amount of time that it was used... ANYWAY.

Ie... They are both equally useless for 8 weeks.

August 05, 2011 9:19 PM  

Post a Comment

<< Home