Evaluating medical studies

Currently, the ScienceBlogs Book Club is discussing Paul Offit's Autism's False Prophets (which I've just ordered). For those unaware of the subject of autism and autism research, the heated discussion going on in the comments sector will probably surprise you.

The heat is because the debate about autism, and the cause of autism, as it happens on the internet, is not about science, but about emotions, and in some cases, about profit. When I say profit, I'm not referring to the pharmaceutical businesses, that makes the vaccinations, that some people claim causes autism, but rather the people pushing dangerous "cures" at high prices to the parents of autistic children. Because, frankly speaking, vaccinations are very low on the profit scale when it comes to the pharmaceutical sector (Viagra and happy pills would be a much better business).

Of course, just because one side might profit from something being true, doesn't mean that it's not true. The makers of penicillin make a profit, yet penicillin has saved more lives than it's possible to count.

When I first heard about the possibility of an autism-vaccination link, from a blogger I respect, I thought it sounded somewhat feasible, since the cause of autism is not particularly well known. Also, the fact that vaccinations contained thimerosal, which is based on a type of mercury, seemed worrisome.

In other words, while I wasn't convinced of the link, I certainly was open to the idea.

However, I began to run into other bloggers, pointing out the problems with this hypothesis, and I began noticing a problematic pattern. There has been a lot of research into autism, and a lot of research into the possibility of an autism-vaccination link. A few studies pointed towards an autism-vaccination link, while the majority showed no link. As such, this might not tell us one way or another, since the minority studies might be the better studies. However, by any objective measure, this was certainly not the case, and what's more, it turned out that, while we don't know everything about the cause of autism, we do know some things, and what we know, invalidates the hypothesis of a link between autism and vaccinations.

So, how does one go about evaluating studies? Well, I obviously have some criteria I follow, but instead of going through them, I think it's much better to point to an excellent article on the very subject in the NY Times by Gina Kolata: Searching for Clarity: A Primer on Medical Studies (might require that you log in).

Kolata explains clearly what kind of things you have to look at when evaluating medical studies, and while the examples she refers to, all had the minority studies overturn the majority studies, it's much more common the other way around, as we can see in the studies about autism-vaccination links.

For more stuff about the proper way to evaluate and do medical studies, I recommend the group-blog Science-Based Medicine.

Why medical experiments matter

A couple of days ago, Orac wrote a long post about animal testing - Bad scientific arguments in the service of "animal rights" activism.

Personally, I dislike furs and am against unnecessary animal testing ("unnecessary" of course being the hard-to-define keyword). Necessary testing, such as medical testing, on the other hand, I'm all for, and Orac gives many good arguments for why I should continue endorsing it.

One of the good examples that Orac mentions Dr. Judah Folkman, who did some critical cancer research on mice.

Folkman found a mouse tumor model that mimicked this behavior and in the early 1990s did a series of pioneering experiments. In a strain Lewis lung carcinoma cells of low metastatic potential (LLC-LM), when cells are injected into C57BL/6 mice and allowed to grow subcutaneously, if the tumor is left alone, mice develop only microscopic lung metastases. These metastases do not grow and kill the mouse. If, however, the primary cancer is removed, then many large lung metastases grow rapidly. The results of the experiment above strongly implied that the primary tumor is secreting something that suppresses the growth of microscopic metastases. After this, the Folkman group did what we like to call "brute force" science, collecting mouse urine and analyzing it for tumor suppressive activity until they were able to purify a single 38 kDa peptide, which they designated angiostatin. This involved analyzing literally gallons of mouse urine. (Who said science isn't glamorous?) Once Folkman's group had a bunch of angiostatin on hand, it peformed the following experiment. Two groups of mice were injected with LLC-LM and the tumors allowed to grow to a certain size, after which they were surgically removed. One group was treated with angiostatin, and the control group with saline. The result was that the control group developed massive lung metastases and died, while the group treated with angiostatin had microsocopic lung metastases that never grew beyond a ball of cells. Dr. Folkman then demonstrated that it was the inhibition of angiogenesis by the angiostatin that kept these tumors in check. Ultimately, he used a similar method to discover endostatin, and later he demonstrated that endostatin could induced tumor dormancy in mice. I trust that the reader can see how these seminal preclinical observations about angiogenesis would have been virtually impossible without animal models, given that angiogenesis requires the interaction between tumor cells, cells in blood vessels, and the surrounding tissue stroma to occur.

Of course, after this was done, Folkman could move on to testing on humans, but the animal testing was vital.

Just to put Orac's example into perspective, the very next day the NY Times, had an article about one of the humans involved in the tests - A Daring Treatment, a Little Girl’s Survival. The treatment is still being researched, and the data is so far anecdotal, but it seems promising. This means that it looks like Folman's experiments on mice will end up saving the lives of many humans, including the small girl mentioned in the NY Times article.

I'm sorry, but if I have to choose between humans and laboratory breed mice, I'll go for the humans.