But some of the phrasing used in the article bugged me, so after a while I decided to go back and read the study for myself. I am glad that I did:
Athletes were placed into ibuprofen (n = 33) and control groups (n = 30) based on their historical use during training and competition, and their willingness to use or avoid ibuprofen before and during the race.
This is a shockingly bad research design. Anyone with any decent knowledge of statistics, or even a good dose of common sense, will immediately see the problem. Athletes were allowed to select into the groups. The ibuprofen group consisted of people who felt that they needed it. The drug-free group consisted of people who felt confident enough to take the race without any chemical help. There is a good reason to believe that these two groups of people are not the same. The drug-free group was probably healthier and/or more tolerant of pain to begin with, and any differences between the two groups will not solely be due to the drug. In statistics lingo, we have a serious identification problem caused by self-selection.
In more common speech, this study is like looking at people who choose to take cough medicine versus people who do not, measuring how much they cough, and concluding that cough medicine makes you cough more. It is entirely possible that ibuprofen is very effective at controlling pain, and that the people who chose to take it would have been in a much worse condition if they had not chosen to take it.
The study authors do provide something of an apology for this:
Permission for a randomized, placebo controlled research design was not granted by the race medical board because of ethical concerns regarding compliance in athletes suffering from pain during the latter stages of the race.
and
Thus further research is warranted using a stronger research design under laboratory conditions to determine what mechanisms best explain the elevated post-race cytokine levels in the ibuprofen users.
In order to really to a proper experiment, you would have to select people at random and secretly replace their ibuprofen with a placebo that looks and tastes and feels exactly the same. Clearly this would be unethical, so the next best thing would be to recruit volunteers and randomly assign them to a placebo or various doses of the real thing. This would not be a true estimate because it would still be a self-selected group (people who are willing to take the chance of getting a placebo) rather than the general population, and it will also be biased by the fact that people know they might be getting a fake. But it would be the best we could do, and we would be reasonably confident that differences between the different groups are due entirely to the drug dose.
This experiment tells us almost nothing. It is true that there were no observable differences between the two groups in terms of body measurements or how well they did on the race. There is no clear 'smoking gun' like a drug-free group that is healthier and ran faster. But they should have asked the participants to fill out a questionnaire on things like pain tolerance, history of pain during races, and attitudes to medication. It would be a serious mistake to say that this study shows that ibuprofen does not help people.
But science proceeds one small step at a time. Hopefully this study will be what it takes to convince someone to do a proper placebo-controlled experiment. Then we will know what the drug really does for people.
*Ibuprofen use, endotoxemia, inflammation, and plasma cytokines during ultramarathon competition; David C. Nieman et al; Brain, Behavior, and Immunity; Volume 20, Issue 6, November 2006, Pages 578-58
You probably can't read it directly unless you are at a university that pays for access to electronic journals.
No comments:
Post a Comment