Religious Orders

The Alternative Medicine Game!

Posted: Thursday August 29 2013 @ 6:52am

Religious Order: Science and Technology

A while back, we went to the Women in Secularism 2 conference and had a great time. The first session was about the vast wasteland of alternative medicine. Surly Amy talked about why alternative medicine can often appear to work at an anecdotal level. The thing is, if you're feeling like crap, random chance says you'll likely feel better the next day. People try cures when they're at extreme lows and then think the cure worked if they feel better the next day. But that can just be random chance. The technical jargon is Regression to the Mean.

I wanted to get a feel for how much of an effect there really is, so I made up a quick dice game. Don't worry, you don't need one of those 20-sided D&D jobs. A normal 6-sided die is fine.

The numbers 1-6 on the die represent how good you feel, with 1 being really good and 6 being really awful. Here's how the game is played:

  1. Roll the die.
  2. If you roll a 5 or a 6, you feel crappy enough to resort to alternative medicine.
  3. Roll the die again and see how you feel the next day, after the alternative medicine. If the number is higher than the previous roll, you feel worse. If the number is the same, you feel the same. If the number is lower, you feel better.
  4. Repeat until you have some good statistics or just get bored.

It should be immediately obvious that, if you rolled a 6 the first time, you won't be rolling an even bigger number the second time. Also plain to see is that there's only one roll to equal a 6, while there are five ways to roll a lower number.

The situation is similar when rolling a 5. There is a way to get worse on the second roll, namely by rolling a 6. There's one way to stay the same and four ways to get a lower number.

When you add up all the percentages, the probabilities about how you'll feel after resorting to alternative medicine look like this:

How You'll FeelIn FractionsAs Decimals
You'll feel worse1/128.3%
You'll feel the same2/1216.7%
You'll feel better9/1275%

As you can see, the odds drastically favor feeling better, despite the alternative medicine in this game not having any role in the outcome of the roll. Also keep in mind that feeling the same isn't any big prize. You started out feeling shitty and still feel shitty.

(As a check, I also wrote a short chunk of Perl to simulate the rolls. It confirmed the percentages.)

But wait, it gets worse!

What if we assume that the alternative medicine actually hurts you a little? To model that, we'll add 1 to the second roll. So, if you roll, say, a 3, we'll treat it as a 4. What to do when you roll a 6? Treat it as a 7, I guess? For this game, we'll just round anything bigger than a 6 to a 6. In the real world, a 7 would mean you die and your story is left untold. We could also just leave the 7 as a 7. The only effect would be to make the numbers even worse, shifting much of the feel the same outcome to feel worse. In general, I mentally group the feel-worse and feel-the-same together anyway.

Obviously, this twist is going to change the outcome and make it less likely to see fake improvement, right? Well, yeah, but just barely.

There's still no way to get worse after an initial 6, but now there are two ways to stay the same, rolling either a 5 or a 6. (Because the 5 will get 1 added to it.) Now there are only four ways to roll a lower number and feel better, namely rolling anything from 1 to 4.

The change is similar when rolling an initial 5. There are now two ways to get worse (5 & 6), one way to stay the same (4), and three ways to get better (1-3). Here are the resulting percentages:

How You'll FeelIn FractionsAs Decimals
You'll feel worse2/1216.7%
You'll feel the same3/1225%
You'll feel better7/1258.3%

Wait, what? Your chances of feeling better are still more than half, even if the so-called medicine actually makes you sicker? That's right!

Okay, so let's make the medicine even worse for you, by adding 2 to the second roll. I'll spare you the details and just show the resulting percentages:

How You'll FeelIn FractionsAs Decimals
You'll feel worse3/1225%
You'll feel the same4/1233.3%
You'll feel better5/1241.7%

Well, at least the chances you'll feel better are below 50% now. Yet feeling better is still the most likely outcome and feeling worse is the least likely. Although, again, feeling the same is no great prize when you already feel shitty. So, at this point, folks might start to doubt the validity of this harmful treatment.

Fine, fuck it, crank up the harm of the medicine to 3. We'll add 3 to the second roll. Given that it's half the range, that's really quite a lot. Here's what you get:

How You'll FeelIn FractionsAs Decimals
You'll feel worse4/1233.3%
You'll feel the same5/1241.7%
You'll feel better3/1225%

Finally, we're getting to where feeling better is the least likely outcome. Note that you're still most likely to feel the same, but that feeling the same level is basically feeling shitty.

If we make the medicine flat out poison and add 4 to the second roll:

How You'll FeelIn FractionsAs Decimals
You'll feel worse5/1241.7%
You'll feel the same6/1250%
You'll feel better1/128.3%

Adding 5 gives you a 50/50 chance of feeling the same or worse, based on whether the first roll was a 6 or 5.

So, that's why folks can be easily fooled into thinking that alternative medicine works. Now, real science has controls and procedures to take regression to the mean into account, as well as placebo effects, which we didn't talk about at all in this post. For more on this sort of stuff, give Trick or Treatment a read, even though it's not that great a read.


Avatar for Anne Anne blathered on about something on Monday October 21 2013 @ 2:06pm

Geezle pete, that is a LOT of math that I am probably not going to pay attention to. But, I love it when a study comes out and says that a placebo works for such & such. Duh! (Or "doi" as kids say nowadays!) That's what a placebo is -- something that is nothing but that works anyway.

Your Name:

Your Email:
(not shown, but logged along with your IP)

Your comments:
(no HTML allowed, at all)

4 plus 0 equals


Secular Stuff

RSS 2.0 Feed