- Non-Sectarian
- Books
- Food
- Movies
- Music
- Photos
- Politics
- PolkaDot
- Religion
- Sports
- Tea
- Technology
- Toys
- Travel

A while back, we went to the Women in Secularism 2 conference and had a great time. The first session was about the vast wasteland of alternative medicine. Surly Amy talked about why alternative medicine can often appear to work at an anecdotal level. The thing is, if you're feeling like crap, random chance says you'll likely feel better the next day. People try cures when they're at extreme lows and then think the cure worked if they feel better the next day. But that can just be random chance. The technical jargon is Regression to the Mean.

I wanted to get a feel for how much of an effect there really is, so I made up a quick dice game. Don't worry, you don't need one of those 20-sided D&D jobs. A normal 6-sided die is fine.

The numbers 1-6 on the die represent how good you feel, with 1 being really good and 6 being really awful. Here's how the game is played:

- Roll the die.
- If you roll a 5 or a 6, you feel crappy enough to resort to alternative medicine.
- Roll the die again and see how you feel the next day, after the alternative medicine. If the number is higher than the previous roll, you feel worse. If the number is the same, you feel the same. If the number is lower, you feel better.
- Repeat until you have some good statistics or just get bored.

It should be immediately obvious that, if you rolled a 6 the first time, you won't be rolling an even bigger number the second time. Also plain to see is that there's only one roll to equal a 6, while there are five ways to roll a lower number.

The situation is similar when rolling a 5. There *is* a way to get worse on the second roll, namely by rolling a 6. There's one way to stay the same and four ways to get a lower number.

When you add up all the percentages, the probabilities about how you'll feel after resorting to alternative medicine look like this:

How You'll Feel | In Fractions | As Decimals |
---|---|---|

You'll feel worse | 1/12 | 8.3% |

You'll feel the same | 2/12 | 16.7% |

You'll feel better | 9/12 | 75% |

As you can see, the odds drastically favor feeling better, despite the alternative medicine in this game not having any role in the outcome of the roll. Also keep in mind that feeling the same isn't any big prize. You started out feeling shitty and still feel shitty.

(As a check, I also wrote a short chunk of Perl to simulate the rolls. It confirmed the percentages.)

But wait, it gets worse!

What if we assume that the alternative medicine actually hurts you a little? To model that, we'll add 1 to the second roll. So, if you roll, say, a 3, we'll treat it as a 4. What to do when you roll a 6? Treat it as a 7, I guess? For this game, we'll just round anything bigger than a 6 to a 6. In the real world, a 7 would mean you die and your story is left untold. We could also just leave the 7 as a 7. The only effect would be to make the numbers even worse, shifting much of the feel the same

outcome to feel worse.

In general, I mentally group the feel-worse and feel-the-same together anyway.

Obviously, this twist is going to change the outcome and make it less likely to see fake improvement,

right? Well, yeah, but just barely.

There's still no way to get worse after an initial 6, but now there are two ways to stay the same, rolling either a 5 or a 6. (Because the 5 will get 1 added to it.) Now there are only four ways to roll a lower number and feel better, namely rolling anything from 1 to 4.

The change is similar when rolling an initial 5. There are now two ways to get worse (5 & 6), one way to stay the same (4), and three ways to get better (1-3). Here are the resulting percentages:

How You'll Feel | In Fractions | As Decimals |
---|---|---|

You'll feel worse | 2/12 | 16.7% |

You'll feel the same | 3/12 | 25% |

You'll feel better | 7/12 | 58.3% |

Wait, what? Your chances of feeling better are *still* more than half, even if the so-called medicine

actually makes you sicker? That's right!

Okay, so let's make the medicine

even worse for you, by adding 2 to the second roll. I'll spare you the details and just show the resulting percentages:

How You'll Feel | In Fractions | As Decimals |
---|---|---|

You'll feel worse | 3/12 | 25% |

You'll feel the same | 4/12 | 33.3% |

You'll feel better | 5/12 | 41.7% |

Well, at least the chances you'll feel better are below 50% now. Yet feeling better is *still* the most likely outcome and feeling worse is the least likely. Although, again, feeling the same is no great prize when you already feel shitty. So, at this point, folks might start to doubt the validity of this harmful treatment.

Fine, fuck it, crank up the harm of the medicine

to 3. We'll add 3 to the second roll. Given that it's half the range, that's really quite a lot. Here's what you get:

How You'll Feel | In Fractions | As Decimals |
---|---|---|

You'll feel worse | 4/12 | 33.3% |

You'll feel the same | 5/12 | 41.7% |

You'll feel better | 3/12 | 25% |

Finally, we're getting to where feeling better is the least likely outcome. Note that you're still most likely to feel the same, but that feeling the same

level is basically feeling shitty.

If we make the medicine

flat out poison and add 4 to the second roll:

How You'll Feel | In Fractions | As Decimals |
---|---|---|

You'll feel worse | 5/12 | 41.7% |

You'll feel the same | 6/12 | 50% |

You'll feel better | 1/12 | 8.3% |

Adding 5 gives you a 50/50 chance of feeling the same or worse, based on whether the first roll was a 6 or 5.

So, that's why folks can be easily fooled into thinking that alternative medicine works. Now, real science has controls and procedures to take regression to the mean into account, as well as placebo effects, which we didn't talk about at all in this post. For more on this sort of stuff, give Trick or Treatment a read, even though it's not that great a read.

No comments yet!