Lies, Damned Lies, and Statistics

I am having trouble assessing the statistics offered by a "Therapeutic Touch" practitioner.I'm afraid I haven't many details of this "study" as the radio host who quoted this "study" (Deborah Ray, Healthy Talk Radio) offered little in the way of information (Harvard, 84 Boston women, half were control).I've been researching this JAMA article, hoping to find out why Dr. Barrett concludes that 3 of the TT studies have inadequate sample size.Two of the cites are dissertations (no online access) and the third referred to two studies; one had a sample size of 19 (9 control) and the other had a sample size of 46 (29 control).It's been a while since I had statistics and even then I just cranked on the numbers (typical biogirl ... I haven't any intuition for #s).Could someone please help me figure out adequate v. inadequate sample size?

Your link points to the landmark paper by Emily Rosa who in 1998 became the youngest person to ever publish a paper in a peer-reviewed medical journal. She was 9 when she conducted the experiment, and she was 11 when it was published. The paper is widely considered to have struck the death knell for therapeutic touch, which still persists but often under a new name, maybe "healing touch", but I'm not sure. The Skeptics Guide to the Universe podcast had an interview with her father, Stephen Barrett, back around September of 2006. Emily's in college now majoring in psychology. I haven't read the Wikipedia article on Emily beyond the 1st paragraph, but I assume you can find more details there.About sample size and statistical measures of validity, it's been so long I no longer remember the equations. I worked in a field where we weren't sampling humans, so we could have huge sample sizes and achieve 99% probability of accurate outcomes. But in the medical field small sample sizes are not unusual because of the expense, and because the outcomes are usually binary (which wasn't the case in my field) you can still obtain good assurance of validity. But sorry, I can't provide any math to back this up. I'm trying to remember who's posted about statistics, maybe it was WK, hopefully someone knowledgeable will pop in.Talking people out of beliefs in all these kinds of nonsense has a low success rate. James Randi talks about demonstrating for people how supposed paranormal practitioners pull off their stunts, and after learning how the trick is done a fair number of these people still don't believe that Randi is just performing a trick - they insist he really does have paranormal powers. There's no hope for these people.--Percy

I love JREF - Home! I check in every Friday to get the latest Swift report.Rrhain was kind enough to drop in on the holistic thread and help me with the stats. Turns out (if I understand his math) that I'm right. The sample size is too small. But, like I mentioned in the holistic thread, Dr. Barrett used Emily's study as a springboard for a literature review (which showed, of course, that there is zero evidence for TT).

What exactly is your question? I might be able to help, or I might not. I'm not a statistician, but I have taught the basic statistics course and the intro to probability course (the two are not the same, by the way). In fact, I wrote a long post in response to your question, then junked it when I reread the paper. Now I wish that I posted it anyway.

---

I've done everything the Bible says, even the stuff that contradicts the other stuff! -- Ned Flanders

Well. My original question (on the holistic thread) had to do with a "study" some radio woomeister was going on about.Rrhain said, "the way you determine sample size is dependent upon the distribution you are expecting. And if you have any information regarding how prevalent the trait you are testing for actually exists in the population, that will have an effect, too."Then he outlined the math for "a binomial distribution with a series of n independent Bernoulli trials". (This bit I remember from stat. Vaguely.)Thing is, I don't know what distribution one would expect for the "mind over body" crap this woman (Deborah Ray, Healthy Talk Radio) was spouting. She claimed (this is a quote) "that the women in the group, who were told that their work was good exercise and that they met the guidelines for healthy active lifestyle, lost weight, saw their blood pressure drop, saw their body fat drop, they reduced their body mass index, and the other group saw absolutely no changes. So, the conclusion, thinking you got a workout made your body healthier when nothing in your lifestyle changed." And then, to make matters worse, this Deborah Ray starts in on "neurotransmitters" and arterial plaque and back pain. In under 30 seconds.I haven't any idea how to proceed re: sample size. And I can't critique the "study" itself because beyond "Harvard" "Dr. Langer" and "84 women", I've got nothing to go on.(Btw, I tried to find something that Langer published on this subject came up empty handed.)

Key word: controlled experiment. One designs an experiment that will measure a rigorously defined outcome. Then one can see what distribution one will expect. If one is making vague claims, then of course it's not going to be clear what distribution one is going to use -- in fact, it won't even be clear what the exact outcomes one should expect or how to measure them.For example, in the JAMA paper to which you linked, the outcome was pretty clear. The practitioner was supposed to tell which hand the subject was presenting; there are only two possible outcomes, left hand or right hand, that is, a success or a failure. This is the Binomial Distribution. (I have no idea why they were using the T Distribution, but then, I'm not a statistician. I don't think it would change the conclusions any, though.)The null hypothesis is that the practitioners were guessing randomly. Thus, the null hypothesis is that the probability for success would be p = 0.5. From here on, everything is a straightforward calculation.Now other experiments that would measure other quantities -- for example, the outcome of a trial might attempt to produce a change in body temperature which would be a continuous variable, would necessity the use of a different distribution -- and, if we were actually going to compare the results with a control group, then that would also factor into which distribution would be appropriate.

---

I've done everything the Bible says, even the stuff that contradicts the other stuff! -- Ned Flanders

I could use some more detail, if you don't mind. You're talking about something different than clinical massage therapy, which is physical therapy (muscular, myofascial, etc). Percy used the word "paranormal."What are the claims being made for this?

It's prana/qi + a nurse named Dolores Krieger (who concocted TT in the 70s).Standard issue stuff. "Imbalances" are sensed by waving hands around, "energy" is transferred ... you know the drill.

Thanks. Gotcha.Body Fengshui.

Heh. I was thinking acupuncture without needles, but this is funnier.

---

I've done everything the Bible says, even the stuff that contradicts the other stuff! -- Ned Flanders