Is it Really True that Only 8% of People Who Buy Herbalife (HLF) Products are Herbalife Distributors?

A reporter emailed me the other day with a question about a case I’d never heard of before, a company called Herbalife (HLF) that is being accused of being a pyramid scheme. The reporter pointed me to this document which describes a survey conducted by “a third party firm called Lieberman Research”:

Two independent studies took place using real time (aka “river”) sampling, in which respondents
were intercepted across a wide array of websites

Sample size of 2,000 adults 18+ matched to U.S. census on age, gender, income, region and ethnicity

“River sampling” in this case appears to mean, according to the reporter, that “people were invited into it through online ads.” The survey found that 5% of U.S. households had purchased Herbalife products during the past three months (with a “0.8% margin of error,” ha ha ha).

They they did a multiplication and a division to estimate that only 8% of households who bought these products were Herbalife distributors: 480,000 active distributorships / (0.05 * 114 million U.S. households) = 0.08.

The reporter asked what I thought about that 8%. Here are some of my reactions (again, recall that I had not heard about this case before, and these reactions are based on the information provided to me):

I think there are some serious problems here.

1. Generalizing from 2000 to the whole population is not a problem—as long as the sample is a representative sample, and as long as the survey responses are correct. I did not look at the details of the survey so I don’t know about the issue of representativeness. I wonder, though, exactly how they “fished” for respondents in their river sampling. It could be that they were more likely to get the sort of person who was interested in dietary supplements. Or maybe not, I don’t know.

2. However the sampling was done, I worry about the survey responses. It’s a well-known problem that rate events get overestimated. For example (from an article by David Hemenway from 1997): “The National Rifle reports 3 million dues-paying members, or about 1.5% of American adults. In national random telephone surveys, however, 4-10% of respondents claim that they are dues-paying NRA members. Similarly, although Sports Illustrated reports that fewer than 3% of American households purchase the magazine, in national surveys 15% of respondents claim that they are current subscribers.” The mathematics is that if there is a small rate of error, it can show up as a large error in the estimate of a small population. So, just because 5% of respondents say they used Herbalife products in the previous three months, that doesn’t mean that 5% of respondents actually used Herbalife products in the previous three months. I’d guess it’s a lot less.

3. I assume that “distributors” buy a lot more Herbalife products than non-distributors. Hence, even if 8% of the Herbalife consumers are distributors, I’d guess that a lot more than 8% of Herbalife products are bought by distributors. This doesn’t address Herbalife’s claims (they talk about the number of households, not the amount of products) but it seems relevant to the discussion.

P.S. Amusingly, one of Herbalife’s points is “Fact: Majority of Former Distributors Would Recommend Herbalife to Friends and Family.” But that’s exactly what you’d expect of a still-active pyramid scheme, no? Existing members want new people below them on the pyramid. I’m not saying this means it is a pyramid scheme, but it doesn’t seem like evidence against the hypothesis!

P.P.S. I tried to choose the most appropriate category for this one.

• Become A Funded Futures Trader