Harvard Management Update (Vol. 11, No. 3, March 2006) passes along Five Guidelines for Using Statistics.

Here they are (with a little InsideWork embellishment):

1. Know what you know–and what you’re only asserting

Well, yeah! What we know is backed up with hard numbers; what we assert has more to do with assumptions (hypotheses, theories, constructs, intuitions, hunches, hopes, dreams, pet projects, dead horses…). There’s nothing wrong with making assertions (which carry the advantage of being free), just don’t label them as as knowledge. Representing assertions as facts may make the sale today but – ask Jeff Skilling, Ken Lay, et.al. – there will be a day of reckoning in the foreseeable future.

2. Be clear about what you want to discover

If you’re not clear on the difference between the mean and the median, we recommend a time out before you make any big announcements based on your research. Going in, ask each other "what do we want to find out?" and measure for that. If we don’t ask the right question – and ask the question right – we may as well be working on hunches (see number one above).

3. Don’t take causality for granted

Correlation is not causality. Harvard’s Frances Frei says, "To establish genuine causation, you need to ask yourself three questions. Is there an association between the two variables? Is the time sequence accurate? Is there any other explanation that could account for the correlation?" Get causality and correlation mixed up and bad things can happen.

4. With Statistics, you can’t prove things with 100 percent certainty

Random samples can yield probabilities above 90 percent. Statistical analyses rely on probabilities because no one can afford to survey everyone in the universe they’re studying unless it is a very tiny universe (say, for instance, all commercial space travelers through the end of 2006). Learn to live with probabilities. Las Vegas has – which is why money is among the things that stays there.

5. A result that is numerically or statistically significant may be managerially useless

When it comes to statistics, the meaning of good numbers is closer to the practice of accounting than, say, engineering. Once you have good numbers, you still have to exercise quite a bit of judgement to discern what they mean and decide what to do with them.

Note: All this, by the way, is why we are working with National Demographics Corporation on InsideWork’s national survey of business people who say they are trying to operate on biblically-based business principles. It’s a million-dollar project because getting and interpreting reliable statistics on a national scale is no job for amateurs. We’ve completed the first phase of the research; stay tuned for details as the study moves forward.