Statistical testing

Juli's Health Food Store has hired you to write a program to do some statistical testing. Juli suspects that the manufacturers of New Age Granola have been cheating her. They guarantee that boxes contain an average of 453 grams of granola. She thinks they are lighter than that and wants you to find out whether her suspicions are well-founded with statistics.

The universe for this problem is a data set including the weight of every box of New Age Granola that the manufacturers have produced. Juli doesn't have access to all of this data, but let's imagine just for the moment that we have the complete set. Let n be the number of boxes that have been produced, and call the weights of the boxes, in grams, x₁, x₂, ..., and x_n.

Some common statistical measurements of this data set are

the mean, M = (x₁ + x₂ + ... + x_n) / n;
the variance, V = ((x₁ - M)² + (x₂ - M)² + ... + (x_n - M)²) / (n - 1); and
the standard deviation, S = sqrt(V).

The mean of a data set is simply the average of its values. Variance and standard deviation are measures of how closely packed the values are around the mean; the variance reflects disparities from the mean (squared, so that large disparities are in effect weighted more heavily than small ones), and the standard deviation is the square root of the variance. (By taking the square root, we ensure that the standard deviation is measured in the same units as the observed values -- in this case, in grams.)

It would be straightforward to verify or refute New Age Granola's claim if we could stand outside their factory, weigh every box that comes out, and take the mean, M, of all the weights -- that is, if we knew all of the values x₁, x₂, ..., and x_n. Unfortunately, this is not practical; but statistics comes to our rescue, giving us a way to test the manufacturers' claim with high probability of being correct (although, since we are not weighing absolutely every box, we cannot be absolutely sure).

The key idea is to use a sample: a selection of, say, k of the n boxes, where k can be much smaller than n. (For instance, we could use the weights of some of the boxes in the shipment that Juli has just received from New Age Granola.) Let's call the mean of these values M_k.

If we were to do this sampling experiment repeatedly, using every possible sample of size k from the universe of n boxes, we would find that the mean of all of our M_ks is exactly M, and that their standard deviation S* is exactly S / sqrt(k). Moreover, the M_ks would be distributed around M in a pattern resembling a ``bell curve,'' with most of the M_ks clustering around the mean and the others tailing off symmetrically above and below the mean.

In fact, the famous Central Limit Theorem of statistics says that, if we take the sample size k to be large enough, the M_ks are very nearly normally distributed. This is a good thing, because statisticians know a lot about normal distributions. For instance, in a normal distribution, 68% of the data lie within one standard deviation on either side of the mean, and 95% of the within two standard deviations.

In this program, we will use the related fact that 95% of the sample means -- the M_ks -- are greater than M - 1.65S*. (The value 1.65, which may seem a little arbitrary, is actually the outcome of a difficult calculation using the formula that describes normal distributions. Like most working statisticians, we'll simply treat it as a constant rather than writing code into the program to compute it.) So if we take the mean M_k of Juli's sample, the probability that it will turn out to be less than M - 1.65S* is just 5% -- one chance in twenty.

Now, the manufacturers of New Age Granola claim that M is 453 grams. We don't know for sure what S* is, but if S_k is the standard deviation of the weights of the boxes in our sample, then S_k / sqrt(k) should be a good estimate of S*, so that M - 1.65S* should be nearly equal to 453 - 1.65(S_k / sqrt(k)).

So our method will be to we compute M_k and S_k for one particular sample of k boxes. If the manufacturer's claim is true, the odds are nineteen to one against our finding that M_k is less than 453 - 1.65(S_k / sqrt(k)). So, if we do get this result, we'll have pretty good reason to doubt the manufacturer's claim; in other words, we'll have pretty good evidence that the real value of M is less than 453 grams.

Here are the weights, in grams, of 30 boxes of New Age Granola, selected at random from the last shipment that Juli received:

Write a Scheme program that applies the statistical test described above and advises Juli whether to return the shipment and call in the Federal Trade Commission. You'll need at least the following procedures: mean, variance, standard-deviation, and reject-order?.

We are indebted to The cartoon guide to statistics, by Larry Gonick and Woollcott Smith, for the premise of this assignment.

This document is available on the World Wide Web as

http://www.cs.grinnell.edu/~stone/courses/scheme/exercises/statistical-testing.xhtml

created October 29, 2001
last revised November 2, 2001

Ben Gum (gum@cs.grinnell.edu) and John David Stone (stone@cs.grinnell.edu)