Introduction to Statistics (MAT/SST 115.03 2008S)

Activity 21-1, Modified

Activity 21-1 asks you to do some simulation using playing cards. While it can be fun to do such simulations, they are also quite time consuming. These instructions, written by Katherine and Kent McClelland and updated by Samuel A. Rebelsky, provide an alternate approach to that activity.

Some Background

What happens when we randomly assign experimental subjects to two groups and then compare the proportions of successes in each group? If there is no difference in the effects of the two treatments, what would you expect, on average, the difference in the two proportions to be?

Of course, we know that sampling variability plays a role and so we do not expect the proportions of successes in the two groups to be identical, even if the treatment has had no effect at all.

Preliminary Activities

Read the introduction to Activity 21-1 and complete steps a through e.

Record the difference between the two proportions you found in step c.

Read the paragraph after step e and the instructions for step f, but do not do step f.

Simulating the Experiment

How do we tell whether the difference in two proportions that we found in doing a study is more likely to be the result of an actual difference in the effects of the two treatments rather than the result of chance? Let's look at the applet called Friendly Observers, to see how differences in proportions are distributed due to chance variation alone. The applet examines the “hand” drawn by Group A, the group where the observer shares the prize. Remember that once you've seen Group A's hand, where black are counted as successes, you know what group B's hand looks like as well, since the total number of successes is fixed.

Open the Simulation and select the circle next to Difference in Proportion of Black/Success. Watch a few hands being played by clicking Randomize with the animation on. Grab a teacher or mentor if you don't understand what is being graphed.

You are now ready to simulate a large number of experiments. Turn off animation by unchecking Animate. Click Reset. Ask for 1000 repetitions and then click Randomize.

What is the mean of the 1000 differences in proportions?

What is the standard deviation of the empirical sampling distribution of 1000 differences?

Analyzing The Results

Use this empirical sampling distribution to estimate the probability that we would get a difference in proportions as extreme or more extreme than the difference we got in the original table, by sampling variability alone. (Remember that you can estimate this by computing a z-score and then looking it up in the standard normal probabilities table.)

When we do a significance test, what do we call that probability?

Back to the Book

Go back to Activity 21-1 and begin reading after part p on page 417.

Creative Commons License

Samuel A. Rebelsky,

Copyright (c) 2007-8 Samuel A. Rebelsky.

This work is licensed under a Creative Commons Attribution-NonCommercial 2.5 License. To view a copy of this license, visit or send a letter to Creative Commons, 543 Howard Street, 5th Floor, San Francisco, California, 94105, USA.