Part One: Probability

Probability Perspectives

Poincare and Chance

"A very small cause which escapes our notice determines a considerable effect that we cannot fail to see, and then we say that that effect is due to chance."₁

Chance can operate at several levels, based upon the nature of that which escapes our notice. In a pragmatic sense, chance is usually viewed at two levels: unobserved factors which could be observed or measured; and irreducible uncertainty. We can have a very naïve sense of chance, in which we claim no prior knowledge. We can modify this naïve sense of chance if we can claim some basic or prior knowledge. The basic idea of chance is that it is driven by the unknown (and unknowable) forces behind a process. Probability views chance from a hypothetical standpoint, while statistics views probability from an empirical/experiential standpoint.

1. H. Poincare, Science and Method, translated by F.Maitland, pp. 67-68.(New York: Dover Publications, 1952).

Prediction and a Fair Die 

Randomness embodies the uncertainty in the prediction of outcomes in selected processes. Consider the fair, six-sided die (d6). A fair d6 has six, equally likely faces, usually with face values {1,2,3,4,5,6} or more commonly with spot-groups in place of numbers. Consider a thought experiment in which we predict the face that will show prior to tossing the d6. Then check the prediction against the actual toss. Try this a few times. Why can't we predict the tosses reliably?

Probability forms the theoretical foundation for Statistics.

Probability underlies Statistics like Algebra underlies Calculus. While some concepts in Statistics do not relate directly to Probability, Statistics is in many ways an extension of Probability. So we study Probability first.

Some loose definitions:

…from Webster's New Collegiate Dictionary… Probable: supported by evidence strong enough to establish presumption but not proof. Probability: the relative degree to which an event is probable.

…from Statistics (Freedman, Pisani, Purves and Adhikari)…Chance: the percentage of time something is expected to happen, when the basic process is repeated independently and under the same conditions.

The basic principle of Probability is that of a likelihood - in fact, the term probability is often used as a synonym or alias for likelihood. Imagine a black box with a red button in front. Each time the button is pressed, something simple happens. The box is in the same state each time - the box resets after each cycle.

A Black Box

Imagine that a team is charge of learning about the box. This team is not given any detailed information about the box, and they are not allowed to harm or damage the box - all they can do is press the button and note the results.

One person pushes the button - over and over and over and over again, ad infinitum. Each time the button is pressed, another person writes down what happens, that is, what the box does when the button is pressed. A third person summarizes what the second person wrote.

After a large number of box-cycles (press button and see what happens), the team can gain some insight into the box 's workings.

Specifically, the team can say something about the following:

What sort of things can the box do when the button is pushed ?

How often does the box seem to do certain specific things ?

The second item is at the heart of Probability - in long runs of activity, how often can we expect specific results from the box? This is the idea of likelihood - on average, how often do we expect a particular event to occur ?