STA301 Handouts Lectures 19-22 by Masters

Relative Frequency Definition of Probability Axiomatic Definition of Probability Laws of Probability Rule of Complementation Addition Theorem

THE RELATIVE FREQUENCY DEFINITION OF PROBABILITY (‘A POSTERIORI’ DEFINITION OF

If a random experiment is repeated a large number of times, say n times, under identical conditions and if an event A is observed to occur m times, then the probability of the event A is defined as the LIMIT of the relative frequency m/n as n tends to infinitely. Symbolically, we write

The definition assumes that as n increases indefinitely, the ratio m/n tends to become stable at the numerical value P(A). The relationship between relative frequency and probability can also be represented as follows: Relative Frequency  Probability as n   As its name suggests, the relative frequency definition relates to the relative frequency with which are event occurs in the long run. In situations where we can say that an experiment has been repeated a very large number of times, the relative frequency definition can be applied. As such, this definition is very useful in those practical situations where we are interested in computing a probability in numerical form but where the classical definition cannot be applied.(Numerous real-life situations are such where various possible outcomes of an experiment are NOT equally likely). This type of probability is also called empirical probability as it is based on EMPIRICAL evidence i.e. on OBSERVATIONAL data. It can also be called STATISTICAL PROBABILITY for it is this very probability that forms the basis of mathematical statistics. Let us try to understand this concept by means of two examples: from a coin-tossing experiment and From data on the numbers of boys and girls born.

EXAMPLE-1

Coin-Tossing: No one can tell which way a coin will fall but we expect the proportion of leads and tails after a large no. of tosses to be nearly equal. An experiment to demonstrate this point was performed by Kerrich in Denmark in 1946. He tossed a coin 10,000 times, and obtained altogether 5067 heads and 4933 tails. The behavior of the proportion of heads throughout the experiment is shown as in the following figure:

.2 · .6 · .5 · .8 · 1.0 · 1000 · Number of tosses (logarithmic scale) · Proportion of heads

The proportion; of heads in a sequence of tosses of a coin (Kerrich, 1946):

n · m · Lim · A · P

n

final term · based on past experience and experiments · mcq · mcq · mcq · mcq · 5067/10,000 = 0.5067 · 4933/10,000 = 0.4933

As you can see, the curve fluctuates widely at first, but begins to settle down to a more or less stable value as the number of spins increases. It seems reasonable to suppose that the fluctuations would continue to diminish if the experiment were continued indefinitely, and the proportion of heads would cluster more and more closely about a limiting value which would be very near, if not exactly, one-half. This hypothetical limiting value is the (statistical) probability of heads. Let us now take an example closely related to our daily lives --- that relating to the sex ratio:- In this context, the first point to note is that it has been known since the eighteenth century that in reliable birth statistics based on sufficiently large numbers (in at least some parts of the world), there is always a slight excess of boys, Laplace records that, among the 215,599 births in thirty districts of France in the years 1800 to 1802, there were 110,312 boys and 105,287 girls. The proportions of boys and girls were thus 0.512 and 0.488 respectively (indicating a slight excess of boys over girls).In a smaller number of births one would, however, expect considerable deviations from these proportions. This point can be illustrated with the help of the following example:

EXAMPLE-2

The following table shows the proportions of male births that have been worked out for the major regions of England as well as the rural districts of Dorset (for the year 1956).

As you can see, the figures for the rural districts of Dorset, based on about 200 births each, fluctuate between 0.38 and

1. 59. While those for the major regions of England, which are each based on about 100,000 births, do not fluctuate much, rather, they range between 0.512 and 0.517 only. The larger sample size is clearly the reason for the greater constancy of the latter. We can imagine that if the sample were increased indefinitely, the proportion of boys would tend to a limiting value which is unlikely to differ much from 0.514, the proportion of male births for the whole country. This hypothetical limiting value is the (statistical) probability of a male birth. The overall discussion regarding the various ways in which probability can be defined is presented in the following diagram:

Proportions of Male Births in various Regions and Rural Districts of England in 1956

Region of · England · Proportion · of Male · Births · Rural Districts of · Dorset · Proportion · of Male · Births · Northern · .514 · Beaminster · .38 · E. & W. Riding · .513 · Blandford · .47 · North Western · .512 · Bridport · .53 · North Midland · .517 · Dorchester · .50 · Midland · .514 · Shaftesbury · .59 · Eastern · .516 · Sherborne · .44

London and S.

Eastern · .514 · Sturminster · .54 · Southern · .514 · Wareham and · Purbeck · .53 · South Western · .513 · Wimborne & · Cranborne · .54 · Whole country · .514 · All Rural District’s · of Dorset · .512

As far as quantifiable probability is concerned, in those situations where the various possible outcomes of our experiment are equally likely, we can compute the probability prior to actually conducting the experiment --- otherwise, as is generally the case, we can compute a probability only after the experiment has been conducted (and this is why it is also called ‘a posteriori’ probability). Non-quantifiable probability is the one that is called Inductive Probability. It refers to the degree of belief which it is reasonable to place in a proposition on given evidence. An important point to be noted is that it is difficult to express inductive probabilities numerically –– to construct a numerical scale of inductive probabilities, with 0 standing for impossibility and for logical certainty. An important point to be noted is that it is difficult to express inductive probabilities numerically –– to construct a numerical scale of inductive probabilities, with 0 standing for impossibility and for logical certainty. Most statisticians have arrived at the conclusion that inductive probability cannot, in general, he measured and, therefore cannot be use in the mathematical theory of statistics. This conclusion is not, perhaps, very surprising since there seems no reason why rational degree of belief should be measurable any more than, say, degrees of beauty. Some paintings are very beautiful, some are quite beautiful, and some are ugly, but it would be observed to try to construct a numerical scale of beauty, on which Mona Lisa had a beauty value of 0.96. Similarly some propositions are highly probable, some are quite probable and some are improbable, but it does not seem possible to construct a numerical scale of such (inductive) probabilities .Because of the fact that inductive probabilities are not quantifiable and cannot be employed in a mathematical argument, this is the reason why the usual methods of statistical inference such as tests of significance and confidence interval are based entirely on the concept of statistical probability. Although we have discussed three different ways of defining probability, the most formal definition is yet to come. This is The Axiomatic Definition of Probability.

THE AXIOMATIC DEFINITION OF PROBABILITY

This definition, introduced in 1933 by the Russian mathematician Andrei N. Kolmogrov, is based on a set of AXIOMS. Let S be a sample space with the sample points E1, E2, … Ei, …En. To each sample point, we assign a real number, denoted by the symbol P(Ei), and called the probability of Ei, that must satisfy the following basic axioms:

Axiom 1:

For any event Ei, 0 < P(Ei) < 1.

Axiom 2:

P(S) =1 for the sure event S.

Axiom 3:

If A and B are mutually exclusive events (subsets of S), then

It is to be emphasized that According to the axiomatic theory of probability:

Non-Quantifiable · (Inductive, · Subjective or · Personalistic · Quantifiable · Statistical · (Empirical or “ A Posteriori ” · (A statistician’s main concern) · “ A Priori ” · (Verifiable · through · Empirical · Evidence) · common causes special causes · mcq · different ways of · talking about the · same things · outcomes are · equally likely · outcomes are not · equally · likely eg: how much beautiful · events are disjoint