STA631 Handouts Lectures 1-45

Topics (1-7)

Statistical Inference and its importance The basic objective of the statistical inference is to learn the characteristics of the population by using sample information. Generally, we call population characteristics as the parameter(s) and sample characteristics as the statistic(s). For illustration purpose, two examples are quoted to estimate the characteristics of a population: the average height of the people of Pakistan or the percentage of the voters in Punjab who think the government is doing well etc.

In real it is not possible to get data from every member of the population. So what should we do? We obtain the data from a sample and on the basis of it we make inferences about the population. According to the International Encyclopedia of Education (3rd edition), statistical inference is

used to draw conclusions about ion based on a random sample some unknown aspect of a populat from that population. Some preliminary conclusions may be drawn by the use of explanatory data analysis (EDA) or by the computation of summary statistics as well, but formal statistical inference uses calculations based on probability theory to validate those conclusions.

Probability theory is the foundation of the statistical inference. In probability theory, we have a random variable (r.v) X having a probability density function (pdf) or probability mass function (pmf) of the form f x; or p x; ; where  is a parameter space), e.g.

X Exp We call is a parameter of the distribution. As is unknown therefore we want to estimate it. The pdf or pmf is sometimes completely unknown or in some situation known down to the parameter only. To drawn inferences about the pdf or the parameter(s), our information comes from a sample based on the unknown distribution of X because sample observation have the same distribution as X.

Introduction to Random Sampling Before starting the properties of the random sampling (r.s), it is important to know about the random sampling.

When we talk about simulating any process with randomness it means that sampling is done by probability distribution. The r.s model describes an experimental situation in which variable of interest has a probability density function f x. Suppose X is a random variable (r.v) of interest

but we do not know its probability density function (pdf) f x or probability mass function

(pmf) p x; i.e. one or more parameters associated with pdf or pmf are unknown. The prime objective of statistical inference is to estimate these unknown parameters or to test certain statements about them. For this purpose we take n independent observations on r.v X, where each i X has a value

i x ,i , ...n . This means 1n X ,...X are independent r.vs with common pdf and

1n x ,...x are their assumed values. The 1n X ,...X is the sample of size n from a population and

1n x ,...x is called the values or realization of sample. In general, the sample observations

1n X ,...X are mutually independent. Definition 1: If the r.vs 1n X ,...X are independent and identically distributed (iid), then these r.vs constitute a random sample of size n from the common distribution.

Statistic with examples The information in the sample is summarized by using functions of sample, which are called statistic.

1nX ,...X denote a sample of size n on a r.v X. Let 1n TTX, X be the

function of a sample, then Tis called a statistic.

The two commonly used statistics are the sample mean and sample variance which are given as:

n i i X Xn and n i i XX S. n

Note: The sample summary given by the statistic includes many types of information such as the smallest or largest value in the sample, the average sample value or a measure of the variability in the sample observation.

Since the dependence of statistic on sample is understood therefore we write S rather than

1n S X ,...X .Generally the observed values of statistic are denoted by lower case so the observed values of X,S etc are denoted by x,s etc.

Remarks: The r.s model in definition 1 is sometimes called sampling from infinite population.

Methods of drawing samples There are many ways of drawing samples but here we discuss only two.

1. Suppose we choose a value from the population in such a way that the probability of selection of each value is same i.e. 1/N. This type of sampling is called sampling with replacement (w.r) and it meets the condition in definition 1. (This type of sampling is used in bootstrap which will be discussed later).
2. The second method of drawing a r.s from a finite population is called sampling without replacement (w.o.r). In this method the probability of selecting first value is 1/N. The first value is recorded and second value is chosen from the remaining Nvalues. The

probability of the remaining N  values is 11 /N The second value is recorded. This process continues for the remaining values till we obtain the samples 1n X ,...X . This type of sampling does not satisfy all the conditions of definition 1, i.e. r.vs 1n X ,...X are not mutually independent.

Example: Suppose a finite population consists of elements, so 1000 N. A sample of

size nis drawn without replacement. What is the probability that all ten sample values are

greater than 200?

If 0 X ,...X are mutually independent, we will proceed as P X ,...X P X ,...P X 1000 1000 1000

times To calculate this probability exactly, we assume that Yis a random variable representing the number of items in the sample which are greater than 200. Since the sampling is done without replacement therefore we use hypergeometric distribution, so 1000 N, M, K

1000 CC P X ,...X P Y C

Note: In future we will use Definition 1 as the definition of a random sample from a population.

Sum of random samples from a random sample When a sample is drawn, some summary of the values is computed. Any well-defined summary of these values is represented by a function 1n T x ,...x whose domain is the sample space of the

random vector 1n X ,...X .The function T may be a real-valued or vector-valued therefore the

summary is a random variable or random vector. Since the r.s. 1n X ,...X has a simple probabilistic structure (as of i.i.d) therefore the distribution of Y is observable and called sampling distribution of Y. This distinguishes the probability distribution of Y from the distribution of population, i.e. the marginal distribution of each i X .

1n X ,...X be a r.s. of size n from a population and let 1n T x ,...x be a real-

valued or vector-valued function whose domain includes the sample space of 1n X ,...X ,then the

random variable or random vector 1n YTX, X is called s statistic. The probability

distribution of a statistic Y is called the sampling distribution of Y.

Some results

Lemma 5.2.5: Let 1n X ,...X be a r.s from a population and let gxbe a function such that

Eg X and Varg X exist. Then

n i i Eg X n E g X  and

n i i Var g X n var g X

Proof: To prove the first part we proceed as:

nn ii ii E gX EgX nEgX