STA406 Slides by Mahar Afaq

Lecturer

Statistical Inference

It is the procedure by which we reach a conclusion about a population on the basis of information contained in the sample drawn from that population. Two broad areas of statistical inference  Estimation  Hypothesis testing

Estimation

It is the process of calculating some statistics based upon the sample data drawn from a certain population. The statistics is used as an approximation to the population parameter.

Types of Estimate

For each of parameters, we can have

Point Estimate

Interval estimate

•  A point estimate is a single numerical value used to estimate the corresponding population parameter.

Example: The mean of the age of men attending a show is 32 years. An interval estimate consists of two numerical values defining a range of values that, with a specified degree of confidence, includes the parameter being estimated. Example: The mean of the age of men attending a show is between 28 and 36 years. A point estimator is said to be good point estimator if it

satisfies the following criteria's :

 Unbiasedness

Unbiasedness

An estimator is defined to be unbiased if statistic used as an

estimator has its expected value equal to the true value of the

population parameter.

Suppose ˆ be an estimator of a parameter. Then ˆ will be called an unbiased estimator if  ˆ E If  ˆ E the statistic is

said to be a biased estimator

Unbiasedness

1. The sample mean is an unbiased estimator of the

population mean

2. The sample proportion is also an unbiased estimator of the

population P.

3. s2 is an unbiased estimator of but S2 is a biased estimator

of . X

p

Example 1

Let n X X X be a random sample from the population with mean of

µ and a variance  . Show that the sample mean x and 2s is unbiased estimator of µ and

Example 2

A random sample of size n is drawn from the rectangular distribution. f x x Show that the statistic 2X is an unbiased estimator for.

An estimator ˆ is said to be a consistent estimator of the parameter

 if, for any arbitrary small positive quantity e,

.1 e ˆ P Lim n

In other words, an estimator ˆ is called a consistent estimator of  if the probability that ˆ is very close to , approaches unity with an increase in the sample size.

It should be noted that consistency is a large sample property. Another point to be noted is that a consistent estimator may or may not be unbiased.  The sample mean n i i 1 X X n , which is an unbiased estimator of , is a

consistent estimator of the mean .  The sample proportion is also a consistent estimator of the parameter

p of a population that has a binomial distribution.  The median is not a consistent estimator of  when the population has a

skewed distribution.  The sample variance

Though a biased estimator but it is a consistent estimator of the population variance X X n S i n

i pˆ

To prove that an estimator is consistent we have to follow the

following criteria:

“Suppose is an estimator of based on a sample of size n.

Then is an consistent estimator of , if Var ( )→0 as n → ”

Example:

Provethat : i Asamplemean X isa consistent estimatorof population mean .

ii Asampleproportion p isa consistent estimatorof population proportion Pfora population that hasbinomialdistribution.

“If and denotes two unbiased estimators for the same parameter

then the estimator will be said to be more efficient than if Variance( )< Variance( )”.

To compare the relative efficiency of two estimators, we examine the ratio of their variances as”

Efficiency( , )=variance( )/variance( )>1

It thus provides a criterion for comparing different unbiased estimators of a parameter. 1

1 1 1

1

is anefiicient estimator of For a normal distribution, both the sample mean and the sample median are unbiased and consistent estimators of µ, but the

variance of the sampling distribution of the sample means n is

smaller than the sampling distribution of sample medians 2n

i.e. 1.57 1. Var median n Var X n Hence the sample

mean is more efficient than the sample median as an estimator of µ,

The sample mean may therefore be preferred as an estimator. Let X X X and X be a random sample of size n  from a N  . A statistician

wishes to estimate the mean by using either of the following estimators of the mean .

X X X X T (Thesamplemean X)

w X 2X 3X X T (A weighted mean,X )

Which estimator should be preferred?

Example 1:

To answer this question correctly, we need to compare the expected values and the

variance of these statistics.

Let us first see if the two estimators are unbiased.

Now,

X X X X E T E E X X X X

E X E X E X E X

1T , the sample mean X , is the unbiased estimator.

Again

2 ( 3 ( X X X X E T E E X E X E X E X

E T

2T , the weighted mean of the observations, is also an unbiased estimator of . Thus we

see that both means are unbiased estimators of .

Next we find the variances of theses estimators. X X X X Var T Var Var X Var X Var X Var X

Var T

(2 (3 X X X X Var T Var Var X Var X Var X Var X

Var T Now,

Var T Var T =1.4

Which is greater than 1, showing that Var T Var T . Thus 1T is more efficient than

2T as an estimator for .

Hence 1T X is a better estimator of  than w T X and should be preferred.

An estimator is defined to be sufficient, if the statistic used as estimator uses all the information that is contained in the sample.

A necessary and sufficient condition for a statistic to be sufficient estimator for an unknown parameter is that the joint distribution of the sample of n values can be factorized into two non-negative functions (factors), one depending on other depending on X but not on and Mathematically, lf x1,x2,..xn be a random sample from a probability distribution f(x, ) and let f(x1,x2,..xn; ) denote its joint distribution.

Then = h(x1,x2,..xn) is a sufficient estimator of if and only if:

f(x1,x2,..xn; )= g( , ) h(x1,x2,..xn)

Example

Let X1, X2, …,Xn be a random sample from the Poisson

distribution f(x; θ) = , (x = 0,1,2, …).

Then show that is a sufficient estimator for θ. x e x

X Given the Poisson distribution

f(x;θ) = x e x , x = 0,1,2, … (θ is used for µ)

The joint distribution of the sample values is

f(x1,x2, …, xn;θ) = f(x1; θ) f(x2; θ) … f(xn; θ)

!...x ! ix n x n i i n e e x x x

!n i i x n i x i n x e n x x x x