Arun K. Tangirala Power of Hypothesis Tests

Computing Pr(Type II error): Two-tailed z-test for mean The test statistic is then Zo = X¯ µ 0 / p n = X¯ µ 1 / p n + p n (2) The distribution of the ...

0 downloads 129 Views 332KB Size
Power of Hypothesis Tests

References

Introduction to Statistical Hypothesis Testing Arun K. Tangirala Power of Hypothesis Tests

Arun K. Tangirala, IIT Madras

Intro to Statistical Hypothesis Testing

1

Power of Hypothesis Tests

References

Learning objectives

I

Computing Pr(Type II error) and Power

I

Choice of sample size

I

Closing remarks (for the course)

Arun K. Tangirala, IIT Madras

Intro to Statistical Hypothesis Testing

2

Power of Hypothesis Tests

References

Opening remarks

The “goodness” or success of a hypothesis tests, in general, I

Proper statement of alternative hypothesis, Ha .

I

True variability, i.e.,of the population(s),

I

Sample size n

I

Choice of significance level ↵.

2

.

A measure of the “strength” of a hypothesis test is its power, which in turn is, dependent on the Type II error.

Arun K. Tangirala, IIT Madras

Intro to Statistical Hypothesis Testing

3

Power of Hypothesis Tests

References

Type II error: One-sample test for mean The hypotheses are H0 :µ = µ0 Ha :µ 6= µ0 We fail to reject the null whenever the observed statistic |Zo | < z↵/2 . Recall, for a two-tailed test for mean with known variance , Pr(Type II error) = Pr(|Zo | < z↵/2 |H0 is false)

(1)

Suppose that the null is false and that the true mean is µ1 = µ0 + .

Arun K. Tangirala, IIT Madras

Intro to Statistical Hypothesis Testing

4

Power of Hypothesis Tests

References

Computing Pr(Type II error): Two-tailed z-test for mean The test statistic is then Zo =

¯ X

p ¯ µ1 µ0 X n p = p + / n / n

The distribution of the test-statistic is thus, Zo ⇠ N

Arun K. Tangirala, IIT Madras

✓ p

n

(2)

◆ ,1

Intro to Statistical Hypothesis Testing

(3)

5

Power of Hypothesis Tests

References

Computing probability of Type II error Introduce the standardized variable, Z1 =

¯ X

. . . contd.

µ p 1 ⇠ N (0, 1). / n

p

Then, in terms of this newly introduced standardized variable, Zo = Z1 +

n

.

Therefore, from (1), we have

= Pr



Z1 +

p

n

< z↵/2



= Pr

✓✓

z↵/2

p ◆ n

< Z1 <



z↵/2

p ◆◆ n

(4)

where the probability is computed using a standard Gaussian distribution.

Arun K. Tangirala, IIT Madras

Intro to Statistical Hypothesis Testing

6

Power of Hypothesis Tests

References

Computing probability of Type II error Introduce the standardized variable, Z1 =

¯ X

. . . contd.

µ p 1 ⇠ N (0, 1). / n

p

Then, in terms of this newly introduced standardized variable, Zo = Z1 +

n

.

Therefore, from (1), we have

= Pr



Z1 +

p

n

< z↵/2



= Pr

✓✓

z↵/2

p ◆ n

< Z1 <



z↵/2

p ◆◆ n

(4)

where the probability is computed using a standard Gaussian distribution. For a given ,

and sample size n, the power of hypothesis test is computed as Power = 1

Arun K. Tangirala, IIT Madras

Intro to Statistical Hypothesis Testing

(5) 7

Power of Hypothesis Tests

References

Type II errors Two-tailed test: =F



p ◆ n

z↵/2

F



z↵/2

p ◆ n

(6)

Lower-tailed test: =1

F



z↵

p ◆ n

(7)

Upper-tailed test: =F

Arun K. Tangirala, IIT Madras



z↵

p ◆ n

Intro to Statistical Hypothesis Testing

(8)

8

Power of Hypothesis Tests

References

Example

Propellant burning rate Consider the motivational and worked out example of the average propellant burning rate. Suppose that the true burning rate is µ = 49 cm/s. What is the Type II error for the two-sided test with ↵ = 0.05, = 2 and sample size n = 25? Solution: The difference between postulated and truth is = 1 and z↵/2 = 1.96. Therefore, from (4), we have ✓ ✓ p ◆ p ◆ n n = F z↵/2 F z↵/2 = F (4.46) F (0.54) = 0.295 Thus, there is a 30% chance that the test will fail to reject H0 , i.e., the truth will be undetected. Observe that the answer would be the same for = 1. The power of the test with this sample size is therefore 1 = 0.7, which is satisfactory.

Arun K. Tangirala, IIT Madras

Intro to Statistical Hypothesis Testing

9

Power of Hypothesis Tests

References

Example

Propellant burning rate Consider the motivational and worked out example of the average propellant burning rate. Suppose that the true burning rate is µ = 49 cm/s. What is the Type II error for the two-sided test with ↵ = 0.05, = 2 and sample size n = 25? Solution: The difference between postulated and truth is = 1 and z↵/2 = 1.96. Therefore, from (4), we have ✓ ✓ p ◆ p ◆ n n = F z↵/2 F z↵/2 = F (4.46) F (0.54) = 0.295 Thus, there is a 30% chance that the test will fail to reject H0 , i.e., the truth will be undetected. Observe that the answer would be the same for = 1. The power of the test with this sample size is therefore 1 = 0.7, which is satisfactory. How can we improve the power of a hypothesis test? Arun K. Tangirala, IIT Madras

Intro to Statistical Hypothesis Testing

10