References

Introduction to Statistical Hypothesis Testing Arun K. Tangirala Power of Hypothesis Tests

Arun K. Tangirala, IIT Madras

Intro to Statistical Hypothesis Testing

1

Power of Hypothesis Tests

References

Learning objectives

I

Computing Pr(Type II error) and Power

I

Choice of sample size

I

Closing remarks (for the course)

Arun K. Tangirala, IIT Madras

Intro to Statistical Hypothesis Testing

2

Power of Hypothesis Tests

References

Opening remarks

The “goodness” or success of a hypothesis tests, in general, I

Proper statement of alternative hypothesis, Ha .

I

True variability, i.e.,of the population(s),

I

Sample size n

I

Choice of significance level ↵.

2

.

A measure of the “strength” of a hypothesis test is its power, which in turn is, dependent on the Type II error.

Arun K. Tangirala, IIT Madras

Intro to Statistical Hypothesis Testing

3

Power of Hypothesis Tests

References

Type II error: One-sample test for mean The hypotheses are H0 :µ = µ0 Ha :µ 6= µ0 We fail to reject the null whenever the observed statistic |Zo | < z↵/2 . Recall, for a two-tailed test for mean with known variance , Pr(Type II error) = Pr(|Zo | < z↵/2 |H0 is false)

(1)

Suppose that the null is false and that the true mean is µ1 = µ0 + .

Arun K. Tangirala, IIT Madras

Intro to Statistical Hypothesis Testing

4

Power of Hypothesis Tests

References

Computing Pr(Type II error): Two-tailed z-test for mean The test statistic is then Zo =

¯ X

p ¯ µ1 µ0 X n p = p + / n / n

The distribution of the test-statistic is thus, Zo ⇠ N

Arun K. Tangirala, IIT Madras

✓ p

n

(2)

◆ ,1

Intro to Statistical Hypothesis Testing

(3)

5

Power of Hypothesis Tests

References

Computing probability of Type II error Introduce the standardized variable, Z1 =

¯ X

. . . contd.

µ p 1 ⇠ N (0, 1). / n

p

Then, in terms of this newly introduced standardized variable, Zo = Z1 +

n

.

Therefore, from (1), we have

= Pr

✓

Z1 +

p

n

< z↵/2

◆

= Pr

✓✓

z↵/2

p ◆ n

< Z1 <

✓

z↵/2

p ◆◆ n

(4)

where the probability is computed using a standard Gaussian distribution.

Arun K. Tangirala, IIT Madras

Intro to Statistical Hypothesis Testing

6

Power of Hypothesis Tests

References

Computing probability of Type II error Introduce the standardized variable, Z1 =

¯ X

. . . contd.

µ p 1 ⇠ N (0, 1). / n

p

Then, in terms of this newly introduced standardized variable, Zo = Z1 +

n

.

Therefore, from (1), we have

= Pr

✓

Z1 +

p

n

< z↵/2

◆

= Pr

✓✓

z↵/2

p ◆ n

< Z1 <

✓

z↵/2

p ◆◆ n

(4)

where the probability is computed using a standard Gaussian distribution. For a given ,

and sample size n, the power of hypothesis test is computed as Power = 1

Arun K. Tangirala, IIT Madras

Intro to Statistical Hypothesis Testing

(5) 7

Power of Hypothesis Tests

References

Type II errors Two-tailed test: =F

✓

p ◆ n

z↵/2

F

✓

z↵/2

p ◆ n

(6)

Lower-tailed test: =1

F

✓

z↵

p ◆ n

(7)

Upper-tailed test: =F

Arun K. Tangirala, IIT Madras

✓

z↵

p ◆ n

Intro to Statistical Hypothesis Testing

(8)

8

Power of Hypothesis Tests

References

Example

Propellant burning rate Consider the motivational and worked out example of the average propellant burning rate. Suppose that the true burning rate is µ = 49 cm/s. What is the Type II error for the two-sided test with ↵ = 0.05, = 2 and sample size n = 25? Solution: The diﬀerence between postulated and truth is = 1 and z↵/2 = 1.96. Therefore, from (4), we have ✓ ✓ p ◆ p ◆ n n = F z↵/2 F z↵/2 = F (4.46) F (0.54) = 0.295 Thus, there is a 30% chance that the test will fail to reject H0 , i.e., the truth will be undetected. Observe that the answer would be the same for = 1. The power of the test with this sample size is therefore 1 = 0.7, which is satisfactory.

Arun K. Tangirala, IIT Madras

Intro to Statistical Hypothesis Testing

9

Power of Hypothesis Tests

References

Example

Propellant burning rate Consider the motivational and worked out example of the average propellant burning rate. Suppose that the true burning rate is µ = 49 cm/s. What is the Type II error for the two-sided test with ↵ = 0.05, = 2 and sample size n = 25? Solution: The diﬀerence between postulated and truth is = 1 and z↵/2 = 1.96. Therefore, from (4), we have ✓ ✓ p ◆ p ◆ n n = F z↵/2 F z↵/2 = F (4.46) F (0.54) = 0.295 Thus, there is a 30% chance that the test will fail to reject H0 , i.e., the truth will be undetected. Observe that the answer would be the same for = 1. The power of the test with this sample size is therefore 1 = 0.7, which is satisfactory. How can we improve the power of a hypothesis test? Arun K. Tangirala, IIT Madras

Intro to Statistical Hypothesis Testing

10