Hypothesis Tests Exercise 1 (#6.2). Let X be a sample from a population P and consider testing hypotheses H0 : P = P0 versus H1 : P = P1 , where Pj is a known population with probability density fj with respect to a σ-ﬁnite measure ν, j = 0, 1. Let β(P ) be the power function of a UMP (uniformly most powerful) test of size α ∈ (0, 1). Show that α < β(P1 ) unless P0 = P1 . Solution. Suppose that α = β(P1 ). Then the test T0 ≡ α is also a UMP test by deﬁnition. By the uniqueness of the UMP test (e.g., Theorem 6.1(ii) in Shao, 2003), we must have f1 (x) = cf0 (x) a.e. ν, which implies c = 1. Therefore, f1 (x) = f0 (x) a.e. ν, i.e., P0 = P1 . Exercise 2 (#6.3). Let X be a sample from a population P and consider testing hypotheses H0 : P = P0 versus H1 : P = P1 , where Pj is a known population with probability density fj with respect to a σ-ﬁnite measure ν, j = 0, 1. For any α > 0, deﬁne ⎧ f1 (X) > c(α)f0 (X) ⎨ 1 Tα (X) = γ(α) f1 (X) = c(α)f0 (X) ⎩ 0 f1 (X) < c(α)f0 (X), where 0 ≤ γ(α) ≤ 1, c(α) ≥ 0, E0 [Tα (X)] = α, and Ej denotes the expectation with respect to Pj . Show that (i) if α1 < α2 , then c(α1 ) ≥ c(α2 ); (ii) if α1 < α2 , then the type II error probability of Tα1 is larger than that of Tα2 , i.e., E1 [1 − Tα1 (X)] > E1 [1 − Tα2 (X)]. Solution. (i) Assume α1 < α2 . Suppose that c(α1 ) < c(α2 ). Then f1 (x) ≥ c(α2 )f0 (x) implies that f1 (x) > c(α1 )f0 (x) unless f1 (x) = f0 (x) = 0. Thus, Tα1 (x) ≥ Tα2 (x) a.e. ν, which implies that E0 [Tα1 (X)] ≥ E0 [Tα2 (X)]. Then α1 ≥ α2 . This contradiction proves that c(α1 ) ≥ c(α2 ). (ii) Assume α1 < α2 . Since Tα1 is of level α2 and Tα2 is UMP, E1 [Tα1 (X)] ≤ E1 [Tα2 (X)]. The result follows if we can show that the equality can not 251

252

Chapter 6. Hypothesis Tests

hold. If E1 [Tα1 (X)] = E1 [Tα2 (X)], then Tα1 is also UMP. By the uniqueness of the UMP and the fact that c(α1 ) ≥ c(α2 ) (part (i)), Pj c(α2 )f0 (X) ≤ f1 (X) ≤ c(α1 )f0 (X) = 0, j = 0, 1. This implies that E0 [Tα2 (X)] = 0 < α2 . Thus, E1 [Tα1 (X)] < E1 [Tα2 (X)], i.e., E1 [1 − Tα1 (X)] > E1 [1 − Tα2 (X)]. Exercise 3 (#6.4). Let X be a sample from a population P and P0 and P1 be two known populations. Suppose that T∗ is a UMP test of size α ∈ (0, 1) for testing H0 : P = P0 versus H1 : P = P1 and that β < 1, where β is the power of T∗ when H1 is true. Show that 1 − T∗ is a UMP test of size 1 − β for testing H0 : P = P1 versus H1 : P = P0 . Solution. Let fj be a probability density for Pj , j = 0, 1. By the uniqueness of the UMP test, 1 f1 (X) > cf0 (X) T∗ (X) = 0 f1 (X) < cf0 (X). Since α ∈ (0, 1) and β < 1, c must be a positive constant. Note that 1 − T∗ (X) =

1 0

f0 (X) > c−1 f1 (X) f0 (X) < c−1 f1 (X).

For testing H0 : P = P1 versus H1 : P = P0 , clearly 1 − T∗ has size 1 − β. The fact that it is UMP follows from the Neyman-Pearson Lemma. Exercise 4 (#6.6). Let (X1 , ..., Xn ) be a random sample from a population on R with Lebesgue density fθ . Let θ0 and θ1 be two constants. Find a UMP test of size α for testing H0 : θ = θ0 versus H1 : θ = θ1 in the following cases: (i) fθ (x) = e−(x−θ) I(θ,∞) (x), θ0 < θ1 ; (ii) fθ (x) = θx−2 I(θ,∞) (x), θ0 = θ1 . Solution. (i) Let X(1) be the smallest order statistic. Since fθ1 (X) = fθ0 (X)

en(θ1 −θ0 ) 0

X(1) > θ1 θ0 < X(1) ≤ θ1 ,

the UMP test is either T1 = or

T2 =

1 γ

X(1) > θ1 θ0 < X(1) ≤ θ1

γ 0

X(1) > θ1 θ0 < X(1) ≤ θ1 .

Chapter 6. Hypothesis Tests

253

When θ = θ0 , P (X(1) > θ1 ) = en(θ0 −θ1 ) . If en(θ0 −θ1 ) ≤ α, then T1 is the UMP test since, under θ = θ0 , E(T1 ) = P (X(1) > θ1 ) + γP (θ0 < X(1) ≤ θ1 ) = en(θ0 −θ1 ) + γ(1 − en(θ0 −θ1 ) ) =α with γ = (α − en(θ0 −θ1 ) )/(1 − en(θ0 −θ1 ) ). If en(θ0 −θ1 ) > α, then T2 is the UMP test since, under θ = θ0 , E(T2 ) = γP (X(1) > θ1 ) = γen(θ0 −θ1 ) = α with γ = α/en(θ0 −θ1 ) . (ii) Suppose θ1 > θ0 . Then fθ1 (X) = fθ0 (X) The UMP test is either

T1 = or

T2 =

θ1n θ0n

X(1) > θ1

0

θ0 < X(1) ≤ θ1 .

1 γ

X(1) > θ1 θ0 < X(1) ≤ θ1

γ 0

X(1) > θ1 θ0 < X(1) ≤ θ1 .

When θ = θ0 , P (X(1) > θ1 ) = θ0n /θ1n . If θ0n /θ1n ≤ α, then T1 is the UMP test since, under θ = θ0 ,

θn θn E(T1 ) = 0n + γ 1 − 0n = α θ1 θ1 with γ = (α − under θ = θ0 ,

θ0n θ1n )/(1

−

θ0n θ1n ).

If θ0n /θ1n > α, then T2 is the UMP test since,

E(T2 ) = γ with γ = αθ1n /θ0n . Suppose now that θ1 < θ0 . Then θn 1 fθ1 (X) θ0n = fθ0 (X) ∞ The UMP test is either T1 =

0 γ

θ0n =α θ1n

X(1) > θ0 θ1 < X(1) ≤ θ0 .

X(1) > θ0 θ1 < X(1) ≤ θ0

254

Chapter 6. Hypothesis Tests

or

T2 =

γ 1

X(1) > θ0 θ1 < X(1) ≤ θ0 .

When θ = θ0 , E(T1 ) = 0 and E(T2 ) = γ. Hence, the UMP test is T2 with γ = α. Exercise 5 (#6.7). Let f1 , ..., fm+1 be Borel functions on Rp that are integrable with respect to a σ-ﬁnite measure ν. For given constants t1 , ..., tm , let T be the class of Borel functions φ (from Rp to [0, 1]) satisfying φfi dν ≤ ti , i = 1, ..., m, and T0 be the set of φ’s in T satisfying φfi dν = ti , i = 1, ..., m. Show that if there are constants c1 , ..., cm such that 1 fm+1 (x) > c1 f1 (x) + · · · + cm fm (x) φ∗ (x) = 0 fm+1 (x) < c1 f1 (x) + · · · + cm fm (x) is a member of T0 , then φ∗ maximizes φfm+1 dν over φ ∈ T0 . Show that if ci ≥ 0 for all i, then φ∗ maximizes φfm+1 dν over φ ∈ T . Solution. Suppose that φ∗ ∈ T0 . By the deﬁnition of φ∗ , for any other φ ∈ T0 , (φ∗ − φ)(fm+1 − c1 f1 − · · · − cm fm ) ≥ 0. Therefore

(φ∗ − φ)(fm+1 − c1 f1 − · · · − cm fm )dν ≥ 0,

i.e.,

(φ∗ − φ)fm+1 dν ≥

m

(φ∗ − φ)fi dν = 0.

ci

i=1

Hence φ∗ maximizes φfm+1 dν over φ ∈ T0 . If ci ≥ 0, for φ ∈ T , we still have (φ∗ − φ)(fm+1 − c1 f1 − · · · − cm fm ) ≥ 0 and, thus, (φ∗ − φ)fm+1 dν ≥

n i=1

ci

(φ∗ − φ)fi dν ≥ 0,

because ci (φ∗ −φ)fi dν ≥ 0 for each i. Therefore φ∗ maximizes φfm+1 dν over φ ∈ T .

Chapter 6. Hypothesis Tests

255

Exercise 6 (#6.9). Let f0 and f1 be Lebesgue integrable functions on R and 1 f0 (x) < 0 or f0 (x) = 0, f1 (x) ≥ 0 φ∗ (x) = 0 otherwise. Show that φ∗ maximizes φ(x)f1 (x)dx over all Borel functions φ on R satisfying 0 ≤ φ(x) ≤ 1 and φ(x)f0 (x)dx = φ∗ (x)f0 (x)dx. Solution. From the deﬁnition of φ∗ , φ∗ (x)f0 (x)dx = {f0 (x)<0} f0 (x)dx. Since 0 ≤ φ(x) ≤ 1 and φ(x)f0 (x)dx = φ∗ (x)f0 (x)dx, φ(x)f0 (x)dx 0≤ {f0 (x)>0} = φ(x)f0 (x)dx − φ(x)f0 (x)dx {f0 (x)<0} = φ∗ (x)f0 (x)dx − φ(x)f0 (x)dx {f0 (x)<0} = f0 (x)dx − φ(x)f0 (x)dx {f0 (x)<0} {f0 (x)<0} = [1 − φ(x)]f0 (x)dx {f0 (x)<0}

≤ 0. That is,

{f0 (x)>0}

φ(x)f0 (x)dx =

{f0 (x)<0}

[1 − φ(x)]f0 (x)dx = 0.

Hence, φ(x) = 0 a.e. on the set {f0 (x) > 0} and φ(x) = 1 a.e. on the set {f0 (x) < 0}. Then, the result follows from [φ∗ (x) − φ(x)]f1 (x)dx = [1 − φ(x)]f1 (x)dx {f0 (x)<0} − φ(x)f1 (x)dx {f0 (x)>0} + [1 − φ(x)]f1 (x)dx {f0 (x)=0,f1 (x)≥0} − φ(x)f1 (x)dx {f0 (x)=0,f1 (x)<0} = [1 − φ(x)]f1 (x)dx {f0 (x)=0,f1 (x)≥0} − φ(x)f1 (x)dx ≥ 0.

{f0 (x)=0,f1 (x)<0}

256

Chapter 6. Hypothesis Tests

Exercise 7 (#6.10). Let F1 and F2 be two cumulative distribution func (x) ≤ F (x) for all x if and only if g(x)dF (x) ≤ tions on R. Show that F 1 2 2 function g. g(x)dF1 (x) for any nondecreasing Solution. If g(x)dF2 (x) ≤ g(x)dF1 (x) for any nondecreasing function g, then 1 − F2 (y) = I(y,∞) (x)dF2 (x) ≤ I(y,∞) (x)dF1 (x) = 1 − F1 (y) for any y, since I(y,∞) (x) is nondecreasing. Assume now that F1 (x) ≤ F2 (x) for all x. Then, for any t ∈ R, {x : F1 (x) ≥ t} ⊂ {x : F2 (x) ≥ t} and, hence, F1−1 (t) = inf{x : F1 (x) ≥ t} ≥ inf{x : F2 (x) ≥ t} = F2−1 (t) for any t. Let U be a random variable having the uniform distribution on (0, 1). Then Fj−1 (U ) has distribution Fj , j = 1, 2. If g is nondecreasing, then g(F1−1 (U )) ≥ g(F2−1 (U )) and, therefore, −1 −1 g(x)dF1 (x) = E[g(F1 (U ))] ≥ E[g(F2 (U ))] = g(x)dF2 (x). Exercise 8 (#6.11). Let X be an observation with a probability density in the family P = {fθ : θ ∈ Θ}, where Θ ⊂ R is the possible values of the parameter θ. (i) Show that P has monotone likelihood ratio in X when Θ = R and fθ is the Lebesgue density of the double exponential distribution with location parameter θ and a known scale parameter c. (ii) Show that P has monotone likelihood ratio in X when Θ = R and fθ is the Lebesgue density of the exponential distribution on the interval (θ, ∞) with a known scale parameter c. (iii) Show that P has monotone likelihood ratio in X when Θ = R and fθ is the Lebesgue density of the logistic distribution with location parameter θ and a known scale parameter c. (iv) Show that P has monotone likelihood ratio in X when Θ = R and fθ is the Lebesgue density of the uniform distribution on (θ, θ + 1). (v) Showthat has monotone likelihood ratio in X when Θ = {1, 2, ...} and P N −θ fθ (x) = xθ N r−x / r when x is an integer between r − θ and min{r, θ}, where r and N are known integers. (vi) Show that P does not have monotone likelihood ratio in X when Θ = R and fθ is the Lebesgue density of the Cauchy distribution with location parameter θ and a known scale parameter c. Solution. (i) We need to show fθ2 (x)/fθ1 (x) is nondecreasing in x for any

Chapter 6. Hypothesis Tests

257

θ1 < θ2 with at least one of fθi (x) is positive. For θ1 < θ2 , ⎧ −(θ2 −θ1 )/c ⎨ e fθ2 (x) −(|θ2 −x|−|θ1 −x|)/c = =e e−(θ2 +θ1 −2x)/c ⎩ (θ2 −θ1 )/c fθ1 (x) e

x ≤ θ1 θ1 < x ≤ θ2 x > θ2 ,

which is a nondecreasing function of x. (ii) For θ1 < θ2 , fθ2 (x) = fθ1 (x)

0 e(θ2 −θ1 )/c

θ1 < x ≤ θ2 x > θ2 ,

which is a nondecreasing function of x. (iii) For θ1 < θ2 , fθ2 (x) = e(θ1 −θ2 )/c fθ1 (x) Since d dx

1 + e(x−θ1 )/c 1 + e(x−θ2 )/c

=

1 + e(x−θ1 )/c 1 + e(x−θ2 )/c

2 .

e(x−θ1 )/c − e(x−θ2 )/c >0 c(1 + e(x−θ2 )/c )2

when θ1 < θ2 , the ratio fθ2 (x)/fθ1 (x) is increasing in x. (iv) For θ1 < θ2 < θ1 + 1, ⎧ 0 fθ2 (x) ⎨ = 1 fθ1 (x) ⎩ ∞

θ1 < x ≤ θ2 θ2 < x < θ1 + 1 θ1 + 1 ≤ x < θ2 + 1.

For θ1 + 1 ≤ θ2 , fθ2 (x) = fθ1 (x)

0 ∞

θ1 < x < θ1 + 1 θ2 ≤ x < θ2 + 1.

In any case, the ratio fθ2 (x)/fθ1 (x) is nondecreasing in x. (v) Note that θ N −θ N / r fθ (x) θ(N − θ − r + x + 1) = θ−1xNr−x N = −θ+1 fθ−1 (x) (θ − x)(N − θ + 1) / r x r−x is an increasing function of x. Hence, for θ1 < θ2 , fθ +1 (x) fθ1 +2 (x) fθ2 (x) fθ2 (x) = 1 ··· fθ1 (x) fθ1 (x) fθ1 +1 (x) fθ2 −1 (x)

258

Chapter 6. Hypothesis Tests

is a product of increasing functions in x and, hence, it is increasing in x. (vi) For θ1 < θ2 , fθ2 (x) c2 + (x − θ1 )2 , = 2 fθ1 (x) c + (x − θ2 )2 which converges to 1 when x → ±∞, is smaller than 1 when x = θ1 , and is larger than 1 when x = θ2 . Hence, the ratio fθ2 (x)/fθ1 (x) is not monotone in x. Exercise 9. Let Θ ⊂ R and P = {fθ (x) : θ ∈ Θ} be a family of functions on X ⊂ R satisfying fθ (x) > 0 for all θ ∈ Θ and x ∈ X . Assume that ∂2 ∂θ∂x log fθ (x) exists. (i) Show that P has monotone likelihood ratio in x is equivalent to one of the following conditions: ∂2 log fθ (x) ≥ 0 for all x and θ; (a) ∂θ∂x ∂2 ∂ ∂ fθ (x) ≥ ∂θ fθ (x) ∂x fθ (x) for all x and θ. (b) fθ (x) ∂θ∂x (ii) Let fθ (x) be the Lebesgue density of the noncentral chi-square distribution χ21 (θ) with the noncentrality parameter θ ≥ 0. Show that the family P = {fθ (x) : θ ≥ 0} has monotone likelihood ratio in x. Solution. (i) Note that ∂ fθ (x) ∂2 ∂ ∂x log fθ (x) = = ∂θ∂x ∂θ fθ (x)

∂2 ∂θ∂x fθ (x)

fθ (x)

−

∂ ∂ ∂x fθ (x) ∂θ fθ (x) . [fθ (x)]2

Since fθ (x) > 0, conditions (a) and (b) are equivalent. Condition (a) is equivalent to ∂ fθ (x) ∂ log fθ (x) = ∂x ∂x fθ (x)

is nondecreasing in θ for any ﬁxed x. Hence, it is equivalent to, for θ1 < θ2 and any x, ∂ ∂ fθ (x) ∂x fθ1 (x) ≤ ∂x 2 , fθ1 (x) fθ2 (x) which is equivalent to, for θ1 < θ2 and any x, ∂ ∂ fθ2 (x) − fθ2 (x) ∂x fθ1 (x) fθ (x) ∂x ∂ fθ2 (x) ≥ 0, = 1 2 ∂x fθ1 (x) [fθ1 (x)]

i.e., P has monotone likelihood ratio in x. √ (ii) Let Z be a random variable having distribution N ( θ, 1). By deﬁnition, Z 2 has the noncentral chi-square distribution χ21 (θ). Hence, + * √ √ 2 2 1 fθ (x) = √ e−( x−θ) /2 + e−( x+θ) /2 . 2 2πx

Chapter 6. Hypothesis Tests

259

For 0 ≤ θ1 < θ2 , √

2

2

√

√

2

e−( x−θ2 ) /2 + e−( x+θ2 ) /2 fθ2 (x) √ = −(√x−θ )2 /2 1 fθ1 (x) e + e−( x+θ1 )2 /2 √

e−θ2 /2 (eθ2 x + e−θ2 x ) = −θ2 /2 √ √ . e 1 (eθ1 x + e−θ1 x ) Hence, we may apply the result in (i) to functions gθ (y) = eθy + e−θy . Note that ∂ gθ (y) = θ(eθy − e−θy ), ∂y ∂ gθ (y) = y(eθy − e−θy ), ∂θ and

∂2 gθ (y) = θy(eθy + e−θy ). ∂θ∂y

Hence, gθ (y)

∂2 gθ (y) = θy(eθy + e−θy )2 ∂θ∂y ≥ θy(eθy − e−θy )2 ∂ ∂ gθ (y) gθ (y), = ∂y ∂θ

i.e., condition √ (b) in (i) holds. Hence P has monotone likelihood ratio in y. Since y = x is an increasing function of x, P also has monotone likelihood ratio in x. Exercise 10 (#6.14). Let X = (X1 , ..., Xn ) be a random sample from a distribution on R with Lebesgue density fθ , θ ∈ Θ = (0, ∞). Let θ0 be a positive constant. Find a UMP test of size α for testing H0 : θ ≤ θ0 versus H1 : θ > θ0 when (i) fθ (x) = θ−1 e−x/θ I(0,∞) (x); (ii) fθ (x) = θ−1 xθ−1 I(0,1) (x); (iii) fθ (x) is the density of N (1, θ); c (iv) fθ (x) = θ−c cxc−1 e−(x/θ) I(0,∞) (x), where c > 0 is known. Solution. n (i) The family of densities has monotone likelihood ratio in T (X) = i=1 Xi , which has the gamma distribution with shape parameter n and scale parameter θ. Under H0 , 2T /θ0 has the chi-square distribution χ22n . Hence, the UMP test is 1 T (X) > θ0 χ22n,α /2 T∗ (X) = 0 T (X) ≤ θ0 χ22n,α /2,

260

Chapter 6. Hypothesis Tests

where χ2r,α is the (1 − α)th quantile of the chi-square distribution χ2r . (ii) n The family of densities has monotone likelihood ratio in T (X) = i=1 log Xi , which has the gamma distribution with shape parameter n and scale parameter θ−1 . Therefore, the UMP test is the same as T∗ in part (i) of the solution but with θ0 replaced by θ0−1 . (iii) of densities has monotone likelihood ratio in T (X) = n The family 2 (X − 1) and T (X)/θ has the chi-square distribution χ2n . Therefore, i i=1 the UMP test is 1 T (X) > θ0 χ2n,α T∗ (X) = 0 T (X) ≤ θ0 χ2n,α . (iv) n Thec family of densities has monotone likelihood ratio in T (X) = i=1 Xi , which has the Gamma distribution with shape parameter n and scale parameter θc . Therefore, the UMP test is the same as T∗ in part (i) of the solution but with θ0 replaced by θ0c . Exercise 11 (#6.15). Suppose that the distribution of X is in a family {fθ : θ ∈ Θ} with monotone likelihood ratio in Y (X), where Y (X) has a continuous distribution. Consider the hypotheses H0 : θ ≤ θ0 versus H1 : θ > θ0 , where θ0 ∈ Θ is known. Show that the p-value of the UMP test is given by Pθ0 (Y ≥ y), where y is the observed value of Y and Pθ is the probability corresponding to fθ . Solution. The UMP test of size α is 1 Y ≥ cα Tα = 0 Y < cα , where cα satisﬁes Pθ0 (Y ≥ cα ) = α. When y is the observed value of Y , the rejection region of the UMP test is {y ≥ cα }. By the deﬁnition of p-value, it is equal to α ˆ = inf{α : 0 < α < 1, Tα = 1} = inf{α : 0 < α < 1, y ≥ cα } = inf Pθ0 (Y ≥ cα ) y≥cα

≥ Pθ0 (Y ≥ y), where the inequality follows from Pθ0 (Y ≥ y) ≤ Pθ0 (Y ≥ cα ) for any α such that y ≥ cα . Let Fθ be the cumulative distribution function of Pθ . Since Fθ0 is continuous, cα = Fθ−1 (1 − α). Let α∗ = Pθ0 (Y ≥ y) = 1 − Fθ0 (y). 0 Since Fθ0 is continuous, cα∗ = Fθ−1 (1 − α∗ ) = Fθ−1 (Fθ0 (y)) ≤ y. 0 0 This means that α∗ ∈ {α : 0 < α < 1, y ≥ cα } and, thus, the p-value α ˆ ≤ α∗ . Therefore, the p-value is equal to α∗ = Pθ0 (Y ≥ y).

Chapter 6. Hypothesis Tests

261

Exercise 12 (#6.17). Let F and G be two known cumulative distribution functions on R and X be a single observation from the cumulative distribution function θF (x) + (1 − θ)G(x), where θ ∈ [0, 1] is unknown. (i) Find a UMP test of size α for testing H0 : θ ≤ θ0 versus H1 : θ > θ0 , where θ0 ∈ [0, 1] is known. (ii) Show that the test T∗ (X) ≡ α is a UMP test of size α for testing H0 : θ ≤ θ1 or θ ≥ θ2 versus H1 : θ1 < θ < θ2 , where θj ∈ [0, 1] is known, j = 1, 2, and θ1 < θ2 . Solution. (i) Let f (x) and g(x) be the Randon-Nikodym derivatives of F (x) and G(x) with respect to the measure ν induced by F (x) + G(x), respectively. The probability density of X is θf (x) + (1 − θ)g(x). For 0 ≤ θ1 < θ2 ≤ 1, (x) θ2 fg(x) + (1 − θ2 ) θ2 f (x) + (1 − θ2 )g(x) = f (x) θ1 f (x) + (1 − θ1 )g(x) θ1 g(x) + (1 − θ1 )

is nondecreasing in Y (x) = f (x)/g(x). Hence, the family of densities of X has monotone likelihood ratio in Y (X) = f (X)/g(X) and a UMP test is given as ⎧ Y (X) > c ⎨ 1 T = γ Y (X) = c ⎩ 0 Y (X) < c, where c and γ are uniquely determined by E[T (X)] = α when θ = θ0 . (ii) For any test T , its power is βT (θ) = T (x)[θf (x) + (1 − θ)g(x)]dν = θ T (x)[f (x) − g(x)]dν + T (x)g(x)du, which is a linear function of θ on [0, 1]. If T has level α, then βT (θ) ≤ α for any θ ∈ [0, 1]. Since the power of T∗ is equal to the constant α, we conclude that T∗ is a UMP test of size α. Exercise 13 (#6.18). Let (X1 , ..., Xn ) be a random sample from the uniform distribution on (θ, θ + 1), θ ∈ R. Suppose that n ≥ 2. (i) Show that a UMP test of size α ∈ (0, 1) for testing H0 : θ ≤ 0 versus H1 : θ > 0 is of the form 0 X(1) < 1 − α1/n , X(n) < 1 T∗ (X(1) , X(n) ) = 1 otherwise, where X(j) is the jth order statistic. (ii) Does the family of all densities of (X(1) , X(n) ) have monotone likelihood

262

Chapter 6. Hypothesis Tests

ratio? Solution A. (i) The Lebesgue density of (X(1) , X(n) ) is fθ (x, y) = n(n − 1)(y − x)n−2 I(θ,y) (x)I(x,θ+1) (y). A direct calculation of βT∗ (θ) = T∗ (x, y)fθ (x, y)dxdy, the power function of T∗ , leads to ⎧ 0 θ < −α1/n ⎪ ⎪ ⎨ 1/n n (θ + α ) −α1/n ≤ θ ≤ 0 βT∗ (θ) = ⎪ 1 + α − (1 − θ)n 0 < θ ≤ 1 − α1/n ⎪ ⎩ 1 θ > 1 − α1/n . For any θ1 ∈ (0, 1 − α1/n ], by the Neyman-Pearson Lemma, the UMP test T of size α for testing H0 : θ = 0 versus H1 : θ = θ1 is ⎧ X(n) > 1 ⎨ 1 T = α/(1 − θ1 )n θ1 < X(1) < X(n) < 1 ⎩ 0 otherwise. The power of T at θ1 is computed as βT (θ1 ) = 1 − (1 − θ1 )n + α, which agrees with the power of T∗ at θ1 . When θ > 1 − α1/n , T∗ has power 1. Therefore T∗ is a UMP test of size α for testing H0 : θ ≤ 0 versus H1 : θ > 0. (ii) The answer is no. Suppose that the family of densities of (X(1) , X(n) ) has monotone likelihood ratio. By the theory of UMP test (e.g., Theorem 6.2 in Shao, 2003), there exists a UMP test T0 of size α ∈ (0, 12 ) for testing H0 : θ ≤ 0 versus H1 : θ > 0 and T0 has the property that, for θ1 ∈ (0, 1−α1/n ), T0 is UMP of size α0 = 1+α−(1−θ1 )n for testing H0 : θ ≤ θ1 versus H1 : θ > θ1 . Using the transformation Xi − θ1 and the result in (i), the test 1/n 0 X(1) < 1 + θ1 − α0 , X(n) < 1 + θ1 Tθ1 (X(1) , X(n) ) = 1 otherwise is a UMP test of size α0 for testing H0 : θ ≤ θ1 versus H1 : θ > θ1 . At θ = θ2 ∈ (θ1 , 1−α1/n ], it follows from part (i) of the solution that the power of T0 is 1+α−(1−θ2 )n and the power of Tθ1 is 1+α0 −[1−(θ2 −θ1 )]n . Since both T0 and Tθ1 are UMP tests, 1 + α − (1 − θ2 )n = 1 + α0 − [1 − (θ2 − θ1 )]n . Because α0 = 1 + α − (1 − θ1 )n , this means that 1 = (1 − θ1 )n − (1 − θ2 )n + [1 − (θ2 − θ1 )]n

Chapter 6. Hypothesis Tests

263

holds for all 0 < θ1 < θ2 ≤ 1−α1/n , which is impossible. This contradiction proves that the family of all densities of (X(1) , X(n) ) does not have monotone likelihood ratio. Solution B. This is an alternative solution to part (i) provided by Mr. Jialiang Li in 2002 as a student at the University of Wisconsin-Madison. Let βT (θ) be the power function of a test T . Since βT∗ (θ) = 1 when θ > 1−α1/n , it suﬃces to show that βT∗ (θ) ≥ βT (θ) for θ ∈ (0, 1 − α1/n ) and any other test T . Deﬁne A = {0 < X(1) ≤ X(n) < 1}, B = {θ < X(1) ≤ X(n) < 1}, and C = {1 < X(1) ≤ X(n) < θ + 1}. Then βT∗ (θ) − βT (θ) = E[(T∗ − T )IB ] + E[(T∗ − T )IC ] = E[(T∗ − T )IB ] + E[(1 − T )IC ] ≥ E(T∗ IB ) − E(T IB ) = E(T∗ IA ) − E(T IB ) ≥ E(T∗ IA ) − E(T IA ) = βT∗ (0) − βT (0) = α − βT (0), where the second equality follows from T∗ = 1 when (X(1) , X(n) ) ∈ C, the third equality follows from T∗ = 0 when (X(1) , X(n) ) ∈ A but (X(1) , X(n) ) ∈ B (since 0 < X(1) ≤ θ ≤ 1 − α1/n ), and the second inequality follows from IA ≥ IB . Therefore, if T has level α, then βT∗ (θ) ≥ βT (θ) for all θ > 0. Exercise 14 (#6.19). Let X = (X1 , ..., Xn ) be a random sample from the discrete uniform distribution on points 1, ..., θ, where θ = 1, 2, .... (i) Consider H0 : θ ≤ θ0 versus H1 : θ > θ0 , where θ0 > 0 is known. Show that 1 X(n) > θ0 T∗ (X) = α X(n) ≤ θ0 is a UMP test of size α. (ii) Consider H0 : θ = θ0 versus H1 : θ = θ0 . Show that 1 X(n) > θ0 or X(n) ≤ θ0 α1/n T∗ (X) = 0 otherwise is a UMP test of size α. (iii) Show that the results in (i) and (ii) still hold if the discrete uniform distribution is replaced by the uniform distribution on the interval (0, θ), θ > 0. Solution A. In (i)-(ii), without loss of generality we may assume that θ0 is an integer. (i) Let Pθ be the probability distribution of the largest order statistic X(n) and Eθ be the expectation with respect to Pθ . The family {Pθ : θ = 1, 2, ...}

264

Chapter 6. Hypothesis Tests

is dominated by the counting measure and has monotone likelihood ratio in X(n) . Therefore, a UMP test of size α is ⎧ ⎨ 1 T1 (X) = γ ⎩ 0

X(n) > c X(n) = c X(n) < c,

where c is an integer and γ ∈ [0, 1] satisfying

n c cn − (c − 1)n Eθ0 (T1 ) = 1 − +γ = α. θ0 θ0n For any θ > θ0 , the power of T1 is Eθ (T1 ) = Pθ (X(n) > c) + γPθ (X(n) = c) cn − (c − 1)n cn + γ θn θn θ0n = 1 − (1 − α) n . θ = 1−

On the other hand, for θ ≥ θ0 , the power of T∗ is Eθ (T∗ ) = Pθ (X(n) > θ0 ) + αPθ (X(n) ≤ θ0 ) = 1 −

θ0n θn + α 0n . n θ θ

Hence, T∗ has the same power as T1 . Since sup Eθ (T∗ ) = sup αPθ (X(n) ≤ θ0 ) = αPθ0 (X(n) ≤ θ0 ) = α,

θ≤θ0

θ≤θ0

T∗ is a UMP test of size α. (ii) Consider H0 : θ = θ0 versus H1 : θ > θ0 . The test T1 in (i) is UMP. For θ > θ0 , Eθ (T∗ ) = Pθ (X(n) > θ0 ) + Pθ (X(n) ≤ θ0 α1/n ) = 1 −

θ0n αθ0n + , θn θn

which is the same as the power of T1 . Now, consider hypotheses H0 : θ = θ0 versus H1 : θ < θ0 . The UMP test is ⎧ ⎨ 1 X(n) < d T2 (X) = η X(n) = d ⎩ 0 X(n) > d with Eθ0 (T2 ) =

(d − 1)n cn − (c − 1)n +η = α. n θ0 θ0n

Chapter 6. Hypothesis Tests

265

For θ ≤ θ0 , Eθ (T∗ ) = Pθ (X(n) > θ0 ) + Pθ (X(n) ≤ θ0 α1/n ) = Pθ (X(n) ≤ θ0 α1/n ) αθ0n = min 1, n . θ On the other hand, the power of T2 when θ ≥ θ0 α1/n is Eθ (T2 ) = Pθ (X(n) < d) + ηPθ (X(n) = d) dn − (d − 1)n (d − 1)n + η θn θn θ0n = α n. θ =

Thus, we conclude that T∗ has size α and its power is the same as the power of T1 when θ > θ0 and is no smaller than the power of T2 when θ < θ0 . Thus, T∗ is UMP. (iii) The results for the uniform distribution on (0, θ) can be established similarly. Instead of providing details, we consider an alternative solution for (i)-(iii). Solution B. (i) Let T be a test of level α. For θ > θ0 , Eθ (T∗ ) − Eθ (T ) = Eθ [(T∗ − T )I{X(n) >θ0 } ] + Eθ [(T∗ − T )I{X(n) ≤θ0 } ] = Eθ [(1 − T )I{X(n) >θ0 } ] + Eθ [(α − T )I{X(n) ≤θ0 } ] ≥ Eθ [(α − T )I{X(n) ≤θ0 } ] = Eθ0 [(α − T )I{X(n) ≤θ0 } ](θ0 /θ)n = Eθ0 (α − T )(θ0 /θ)n ≥ 0, where the second equality follows from the deﬁnition of T∗ and the third equality follows from a scale transformation. Hence, T∗ is UMP. It remains to show that the size of T∗ is α, which has been shown in part (i) of Solution A. (ii) Let T be a test of level α. For θ > θ0 , Eθ (T∗ ) − Eθ (T ) = Eθ [(1 − T )I{X(n) >θ0 } ] + Eθ [(T∗ − T )I{X(n) ≤θ0 } ] ≥ Eθ [(T∗ − T )I{X(n) ≤θ0 } ] = Pθ (X(n) ≤ θ0 α1/n ) − Eθ (T I{X(n) ≤θ0 } ) = α(θ0 /θ)n − Eθ0 (T I{X(n) ≤θ0 } )(θ0 /θ)n ≥ 0.

266

Chapter 6. Hypothesis Tests

Similarly, for θ < θ0 , Eθ (T∗ ) − Eθ (T ) ≥ Pθ (X(n) ≤ θ0 α1/n ) − Eθ (T ), which is equal to 1 − Eθ (T ) ≥ 0, if θ ≤ θ0 α1/n , and is equal to α(θ0 /θ)n − Eθ (T ) = α(θ0 /θ)n − Eθ0 (T )(θ0 /θ)n ≥ 0 if θ0 > θ > θ0 α1/n . Hence, T∗ is UMP. It remains to show that the size of T∗ is α, which has been shown in part (ii) of Solution A. (iii) Note that the results for the power hold for both discrete uniform distribution and uniform distribution on (0, θ). Hence, it remains to show that T∗ has size α. For T∗ in (i), sup Eθ (T∗ ) = sup αPθ (X(n) ≤ θ0 ) = αPθ0 (X(n) ≤ θ0 ) = α.

θ≤θ0

For T∗ in (ii),

θ≤θ0

Eθ0 (T∗ ) = Pθ0 (X(n) ≤ θ0 α1/n ) = α.

Exercise 15 (#6.20). Let (X1 , ..., Xn ) be a random sample from the exponential distribution on the interval (a, ∞) with scale parameter θ, where a ∈ R and θ > 0. (i) Derive a UMP test of size α for testing H0 : a = a0 versus H1 : a = a0 , when θ is known. (ii) For testing H0 : a = a0 versus H1 : a = a1 < a0 , show that any UMP test T∗ of size α has power βT∗ (a1 ) = 1 − (1 − α)e−n(a0 −a1 )/θ . (iii) For testing H0 : a = a0 versus H1 : a = a1 < a0 , show that the power of any size α test that rejects H0 when Y≤ c1 or Y ≥ c2 is the same as n that in part (ii), where Y = (X(1) − a0 )/ i=1 (Xi − X(1) ) and X(1) is the smallest order statistic and 0 ≤ c1 < c2 are constants. (iv) Derive a UMP test of size α for testing H0 : a = a0 versus H1 : a = a0 . (v) Derive a UMP test of size α for testing H0 : θ = θ0 , a = a0 versus H1 : θ < θ0 , a < a0 . Solution. (i) Let Yi = e−Xi /θ , i = 1, ..., n. Then (Y1 , ..., Yn ) is a random sample from the uniform distribution on (0, e−a/θ ). Note that the hypotheses H0 : a = a0 versus H1 : a = a0 are the same as H0 : e−a/θ = e−a0 /θ versus H1 : e−a/θ = e−a0 /θ . Also, the largest order statistic of Y1 , ..., Yn is equal to e−X(1) /θ . Hence, it follows from the previous exercise that a UMP test of size α is 1 X(1) < a0 or X(1) ≥ a0 − nθ log α T = 0 otherwise. (ii) A direct calculation shows that, at a1 < a0 , the power of the UMP test in part (i) of the solution is 1 − (1 − α)e−n(a0 −a1 )/θ . Hence, for each ﬁxed

Chapter 6. Hypothesis Tests

267

θ, the power of T∗ at a1 can not be larger than 1 − (1 − α)e−n(a0 −a1 )/θ . On the other hand, in part (iii) it is shown that there are tests for testing H0 : a = a0 versus H1 : a = a1 < a0 that have power 1−(1−α)e−n(a0 −a1 )/θ at a1 < a0 . Therefore, the power of T∗ at a1 can not be smaller than 1 − (1 − α)e−n(a0 −a1 )/θ . (iii) Let 1 Y ≤ c1 or Y ≥ c2 T = 0 otherwise be a test of size α for testing H0 : a = a0 versus H1 : a = a1 < a0 . n Let Z = i=1 (Xi − X(1) ). By Exercise 27 in Chapter 2, Z and X(1) are independent. Then, the power of T at a1 is E(T ) = 1 − P (c1 < Y < c2 ) = 1 − P (a0 + c1 Z < X(1) < a0 + c2 Z) a0 +c2 Z n −n(x−a1 )/θ e dx = 1− E θ a0 +c1 Z = 1 − E e−n(a0 −a1 +c1 Z)/θ − e−n(a0 −a1 +c2 Z)/θ = 1 − e−n(a0 −a1 )/θ E e−nc1 Z/θ − e−nc2 Z/θ . Since 2Z/θ has the chi-square distribution χ22(n−1) (Exercise 7 in Chapter 2), b = E e−nc1 Z/θ − e−nc2 Z/θ does not depend on θ. Since T has size α, E(T ) at a = a0 , which is 1 − b, is equal to α. Thus, b = 1 − α and E(T ) = 1 − (1 − α)e−n(a0 −a1 )/θ . (iv) Consider the test T in (iii) with c1 = 0 and c2 = c > 0. From the result in (iii), T has size α and is UMP for testing H0 : a = a0 versus H1 : a < a0 . Hence, it remains to show that T is UMP for testing H0 : a = a0 versus H1 : a > a0 . Let a1 > a0 be ﬁxed and θ be ﬁxed. From the NeymanPearson lemma, a UMP test for H0 : a = a0 versus H1 : a = a1 has the rejection region " ea1 /θ I(a1 ,∞) (X(1) ) > c0 ea0 /θ I(a0 ,∞) (X(1) ) for some constant c0 . Since a1 > a0 , this rejection region is the same as {Y > c} for some constant c. Since the region {Y > c} does not depend on (a, θ), T is UMP for testing H0 : a = a0 versus H1 : a > a0 . (v) For ﬁxed θ1 < θ0 and a1 < a0 , by the Neyman-Pearson lemma, the UMP test of size α for H0 : a = a0 , θ = θ0 versus H1 : a = a1 , θ = θ1 has the rejection region n " θ0n e− i=1 (Xi −a1 )/θ1 I(a1 ,∞) (X(1) ) n R= > c0 θ1n e− i=1 (Xi −a0 )/θ0 I(a0 ,∞) (X(1) )

268

Chapter 6. Hypothesis Tests

for some c0 . The ratio in the previous expression is equal to ∞ when a < X(1) ≤ a0 and θ0n na1 /θ1 −na0 /θ0 (θ0−1 −θ1−1 ) n Xi i=1 e e e θ1n when X(1) > a0 . Since θ0−1 − θ1−1 < 0,

R = X(1) ≤ a0 ∪

n

" Xi < c

i=1

for some constant c satisfying P (R) = α when a = a0 and θ = θ0 . Hence, c depends on a0 and θ0 . Since this test does not depend on (a1 , θ1 ), it is UMP for testing H0 : θ = θ0 , a = a0 versus H1 : θ < θ0 , a < a0 . Exercise 16 (#6.22). In Exercise 11(i) in Chapter 3, derive a UMP test of size α ∈ (0, 1) for testing H0 : θ ≤ θ0 versus H1 : θ > θ0 , where θ0 > 1 is known. Solution. From Exercise 11(i) in Chapter 3, the probability density (with respect to the sum of Lebesgue measure and point mass at 1) of the suﬃcient and complete statistic X(n) , the largest order statistic, is fθ (x) = θ−n I{1} (x) + nθ−n xn−1 I(1,θ) (x). The family {fθ : θ > 1} has monotone likelihood ratio in X(n) . Hence, a UMP test of size α is ⎧ X(n) > c ⎨ 1 T = γ X(n) = c ⎩ 0 X(n) < c, where c and γ are determined by the size of T . When θ = θ0 and 1 < c ≤ θ0 , θ0 n cn E(T ) = P (X(n) > c) = n xn−1 dx = 1 − n . θ0 c θ0 If θ0 > (1 − α)−1/n , then T has size α with c = θ0 (1 − α)1/n and γ = 0. If θ0 > (1 − α)−1/n , then the size of T is P (X(n) > 1) + γP (X(n) = 1) = 1 −

1 γ + n. θ0n θ0

Hence, T has size α with c = 1 and γ = 1 − (1 − α)θ0n . Exercise 17 (#6.25). Let (X1 , ..., Xn ) be a random sample from N (θ, 1). ¯ is a UMP test of size α ∈ (0, 1 ) for testing Show that T = I(−c,c) (X) 2

Chapter 6. Hypothesis Tests

269

¯ is the sample mean and θ1 > 0 H0 : |θ| ≥ θ1 versus H1 : |θ| < θ1 , where X is a constant. Provide a formula for determining c. Solution. From Theorem 6.3 in Shao (2003), the UMP test is of the form ¯ where c1 and c2 satisfy T = I(c1 ,c2 ) (X), Φ Φ

√

√ n(c2 + θ1 ) − Φ n(c1 + θ1 ) = α,

√

√ n(c2 − θ1 ) − Φ n(c1 − θ1 ) = α,

and Φ is the cumulative distribution function of N (0, 1). Let Yi = −Xi , i = 1, ..., n. Then (Y1 , ..., Yn ) is a random sample from N (−θ, 1). Since the hypotheses are not changed with θ replaced by −θ, the UMP test for ¯ with testing the same hypotheses but based on Yi ’s is T1 = I(c1 ,c2 ) (−X) the same c1 and c2 . By the uniqueness of the UMP test, T = T1 and, thus, c1 = −c and c2 = c > 0. The constraints on ci ’s reduce to Φ

√

√ n(θ1 + c) − Φ n(θ1 − c) = α.

Exercise 18 (#6.29). Consider Exercise 12 with H0 : θ ∈ [θ1 , θ2 ] versus H1 : θ ∈ [θ1 , θ2 ], where 0 < θ1 ≤ θ2 < 1 are constants. (i) Show that a UMP test does not exist. (ii) Obtain a UMPU (uniformly most power unbiased) test of size α. Solution. (i) Let βT (θ) be the power function of a test T . For any test T of level α such that βT (θ) is not constant, either βT (0) or βT (1) is strictly less than α. Without loss of generality, assume that βT (0) < α. This means that at θ = 0, which is one of parameter values under H1 , the power of T is smaller than T∗ ≡ α. Hence, any T with nonconstant power function can not be UMP. From Exercise 12, the UMP test of size α for testing H0 : θ ≤ θ1 versus H1 : θ > θ1 clearly has power larger than α at θ = 1. Hence, T∗ ≡ α is not UMP. Therefore, a UMP test does not exists. (ii) If a test T of level α has a nonconstant power function, then either βT (0) or βT (1) is strictly less than α and, hence, T is not unbiased. Therefore, only tests with constant power functions may be unbiased. This implies that T∗ ≡ α is a UMPU test of size α. Exercise 19. Let X be a random variable with probability density fθ . Assume that {fθ : θ ∈ Θ} has monotone likelihood ratio in X, where Θ ⊂ R. Suppose that for each θ0 ∈ Θ, a UMPU test of size α for testing H0 : θ = θ0 has the acceptance region {c1 (θ0 ) ≤ X ≤ c2 (θ0 )} and is strictly unbiased (i.e., its power is larger than α when θ = θ0 ). Show that the functions c1 (θ) and c2 (θ) are increasing in θ. Solution. Let θ0 < θ1 be two values in Θ and T0 and T1 be the UMPU tests with acceptance regions {c1 (θ0 ) ≤ X ≤ c2 (θ0 )} and {c1 (θ1 ) ≤ X ≤ c2 (θ1 )},

270

Chapter 6. Hypothesis Tests

respectively. Let ψ(X) = T1 (X) − T0 (X) and Eθ be the expectation with respect to fθ . It follows from the strict unbiasedness of the tests that Eθ0 ψ(X) = Eθ0 (T1 ) − α > 0 > α − Eθ1 = Eθ1 ψ(X). If [c1 (θ0 ), c2 (θ0 )] ⊂ [c1 (θ1 ), c2 (θ1 )], then ψ(X) ≤ 0 and Eθ0 ψ(X) ≤ 0, which is impossible. If [c1 (θ1 ), c2 (θ1 )] ⊂ [c1 (θ0 ), c2 (θ0 )], then ψ(X) ≥ 0 and Eθ1 ψ(X) ≥ 0, which is impossible. Hence, neither of the two intervals contain the other. If c1 (θ1 ) ≤ c1 (θ0 ) ≤ c2 (θ1 ) ≤ c2 (θ0 ), then there is a x0 ∈ [c1 (θ0 ), c2 (θ1 )] such that ψ(X) ≥ 0 if X < x0 and ψ(X) ≤ 0 if X ≥ x0 , i.e., the function ψ has a single change of sign. Since the family has monotone likelihood ratio in X, it follows from Lemma 6.4(i) in Shao (2003) that there is a θ∗ such that Eθ ψ(X) ≤ 0 for θ < θ∗ and Eθ ψ(X) ≥ 0 for θ > θ∗ . But this contradicts to the fact that Eθ0 ψ(X) > 0 > Eθ1 ψ(X) and θ0 < θ1 . Therefore, we must have c1 (θ1 ) > c1 (θ0 ) and c2 (θ1 ) > c2 (θ0 ), i.e., both c1 (θ) and c2 (θ) are increasing in θ. Exercise 20 (#6.34). Let X be a random variable from the geometric distribution with mean p−1 . Find a UMPU test of size α for H0 : p = p0 versus H1 : p = p0 , where p0 ∈ (0, 1) is known. Solution. The probability density of X with respect to the counting measure is p I{1,2,...} (X), f (x) = exp x log(1 − p) + log 1−p which is in an exponential family. Applying Theorem 6.4 in Shao (2003), we conclude that the UMPU test of size α is ⎧ X < c1 or X > c2 ⎨ 1 T∗ = γi X = ci i = 1, 2 ⎩ 0 otherwise, where ci ’s are positive integers and ci ’s and γi ’s are uniquely determined by c ∞ 1 −1 α = (1 − p0 )k−1 + (1 − p0 )k−1 + γi (1 − p0 )ci −1 p0 i=1,2 k=1

k=c2 +1

and c ∞ 1 −1 α k−1 = k(1 − p0 ) + k(1 − p0 )k−1 + γi ci (1 − p0 )ci −1 . 2 p0 i=1,2 k=1

k=c2 +1

Exercise 21 (#6.36). Let X = (X1 , ..., Xn ) be a random sample from N (µ, σ 2 ) with unknown µ and σ 2 .

Chapter 6. Hypothesis Tests

271

(i) Show that the power of the one-sample t-test depends on a noncentral t-distribution. (ii) Show that the power of the one-sample t-test is an increasing function of (µ − µ0 )/σ for testing H0 : µ ≤ µ0 versus H1 : µ > µ0 and of |µ − µ0 |/σ for testing H0 : µ = µ0 versus H1 : µ = µ0 , where µ0 is a known constant. Note. For testing H0 : µ ≤ µ0 versus H1 : µ > µ0 , the one-sample √ ¯ t-test of size α rejects H0 if and only if t(X) > tn−1,α , where t(X) = n(X −µ0 )/S, ¯ is the sample mean, S 2 is the sample variance, and tr,α is the (1 − α)th X quantile of the t-distribution tr . For testing H0 : µ = µ0 versus H1 : µ = µ0 , the one-sample t-test of√ size α rejects H0 if and only if |t(X)| > tn−1,α/2 . ¯ − µ0 )/σ, U = S/σ, and δ = √n(µ − µ0 )/σ. Solution. (i) Let Z = n(X Then Z is distributed as N (δ, 1), (n − 1)U 2 has the chi-square distribution χ2n−1 , and Z and U are independent. By deﬁnition, t(X) = Z/U has the noncentral t-distribution tn−1 (δ) with the noncentrality parameter δ. (ii) For testing H0 : µ ≤ µ0 versus H1 : µ > µ0 , the power of the one-sample t-test is P t(X) > tn−1,α = P Z > tn−1,α U = E [Φ(δ − tn−1,α U )] , where Φ is the cumulative distribution function of N (0, 1). Since Φ is an increasing function, the power is an increasing function of δ. For testing H0 : µ = µ0 versus H1 : µ = µ0 , the power of the one-sample t-test is P |t(X)| > tn−1,α/2 = P Z > tn−1,α/2 U + P Z < −tn−1,α/2 U ) ( = E Φ(δ − tn−1,α/2 U ) + Φ(−δ − tn−1,α/2 U ) ) ( = E Φ(|δ| − tn−1,α/2 U ) + Φ(−|δ| − tn−1,α/2 U ) . To show that the power is an increasing function of |δ|, it suﬃces to show that Φ(x − a) + Φ(−x − a) is increasing in x > 0 for any ﬁxed a > 0. The result follows from 2

2

d e−(x−a) /2 − e−(x+a) /2 [Φ(x − a) + Φ(−x − a)] = dx 2π −(x2 +a2 )/2 ax (e − e−ax ) e = 2π > 0. Exercise 22. Let X = (X1 , ..., Xn ) be√a random sample from N (µ, σ 2 ) ¯ ¯ is the sample with unknown µ and σ 2 and t(X) = nX/S, where X 2 mean and S is the sample variance. For testing H0 : µ/σ ≤ θ0 versus H1 : µ/σ > θ0 , ﬁnd a test of size α that is UMP among all tests based on t(X), where θ0 is a known constant.

272

Chapter 6. Hypothesis Tests

Solution. From the previous exercise,√we know that t(X) has the noncentral t-distribution tn−1 (δ), where δ = nµ/σ is the noncentrality parameter. The Lebesgue density of t(X) is (e.g., Shao, 2003, p. 26) ∞ gδ (t, y)dy, fδ (t) = 0

√ 2 y (n−2)/2 e−{[t/ y/(n−1)−δ] +y}/2 gδ (t, y) = . 2n/2 Γ( n−1 2 ) π(n − 1)

where

We now show that the family {fδ (t) : δ ∈ R} has monotone likelihood ratio in t. For δ1 < δ2 , it suﬃces to show that f (t)fδ1 (t) − fδ2 (t)fδ1 (t) d fδ2 (t) = δ2 ≥ 0, dt fδ1 (t) [fδ1 (t)]2 Since fδ (t) =

0

∞

[δ − t/

y/(n − 1)]gδ (t, y)dy = δfδ (t) − tf˜δ (t),

where

f˜δ (t) =

we obtain that

t ∈ R.

0

∞

3

y gδ (t, y)dy, n−1 &

fδ2 (t)fδ1 (t) − fδ2 (t)fδ1 (t) = fδ1 (t)fδ2 (t) δ2 − δ1 + t

f˜δ1 (t) f˜δ2 (t) − fδ1 (t) fδ2 (t)

' .

: δ ∈ R is an exponen√ tial family having monotone likelihood ratio in t/ y. Hence, by Lemma 6.3 in Shao (2003), the integral f˜δ (t) is nonincreasing in δ when t > 0 and is nondecreasing in δ when t < 0. Hence, for t > 0 and t < 0, f˜δ1 (t) f˜δ2 (t) t − ≥0 fδ1 (t) fδ2 (t) For any ﬁxed t ∈ R, the family of densities

gδ (t,y) fδ (t)

and, therefore, fδ2 (t)fδ1 (t) − fδ2 (t)fδ1 (t) ≥ 0. Consequently, for testing H0 : µ/σ ≤ θ0 versus H1 : µ/σ > θ0 , a test of size α that is UMP among all tests based on t(X) rejects H0 when t(X) > √ c, where c is the (1 − α)th quantile of the noncentral t-distribution tn−1 ( nθ0 ). Exercise 23 (#6.37). Let (X1 , ..., Xn ) be a random sample from the gamma distribution with unknown shape parameter θ and unknown scale

Chapter 6. Hypothesis Tests

273

parameter γ. Let θ0 > 0 and γ0 > 0 be known constants. (i) For testing H0 : θ ≤ θ0 versus H1 : θ > θ0 and H0 : θ = θ0 versus H1 : θ = θ0 , show exist UMPU tests whose rejection regions are !n that there ¯ where X ¯ is the sample mean. based on V = i=1 (Xi /X), (ii) For testing H : γ ≤ γ versus H 0 0 1 : γ > γ0 , show that a UMPU test !n n X C. rejects H0 when i=1 Xi > C( !n i=1 i ) for some function ¯ The joint density Solution. (i) Let Y = log( i=1 Xi ) and U = nX. )−n θY −U/γ−Y ( of (X1 , ..., Xn ) can be written as Γ(θ)γ θ e , which belongs to an exponential family. From Theorem 6.4 in Shao (2003), UMPU tests are functions of Y and U . By Basu’s theorem, V1 = Y − n log(U/n) = !n ¯ satisﬁes the conditions in Lemma 6.7 of Shao (2003). log (X / X) i i=1 Hence, the rejection regions of the UMPU tests can be determined by using V1 . Since V = eV1 , the rejection regions of the UMPU tests can also be determined by using !n V . ¯ The joint density of (X1 , ..., Xn ) (ii) Let U = log( i=1 Xi ) and Y = nX. ( ) −1 −n can be written as Γ(θ)γ θ e−γ Y +θU −U , From Theorem 6.4 in Shao (2003), for testing H0 : γ ≤ γ0 versus H1 : γ > γ0 , the UMPU test is ∗

T =

1 0

Y > C1 (U ) Y ≤ C1 (U ),

where C1 is a function such that E(T ∗ |U ) = α when γ = γ0 . The result follows by letting C(x) = C1 (log x). Exercise 24 (#6.39). Let X1 and X2 be independent observations from the binomial distributions with sizes n1 and n2 and probabilities p1 and p2 , respectively, where ni ’s are known and pi ’s are unknown. (i) Let Y = X2 and U = X1 + X2 . Show that

n2 θy n1 e IA (y), u = 0, 1, ..., n1 + n2 , P (Y = y|U = u) = Ku (θ) u−y y 1) where A = {y : y = 0, 1, ..., min{u, n2 }, u − y ≤ n1 }, θ = log pp21 (1−p (1−p2 ) , and

⎤−1 n1 n2 eθy ⎦ . Ku (θ) = ⎣ u−y y ⎡

y∈A

(ii) Find a UMPU test of size α for testing H0 : p1 ≥ p2 versus H1 : p1 < p2 . (iii) Repeat (ii) for H0 : p1 = p2 versus H1 : p1 = p2 . Solution. (i) When u = 0, 1, ..., n1 + n2 and y ∈ A,

n2 u−y n1 p1 (1 − p1 )n1 −u+y py2 (1 − p2 )n2 −y P (Y = y, U = u) = u−y y

274

and

Chapter 6. Hypothesis Tests

n1 n2 u−y p1 (1 − p1 )n1 −u+y py2 (1 − p2 )n2 −y . P (U = u) = u−y y y∈A

Then, when y ∈ A, P (Y = y, U = u) P (Y = y|U = u) = = P (U = u)

n1 u−y

n2 θy e Ku (θ). y

1) (ii) Since θ = log pp21 (1−p (1−p2 ) , the testing problem is equivalent to testing H0 : θ ≤ 0 versus H1 : θ > 0. By Theorem 6.4 in Shao (2003), the UMPU test is ⎧ Y > C(U ) ⎨ 1 T∗ (Y, U ) = γ(U ) Y = C(U ) ⎩ 0 Y < C(U ),

where C and γ are functions of U such that E(T∗ |U ) = α when θ = 0 (p1 = p2 ), which can be determined using the conditional distribution of Y given U . When θ = 0, this conditional distribution is, by the result in (i), −1

n2 n1 n1 + n 2 IA (y), u = 0, 1, ..., n1 + n2 . P (Y = y|U = u) = u u−y y (iii) The testing problem is equivalent to testing H0 : θ = 0 versus H1 : θ = 0. Thus, the UMPU test is ⎧ Y > C1 (U ) or Y < C2 (U ) ⎨ 1 T∗ = γi (U ) Y = Ci (U ) i = 1, 2 ⎩ 0 C1 (U ) < Y < C2 (U ), where Ci ’s and γi ’s are functions such that E(T∗ |U ) = α and E(T∗ Y |U ) = αE(Y |U ) when θ = 0, which can be determined using the conditional distribution of Y given U in part (ii) of the solution. Exercise 25 (#6.40). Let X1 and X2 be independently distributed as the negative binomial distributions with sizes n1 and n2 and probabilities p1 and p2 , respectively, where ni ’s are known and pi ’s are unknown. (i) Show that there exists a UMPU test of size α for testing H0 : p1 ≤ p2 versus H1 : p1 > p2 . (ii) Let Y = X1 and U = X1 + X2 . Determine the conditional distribution of Y given U when n1 = n2 = 1. Solution. (i) The joint probability density of X1 and X2 is x1 −1x2 −1 n1 n2 n1 −1 n2 −1 p1 p2 eθY +U log(1−p2 ) (1 − p1 )n1 (1 − p2 )n2

Chapter 6. Hypothesis Tests

275

1−p1 where θ = log( 1−p ), Y = X1 , and U = X1 + X2 . The testing problem is 2 equivalent to testing H0 : θ ≥ 0 versus H1 : θ < 0. By Theorem 6.4 in Shao (2003), the UMPU test is ⎧ Y < C(U ) ⎨ 1 T∗ (Y, U ) = γ(U ) Y = C(U ) ⎩ 0 Y > C(U ),

where C(U ) and γ(U ) satisfy E(T∗ |U ) = α when θ = 0. (ii) When n1 = n2 = 1, P (U = u) =

u−1

P (X1 = k, X2 = u − k)

k=1

k u−1 p1 p2 (1 − p2 )u−1 1 − p1 = 1 − p1 1 − p2 k=1

=

u−1 u−1

p1 p2 (1 − p2 ) 1 − p1

eθk

k=1

for u = 2, 3, ..., and P (Y = y, U = u) = (1 − p1 )y−1 p1 (1 − p2 )u−y−1 p2 for y = 1, ..., u − 1, u = 2, 3, .... Hence P (Y = y|U = u) =

eθy (1 − p1 )y−1 p1 (1 − p2 )u−y−1 p2 = u−1 u−1 u−1 θk p1 p2 (1−p2 ) θk k=1 e k=1 e 1−p1

for y = 1, ..., u − 1, u = 2, 3, .... When θ = 0, this conditional distribution is the discrete uniform distribution on {1, ..., u − 1}. Exercise 26 (#6.44). Let Xj , j = 1, 2, 3, be independent from the Poisson distributions with means λj , j = 1, 2, 3, respectively. Show that there exists a UMPU test of size α for testing H0: λ1 λ2 ≤ λ23 versus H1: λ1 λ2 > λ23 . Solution. The joint probability density for (X1 , X2 , X3 ) is e−(λ1 +λ2 +λ3 ) X1 log λ1 +X2 log λ2 +X3 log λ3 , e X1 !X2 !X3 ! which is the same as e−(λ1 +λ2 +λ3 ) θY +U1 log λ2 +U2 log λ3 , e X1 !X2 !X3 ! where θ = log λ1 + log λ2 − 2 log λ3 , Y = X1 , U1 = X2 − X1 , and U2 = X3 +2X1 . By Theorem 6.4 in Shao (2003), there exists a UMPU test of size

276

Chapter 6. Hypothesis Tests

α for testing H0 : θ ≤ 0 versus H1 : θ > 0, which is equivalent to testing H0 : λ1 λ2 ≤ λ23 versus H1 : λ1 λ2 > λ23 . Exercise 27 (#6.49). Let (Xi1 , ..., Xini ), i = 1, 2, be two independent random samples from N (µi , σ 2 ), respectively, where ni ≥ 2 and µi ’s and σ are unknown. Show that a UMPU test of size α for H0 : µ1 = µ2 versus H1 : µ1 = µ2 rejects H0 when |t(X)| > tn1 +n2 −1,α/2 , where ;2 −1 ¯1) ¯2 − X n1 + n−1 (X 2 t(X) = , 2 2 [(n1 − 1)S1 + (n2 − 1)S2 ]/(n1 + n2 − 2) ¯ i and S 2 are the sample mean and variance based on Xi1 , ..., Xin , i = 1, 2, X i i and tn1 +n2 −1,α is the (1 − α)th quantile of the t-distribution tn1 +n2 −1 . Derive the power function of this test. ¯ 1 , U1 = n1 X ¯2 − X ¯ 1 + n2 X ¯ 2 , U2 = 2 n X 2 , Solution. Let Y = X i=1 j=1 ij −1 2 2 θ = (µ1 − µ2 )/[(n−1 1 + n2 )σ ], ϕ1 = (n1 µ1 + n2 µ2 )/[(n1 + n2 )σ ], and 2 −1 ϕ2 = −(2σ ) . Then, the joint density of Xi1 , ..., Xini , i = 1, 2, can be written as √ ( 2πσ)n1 +n2 eθY +ϕ1 U1 +ϕ2 U2 . The statistic V = Y / U2 − U12 /(n1 + n2 ) satisﬁes the conditions in Lemma 6.7(ii) in Shao (2003). Hence, the UMPU test has the rejection region V < c1 or V > c2 . Under H0 , V is symmetrically distributed around 0, i.e., V and −V have the same distribution. Thus, a UMPU test rejects H0 when −V < c1 or −V > c2 , which is the same as rejecting H0 when V < −c2 or V > −c1 . By the uniqueness of the UMPU test, we conclude that c1 = −c2 , i.e., the UMPU test rejects when |V | > c. Since U2 −

U12 n1 n2 Y 2 = (n1 − 1)S12 + (n2 − 1)S22 + , n1 + n2 n1 + n2

we obtain that 1 n1 n2 n1 n2 1 = + . V2 (n1 + n2 )(n1 + n2 − 2) [t(X)]2 n1 + n2 Hence, |V | is an increasing function of |t(X)|. Also, t(X) has the tdistribution with tn1 +n2 −2 under H0 . Thus, the UMPU test rejects H0 when |t(X)| > tn1 +n2 −2,α/2 . Under H1 , t(X) is distributed as the noncentral t-distribution tn1 +n2 −2 (δ) with noncentrality parameter δ=

µ − µ1 2 2 . −1 σ n−1 1 + n2

Thus the power function of the UMPU test is 1 − Gδ (tn1 +n2 −2,α/2 ) + Gδ (−tn1 +n2 −2,α/2 ),

Chapter 6. Hypothesis Tests

277

where Gδ denotes the cumulative distribution function of the noncentral t-distribution tn1 +n2 −2 (δ). Exercise 28 (#6.50). Let (Xi1 , ..., Xin ), i = 1, 2, be two independent random samples from N (µi , σi2 ), respectively, where n > 1 and µi ’s and σi ’s are unknown. Show that a UMPU test of size α for testing H0 : σ22 = ∆0 σ12 versus H1 : σ22 = ∆0 σ12 rejects H0 when 2 S2 ∆0 S12 1−c , max > , ∆0 S12 S22 c c where ∆0 > 0 is a constant, 0 f(n−1)/2,(n−1)/2 (v)dv = α/2 and fa,b is the Lebesgue density of the beta distribution with parameter (a, b). Solution. From Shao (2003, p. 413), the UMPU test rejects H0 if V < c1 or V > c2 , where S 2 /∆0 V = 2 2 2 S1 + S2 /∆0 and Si2 is the sample variance based on Xi1 , ..., Xin . Under H0 (σ12 = σ22 ), n−1 V has the beta distribution with parameter ( n−1 2 , 2 ), which is symmetric 1 about 2 , i.e., V has the same distribution as 1 − V . Thus, a UMPU test rejects H0 when 1 − V < c1 or 1 − V > c2 , which is the same as rejecting H0 when V < 1 − c2 or V > 1 − c1 . By the uniqueness of the UMPU test, we conclude that c1 + c2 = 1. Let c1 = c. Then the UMPU test rejects H0 c when V < c or V > 1 − c and c satisﬁes 0 f(n−1)/2,(n−1)/2 (v)dv = α/2. Let 2 2 F = S2 /(∆0 S1 ). Then V = F/(1+F ), V < c if and only if F −1 > (1−c)/c, and V > 1 − c if and only if F > (1 − c)/c. Hence, the UMPU test rejects when max{F, F −1 } > (1 − c)/c, which is the desired result. Exercise 29 (#6.51). Suppose that Xi = β0 + β1 ti + εi , i = 1, ..., n, where ti ’s are ﬁxed constants that are not all the same, εi ’s are independent and identically distributed as N (0, σ 2 ), and β0 , β1 , and σ 2 are unknown parameters. Derive a UMPU test of size α for testing (i) H0 : β0 ≤ θ0 versus H1 : β0 > θ0 ; (ii) H0 : β0 = θ0 versus H1 : β0 = θ0 ; (iii) H0 : β1 ≤ θ0 versus H1 : β1 > θ0 ; (iv) H0 : β1 = θ0 versus H1 : β1 = θ0 . Solution: Note that (X1 , ..., Xn ) follows a simple linear regression model. n n 2 Let D = n i=1 t2i − ( i=1 ti ) , n n n n 1 2 βˆ0 = t Xi − ti ti X i D i=1 i i=1 i=1 i=1 and 1 βˆ1 = D

n

n i=1

ti X i −

n i=1

ti

n i=1

Xi

278

Chapter 6. Hypothesis Tests

be the least squares estimators, and let 1 (yi − βˆ0 βˆ1 ti )2 . n − 2 i=1 n

σ ˆ2 =

ˆ From the theory of linear n models, β0 has the normal distribution with mean β0 and variance σ 2 i=1 t2i /D, βˆ1 has the normal distribution with mean σ 2 /σ 2 has the chi-square distribution χ2n−2 , β1 and variance σ 2 n/D, (n − 2)ˆ ˆ 2 are independent. Thus, the following results follow and (βˆ0 , βˆ1 ) and σ from the example in Shao (2003, p. 416). (i) The UMPU test of size α for testing H0 : β0 ≤ θ0 versus H1 : β0 > θ0 rejects H0 when t0 > tn−2,α , where √ t0 =

D(βˆ0 − θ0 ) n 2 σ ˆ i=1 ti

and tr,α is the (1 − α)th quantile of the t-distribution tr . (ii) The UMPU test of size α for testing H0 : β0 = θ0 versus H1 : β0 =

θ0 rejects H0 when |t0 | > tn−2,α/2 . (iii) The UMPU test of size α for testing H0 : β1 ≤ θ0 versus H1 : β1 > θ0 rejects H0 when t1 > tn−2,α , where √ t1 =

D(βˆ1 − θ0 ) √ . nˆ σ

(iv) The UMPU test of size α for testing H0 : β1 = θ0 versus H1 : β1 = θ0 rejects H0 when |t1 | > tn−2,α/2 . Exercise 30 (#6.53). Let X be a sample from Nn (Zβ, σ 2 In ), where β ∈ Rp and σ 2 > 0 are unknown and Z is an n × p known matrix of rank r ≤ p < n. For testing H0 : σ 2 ≤ σ02 versus H1 : σ 2 > σ02 and H0 : σ 2 = σ02 versus H1 : σ 2 = σ02 , show that UMPU tests of size α are functions of ˆ 2 , where βˆ is the least squares estimator of β, and their SSR = X − Z β rejection regions can be determined using chi-square distributions. Solution. Since H = Z(Z τ Z)− Z τ is a projection matrix of rank r, there exists an n × n orthogonal matrix Γ such that Γ = ( Γ1 Γ2 )

and

HΓ = ( Γ1 0 ),

where Γ1 is n×r and Γ2 is n×(n−r). Let Yj = Γτj X, j = 1, 2. Consider the transformation (Y1 , Y2 ) = Γτ X. Since Γτ Γ = In , (Y1 , Y2 ) has distribution Nn (Γτ Zβ, σ 2 In ). Note that E(Y2 ) = E(Γτ2 X) = Γτ2 Zβ = Γτ2 HZβ = 0.

Chapter 6. Hypothesis Tests

279

Let η = Γτ1 Zβ = E(Y1 ). Then the density of (Y1 , Y2 ) is 1 Y1 2 + Y2 2 η τ Y1 η2 . exp + − 2σ 2 σ2 2σ 2 (2πσ 2 )n/2 From Theorem 6.4 in Shao (2003), the UMPU tests are based on Y = Y1 2 + Y2 2 and U = Y1 . Let V = Y2 2 . Then V = Y − U 2 satisﬁes the conditions in Lemma 6.7 of Shao (2003). Hence, the UMPU test for H0 : σ 2 ≤ σ02 versus H1 : σ 2 > σ02 rejects when V > c and the UMPU test for H0 : σ 2 = σ02 versus H1 : σ 2 = σ02 , rejects when V < c1 or V > c2 . Since Y1 − η2 + Y2 2 = X − Zβ2 , min Y1 − η2 + Y2 2 = min X − Zβ2 η

β

and, therefore,

ˆ 2 = SSR. V = Y2 2 = X − Z β

Finally, by Theorem 3.8 in Shao (2003), SSR/σ 2 has the chi-square distribution χ2n−r . Exercise 31 (#6.54). Let (X1 , ..., Xn ) be a random sample from a bivariate normal distribution with unknown means µ1 and µ2 , variances σ12 and σ22 , and correlation coeﬃcient ρ. Let Xij be the jth component of Xi , ¯ and S 2 be the sample mean and variance based on X1j , ..., Xnj , j = 1, 2, X j √ √j and V = n − 2R/ 1 − R2 , where 1 ¯ 1 )(Xi2 − X ¯2) (Xi1 − X S1 S2 (n − 1) i=1 n

R=

is the sample correlation coeﬃcient. Show that the UMPU test of size α for H0 : ρ ≤ 0 versus H1 : ρ > 0 rejects H0 when V > tn−2,α and the UMPU test of size α for H0 : ρ = 0 versus H1 : ρ = 0 rejects H0 when |V | > tn−2,α/2 , where tn−2,α is the (1 − α)th quantile of the t-distribution tn−2 . Solution. The Lebesgue density of (X1 , ..., Xn ) can be written as C(µ1 , µ2 , σ1 , σ2 , ρ) exp {θY + ϕτ U } , where C(·) is a function of (µ1 , µ2 , σ1 , σ2 , ρ), Y =

n

Xi1 Xi2 ,

θ=

i=1

U=

n i=1

2 Xi1 ,

n i=1

2 Xi2 ,

n i=1

ρ , σ1 σ2 (1 − ρ2 ) Xi1 ,

n i=1

Xi2

,

280

Chapter 6. Hypothesis Tests

and

ϕ=

−

1 µ2 1 µ1 − θµ . , , − , − θµ 1 2 2σ12 (1 − ρ2 ) 2σ22 (1 − ρ2 ) σ12 (1 − ρ2 ) σ22 (1 − ρ2 )

By Basu’s theorem, R is independent of U when ρ = 0. Also, R=

Y − U3 U4 /n (U1 − U32 /n)(U2 − U42 /n)

,

which is linear in Y , where Uj is the jth component of U . Hence, we may apply Theorem 6.4 and Lemma 6.7 in Shao (2003). It remains to show that V has the t-distribution tn−2 when ρ = 0, which is a consequence of the result in the note of Exercise 17 in Chapter 1 and the result in Exercise 22(ii) in Chapter 2. Exercise 32 (#6.55). Let (X1 , ..., Xn ) be a random sample from a bivariate normal distribution with unknown means µ1 and µ2 , variances σ12 and σ22 , and correlation coeﬃcient ρ. Let Xij be the jth component of Xi , ¯ j and S 2 be the sample mean and variance based on X1j , ..., Xnj , j = 1, 2, X j n ¯ 1 )(Xi2 − X ¯ 2 ). and S12 = (n − 1)−1 i=1 (Xi1 − X (i) Let ∆0 > 0 be a known constant. Show that a UMPU test for testing H0 : σ2 /σ1 = ∆0 versus H1 : σ2 /σ1 = ∆0 rejects H0 when ;2 2 2 2 > c. (∆0 S1 + S22 )2 − 4∆20 S12 R = |∆20 S12 − S22 | (ii) Find the Lebesgue density of R in (i) when σ2 /σ1 = ∆0 . (iii) Assume that σ1 = σ2 . Show that a UMPU test for H0 : µ1 = µ2 versus H1 : µ1 = µ2 rejects H0 when ;2 ¯2 − X ¯1| V = |X (n − 1)(S12 + S22 − 2S12 ) > c. (iv) Find the Lebesgue density of V in (iii) when µ1 = µ2 . Solution. (i) Let

Yi1 Yi2

=

∆0 ∆0

1 −1

Xi1 Xi2

.

Then Cov(Yi1 , Yi2 ) = ∆20 σ12 − σ22 . If we let ρY be the correlation between Yi1 and Yi2 , then testing H0 : σ2 /σ1 = ∆0 versus H1 : σ2 /σ1 = ∆0 is equivalent to testing H0 : ρY = 0 versus H1 : ρY = 0. By the result in the previous exercise, the UMPU test rejects when |VY | > c, where VY is the sample correlation coeﬃcient based on the Y -sample. The result follows from the fact that |VY | = R.

Chapter 6. Hypothesis Tests

281

(ii) From Exercise 22(ii) in Chapter 2, the Lebesgue density of R in (i) when σ2 /σ1 = ∆0 is 2Γ( n−1 n−4 2 ) (1 − r2 ) 2 I(0,1) (r). √ πΓ( n−2 ) 2 (iii) Let

Yi1 Yi2

=

1 1

1 −1

Xi1 Xi2

.

Then (Yi1 , Yi2 ) has the bivariate normal distribution

µ1 + µ2 2(1 + ρ)σ 2 0 , . N2 0 2(1 − ρ)σ 2 µ1 − µ2 Since Yi1 and Yi2 are independent, the UMPU test of size α for testing H0 : µ1 √ = µ2 versus H1 : µ1 = µ2 rejects H0 when |t(Y )| > tn−1,α/2 , where t(Y ) = nY¯2 /SY2 , Y¯2 and SY22 are the sample mean and variance based on Y12 , ..., Yn2 , and tn−1,α is the (1 − α)th quantile of the t-distribution tn−1 . A direct calculation shows that √ ¯ ¯2| n|X1 − X |t(Y )| = 2 n(n − 1)V. = (S1 + S22 − 2S12 ) (iv) Since t(Y ) has the t-distribution tn−1 under H0 , the Lebesgue density of V when µ1 = µ2 is √ nΓ( n2 ) (1 + nv 2 )−n/2 . √ πΓ( n−1 ) 2 Exercise 33 (#6.57). Let (X1 , ..., Xn ) be a random sample from the exponential distribution on the interval n (a, ∞) with scale parameter θ, where a and θ are unknown. Let V = 2 i=1 (Xi − X(1) ), where X(1) is the smallest order statistic. (i) For testing H0 : θ = 1 versus H1 : θ = 1, show that a UMPU test of size α rejects H0 when V < c1 or V > c2 , where ci ’s are determined by c2 c2 f2n−2 (v)dv = f2n (v)dv = 1 − α c1

c1

and fm (v) is the Lebesgue density of the chi-square distribution χ2m . (ii) For testing H0 : a = 0 versus H1 : a = 0, show that a UMP test of size α rejects H0 when X(1) < 0 or 2nX(1) /V > c, where c is determined by c (n − 1) (1 + v)−n dv = 1 − α. 0

282

Chapter 6. Hypothesis Tests

Solution. Since (X(1) , V ) is suﬃcient and complete for (a, θ), we may consider tests that are functions of (X(1) , V ). (i) When θ = 1, X(1) is complete and suﬃcient for a, V is independent of X(1) , and V /2 has the Gamma distribution with shape parameter n − 1 and scale parameter θ. By Lemma 6.5 and Lemma 6.6 in Shao (2003), the UMPU test is the UMPU test in the problem where V /2 is an observation from the Gamma distribution with shape parameter n − 1 and scale parameter θ, which has monotone likelihood ratio in V . Hence, the UMPU test of size α rejects H0 when V < c1 or V > c2 , where V has the chi-square distribution χ22(n−1) when θ = 1 and, hence, ci ’s are determined by

c2

f2n−2 (v)dv = 1 − α

c1

and

c2

vf2n−2 (v)dv = (1 − α)

c1

0

∞

vf2n−2 (v)dv = (1 − α)(2n − 2).

Since (2n − 2)−1 vf2n−2 (v) = f2n (v), ci ’s are determined by c2 c2 f2n−2 (v)dv = f2n (v)dv = 1 − α. c1

c1

(ii) From Exercise 15, for testing H0 : a = 0 versus H1 : a = 0, a UMP test of size α rejects H0 when X(1) < 0 or 2nX(1) /V > c. It remains to determine c. When a = 0, X(1) /θ has the chi-square distribution χ22 , V /θ has the chi-square distribution χ22n−2 , and they are independent. Hence, 2nX(1) /[V (n − 1)] has the F -distribution F2,2(n−1) . Hence, 2nX(1) /V has Lebesgue density f (y) = (n − 1)(1 + y)−n . Therefore, c (n − 1) (1 + y)−n dy = 1 − α. 0

Exercise 34 (#6.58). Let (X1 , ..., Xn ) be a random sample from the uniform distribution on the interval (θ, ϑ), where −∞ < θ < ϑ < ∞. (i) Show that the conditional distribution of the smallest order statistic X(1) given the largest order statistic X(n) = x is the distribution of the minimum of a random sample of size n − 1 from the uniform distribution on the interval (θ, x). (ii) Find a UMPU test of size α for testing H0 : θ ≤ 0 versus H1 : θ > 0. Solution. (i) The joint Lebesgue density of (X(1) , X(n) ) is f (x, y) =

n(n − 1)(x − y)n−2 I(θ,x) (y)I(y,ϑ) (x) (ϑ − θ)n

Chapter 6. Hypothesis Tests

283

and the Lebesgue density of X(n) is g(x) =

n(x − θ)n−1 I(θ,ϑ) (x). (ϑ − θ)n

Hence, the conditional density of X(1) given X(n) = x is (n − 1)(x − y)n−2 f (x, y) = I(θ,x) (y), g(x) (x − θ)n−1 which is the Lebesgue density of the smallest order statistic based on a random sample of size n − 1 from the uniform distribution on the interval (θ, x). (ii) Note that (X(1) , X(n) ) is complete and suﬃcient for (θ, ϑ) and when θ = 0, X(n) is complete for ϑ. Thus, by Lemmas 6.5 and 6.6 in Shao (2003), the UMPU test is the same as the UMPU test in the problem where X(1) is the smallest order statistic of a random sample of size n − 1 from the uniform distribution on the interval (θ, x). Let Y = x − X(1) . Then Y is the largest order statistic of a random sample of size n − 1 from the uniform distribution on the interval (0, η), where η = x − θ. Thus, by the result in Exercise 14(i), a UMPU test of size α is T =

α 0

Y

conditional on X(n) = x. Since x − X(1) = Y , T =

α 0

X(1) > 0 X(1) < 0.

Exercise 35 (#6.82). Let X be a random variable having probability density fθ (x) = exp{η(θ)Y (x)−ξ(θ)}h(x) with respect to a σ-ﬁnite measure ν, where η is an increasing and diﬀerentiable function of θ ∈ Θ ⊂ R. ˆ − log (θ0 ) is increasing (or decreasing) in Y when (i) Show that log (θ) θˆ > θ0 (or θˆ < θ0 ), where (θ) = fθ (x), θˆ is an MLE of θ, and θ0 ∈ Θ. (ii) For testing H0 : θ1 ≤ θ ≤ θ2 versus H1 : θ < θ1 or θ > θ2 or for testing H0 : θ = θ0 versus H1 : θ = θ0 , show that there is a likelihood ratio (LR) test whose rejection region is equivalent to Y (X) < c1 or Y (X) > c2 for some constants c1 and c2 . Solution. (i) From the property of exponential families, θˆ is a solution of the likelihood equation ∂ log (θ) = η (θ)Y (X) − ξ (θ) = 0 ∂θ

284

Chapter 6. Hypothesis Tests

ˆ − and ψ(θ) = ξ (θ)/η (θ) has a positive derivative ψ (θ). Since η (θ)Y dθˆ ˆ ˆ ξ (θ) = 0, θ is an increasing function of Y and dY > 0. Consequently, for any θ0 ∈ Θ, * + + d * ˆ − log (θ0 ) = d η(θ)Y ˆ − ξ(θ) ˆ − η(θ0 )Y + ξ(θ0 ) log (θ) dY dY ˆ dθˆ ˆ ˆ − dθ ξ (θ) ˆ − η(θ0 ) η (θ)Y + η(θ) = dY dY dθˆ ˆ ˆ + η(θ) ˆ − η(θ0 ) = [η (θ)Y − ξ (θ)] dY ˆ − η(θ0 ), = η(θ) which is positive (or negative) if θˆ > θ0 (or θˆ < θ0 ). ˆ (ii) Since (θ) is increasing when θ ≤ θˆ and decreasing when θ > θ, ⎧ ⎪ ⎪ ⎨

λ(X) =

(θ1 ) ˆ (θ)

θˆ < θ1

ˆ (θ)

θ1 ≤ θˆ ≤ θ2 θˆ > θ2

supθ1 ≤θ≤θ2 (θ) = 1 ⎪ supθ∈Θ (θ) ⎪ ⎩ (θ2 )

for θ1 ≤ θ2 . Hence, λ(X) < c if and only if θˆ < d1 or θˆ > d2 for some constants d1 and d2 . From the result in (i), this means that λ(X) < c if and only if Y < c1 or Y > c2 for some constants c1 and c2 . Exercise 36 (#6.83). In Exercises 55 and 56 of Chapter 4, consider H0 : j = 1 versus H1 : j = 2. (i) Derive the likelihood ratio λ(X). (ii) Obtain an LR test of size α in Exercise 55 of Chapter 4. Solution. Following the notation in Exercise 55 of Chapter 4, we obtain that ⎧ ˆj = 1 ⎨ 1 λ(X) = ˆ ˆj = 2 ⎩ (θˆ1 ,j=1) (θ2 ,j=2) ⎧ √ 2 T1 /n ⎪ 2e ⎪ ≤ ⎨ 1 T2 /n π

√ n 2 2 = T /n ⎪ 1 T /n 2e √ 2 2e ⎪ ⎩ π T2 /n > π , T1 /n

where T1 = Chapter 4,

n i=1

Xi2 and T2 =

λ(X) =

n i=1

|Xi |. Similarly, for Exercise 56 of

1

h(X) = 0 ¯

e−nX ¯ ¯ n(1−X) (1−X)

h(X) = 1,

Chapter 6. Hypothesis Tests

285

¯ is the sample mean, h(X) = 1 if all Xi ’s are not larger than 1 and where X h(X) = 0 otherwise. (ii) Let c ∈ [0, 1]. Then the LR test of size α rejects H0 when 3 2e T2 /n < c1/n , π T1 /n where c1/n is the αth quantile of the distribution of

2

T2 /n 2e √ π T1 /n

when

X1 , ..., Xn are independent and identically distributed as N (0, 1). Exercise 37 (#6.84). In Exercise 12, derive the likelihood ratio λ(X) when (a) H0 : θ ≤ θ0 ; (b) H0 : θ1 ≤ θ ≤ θ2 ; and (c) H0 : θ ≤ θ1 or θ ≥ θ2 . Solution. Let f and g be the probability densities of F and G, respectively, with respect to the measure corresponding to F + G. Then, the likelihood function is (θ) = θ[f (X) − g(X)] + g(X) and

sup (θ) = 0≤θ≤1

f (X) g(X)

f (X) ≥ g(X) f (X) < g(X).

For θ0 ∈ [0, 1],

θ0 [f (X) − g(X)] + g(X) g(X)

sup (θ) = 0≤θ≤θ0

Hence, for H0 : θ ≤ θ0 , λ(X) =

θ0 [f (X)−g(X)]+g(X) f (X)

f (X) ≥ g(X) f (X) < g(X).

f (X) ≥ g(X) f (X) < g(X).

1

For 0 ≤ θ1 ≤ θ2 ≤ 1, sup 0≤θ1 ≤θ2 ≤1

(θ) =

θ2 [f (X) − g(X)] + g(X) θ1 [f (X) − g(X)] + g(X)

and, thus, for H0 : θ1 ≤ θ ≤ θ2 , ⎧ ⎨ θ2 [f (X)−g(X)]+g(X) f (X) λ(X) = ⎩ θ1 [f (X)−g(X)]+g(X)

f (X) ≥ g(X) f (X) < g(X).

g(X)

Finally, sup 0≤θ≤θ1 ,θ2 ≤θ≤1

f (X) ≥ g(X) f (X) < g(X)

(θ) = sup (θ). 0≤θ≤1

Hence, for H0 : θ ≤ θ1 or θ ≥ θ2 , λ(X) = 1.

286

Chapter 6. Hypothesis Tests

Exercise 38 (#6.85). Let (X1 , ..., Xn ) be a random sample from the discrete uniform distribution on {1, ..., θ}, where θ is an integer ≥ 2. Find a level α LR test for (i) H0 : θ ≤ θ0 versus H1 : θ > θ0 , where θ0 is a known integer ≥ 2; (ii) H0 : θ = θ0 versus H1 : θ = θ0 . Solution. (i) The likelihood function is (θ) = θ−n I{X(n) ,X(n) +1,...} (θ), where X(n) is the largest order statistic. Then, −n sup (θ) = X(n) ,

θ=2,3,...

sup

(θ) =

θ=2,...,θ0

and

λ(X) =

1 0

−n X(n) 0

X(n) ≤ θ0 X(n) > θ0 ,

X(n) ≤ θ0 X(n) > θ0 .

Hence, a level α test rejects H0 when X(n) > θ0 , which has size 0. (ii) From part (i) of the solution, we obtain that X(n) n X(n) ≤ θ0 θ 0 λ(X) = 0 X(n) > θ0 . Then, λ(X) < c is equivalent to X(n) > θ0 or X(n) < θ0 c1/n . Let c = α. When θ = θ0 , the type I error rate is * +n P X(n) < θ0 α1/n = P (X1 < θ0 α1/n ) n

the integer part of θ0 α1/n = θ0

1/n n θ0 α ≤ θ0 = α. Hence, an LR test of level α rejects H0 if X(n) > θ0 or X(n) < θ0 α1/n . Exercise 39 (#6.87). Let X = (X1 , ..., Xn ) be a random sample from the exponential distribution on the interval (a, ∞) with scale parameter θ. (i) Suppose that θ is known. Find an LR test of size α for testing H0 : a ≤ a0 versus H1 : a > a0 , where a0 is a known constant. (ii) Suppose that θ is known. Find an LR test of size α for testing H0 : a =

Chapter 6. Hypothesis Tests

287

a0 versus H1 : a = a0 . (iii) Repeat part (i) for the case where θ is also unknown. (iv) When both θ and a are unknown, ﬁnd an LR test of size α for testing H0 : θ = θ0 versus H1 : θ = θ0 . (v) When a > 0 and θ > 0 are unknown, ﬁnd an LR test of size α for testing H0 : a = θ versus H1 : a = θ. Solution. (i) The likelihood function is ¯

(a, θ) = θ−n ena/θ e−nX/θ I(a,∞) (X(1) ), ¯ is the sample mean and X(1) is the smallest order statistic. When where X θ is known, the MLE of a is X(1) . When a ≤ a0 , the MLE of a is min{a0 , X(1) }. Hence, the likelihood ratio is 1 X(1) < a0 λ(X) = e−n(X(1) −a0 )/θ X(1) ≥ a0 . Then λ(X) < c is equivalent to X(1) > d for some d ≥ a0 . To determine d, note that nena/θ ∞ −nx/θ e dx sup P (X(1) > d) = sup θ a≤a0 a≤a0 d nena0 /θ ∞ −nx/θ e dx = θ d = en(a0 −d)/θ . Setting this probability to α yields d = a0 − n−1 θ log α. (ii) Note that (a0 , θ) = 0 when X(1) < a0 . Hence the likelihood ratio is 0 X(1) < a0 λ(X) = e−n(X(1) −a0 )/θ X(1) ≥ a0 . Therefore, λ(X) < c is equivalent to X(1) ≤ a0 or X(1) > d for some d ≥ a0 . From part (i) of the solution, d = a0 − n−1 θ log α leads to an LR test of size α. ¯ − X(1) ). When a ≤ a0 , the MLE of a0 is (iii) The MLE of (a, θ) is (X(1) , X min{a0 , X(1) } and the MLE of θ is ¯ − X(1) X X(1) < a0 ˆ θ0 = ¯ − a0 X(1) ≥ a0 . X Therefore, the likelihood ratio is [T (X)]n λ(X) = 1

X(1) ≥ a0 X(1) < a0 ,

288

Chapter 6. Hypothesis Tests

¯ ¯ where T (X) = (X−X (1) )/(X−a0 ). Hence the LR test rejects H0 if and only 1/n ¯ − X(1) )/(X ¯ − a) if T (X) < c . From the solution to Exercise 33, Y = (X has the beta distribution with parameter (n − 1, 1). Then,

¯ − X(1) X sup P T (X) < c1/n = sup P ¯ < c1/n X − a + a − a0 a≤a0 a≤a0 = P Y < c1/n c1/n = (n − 1) xn−2 dx 0

= c(n−1)/n . Setting this probability to α yields c = αn/(n−1) . ¯ − X(1) ). Then the (iv) Under H0 , the MLE is (X(1) , θ0 ). Let Y = θ0−1 n(X likelihood ratio is λ(X) = en Y n e−Y . Thus, λ(X) < c is equivalent to Y < c1 or Y > c2 . Under H0 , 2Y has the chi-square distribution χ22n−2 . Hence, an LR test of size α rejects H0 when 2Y < χ22(n−1),1−α/2 or 2Y > χ22(n−1),α/2 , where χ2r,α is the (1 − α)th quantile of the chi-square distribution χ2r . ¯ X). ¯ Let Y = X(1) /(X ¯ − X(1) ). Then the (v) Under H0 , the MLE is (X, likelihood ratio is ¯ − X(1) )n = en (1 + Y )−n , ¯ −n (X λ(X) = en X ¯ − X(1) ). Then λ(X) < c is equivalent to Y > b for where Y = X(1) /(X some b. Under H0 , the distribution Y is the same as that of the ratio Y1 /Y2 , where Y1 has the exponential distribution on the interval (n, ∞) and scale parameter 1, Y2 has the gamma distribution with shape parameter n − 1 and scale parameter 1, and Y1 and Y2 are independent. Hence, b satisﬁes ∞ ∞ en y2n−2 e−y2 e−y1 dy1 dy2 α= Γ(n − 1) 0 max{n,by2 } ∞ en n−2 −y2 − max{n,by2 } = y2 e e dy2 . Γ(n − 1) 0 Exercise 40. Let (X1 , ..., Xn ) and (Y1 , ..., Yn ) be independent random samples from N (µ1 , 1) and N (µ2 , 1), respectively, where −∞ < µ2 ≤ µ1 < ∞. (i) Derive the likelihood ratio λ and an LR test of size α for H0 : µ1 = µ2 versus H1 : µ1 > µ2 . (ii) Derive the distribution of −2 log λ and the power of the LR test in (ii).

Chapter 6. Hypothesis Tests

289

(iii) Verify that the LR test in (ii) is a UMPU test. Solution. (i) The likelihood function is " n n 1 1 1 (µ1 , µ2 ) = exp − (Xi − µ1 )2 − (Yi − µ2 )2 . (2π)n 2 i=1 2 i=1 ¯ be the sample mean based on Xi ’s and Y¯ be the sample mean based Let X ¯ + Y¯ )/2. The ¯ = (X on Yi ’s. When µ1 = µ2 , (µ1 , µ2 ) is maximized at µ ¯ ¯ ¯ ¯ ¯ < Y¯ . MLE of (µ1 , µ2 ) is equal to (X, Y ) when X ≥ Y . Consider the case X ¯ For any ﬁxed µ1 ≤ Y , (µ1 , µ2 ) increases in µ2 and, hence, sup

µ1 ≤Y¯ , µ2 ≤µ1

(µ1 , µ2 ) = sup (µ1 , µ1 ). µ1 ≤Y¯

Also, sup

µ1 >Y¯ , µ2 ≤µ1

(µ1 , µ2 ) = sup (µ1 , Y¯ ) ≤ sup (µ1 , µ1 ), µ1 >Y¯

µ1 ≤Y¯

¯ < Y¯ . Hence, since X µ, µ ¯), sup (µ1 , µ1 ) = sup (µ1 , µ1 ) = (¯ µ1 ≤Y¯

µ2 ≤µ1

¯ < Y¯ . This shows that the MLE is (¯ ¯ < Y¯ . since µ ¯ < Y¯ when X µ, µ ¯) when X ¯ < Y¯ and Therefore, the likelihood ratio λ = 1 when X n n exp − 12 i=1 (Xi − µ ¯)2 − 12 i=1 (Yi − µ ¯)2 n λ= ¯ 2 − 1 n (Yi − Y¯ )2 exp − 12 i=1 (Xi − X) i=1 2 ¯ − Y¯ )2 = exp − n (X 4

¯ ≥ Y¯ . Hence, when X

n ¯ 2 (X

− Y¯ )2

¯ ≥ Y¯ X ¯ < Y¯ . 0 X n ¯ ¯ Note that λ < c for some c ∈ (0, 1) is equivalent to 2 (X − Y ) > d for n ¯ ¯ some d > 0. Under H0 , 2 (X − Y ) has the standard normal distribution. Hence, setting d = Φ−1 (1 − α) yields an LR test of size α, where Φ is the cumulative distribution function of the standard normal distribution. ¯ − Y¯ ) (ii) Note that −2 log λ ≥ 0. Let δ = n2 (µ1 − µ2 ). Then Z = n2 (X has distribution N (δ, 1). For t > 0, −2 log λ =

P (−2 log λ ≤ t) = P (−2 log λ ≤ t, Z > 0) + P (−2 log λ ≤ t, Z ≤ 0) √ √ = P (Z ≤ t, Z > 0) + P (Z ≤ t, Z ≤ 0) √ = P (0 < Z ≤ t) + P (Z ≤ 0) √ = P (Z ≤ t) √ = Φ( t − δ).

290

Chapter 6. Hypothesis Tests

The power of the LR test in (ii) is then 1 − Φ(d − δ) = 1 − Φ(Φ−1 (1 − α) − δ). (iii) The likelihood can be written as ¯ + nµ2 U − n(µ1 + µ2 )/2 , CX,Y exp θX ¯ + Y¯ , and CX,Y is a quantity does not depend where θ = n(µ1 − µ2 ), U = X on any parameters. When θ = 0 (µ1 = µ2 ), Z is independent of U . Also, ¯ − U ). Hence, by Theorem 6.4 and Lemma 6.7 in Shao (2003), Z = n2 (2X the UMPU test for H0 : θ = 0 versus H1 : θ > 0 rejects H0 when Z > c for some c, which is the same as the LR test in (ii). Exercise 41 (#6.89). Let Xi1 , ..., Xini , i = 1, 2, be two independent random samples from the uniform distributions on (0, θi ), i = 1, 2, respectively, where θ1 > 0 and θ2 > 0 are unknown. (i) Find an LR test of size α for testing H0 : θ1 = θ2 versus H1 : θ1 = θ2 . (ii) Derive the limiting distribution of −2 log λ when n1 /n2 → κ ∈ (0, ∞), where λ is the likelihood ratio in part (i). Solution. (i) Let Yi = maxj=1,...,ni Xij , i = 1, 2, and Y = max{Y1 , Y2 }. The likelihood function is (θ1 , θ2 ) = θ1−n1 θ2−n2 I(0,θ1 ) (Y1 )I(0,θ2 ) (Y2 ) and the MLE of θi is Yi , i = 1, 2. When θ1 = θ2 , (θ1 , θ1 ) = θ1−n1 −n2 I(0,θ1 ) (Y ) and the MLE of θ1 is Y . Hence, the likelihood ratio is λ=

Y1n1 Y2n2 . Y n1 +n2

Assume that θ1 = θ2 . For any t ∈ (0, 1), P (λ < t) = P (λ < t, Y1 ≥ Y2 ) + P (λ < t, Y1 < Y2 ) = P Y2 < t1/n2 Y1 , Y1 ≥ Y2 + P Y1 < t1/n1 Y2 , Y1 ≥ Y2 = P Y2 < t1/n2 Y1 + P Y1 < t1/n1 Y2 1 t1/n2 y1 = n1 n 2 y2n2 −1 y1n1 −1 dy2 dy1 0

+ n1 n2

0

0

1

0

t1/n1 y2

y1n1 −1 y2n2 −1 dy1 dy2

Chapter 6. Hypothesis Tests = n1 t

0

1

291

y1n1 +n2 −1 dy1 + n2 t

= (n1 + n2 )t

1

1

0

y2n1 +n2 −1 dy2

y n1 +n2 −1 dy

0

= t. Hence, the LR test of size α rejects H0 when λ < α. (ii) Under H0 , by part (i) of the solution, λ has the uniform distribution on (0, 1), which does not depend on (n1 , n2 ). The distribution of −2 log λ is then the exponential distribution on the interval (0, ∞) with scale parameter 2−1 , which is also the chi-square distribution χ21 . Consider now the limiting distribution of −2 log λ when n1 /n2 → κ ∈ (0, ∞). Assume that θ1 < θ2 . Then P (Y1 > Y2 ) = P (Y2 − Y1 − (θ2 − θ1 ) < −(θ2 − θ1 )) → 0 since Y2 − Y1 →p θ2 − θ1 . Thus, for the limiting distribution of −2 log λ, we may assume that Y1 ≤ Y2 and, consequently, −2 log λ = 2n1 (log Y2 −log Y1 ). Note that ni (θi − Yi ) →d θi Zi , i = 1, 2, where Zi has the exponential distribution on the interval (0, ∞) with scale parameter 1. By the δ-method, ni (log θi − log Yi ) →d Zi ,

i = 1, 2.

Because Y1 and Y2 are independent, −2 log λ + 2 log(θ1 /θ2 )n1 = 2[n1 (log θ1 − log Y1 ) − →d 2(Z1 − κZ2 ),

n1 n2 n2 (log θ2

− log Y2 )]

where Z1 and Z2 are independent. The limiting distribution of −2 log λ for the case of θ1 > θ2 can be similarly obtained. Exercise 42 (#6.90). Let (Xi1 , ..., Xini ), i = 1, 2, be two independent random samples from N (µi , σi2 ), i = 1, 2, respectively, where µi ’s and σi2 ’s are unknown. For testing H0 : σ22 /σ12 = ∆0 versus H1 : σ22 /σ12 = ∆0 with a known ∆0 > 0, derive an LR test of size α and compare it with the UMPU test. ¯1, X ¯2, σ ¯ i is the Solution. The MLE of (µ1 , µ2 , σ12 , σ22 ) is (X ˆ12 , σ ˆ22 ), where X sample mean based on Xi1 , ..., Xini and σ ˆi2 =

ni 1 ¯ j )2 , (Xij − X ni j=1

292

Chapter 6. Hypothesis Tests

¯1, X ¯2, σ i = 1, 2. Under H0 , the MLE of (µ1 , µ2 , σ12 ) is (X ˜12 ), where n1 n 2 ¯ 1 )2 + ¯ 2 ∆0 j=1 (X1j − X j=1 (X2j − X2 ) . σ ˜12 = ∆0 (n1 + n2 ) Then the likelihood ratio is proportional to

n1 /2 n2 /2 1 F , 1+F 1+F where F = S22 /(∆0 S12 ) and Sj2 is the sample variance based on Xi1 , ..., Xini , i = 1, 2. Under H0 , F has the F-distribution Fn2 −1,n1 −1 . Since the likelihood ratio is a unimodal function in F , an LR test is equivalent to the one that rejects the null hypothesis when F < c1 or F > c2 for some positive constants c1 < c2 chosen so that P (F < c1 ) + P (F > c2 ) = α under H0 . Note that the UMPU test is one of these tests with an additional requirement being unbiased, i.e., the ci ’s must satisfy P (B < c1 )+P (B > c2 ) = α, where B has the beta distribution with parameter ( n22+1 , n12−1 ) (e.g., Shao, 2003, p. 414). Exercise 43 (#6.91). Let (Xi1 , Xi2 ), i = 1, ..., n, be a random sample from the bivariate normal distribution with unknown mean and covariance matrix. For testing H0 : ρ = 0 versus H1 : ρ = 0, where ρ is the correlation coeﬃcient, show that the test rejecting H0 when |R| > c is an LR test, where ' -& n n n ¯ 1 )(Xi2 − X ¯2) ¯ 1 )2 + ¯ 2 )2 R= (Xi1 − X (Xi1 − X (Xi2 − X i=1

i=1

i=1

¯ j is the sample mean based on is the sample correlation coeﬃcient and X X1j , ..., Xnj . Discuss the form of the limiting distribution of −2 log λ, where λ is the likelihood ratio. Solution. From the normal distribution theory, the MLE of the means are ¯ 1 and X ¯ 2 and the MLE of the variances are n−1 n (Xi1 − X ¯ 1 )2 and X i=1 n −1 2 ¯ n i=1 (Xi2 − X2 ) , regardless of whether H0 holds or not. The MLE of ρ is the sample correlation coeﬃcient R. Under H0 , ρ = 0. Using these results, the likelihood ratio is λ = (1 − R2 )n/2 . Hence, an LR test rejects H0 when |R| > c for some c > 0. The distribution of R under H0 is given in Exercise 9(ii) in Chapter 2. Hence, the Lebesgue density of −2 log λ is Γ( n−1 2 ) (1 − e−x )−1/2 e−(n−2)x/2 I(0,∞) (x). √ πΓ( n−2 2 ) When ρ = 0, it follows from the result in Exercise 9(i) of Chapter 2 that √ n(R − ρ) →d N (0, (1 − ρ2 )2 /(1 + ρ2 )). By the δ-method, √ −2 log λ 4ρ2 − log(1 − ρ2 ) →d N 0, 1+ρ . n 2 n

Chapter 6. Hypothesis Tests

293

Exercise 44. Consider the problem in Exercise 63 of Chapter 4. Find an LR test of size α for testing H0 : θ1 = θ2 versus H1 : θ1 = θ2 . Discuss the limiting distribution of −2 log λ, where λ is the likelihood ratio. Solution. the MLE of (θ1 , θ2 ) is equal √ From the√solution of Exercise 44, n to n−1 ( T1 T2 + T1 , T1 T2 + T2 ), where T1 = i=1 Xi I(0,∞) (Xi ) and T2 = n − i=1 Xi I(−∞,0] (Xi ). When θ1 = θ2 , the distribution of Xi is the double exponential distribution with mean 0 and scale parameter θ1 = θ2 . Hence, the MLE of θ1 = θ2 is (T1 + T2 )/n. Since the likelihood function is T1 T2 (θ1 , θ2 ) = (θ1 + θ2 )−n exp − − , θ1 θ2 √ √ ( T1 + T2 )2n . λ= n 2 (T1 + T2 )n Under H0 , the distribution of λ does not depend on any unknown parameter. Hence, an LR test of size α rejects H0 when λ < c, where c is the (1 − α)th quantile of λ under H0 . Note that the likelihood ratio is

E(Ti ) =

nθi2 n[2θi3 (θ1 + θ2 ) − θi4 ] , Var(Ti ) = , i = 1, 2, θ1 + θ2 (θ1 + θ2 )2

and Cov(T1 , T2 ) = −E(T1 )E(T2 ) = −

n2 θ12 θ22 . (θ1 + θ2 )2

Hence, by the central limit theorem, ⎛ ⎛ 3 ⎞⎞ ⎡ ⎛ 2 ⎞⎤ θ1 (θ1 +2θ2 ) θ12 θ22

θ1 − √ 2 2 T /n (θ1 +θ2 ) (θ1 +θ2 ) 2 ⎠⎦ ⎠⎠ . →d N2 ⎝0, ⎝ n⎣ 1 − ⎝ θ1 +θ θ22 θ12 θ22 θ23 (θ2 +2θ1 ) T2 /n − 2 2 θ1 +θ2 (θ1 +θ2 ) (θ1 +θ2 ) √ √ Let g(x, y) = 2 log( x + y) − log(x + y) − log 2. Then n−1 log λ = g(T1 /n, T2 /n). The derivatives ∂g(x, y) 1 1 = √ − ∂x x + xy x + y

and

∂g(x, y) 1 1 = √ − ∂y y + xy x + y

(θ2 −θ1 ) (θ1 −θ2 ) at x = E(T1 ) and x = E(T2 ) are equal to θθ12 (θ and θθ21 (θ 2 2 2 2 , respec1 +θ2 ) 1 +θ2 ) tively. Hence, by the δ-method, + √ * −1 (θ1 +θ2 )2 →d N (0, τ 2 ), n n log λ − log 2(θ 2 +θ 2 ) 1

2

where τ2 =

[θ1 θ22 (θ1 + 2θ2 ) + θ2 θ12 (θ2 + 2θ1 ) + 2θ12 θ22 ](θ1 − θ2 )2 . (θ1 + θ2 )2 (θ12 + θ22 )2

294

Chapter 6. Hypothesis Tests

Exercise 45 (#6.93). Let X1 and X2 be independent observations from the binomial distributions with sizes n1 and n2 and probabilities p1 and p2 , respectively, where ni ’s are known and pi ’s are unknown. (i) Find an LR test of level α for testing H0 : p1 = p2 versus H1 : p1 = p2 . (ii) Find an LR test of level α for testing H0 : p1 ≥ p2 versus H1 : p1 < p2 . Is this test a UMPU test? Solution. (i) The likelihood function is n1 −X1 X2 1 (p1 , p2 ) = CX pX p2 (1 − p2 )n2 −X2 , 1 (1 − p1 )

where CX is a quantity not depending on (p1 , p2 ). The MLE of (p1 , p2 ) is (X1 /n1 , X2 /n2 ). Under H0 , the MLE of p1 = p2 is U/n, where U = X1 +X2 and n = n1 + n2 . Then, the likelihood ratio is U U λ1 =

X1 n1

X1

n−U 1 − Un n1 −X1 X2

n

1−

X1 n1

X2 n2

1−

X2 n2

n2 −X2 .

An LR test rejects H0 when λ1 < c, which is equivalent to ψ(X1 , X2 ) > g(U ) for some function g, where ψ(X1 , X2 ) =

X1 n1

X1

1−

X1 n1

n1 −X1

X2 n2

X2

1−

X2 n2

n2 −X2

is the denominator of λ1 . To determine g(U ), we note that, under H0 , the conditional distribution of ψ(X1 , X2 ) given U does not depend on any unknown parameter (which follows from the suﬃciency of U under H0 ). Hence, if we choose g(U ) such that P (ψ(X1 , X2 ) > g(U )|U ) ≤ α, then P (ψ(X1 , X2 ) > g(U )) ≤ α. (ii) Using the same argument used in the solution of Exercise 40, we can show that the MLE of (p1 , p2 ) under H0 (p1 ≥ p2 ) is equal to (X1 /n1 , X2 /n2 ) if X1 /n1 ≥ X2 /n2 and is equal to (U/n, U/n) if X1 /n1 < X2 /n2 . Hence, the likelihood ratio is λ1 X1 /n1 < X2 /n2 λ= 1 X1 /n1 ≥ X2 /n2 , where λ1 is given in part (i) of the solution. Hence, an LR test rejects H0 when λ1 < c and X1 /n1 < X2 /n2 , which is equivalent to the test > c−1 and U/(1 + n1 /n2 ) < X2 . Note that that rejects H0 when λ−1 1 −1 λ1 = h(X2 , U )/q(U ), where h(X2 , U ) =

U −X2 n1

U −X2

1−

U −X2 n1

n1 −U +X2

X2 n2

X2

1−

X2 n2

n2 −X2

Chapter 6. Hypothesis Tests

295

and q(U ) = Since

X2 (n1 −U +X2 ) (n2 −X2 )(U −X2 )

U U n

1−

U n−U . n

> 1 when U/(1 + n1 /n2 ) < X2 , the derivative

∂ log h(X2 , U ) = log ∂X2

X2 (n1 − U + X2 ) (n2 − X2 )(U − X2 )

> 0,

i.e., h(X2 , U ) is increasing when U/(1 + n1 /n2 ) < X2 . Thus, the LR test is equivalent to the test that rejects H0 when X2 > c(U ) for some function c(U ). The diﬀerence between this test and the UMPU test derived in Exercise 24 is that the UMPU test is of size α and possibly randomized, whereas the LR test is of level α and nonrandomized. Exercise 46 (#6.95). Let X1 and X2 be independently distributed as the exponential distributions on the interval (0, ∞) with unknown scale parameters θi , i = 1, 2, respectively. Deﬁne θ = θ1 /θ2 . Find an LR test of size α for testing (i) H0 : θ = 1 versus H1 : θ = 1; (ii) H0 : θ ≤ 1 versus H1 : θ > 1. Solution. (i) Since the MLE of (θ1 , θ2 ) is (X1 , X2 ) and, under H0 , the MLE of θ1 = θ2 is (X1 + X2 )/2, the likelihood ratio is X1 X2 λ= = X +X 2 1

2

2

4F , (1 + F )2

where F = X2 /X1 . Note that λ < c is equivalent to F < c1 or F > c2 for 0 < c1 < c2 . Under H0 , F is has the F-distribution F2,2 . Hence, an LR test of size α rejects H0 when F < c1 or F > c2 with ci ’s determined by P (F < c1 ) + P (F > c2 ) = α under H0 . (ii) Using the same argument used in Exercises 40 and 45, we obtain the likelihood ratio 1 X 1 < X2 λ= 4F X1 ≥ X2 . (1+F )2 Note that λ < c if and only if 4F/(1 + F )2 < c and X1 ≥ X2 , which is equivalent to F < b for some b. Let F2,2 be a random variable having the F-distribution F2,2 . Then

sup P (F < b) = sup P θ1 ≤θ2

θ1 ≤θ2

F2,2 <

bθ1 θ2

= P (F2,2 < b) .

Hence, an LR test of size α rejects H0 when F < b with b being the αth quantile of the F-distribution F2,2 .

296

Chapter 6. Hypothesis Tests

Exercise 47 (#6.98). Let (X1 , ..., Xn ) be a random sample from N (µ, σ 2 ). (i) Suppose that σ 2 = γµ2 with unknown γ > 0 and µ ∈ R. Find an LR test for testing H0 : γ = 1 versus H1 : γ = 1. (ii) In the testing problem in (i), ﬁnd the forms of Wald’s test and Rao’s score test. (iii) Repeat (i) and (ii) when σ 2 = γµ with unknown γ > 0 and µ > 0. Solution. (i) The likelihood function is " n 1 −n 2 (µ, γ) = ( 2πγ|µ|) exp − (Xi − µ) . 2γµ2 i=1 ¯ σ ¯ 2 ), where X ¯ is the sample mean The MLE of (µ, µ, γˆ ) = (X, ˆ 2 /X nγ) is (ˆ 2 −1 2 ¯ and σ ˆ =n i=1 (Xi − X) . Under H0 , by Exercise 41(viii) in Chapter 4, the MLE of µ is µ+ (µ+ , 1) > (µ− , 1) µ ˆ0 = µ− (µ+ , 1) ≤ (µ− , 1), where µ± =

¯± −X

√

¯ 2 + 4ˆ 5X σ2 . 2

The likelihood ratio is λ=

¯ 2 en/2 σ ˆn µ0 − X) (ˆ µ0 , 1) nˆ σ 2 + n(ˆ = , exp − (ˆ µ, γˆ ) |ˆ µ0 |n 2ˆ µ20

¯ 2 /ˆ ¯ 2 /ˆ σ 2 . Under H0 , the distribution of X σ 2 does not which is a function of X depend on any unknown parameter. Hence, an LR test can be constructed with rejection region λ < c. (ii) Let n ⎞ ⎛ (Xi −µ)2 ¯ n(X−µ) n i=1 ∂ log (µ, γ) ⎜ − µ + γµ2 + γµ3 ⎟ s(µ, γ) = =⎝ n ⎠. 2 ∂(µ, γ) (X −µ) i n i=1 − 2γ + 2µ2 γ 2 The Fisher information about (µ, γ) is In (µ, γ) = E{s(µ, γ)[s(µ, γ)]τ } = n

1 µ2 γ

+

2 µ2

1 µγ

Then, Rao’s score test statistic is Rn = [s(ˆ µ0 , 1)]τ [In (ˆ µ0 , 1)]−1 s(ˆ µ0 , 1), where µ ˆ0 is given in part (i) of the solution.

1 µγ 1 2γ 2

.

Chapter 6. Hypothesis Tests

297

Let R(µ, γ) = γ − 1. Then ∂R/∂µ = 0 and ∂R/∂γ = 1. Wald’s test statistic Wn is equal to [R(ˆ µ, γˆ )]2 divided by the last element of the inverse µ, γˆ ), where µ ˆ and γˆ are given in part (i) of the solution. Hence, of In (ˆ Wn = (iii) Let T = n−1

n(ˆ γ − 1)2 . 2ˆ γ 2 + 4ˆ γ3

n

Xi2 . The likelihood function is ¯ nX nµ nT + − . (µ, γ) = ( 2πγµ)−n exp − 2γµ γ 2γ i=1

When γ = √ 1, it is shown in Exercise 60 of Chapter 4 that the MLE of µ is µ ˆ0 = ( 1 + 4T − 1)/2. For the MLE of (µ, γ), it is equal to (ˆ µ, γˆ ) = ¯ when X ¯ > 0. If X ¯ ≤ 0, however, the likelihood is unbounded in ¯ σ (X, ˆ 2 /X) γ. Hence, the likelihood ratio is nT ¯ >0 ¯ − nˆµ0 X ˆnµ ˆ0 )−n/2 exp − 2ˆ + n X en/2 σ µ 2 0 λ= ¯ ≤ 0. 0 X To construct an LR test, we note that, under H0 , T is suﬃcient for µ. Hence, under H0 , we may ﬁnd a c(T ) such that P (λ < c(T )|T ) = α for every T . The test rejecting H0 when λ < c(T ) has size α, since P (λ < c(T )) = α under H0 . Note that n nT n + 2γµ − 2µ ∂ log (µ, γ) 2 − 2γ s(µ, γ) = = , ¯ nµ n ∂(µ, γ) − 2γ + 2γnT2 µ − nγX 2 + 2γ 2 ∂ 2 log (µ, γ) n nT = − , ∂µ2 2µ2 γµ3 n ∂ 2 log (µ, γ) nT = − 2 2 + 2, ∂µ∂γ 2γ µ 2γ and

¯ n nT nµ ∂ 2 log (µ, γ) 2nX = − − 3. + 2 2 3 3 ∂γ 2γ γ µ γ γ

Hence, the Fisher information about (µ, γ) is 1 1 2µ2 + γµ In (µ, γ) = n 1 2γµ

1 2γµ 1 2γ 2

and Rao’s score test statistic is Rn = [s(ˆ µ0 , 1)]τ [In (ˆ µ0 , 1)]−1 s(ˆ µ0 , 1).

298

Chapter 6. Hypothesis Tests

Similar to that in part (ii), Wald’s test statistic Wn is equal to (ˆ γ − 1)2 divided by the last element of the inverse of In (ˆ µ, γˆ ), i.e., Wn =

n(ˆ γ − 1)2 . 2ˆ γ 2 + γˆ 3 /ˆ µ

¯ ≤ 0. But limn P (X ¯ ≤ 0) = 0 since Note that Wn is not deﬁned when X µ > 0. Exercise 48 (#6.100). Suppose that X = (X1 , ..., Xk ) has the multinomial distribution with a known size n and an unknown probability vector (p1 , ..., pk ). Consider the problem of testing H0 : (p1 , ..., pk ) = (p01 , ..., p0k ) versus H1 : (p1 , ..., pk ) = (p01 , ..., p0k ), where (p01 , ..., p0k ) is a known probability vector. Find the forms of Wald’s test and Rao’s score test. Solution. The MLE of θ = (p1 , ..., pk−1 ) is θˆ = (X1 /n, ..., Xk−1 /n). The Fisher information about θ is In (θ) = n[D(θ)]−1 +

n τ Jk−1 Jk−1 , pk

where D(θ) denotes the (k−1)×(k−1) diagonal matrix whose k−1 diagonal elements are the components of the vector θ and Jk−1 is the (k − 1)-vector of 1’s. Let θ0 = (p01 , ..., p0(k−1) ). Then H0 : θ = θ0 and the Wald’s test statistic is ˆ θˆ − θ0 ) Wn = (θˆ − θ0 )τ In (θ)( 2 ˆ −1 (θˆ − θ0 ) + n [J τ (θˆ − θ0 )]2 = n(θˆ − θ0 )τ [D(θ)] Xk k−1 k (Xj − np0j )2 = , Xj j=1 τ (θˆ − θ0 ) = p0k − Xk /n. Let (θ) be the likelihood using the fact that Jk−1 function. Then

∂ log (θ) Xk Xk−1 Xk X1 . s(θ) = − , ..., − = ∂θ p1 pk pk−1 pk

Note that [In (θ)]−1 = n−1 D(θ) − n−1 θθτ and θτ s(θ) =

k−1 j=1

Xj −

Xk pk

= n − Xk −

Xk npk − Xk (1 − pk ) = . pk pk

Chapter 6. Hypothesis Tests

299

Also, [s(θ)]τ D(θ)s(θ) =

k−1

pj

j=1

=

k−1

pj

j=1

+2

k−1

Xj Xk − pj pk Xj −n pj

pj

j=1

=

=

2

2 +

k−1

pj

j=1

Xj −n pj

n−

Xk −n pk

Xk pk

2

k−1

(Xj − npj )2 (Xk − npk )2 + (1 − pk ) pj p2k j=1

Xk + 2[(n − Xk ) − n(1 − pk )] n − pk k−1 j=1

(Xj − npj )2 (Xk − npk )2 + [θτ s(θ)]2 − pj pk

2(Xk − npk )2 pk k (Xj − npj )2 +

=

pj

j=1

+ [θτ s(θ)]2 .

Hence, Rao’s score test statistic is Rn = [s(θ0 )]τ [In (θ0 )]−1 s(θ0 ) = n−1 [s(θ0 )]τ D(θ0 )s(θ0 ) − n−1 [θ0τ s(θ0 )]2 =

k (Xj − np0j )2 . np0j j=1

Exercise 49 (#6.101). Let A and B be two diﬀerent events in a probability space related to a random experiment. Suppose that n independent trials of the experiment are carried out and the frequencies of the occurrence of the events are given in the following 2 × 2 contingency table: B Bc

A X11 X21

Ac X12 X22

Consider testing H0 : P (A) = P (B) versus H1 : P (A) = P (B). (i) Derive the likelihood ratio λ and the limiting distribution of −2 log λ

300

Chapter 6. Hypothesis Tests

under H0 . (ii) Find the forms of Wald’s test and Rao’s score test. Solution. Let pij = E(Xij /n), i = 1, 2, j = 1, 2, and θ = (p11 , p12 , p21 ) be the parameter vector (p22 = 1 − p11 − p12 − p21 ). The likelihood function is proportional to n−X11 −X12 −X21 11 X12 X21 (θ) = pX . 11 p12 p21 (1 − p11 − p12 − p21 )

Note that H0 is equivalent to H0 : p21 = p12 . (i) The MLE of θ is θˆ = n−1 (X11 , X12 , X21 ). Under H0 , The MLE of p11 is still X11 /n, but the MLE of p12 = p21 is (X12 + X21 )/(2n). Then λ=

[(X12 + X21 )/2]X12 +X21 . X12 X21 X12 X21

Note that there are two unknown parameters under H0 . By Theorem 6.5 in Shao (2003), under H0 , −2 log λ →d χ21 . (ii) Let R(θ) = p12 − p21 . Then C(θ) = ∂R/∂θ = (0, 1, −1). The Fisher information matrix about θ is ⎞ ⎛ ⎞ ⎛ −1 1 1 1 0 0 p11 n ⎝ In (θ) = n ⎝ 0 1 1 1 ⎠ p−1 0 ⎠+ 12 p 22 −1 1 1 1 0 0 p21 with

⎛

[In (θ)]−1

p11 1 = ⎝ 0 n 0

0 p12 0

⎞ 0 1 0 ⎠ − θθτ . n p21

Therefore, Wald’s test statistic is ˆ τ {[C(θ)] ˆ τ [In (θ)] ˆ −1 C(θ)} ˆ −1 R(θ) ˆ Wn = [R(θ)] 2 n(X12 − X21 ) = . n(X12 + X21 ) − (X12 − X21 )2 Note that

⎛ s(θ) =

∂ log (θ) ⎜ =⎜ ⎝ ∂θ

X11 p11

−

X22 1−p11 −p12 −p21

X12 p12

−

X22 1−p11 −p12 −p21

X21 p21

−

X22 1−p11 −p12 −p21

⎞ ⎟ ⎟ ⎠

and, hence, s(θ) evaluated at θ˜ = n−1 (X11 , (X12 + X21 )/2, (X12 + X21 )/2) is ⎛ ⎞ 0 ⎜ ⎟ ˜ = ⎜ n(X12 −X21 ) ⎟ . s(θ) ⎝ X12 +X21 ⎠ n(X21 −X12 ) X12 +X21

Chapter 6. Hypothesis Tests

301

Therefore, Rao’s score test statistic is 2 ˜ τ [In (θ)] ˜ −1 s(θ) ˜ = (X12 − X21 ) . Rn = [s(θ)] X12 + X21

Exercise 50 (#6.102). Consider the r × c contingency table 1 2 .. . r

1 X11 X21 .. . Xr1

··· ··· ··· .. . ···

2 X12 X22 .. . Xr2

c X1c X2c .. . Xrc

with unknown pij = E(Xij )/n, where n is a known positive integer. (i) Let A1 , ..., Ac be disjoint events with A1 ∪ · · · ∪ Ac = Ω (the sample space of a random experiment), and let B1 , ..., Br be disjoint events with B1 ∪ · · · ∪ Br = Ω. Suppose that Xij is the frequency of the occurrence of Aj ∩ Bi in n independent trials of the experiment. Derive the χ2 goodnessof-ﬁt test for testing independence of {A1 , ..., Ac } and {B1 , ..., Br }, i.e., H0 : pij = pi· p·j for all i, j

versus

H1 : pij = pi· p·j for some i, j,

where pi· = P (Bi ) and p·j = P (Aj ), i = 1, ..., r, j = 1, ..., c. (ii) Let (X1j , ..., Xrj ), j = 1, ..., c, be c independent random vectors having the multinomial distributions with sizes nj and unknown probability vectors (p1j , ..., prj ), j = 1, ..., c, respectively. Consider the problem of testing whether c multinomial distributions are the same, i.e., H0 : pij = pi1 for all i, j

versus

H1 : pij = pi1 for some i, j.

Show that the χ2 goodness-of-ﬁt test is the same as that in (i). Solution. (i) Using the Lagrange multiplier method, we can obtain the MLE of pij ’s by maximizing ⎛ ⎞ r r c c Xij log pij − λ ⎝ pij − 1⎠ , i=1 j=1

i=1 j=1

where λ is the Lagrange multiplier. Thus, the MLE of pij is Xij /n. Under H0 , the MLE’s of pi· ’s and p·j ’s can be obtained by maximizing ⎛ ⎞ r r c c Xij (log pi· + log p·j ) − λ1 pi· − 1 − λ2 ⎝ p·j − 1⎠ , i=1 j=1

i=1

j=1

302

Chapter 6. Hypothesis Tests

where λ1 and λ2 are the Lagrange multipliers. Thus, the MLE of pi· is ¯ i· = c Xij /n and the MLE of p·j is X ¯ ·j = r Xij /n. Hence, the X j=1 i=1 χ2 statistic is r c ¯ i· X ¯ ·j )2 (Xij − nX χ2 = . ¯ i· X ¯ ·j nX i=1 j=1 The number of free parameters is rc − 1. Under H0 , the number of free parameters is r − 1 + c − 1 = r + c − 2. The diﬀerence of the two is rc − r − c + 1 = (r − 1)(c − 1). By Theorem 6.9 in Shao (2003), under H0 , χ2 →d χ2(r−1)(c−1) . Therefore, the χ2 goodness-of-ﬁt test rejects H0 when χ2 > χ2(r−1)(c−1),α , where χ2(r−1)(c−1),α is the (1 − α)th quantile of the chi-square distribution χ2(r−1)(c−1) . (ii) Since (X1j , ..., Xrj ) has the multinomial distribution with size c nj and probability vector (p1j , ..., prj ), the MLE of pij is Xij /n. Let Yi = j=1 Xij . Under H0 , (Y1 , ..., Yr ) has the multinomial distribution with size n and ¯ i· = probability vector (p11 , ..., pr1 ). Hence, the MLE of pi1 under H0 is X ¯ Yi /n. Note that nj = nX·j , j = 1, ..., c. Hence, under H0 , the expected ¯ i· X ¯ ·j . Thus, the χ2 (i, j)th frequency estimated by the MLE under H0 is nX statistic is the same as that in part (i) of the solution. The number of free parameters in this case is c(r−1). Under H0 , the number of free parameters is r − 1. The diﬀerence of the two is c(r − 1) − (r − 1) = (r − 1)(c − 1). Hence, χ2 →d χ2(r−1)(c−1) under H0 and the χ2 goodness-of-ﬁt test is the same as that in (i). Exercise 51 (#6.103). In Exercise 50(i), derive Wald’s test and Rao’s score test statistics. Solution. For a set {aij , i = 1, ..., r, j = 1, ..., c, (i, j) = (r, c)} of rc − 1 numbers, we denote vec(aij ) to be the (rc − 1)-vector whose components are aij ’s and D(aij ) to be the diagonal matrix whose diagonal elements are the components of vec(aij ). Let θ = vec(pij ), J be the (rc − 1)-vector of 1’s, and (θ) be the likelihood function. Then,

∂ log (θ) Xrc Xij s(θ) = − = vec J ∂θ pij 1 − Jτ θ and

∂ 2 log (θ) = −D ∂θ∂θτ

Xij p2ij

−

Xrc JJ τ . (1 − J τ θ)2

Since E(Xij ) = npij , the Fisher information about θ is −1 τ In (θ) = nD(p−1 ij ) + nprc JJ

with

[In (θ)]−1 = n−1 D(pij ) − n−1 θθτ .

Chapter 6. Hypothesis Tests

303

Let p˜ij be the MLE of pij under H0 and θ˜ = vec(˜ pij ). Then Rao’s score test statistic is ˜ τ [D(˜ ˜ Rn = n−1 [s(θ)] pij ) − θ˜θ˜τ ]s(θ). Note that ˜ = θ˜τ s(θ)

Xij −

(i,j) =(r,c)

Xrc p˜ij 1 − J τ θ˜

= n − Xrc −

Xrc J τ θ˜ Xrc , =n− p˜rc 1 − J τ θ˜

¯ r· X ¯ ·c . Also, where p˜rc = 1 − J τ θ˜ = X

2 Xij Xrc τ ˜ ˜ [s(θ)] D(˜ pij )s(θ) = p˜ij − p˜ij p˜rc (i,j) =(r,c)

2

Xij = p˜ij − n + (1 − p˜rc ) n − p˜ij (i,j) =(r,c)

Xij Xrc +2 p˜ij −n n− p˜ij p˜rc (i,j) =(r,c)

2

Xij = p˜ij − n + (1 + p˜rc ) n − p˜ij (i,j) =(r,c)

Xrc p˜rc

Xrc p˜rc

2

2 .

Hence,

2 2 Xij Xrc − n + p˜rc n − p˜ij p˜rc (i,j) =(r,c)

2 r c Xij 1 = p˜ij −n n i=1 j=1 p˜ij

Rn =

=

1 n

p˜ij

r c ¯ i· X ¯ ·j )2 (Xij − nX , ¯ i· X ¯ ·j nX i=1 j=1

¯ i· X ¯ ·j (part (i) of the solution to Exercise 50). using the fact that p˜ij = X 2 Hence, Rn is the same as χ in part (i) of the solution to Exercise 50. Let η(θ) be the (r − 1)(c − 1)-vector obtained by deleting components prj and pic , j = 1, ..., c − 1, i = 1, ..., r − 1, from the vector θ and let ζ(θ) be η(θ) with pij replaced by pi· p·j , i = 1, ..., r − 1, j = 1, ..., c − 1. Let ∂ζ θˆ = vec(Xij /n), the MLE of θ, R(θ) = η(θ) − ζ(θ), and C(θ) = ∂η ∂θ − ∂θ . Then, Wald’s test statistic is ˆ τ {[C(θ)] ˆ τ [In (θ)] ˆ −1 C(θ)} ˆ −1 R(θ). ˆ Wn = [R(θ)]

304

Chapter 6. Hypothesis Tests

Exercise 52 (#6.105). Let (X1 , ..., Xn ) be a random sample of binary random variables with θ = P (X1 = 1). (i) Let Π(θ) be the beta distribution with parameter (a, b). When Π is used as the prior for θ, ﬁnd the Bayes factor and the Bayes test for H0 : θ ≤ θ0 versus H1 : θ > θ0 . (ii) Let π0 I[θ0 ,∞) (θ) + (1 − π0 )Π(θ) be the prior cumulative distribution, where Π is the same as that in (i) and π0 ∈ (0, 1) is a constant. Find the Bayes factor and the Bayes test for H0 : θ = θ0 versus H1 : θ = θ0 . Solution. (i) Under prior Π, the posterior of θ is the beta distribution n with parameter (a + T, b + n − T ), where T = i=1 Xi . Then, the posterior probability of the set (0, θ0 ] (the null hypothesis H0 ) is Γ(a + b + n) p(T ) = Γ(a + T )Γ(b + n − T )

0

θ0

ua+T −1 (1 − u)b+n−T −1 du.

Hence, the Bayes test rejects H0 if and only if p(T ) < factor is posterior odds ratio p(T )[1 − π(0)] = , prior odds ratio [1 − p(T )]π(0) where π(0) =

Γ(a + b) Γ(a)Γ(b)

0

θ0

1 2

and the Bayes

ua−1 (1 − u)b−1 du

is the prior probability of the set (0, θ0 ]. (ii) Let θT (1 − θ)n−T dΠ(θ) m1 (T ) = θ =θ0

Γ(a + b) 1 a+T −1 θ (1 − θ)b+n−T −1 dθ Γ(a)Γ(b) 0 Γ(a + b)Γ(a + T )Γ(b + n − T ) = . Γ(a + b + n)Γ(a)Γ(b) Since the likelihood function is (θ) = Tn θT (1 − θ)n−T , the posterior probability of the set {θ0 } (the null hypothesis H0 ) is (θ)d[π0 I[θ0 ,∞) (θ) + (1 − π0 )Π(θ)] 0 p(T ) = θ=θ (θ)d[π0 I[θ0 ,∞) (θ) + (1 − π0 )Π(θ)] =

=

π0 θ0T (1

π0 θ0T (1 − θ0 )n−T . − θ0 )n−T + (1 − π0 )m1 (T )

Hence, the Bayes test rejects H0 if and only if p(T ) < )(1−π0 ) factor is p(T [1−p(T )]π0 .

1 2

and and the Bayes

Chapter 6. Hypothesis Tests

305

Exercise 53 (#6.114). Let (X1 , ..., Xn ) be a random sample from a continuous distribution F on R, Fn be the empirical distribution, Dn+ = supx∈R [Fn (x)−F (x)], and Dn− = supx∈R [F (x)−Fn (x)]. Show that Dn− (F ) and Dn+ (F ) have the same distribution and, for t ∈ (0, 1), P

Dn+ (F )

n

≤ t = n! i=1

un−i+2

max{0, n−i+1 −t} n

du1 · · · dun .

Proof. Let X(i) be the ith order statistic, i = 1, ..., n, X(0) = −∞, and X(n+1) = ∞. Note that i − F (x) Dn+ (F ) = max sup 0≤i≤n X(i) ≤x

i P (Dn+ (F ) ≤ t) = P max − U(i) ≤ t 0≤i≤n n

i = P U(i) ≥ − t, i = 1, ..., n n n un−i+2 du1 · · · dun . = n! i=1

max{0, n−i+1 −t} n

306

Chapter 6. Hypothesis Tests

Exercise 54 (#6.116). Let (X1 , ..., Xn ) be a random sample from a continuous distribution F on R, Fn be the empirical distribution, and Cn (F ) = [Fn (x) − F (x)]2 dF (x). Show that the distribution of Cn (F ) does not vary with F . Solution. Note that 1 [Fn (F −1 (t)) − t]2 dt Cn (F ) = 0

&

'2 n 1 −1 = I (Xi ) − t dt n i=1 (−∞,F (t)] 0 '2 1& n 1 I(−∞,t] (F (Xi )) − t dt, = n i=1 0

1

where the last equality follows from the fact that Xi ≤ F −1 (t) if and only if F (Xi ) ≤ t. Since (F (X1 ), ..., F (Xn )) is a random sample from the uniform distribution on (0, 1), the distribution of Cn (F ) does not depend on F . Exercise 55 (#6.123). Let θˆn be an estimator of a real-valued parameter −1/2 ˆ θ such that Vn (θn − θ) →d N (0, 1) for any θ and let Vˆn be a consistent estimator of Vn . Suppose that Vn → 0. −1/2 ˆ (θn − θ0 ) > zα is a con(i) Show that the test with rejection region Vˆn sistent asymptotic level α test for testing H0 : θ ≤ θ0 versus H1 : θ > θ0 , where zα is the (1 − α)th quantile of N (0, 1). (ii) Apply the result in (i) to show that the one-sample one-sided t-test for the testing problem in (i) is a consistent asymptotic level α test. −1/2 ˆ Solution. (i) Under H1 : θ > θ0 , Vˆn (θn − θ) →d N (0, 1). Therefore, −1/2 ˆ the test with rejection region Vˆn (θn − θ0 ) > zα is an asymptotic level α test. Also, P Vˆn−1/2 (θˆn − θ0 ) > zα = P Vˆn−1/2 (θˆn − θ) > zα − Vˆn−1/2 (θ − θ0 ) → 1 −1/2 as n → ∞, since Vˆn (θ − θ0 ) →p ∞. Hence, the test is consistent. (ii) Let (X1 , ..., Xn ) be a random sample from a population with ﬁnite mean θ and variance σ 2 . For testing H0 : θ ≤ θ0 versus 1 : θ > θ0 , √ H ¯ − θ0 )/S > the one-sample t-test rejects H0 if and only if t(X) = n(X ¯ and S 2 are the sample mean and variance and tn−1,α tn−1,α , where X is the (1 − quantile of the t-distribution tn−1 . By the central limit √ α)th ¯ − θ) →d N (0, σ 2 ). Hence Vn = σ 2 /n → 0. By the law of theorem, n(X large numbers, S 2 →p σ 2 . Hence Vˆn = S 2 /n is a consistent estimator of Vn . Note that the t-distribution tn−1 converges to N (0, 1). Then, by the

Chapter 6. Hypothesis Tests

307

result in Exercise 28 of Chapter 5, limn tn−1,α = zα . From the result in (i), the one-sample t-test is a consistent asymptotic level α test. Exercise 56 (#6.124). Let (X1 , ..., Xn ) be a random sample from the gamma distribution with shape parameter θ and γ, where n scale parameter n θ > 0 and γ > 0 are unknown. Let Tn = n i=1 Xi2 /( i=1 Xi )2 . Show how to use Tn to obtain an asymptotic level α and consistent test for testing H0 : θ = 1 versus H1 : θ = 1. Solution. From the central limit theorem, 1 n √ E(X1 ) Xi i=1 n n − →d N2 (0, Σ) , n 1 2 E(X12 ) i=1 Xi n where

Σ=

E(X12 ) − [E(X1 )]2 E(X13 ) − E(X1 )E(X12 ) 3 2 E(X1 ) − E(X1 )E(X1 ) E(X14 ) − [E(X12 )]2

.

Since the moment generating function of the gamma distribution is g(t) = (1 − γt)−θ , E(X1 ) = g (0) = θγ, E(X12 ) = g (0) = θ(θ + 1)γ 2 , E(X13 ) = g (0) = θ(θ + 1)(θ + 2)γ 3 , and

E(X14 ) = g (0) = θ(θ + 1)(θ + 2)(θ + 3)γ 4 .

Hence,

Σ=

θγ 2 2θ(θ + 1)γ 3

2θ(θ + 1)γ 3 2θ(θ + 1)(2θ + 3)γ 4

.

Note that T =

n 1 2 i=1 Xi n 1 n 2 i=1 Xi n

=h

1 1 2 Xi , X n i=1 n i=1 i n

n

with h(x, y) = y/x2 , H(x, y) =

∂h ∂x ∂h ∂y

=

− x2y3

,

1 x2

and

H(E(X1 ), E(X12 ))

2

= H(θγ, θ(θ + 1)γ ) =

− 2(θ+1) θ2 γ 1 θ2 γ 2

.

308

Chapter 6. Hypothesis Tests

By the δ-method, √

n[T − h(E(X1 ), E(X12 ))] →d N (0, σ 2 ),

where σ 2 = [H(θγ, θ(θ + 1)γ 2 )]τ ΣH(θγ, θ(θ + 1)γ 2 ) 4(θ + 1)2 2(θ + 1)(2θ + 3) 8(θ + 1)2 = + − 3 θ θ3 θ3 2(θ + 1) = . θ3 Note that h(E(X1 ), E(X12 )) = h(θγ, θ(θ + 1)γ 2 ) =

θ(θ + 1)γ 2 1 =1+ . θ2 γ 2 θ

Combining all the results, we obtain that

1 −1/2 →d N (0, 1) T −1− Vn θ with Vn = 2(θ + 1)/(θ3 n). From the asymptotic normality of T , 1 →p θ. T −1 Hence, a consistent estimator of Vn is 1 +1 2 T −1 2T (T − 1)2 = Vˆn = . n n (T −1)3 From Theorem 6.12 in Shao (2003), an asymptotic level α and consistent test for H0 : θ = 1 versus H1 : θ = 1 rejects H0 if and only if Vˆn−1 (T − 2)2 > χ21,α , which is the same as

n(T − 2)2 > χ21,α , 2T (T − 1)2

where χ21,α is the (1 − α)th quantile of the chi-square distribution χ21 .