doi:10.58079/ov0t

Tests, Power and Significance

14/10/2015 Arthur Charpentier 3 Comments

In the mathematical statistics course today, we started talking about tests, and decision rules. To illustrate all the concepts introduced today, we considered the case where we have a sample $\boldsymbol{x}=\{x_1,\cdots,x_n\}$ with $U_i\sim\mathcal{U}([0,\theta])$ . And we want to test

$H_0:\theta\leq \theta_0$ against $H_1:\theta> \theta_0$

In the course, we’ve seen that we could use a test based on the order statistics $x_{n:n}=\max\{x_i\}$ . The test would be

$\psi(\boldsymbol{x})=\boldsymbol{1}_{(c\ ;\ +\infty)}(x_{n:n})$

i.e. if $\psi(\boldsymbol{x})=1$ we choose $H_1$ , and if $\psi(\boldsymbol{x})=0$ , we choose $H_0$ .

From the definition of the first order risk,

$\alpha=\sup_{\theta\in\Theta_0}\left\lbrace{\mathbb E}_{\theta}\lbrack \psi(\boldsymbol{X})\rbrack\right\rbrace={\mathbb E}_{\theta_0}\lbrack \psi(\boldsymbol{X})\rbrack$

we can easily get that

$c=\theta_0\cdot(1-\alpha)^{\frac{1}{n}}$

Thus, the power is then

$p(\theta)=\left(1-\left(\frac{\theta_0}{\theta}\right)^n(1-\alpha)\right)\boldsymbol{1}_{(c\ ;\ +\infty)}(\theta)$

To visualize it, use the following parameters

n=5
alpha=.1
theta0=1

Then

C1=theta0*(1-alpha)^(1/n)
theta=seq(0,2,by=.01)
P1=(1-(theta0/theta)^n*(1-alpha))*(theta>C1)
plot(theta,P1,type="l",lwd=2,col="blue",xlab="",ylab="Power")

Note that, so far, we did never consider the maximum of our sample. Assume that the maximum is $x_{n:n}$ , then we can compute the $p$ -value,

$p=\mathbb{P}(X_{n:n}>x_{n:n})=1-\left(\frac{x_{n:n}}{\theta_0}\right)^n$

Here it is

PV=(1-theta^n)*(theta<=1)
plot(theta,PV,type="l",lwd=2,col="blue",xlab="",ylab="p-value")

Now, why not consider another test, based on the minimum (since we have the distribution of the minimum of a sample from a uniform distribution). The test is the same as before

$\psi(\boldsymbol{x})=\boldsymbol{1}_{(c\ ;\ +\infty)}(x_{1:n})$

but here, the threshold is

$c=\theta_0\cdot (1-\alpha^{\frac{1}{n}})$

The power of the test is here

$\quad p(\theta)=\left(1-\frac{\theta_0}{\theta}(1-\alpha^{\frac{1}{n}})\right)^n \boldsymbol{1}_{( c\ ;\ +\infty)}(\theta)$

This test has the same significance level (by construction), but the power of the test is clearly lower than the one we got using the maximum of our sample, when $\theta\in\Theta_1$

C2=theta0*(1-alpha^(1/n))
P2=(1-(theta0/theta)*(1-alpha^(1/n)))^n*(theta>C2)
lines(theta,P2,type="l",lwd=2,col="red")

Why not consider a test based on $\overline{x}$ ? The problem is that we need the distribution (more specifically the survival distribution) of $\overline{X}$ . We can compute it, numerically. But that might be painful. An alternative is to consider some approximation, based on the central limit theorem, i.e.

$2\overline{X}\sim\mathcal{N}\left(\theta,2^2 \frac{\theta^2}{12n}\right)$

Our test is based on $\psi(\boldsymbol{x})=\boldsymbol{1}_{(c\ ;\ +\infty)}(2\overline{x})$ , and to get the same significance as before, use

$c=\Phi_{\mu,\sigma^2}^{-1}(1-\alpha)=\mu+\sigma \Phi^{-1}(1-\alpha)$

The power of the test is then

$p(\theta)=1-\Phi_{\theta,\sigma^2}(c)\cdot \boldsymbol{1}_{(c,+\infty)}(\theta)$

Here it is

mu=2*(theta0/2)
s2=2^2*(theta0^2/12)/n
C3=qnorm(1-alpha,mu,sqrt(s2))
(P=1-pnorm(C3,theta,sqrt(s2)))*(theta>C3)
lines(theta,P)

Observe here that the test based on the maximum is not more powerful than the one based on the average (I just wonder if it could be due to the Gaussian approximation…).

OpenEdition suggests that you cite this post as follows:
Arthur Charpentier (October 14, 2015). Tests, Power and Significance. Freakonometrics. Retrieved January 23, 2025 from https://doi.org/10.58079/ov0t

M	T	W	T	F	S	S
		1	2	3	4	5
6	7	8	9	10	11	12
13	14	15	16	17	18	19
20	21	22	23	24	25	26
27	28	29	30	31

Freakonometrics

Tests, Power and Significance

3 thoughts on “Tests, Power and Significance”

Leave a Reply Cancel reply

"sendo l'intento mio scrivere cosa utile a chi la intende…"