Where does that 2 come from in the likelihood ratio test?

This afternoon, in class, we’ve seen Wald test, the likelihood-ratio test, and finally the score test. All of them rely on the same idea

and then, use that if   with , we can write

Or – slightly more interesting – if  , then

Then one can get that

Based on that property, we can derive Wald statistics,

that can be visualized below

The score test is a test on the square of the slope

The idea for the likelihood ratio test is to consider

Observe that $\log\mathcal{L}(\widehat{\theta})-\log\mathcal{L}({\theta}_0)$ can be written, using Taylor’s expansion

$[\widehat{\theta}-{\theta}_0]\frac{\partial \log\mathcal{L}(\widehat{\theta})}{\partial \theta}+\frac{1}{2}[\widehat{\theta}-{\theta}_0]^2\frac{\partial^2 \log\mathcal{L}(\widetilde{\theta})}{\partial \theta^2}$

for some $\widetilde{\theta}\in[\widehat{\theta},\theta_0]$. The first term is null, since the maximum likelihood estimator is precisely at the maximum of the (log) likelihood. So

$2[\log\mathcal{L}(\widehat{\theta})-\log\mathcal{L}(\theta_0)]=[\widehat{\theta}-{\theta}_0]^2\frac{\partial^2 \log\mathcal{L}(\widetilde{\theta})}{\partial \theta^2}$

That’s more or less where the 2 comes from. Then observe that

$\frac{\partial^2 \log\mathcal{L}(\widetilde{\theta})}{\partial \theta^2}\sim nI(\theta_0)$

and therefore

$2[\log\mathcal{L}(\widehat{\theta})-\log\mathcal{L}(\theta_0)]\sim n[\widehat{\theta}-{\theta}_0]^2I(\theta_0)\sim \chi^2(k)$

This test will be discussed further next week (since it is related to Neyman-Pearson’s theorem), but also, that result can be used to derive confidence intervals. With a log-likelihood as follows

it is possible to get a confidence interval for the parameter by looking for$\theta$‘s such that

$\log\mathcal{L}(\theta)\geq\log\mathcal{L}(\widehat{\theta})-q$

We will discuss that idea later on, in the context of profile likelihood.

4 thoughts on “Where does that 2 come from in the likelihood ratio test?”

1. Paul says:

Hello, we can’t see well the pictures, maybe because this is an old thread..? Thank you.

1. sorry… hyperlink issues follwing a blog transfer (and the fact that I removed some folders of the previous blog)

I will fix it tonight

This site uses Akismet to reduce spam. Learn how your comment data is processed.