# Stationarity of ARCH processes

In the context of AR(1) processes, we spent some time to explain what happens when $\phi$ is close to 1.

• if $\phi<1$ the process is stationary,
• if $\phi=1$ the process is a random walk
• if $\phi>1$ the process will explode

Again, random walks are extremely interesting processes, with puzzling properties. For instance,

$\text{Var}(X_{t+h}\vert X_t)\sim \sigma^2 h\rightarrow\infty$

as $h\rightarrow\infty$, and the process will cross the x-axis an infinite number of times…

Recently, in the MAT8181 course, we studied carefully properties of the ARCH(1) process, especially when $\alpha\sim 1$. And again, what we get might be puzzling.

Consider some ARCH(1) process $(\varepsilon_t)$, with a Gaussian noise, i.e.

$\varepsilon_t=\sigma_t\cdot \eta_t$

where

$\sigma^2_t=\omega+\alpha \varepsilon_{t-1}^2$

and $(\eta_t)$ is a sequence of i.i.d. $\mathcal{N}(0,1)$ variables. Here both $\omega$ and $\alpha$ have to be positive.

Recall that $\mathbb{E}(\varepsilon_t)=0$ since $\mathbb{E}(\eta_t)=0$. Further

$\text{var}(\varepsilon_t)=\mathbb{E}(\varepsilon_t^2)=\omega+\alpha \mathbb{E}(\varepsilon_{t-1}^2)$

since $\mathbb{E}(\eta_t^2)=1$, so the variance exists, and is constant only if $\alpha\in(0,1)$, and in that case

$\sigma^2=\text{var}(\varepsilon_t)=\frac{\omega}{1-\alpha}\in[0,+\infty)$

Further, if $3\alpha^2<1$, then the fourth moment can be obtained,

$\mathbb{E}(\varepsilon_t^4)=\frac{3\omega^2}{1-\alpha^2}\frac{1-\alpha^2}{1-3\alpha^2}$

since$\mathbb{E}(\eta_t^4)=3$. Now, if we get back on the property obtained while studying the variance, what does that mean if $\alpha=1$, or $\alpha>1$ ?

If we look at simulations, we can generate an ARCH(1) process with $\alpha=2$ for instance.

> n=600
> a=2
> w=0.2
> set.seed(1)
> eta=rnorm(n)
> epsilon=rnorm(n)
> sigma2=rep(w,n)
> for(t in 2:n){
+ sigma2[t]=w+a*epsilon[t-1]^2
+ epsilon[t]=eta[t]*sqrt(sigma2[t])
+ }
> plot(epsilon,type="l")

In order to understand what’s going on, we should keep in mind that, what we good is that $\alpha$ has to lie in $(0,1)$ to be able to compute the second moment of $(\varepsilon_t)$. But it is possible to have a stationary process with infinite variance. And actually, this is what we have here.

Write

$\sigma^2_t=\omega+\alpha \varepsilon_{t-1}^2 = \omega+[\alpha \eta_{t-1}^2] \sigma_{t-1}^2$

and them, iterate

$\sigma^2_t= \omega+[\alpha \eta_{t-1}^2] \left( \omega+[\alpha \eta_{t-2}^2] \sigma_{t-2}^2\right)$

and iterate again, and again, and again…

$\sigma^2_t= \underbrace{\omega\left[ 1+\sum_{i=1}^h[\alpha \eta_{t-1}^2]\cdots [\alpha \eta_{t-i}^2] \right]}_{\Sigma_t(h)} +[\alpha \eta_{t-1}^2]\cdots[\alpha \eta_{t-h-1}^2]\sigma_{t-h-1}^2$

where

$\Sigma_t(h)=\sum_{i=1}^h\underbrace{[\alpha \eta_{t-1}^2]\cdots [\alpha \eta_{t-i}^2]}_{u_i}$

Here, we have a sum of positive terms, and we can use the so-called Cauchy rule: define

$\lambda=\text{limsup}\{ u_n^{1/n}\}$

then, if $\lambda<1$, the series $\sum u_n$ converges. Here,

$u_n^{1/n}=\left[[\alpha \eta_{t-1}^2]\cdots [\alpha \eta_{t-n}^2]\right]^{1/n}$

which can also be written

$u_n^{1/n}=\exp\left[\frac{1}{n}\sum_{i=1}^n\log[\alpha \eta_{t-i}^2]\right]$

and from the law of large numbers, since we have here a sum of i.i.d. terms,

$u_n^{1/n}\rightarrow\exp\left[\mathbb{E}(\log[\alpha \eta^2])\right]$

So, if $\exp\left[\mathbb{E}(\log[\alpha \eta^2])\right]<1$, then $\Sigma_t(h)$ will have a limit when $h$ goes to infinity.

The condition above can be written

$\gamma=\mathbb{E}(\log[\alpha \eta^2]<0$

which is called Lyapunov coefficient.

The equation

$\exp\left[\mathbb{E}(\log[\alpha \eta^2])\right]=\alpha\exp\left[-\mathbb{E}(\log[\eta^2])\right]<1$

is a condition on $\alpha$.

In the case where $\eta\sim\mathcal{N}(0,1)$, the numerical value of this upper bound is 3.56.

> 1/exp(mean(log(rnorm(1e7)^2)))
[1] 3.562517

In that case ($\gamma<0$), the variance may be infinite, but the series is stationary. On the other hand, if $\gamma>0$, then $\varepsilon_t^2$ will go to infinity almost surely, as $t$ goes to infinity.

But in order to observe this difference, we need a lot of observations. For instance, with $\alpha=0.8$,

and $\alpha=1.2$,

we can easily see a difference. I do not say that it’s easy to see that the distribution above has an infinite variance, but still. Actually, if we consider Hill’s plot on the series above, on the tails of positive $\varepsilon_t$‘s

> library(evir)
> hill(epsilon)

or on the tails of negative $\varepsilon_t$‘s

> hill(-epsilon)

we can see that the tail index is (strictly) smaller than 2 (meaning that the moment of order 2 does not exist).

Why is it puzzling? Maybe because here, $(\varepsilon_t)$ is not weakly stationary (in the $L^2$ sense), but it is strongly stationary. Which is not the usual way weak and strong are related. This might be why we will not call this strong stationarity, but strict.