Tag Archives: Cauchy

Stationarity of ARCH processes

06/04/2014 Arthur Charpentier Leave a comment

In the context of AR(1) processes, we spent some time to explain what happens when $\phi$ is close to 1.

if $\phi<1$ the process is stationary,
if $\phi=1$ the process is a random walk
if $\phi>1$ the process will explode

Again, random walks are extremely interesting processes, with puzzling properties. For instance,

$\text{Var}(X_{t+h}\vert X_t)\sim \sigma^2 h\rightarrow\infty$

as $h\rightarrow\infty$ , and the process will cross the x-axis an infinite number of times…

Recently, in the MAT8181 course, we studied carefully properties of the ARCH(1) process, especially when $\alpha\sim 1$ . And again, what we get might be puzzling.

Consider some ARCH(1) process $(\varepsilon_t)$ , with a Gaussian noise, i.e.

$\varepsilon_t=\sigma_t\cdot \eta_t$

where

$\sigma^2_t=\omega+\alpha \varepsilon_{t-1}^2$

and $(\eta_t)$ is a sequence of i.i.d. $\mathcal{N}(0,1)$ variables. Here both $\omega$ and $\alpha$ have to be positive.

Recall that $\mathbb{E}(\varepsilon_t)=0$ since $\mathbb{E}(\eta_t)=0$ . Further

$\text{var}(\varepsilon_t)=\mathbb{E}(\varepsilon_t^2)=\omega+\alpha \mathbb{E}(\varepsilon_{t-1}^2)$

since $\mathbb{E}(\eta_t^2)=1$ , so the variance exists, and is constant only if $\alpha\in(0,1)$ , and in that case

$\sigma^2=\text{var}(\varepsilon_t)=\frac{\omega}{1-\alpha}\in[0,+\infty)$

Further, if $3\alpha^2<1$ , then the fourth moment can be obtained,

$\mathbb{E}(\varepsilon_t^4)=\frac{3\omega^2}{1-\alpha^2}\frac{1-\alpha^2}{1-3\alpha^2}$

since $\mathbb{E}(\eta_t^4)=3$ . Now, if we get back on the property obtained while studying the variance, what does that mean if $\alpha=1$ , or $\alpha>1$ ?

If we look at simulations, we can generate an ARCH(1) process with $\alpha=2$ for instance.

> n=600
> a=2
> w=0.2
> set.seed(1)
> eta=rnorm(n)
> epsilon=rnorm(n)
> sigma2=rep(w,n)
> for(t in 2:n){
+ sigma2[t]=w+a*epsilon[t-1]^2
+ epsilon[t]=eta[t]*sqrt(sigma2[t])
+ }
> plot(epsilon,type="l")

In order to understand what’s going on, we should keep in mind that, what we good is that $\alpha$ has to lie in $(0,1)$ to be able to compute the second moment of $(\varepsilon_t)$ . But it is possible to have a stationary process with infinite variance. And actually, this is what we have here.

Write

$\sigma^2_t=\omega+\alpha \varepsilon_{t-1}^2 = \omega+[\alpha \eta_{t-1}^2] \sigma_{t-1}^2$

and them, iterate

$\sigma^2_t= \omega+[\alpha \eta_{t-1}^2] \left( \omega+[\alpha \eta_{t-2}^2] \sigma_{t-2}^2\right)$

and iterate again, and again, and again…

$\sigma^2_t= \underbrace{\omega\left[ 1+\sum_{i=1}^h[\alpha \eta_{t-1}^2]\cdots [\alpha \eta_{t-i}^2] \right]}_{\Sigma_t(h)} +[\alpha \eta_{t-1}^2]\cdots[\alpha \eta_{t-h-1}^2]\sigma_{t-h-1}^2$

where

$\Sigma_t(h)=\sum_{i=1}^h\underbrace{[\alpha \eta_{t-1}^2]\cdots [\alpha \eta_{t-i}^2]}_{u_i}$

Here, we have a sum of positive terms, and we can use the so-called Cauchy rule: define

$\lambda=\text{limsup}\{ u_n^{1/n}\}$

then, if $\lambda<1$ , the series $\sum u_n$ converges. Here,

$u_n^{1/n}=\left[[\alpha \eta_{t-1}^2]\cdots [\alpha \eta_{t-n}^2]\right]^{1/n}$

which can also be written

$u_n^{1/n}=\exp\left[\frac{1}{n}\sum_{i=1}^n\log[\alpha \eta_{t-i}^2]\right]$

and from the law of large numbers, since we have here a sum of i.i.d. terms,

$u_n^{1/n}\rightarrow\exp\left[\mathbb{E}(\log[\alpha \eta^2])\right]$

So, if $\exp\left[\mathbb{E}(\log[\alpha \eta^2])\right]<1$ , then $\Sigma_t(h)$ will have a limit when $h$ goes to infinity.

The condition above can be written

$\gamma=\mathbb{E}(\log[\alpha \eta^2]<0$

which is called Lyapunov coefficient.

The equation

$\exp\left[\mathbb{E}(\log[\alpha \eta^2])\right]=\alpha\exp\left[-\mathbb{E}(\log[\eta^2])\right]<1$

is a condition on $\alpha$ .

In the case where $\eta\sim\mathcal{N}(0,1)$ , the numerical value of this upper bound is 3.56.

> 1/exp(mean(log(rnorm(1e7)^2)))
[1] 3.562517

In that case ( $\gamma<0$ ), the variance may be infinite, but the series is stationary. On the other hand, if $\gamma>0$ , then $\varepsilon_t^2$ will go to infinity almost surely, as $t$ goes to infinity.

But in order to observe this difference, we need a lot of observations. For instance, with $\alpha=0.8$ ,

and $\alpha=1.2$ ,

we can easily see a difference. I do not say that it’s easy to see that the distribution above has an infinite variance, but still. Actually, if we consider Hill’s plot on the series above, on the tails of positive $\varepsilon_t$ ‘s

> library(evir)
> hill(epsilon)

or on the tails of negative $\varepsilon_t$ ‘s

> hill(-epsilon)

we can see that the tail index is (strictly) smaller than 2 (meaning that the moment of order 2 does not exist).

Why is it puzzling? Maybe because here, $(\varepsilon_t)$ is not weakly stationary (in the $L^2$ sense), but it is strongly stationary. Which is not the usual way weak and strong are related. This might be why we will not call this strong stationarity, but strict.

MAT8181, Time Series

Defining Properly MA(∞) Time Series

28/01/2014 Arthur Charpentier Leave a comment

In order to properly define $https://latex.codecogs.com/gif.latex?MA(\infty)$ series, we need to get back on some properties of infinite sequences, as briefly mentioned yesterday in the MAT8181 course. Consider some sequence $https://latex.codecogs.com/gif.latex?(a_i)_{i\in\mathbb{N}}$ . The sequence is said to be summable if

$https://latex.codecogs.com/gif.latex?S_n=\sum_{i=0}^n%20a_i$

is convergent, i.e. if the limit of $https://latex.codecogs.com/gif.latex?S_n$ exists when $https://latex.codecogs.com/gif.latex?n\rightarrow\infty$ .

From Cauchy criterion, $https://latex.codecogs.com/gif.latex?\sum%20a_i$ converges if and only if for each $https://latex.codecogs.com/gif.latex?\eta%3E0$ , there is $https://latex.codecogs.com/gif.latex?m\in\mathbb{N}$ for which

$https://latex.codecogs.com/gif.latex?\vert%20a_i+a_{i+1}+\cdots+a_{j-1}+a_j\vert%3C\eta$

when $https://latex.codecogs.com/gif.latex?%20i,j%3Em$ . The sequence $https://latex.codecogs.com/gif.latex?(a_i)_{i\in\mathbb{N}}$ is said to be absolutely summable if

$https://latex.codecogs.com/gif.latex?%20\sum_{i=0}^\infty%20\vert%20a_i\vert%20%3C\infty$

and square-summable if

$https://latex.codecogs.com/gif.latex?%20\sum_{i=0}^\infty%20a_i^2%20%3C\infty$

Observe that absolute summability will imply square summability (since for $https://latex.codecogs.com/gif.latex?j$ ‘s large enough $https://latex.codecogs.com/gif.latex?%20\vert%20a_j\vert%20\leq1$ , and then $https://latex.codecogs.com/gif.latex?%20a_j^2\leq\vert%20a_j\vert$ )

Consider now some $https://latex.codecogs.com/gif.latex?MA(\infty)$ time series

$https://latex.codecogs.com/gif.latex?%20X_t=\sum_{h=0}^\infty%20\theta_h%20\varepsilon_{t-h}$

If the sequence of coefficients $https://latex.codecogs.com/gif.latex?%20(\theta_i)$ is square-summable, then

$https://latex.codecogs.com/gif.latex?%20S_T%20=%20\sum_{h=0}^T%20\theta_h%20\varepsilon_{t-h}$

converges in $https://latex.codecogs.com/gif.latex?%20L_2$ to some random varible as $https://latex.codecogs.com/gif.latex?%20T\rightarrow\infty$ . This can be proved easily using Cauchy criteria, in the sense that for any $https://latex.codecogs.com/gif.latex?\eta%3E0$ , there is a $https://latex.codecogs.com/gif.latex?%20T$ large enough such that, for any $https://latex.codecogs.com/gif.latex?%20h$ ,

$\underbrace{\mathbb{E}\left(\sum_{i=T}^h\theta_i \varepsilon_{t-i}\right)^2}_{\sigma^2 \cdot \sum_{i=T}^h \theta_i^2} \leq \eta$

In that case, if the sequence of coefficients $https://latex.codecogs.com/gif.latex?%20(\theta_i)$ is square-summable, then $https://latex.codecogs.com/gif.latex?%20(X_t)$ is stationary (in the $https://latex.codecogs.com/gif.latex?%20L_2$ sense) since the process is centered, and

$https://latex.codecogs.com/gif.latex?\gamma(h)=\sigma^2%20\cdot%20\sum_{i=0}^\infty%20\theta_i%20\theta_{i+h}$

for all $https://latex.codecogs.com/gif.latex?h\in\mathbb{N}$ .

Further, ergodicity of the time series, define as the absolute summability of the autocovariance sequence, is obtained when the sequence of coefficients $https://latex.codecogs.com/gif.latex?%20(\theta_i)$ is absolutely summable.

M	T	W	T	F	S	S
						1
2	3	4	5	6	7	8
9	10	11	12	13	14	15
16	17	18	19	20	21	22
23	24	25	26	27	28	29
30

Freakonometrics

Tag Archives: Cauchy

Stationarity of ARCH processes

Defining Properly MA(∞) Time Series

"sendo l'intento mio scrivere cosa utile a chi la intende…"