# Filtering a Stationary Time Series

In the first part of the MAT8181 course, on linear (univariate) time series, I forgot to mention an important theorem. Let $(X_t)_{t\in\mathbb{Z}}$ be a stationary time series, and $(a_t)_{t\in\mathbb{Z}}$ a sequence of real numbers such that

$\sum_{t\in\mathbb{Z}} \vert a_t\vert < \infty$

then the time series $(Y_t)_{t\in\mathbb{Z}}$ defined as

$Y_t=\sum_{k\in\mathbb{Z}} a_k X_{t-k}= \left( \sum_{k\in\mathbb{Z}} a_k L^k\right) X_t$

is a stationary time series. Further, one can get easily that

$\gamma_Y(h)=\text{cov}(Y_t,Y_{t-k})=\sum_{r,s\in\mathbb{Z}} a_r a_s \underbrace{\gamma_X(h+r-s)}_{\text{cov}(X_{t-r},X_{t-s+h})}$

This result can be used, if necessary in the exercises (that might save some time actually). I did not include this property in my notes because it is a bit technical to establish that this sum exists, and that the time series is stationary. It is rather simple with the spectral density (since $f_Y(\omega)=\vert A(e^{i\omega})\vert^2f_X(\omega)$ where $A(\cdot)$ stands for the filter generating function), but I did not mention the spectral density since it requires some knowledge on Fourier analysis…

# Causal Autoregressive Time Series

In the MAT8181 graduate course on Time Series, we will discuss (almost) only causal models. For instance, with $AR(1)$,

$X_t=\phi X_{t-1}+\varepsilon_t$

with some white noise $(\varepsilon_t)$, those models are obtained when $\vert \phi\vert <1$. In that case, we’ve seen that $(\varepsilon_t)$ was actually the innovation process, and we can write

$X_t = \sum_{h=0}^{+\infty} \phi^h \varepsilon_{t-h}$

which is actually a mean-square convergent series (using simple Analysis arguments on series). From that expression, we can easily see that $(X_t)$ is stationary, since $\mathbb{E}(X_t)=0$ (which does not depend on $t$) and

$\text{cov}(X_t,X_{t-h})=\frac{\phi^h}{1-\phi^2}\sigma^2$(which does not depend on $t$).

Consider now the case where $\vert \phi\vert >1$. Clearly, we have some problem here, since

$X_t = \sum_{h=0}^{+\infty} \phi^h \varepsilon_{t-h}$

cannot be defined (the series does not converge, in $L^2$). Nevertheless, it is still possible to write

$X_t=\frac{1}{\phi} X_{t{\color{Red} +1}}{\color{Red} -\frac{1}{\phi}}\varepsilon_{t{\color{Red} +1}}$But it is possible to iterate (as in the previous case) and write

$X_t = \sum_{h={\color{Red} 1}}^{+\infty} \frac{-1}{\phi^h} \varepsilon_{t{\color{Red} +h}}$

which is actually well defined. And in that case, the sequence of random variables $(X_t)$ obtained from this equation is the unique stationary solution of the recursive equation $X_t=\phi X_{t-1}+\varepsilon_t$. This might be confusing, but the thing is this solution should not be confused with the usual non-stationary solution of $X_t=\phi X_{t-1}+\varepsilon_t$ obtained from $X_0$. As in the code writen to generate a time series, from some starting value $X_0$ in the previous post.

Now, let us spent some time with this stationary time series, considered as unatural in Brockwell and Davis (1991). One point is that, in the previous case (where $\vert \phi\vert <1$) $(\varepsilon_t)$ was the innovation process. So variable $X_t$ was not correlated with the future of the noise, $\sigma\{\varepsilon_{t+1},\varepsilon_{t+2},\cdots\}$. Which is not the case when $\vert \phi\vert >1$.

All that looks nice, if you’re willing to understand thing at some theoretical level. What does all that mean from a computational perspective ? Consider some white noise (this noise actually does exist whatever you want to define, based on that time series)

> n=10000
> e=rnorm(n)
> plot(e,type="l",col="red")

> phi=.8
> X=rep(0,n)
> for(t in 2:n) X[t]=phi*X[t-1]+e[t]

The time series – the latest 1,000 observations – looks like

Now, if we use the cumulated sum of the noise,

> Y=rep(0,n)
> for(t in 2:n) Y[t]=sum(phi^((0:(t-1)))*e[t-(0:(t-1))])

we get

Which is exactly the same process ! This should not surprise us because that’s what the theory told us. Now, consider the problematic case, where $\vert \phi\vert >1$

> phi=1.1
> X=rep(0,n)
> for(t in 2:n) X[t]=phi*X[t-1]+e[t]

Clearly, that series is non-stationary (just look at the first 1,000 values)

Now, if we look at the series obtained from the cumulated sum of future values of the noise

> Y=rep(0,n)
> for(t in 1:(n-1)) Y[t]=sum((1/phi)^((1:(n-t)))*e[t+(1:(n-t))])

We get something which is, actually, stationary,

So, what is this series exactly ? If you look that the autocorrelation function,

> acf(Y)

we get the autocorrelation function of a (stationary) $AR(1)$ process,

> acf(Y)[1]

Autocorrelations of series ‘Y’, by lag

1
0.908

> 1/phi
[1] 0.9090909

Observe that there is a white noise – call it $(\eta_t)$ – such that

$X_t=\frac{1}{\phi}X_{t-1}+\eta_t$

This is what we call the canonical form of the stationary process $(X_t)$.