# Linear ‘Prediction’ for AR Time Series

In the exercises for the MAT8181 course, there are two Exercises (16 and 17) about prediction and extrapolation based on MA(1) and AR(1) time series. But before discussing those exercises (I had some request for hints), I wanted to recall the definition of the linear prediction,

$EL(Y\vert \boldsymbol{X})=\boldsymbol{\alpha}^{\star \text{\sffamily T}} \boldsymbol{X}$

where

$\boldsymbol{\alpha}^{\star}=\text{argmin}\left\{\mathbb{E}([Y-\boldsymbol{\alpha}^{\text{\sffamily T}} \boldsymbol{X}]^2)\right\}$

As discussed previously on this blog, we consider here on projection not on $\sigma\{\boldsymbol{X}\}$ (this would be the conditional expectancy) but on the linear subset.

The goal of Exercise 2 was to establish an important result, in the context of Gaussian random vectors. If $\boldsymbol{X}$ is a (multivariate) Gaussian vector, $\mathcal{N}(\boldsymbol{0},\boldsymbol{\Sigma})$, then

$\boldsymbol{\alpha}^{\star}=\boldsymbol{\Sigma}^{-1}\boldsymbol{\gamma}$

where $\boldsymbol{\gamma}$ is the vector $\text{cov}(Y,\boldsymbol{X})$.

Keep those results in mind, and let us look at Exercise 17, for instance. Here, $(X_t)$ is an AR(1) process, with innovation $(\varepsilon_t)$,

$X_{t}=\phi X_{t-1}+\varepsilon_t$

One observation (say $X_\tau$) is missing. We have here 3 questions:

• what is the best linear prediction of $X_\tau$ given $X_{\tau-2}$ and $X_{\tau-1}$
• what is the best linear prediction of $X_\tau$ given $X_{\tau+2}$ and $X_{\tau+1}$
• what is the best linear prediction of $X_\tau$ given $X_{\tau-1}$ and $X_{\tau+1}$

Case 1. Here, we have to compute

$EL(X_{\tau}\vert X_{\tau-1}, X_{\tau-2})=\alpha_{-1}^\star X_{\tau-1}+\alpha_{-2}^\star X_{\tau-2}$

Since we have an AR(1) process, $\text{cov}(X_\tau,X_{\tau-1})=\phi$$\text{cov}(X_\tau,X_{\tau-2})=\phi^2$ and $\text{cov}(X_{\tau-1},X_{\tau-2})=\phi$. Thus, from the relationship above

$\left(\begin{array}{ll}1 & \phi\\\phi & 1\end{array}\right)\left(\begin{array}{l}\alpha_{-2}^\star\\ \alpha_{-1}^\star\end{array}\right)=\left(\begin{array}{l}\phi^2\\\phi\end{array}\right)$

which can be written

$\left(\begin{array}{l}\alpha_{-2}^\star\\ \alpha_{-1}^\star\end{array}\right)=\left(\begin{array}{ll}1 & \phi\\\phi & 1\end{array}\right)^{-1}\left(\begin{array}{l}\phi^2\\\phi\end{array}\right)=\frac{1}{\phi^2-1}\left(\begin{array}{ll}-1 & \phi\\\phi & -1\end{array}\right)\left(\begin{array}{l}\phi^2\\\phi\end{array}\right)$

i.e. $(\alpha_{-2}^\star,\alpha_{-1}^\star)=(0,\phi)$. Which makes sense actually: the AR(1) process is Markovian of order one, so

$EL(X_\tau\vert \underline{\boldsymbol{X}}_{\tau-1})=EL(X_\tau\vert X_{\tau-1})$

And we seen in class that for an AR(1) process

${}_{\tau}\widehat{X}_{\tau+1}=EL(X_\tau\vert \underline{\boldsymbol{X}}_{\tau-1})=\phi X_{\tau-1}$

So far, so good.

Case 2. Here, we have to compute

$EL(X_{\tau}\vert X_{\tau+1}, X_{\tau+2})=\alpha_{+1}^\star X_{\tau+1}+\alpha_{+2}^\star X_{\tau+2}$

Since we have an AR(1) process, $\text{cov}(X_\tau,X_{\tau+1})=\phi$$\text{cov}(X_\tau,X_{\tau+2})=\phi^2$ and $\text{cov}(X_{\tau+1},X_{\tau+2})=\phi$. Thus, from the relationship above

$\left(\begin{array}{ll}1 & \phi\\\phi & 1\end{array}\right)\left(\begin{array}{l}\alpha_{+1}^\star\\ \alpha_{+2}^\star\end{array}\right)=\left(\begin{array}{l}\phi\\\phi^2\end{array}\right)$

i.e. $(\alpha_{+1}^\star,\alpha_{+2}^\star)=(\phi,0)$.

Case 3. Finally, we have to compute

$EL(X_{\tau}\vert X_{\tau-1}, X_{\tau+1})=\alpha_{-1}^\star X_{\tau-1}+\alpha_{+1}^\star X_{\tau+1}$

One more time, since we have an AR(1) process, $\text{cov}(X_\tau,X_{\tau-1})=\phi$$\text{cov}(X_{\tau-1},X_{\tau+1})=\phi^2$ and $\text{cov}(X_{\tau},X_{\tau+1})=\phi$. So here, the relationship above becomes

$\left(\begin{array}{ll}1 & \phi^2\\\phi^2 & 1\end{array}\right)\left(\begin{array}{l}\alpha_{-1}^\star\\ \alpha_{+1}^\star\end{array}\right)=\left(\begin{array}{l}\phi\\\phi\end{array}\right)$Here, we can write

$\left(\begin{array}{l}\alpha_{-1}^\star\\ \alpha_{+1}^\star\end{array}\right)=\left(\begin{array}{ll}1 & \phi^2\\\phi^2 & 1\end{array}\right)^{-1}\left(\begin{array}{l}\phi\\\phi\end{array}\right)=\frac{1}{\phi^4-1}\left(\begin{array}{ll}-1 & \phi^2\\\phi^2 & -1\end{array}\right)\left(\begin{array}{l}\phi\\\phi\end{array}\right)$

i.e.

$\alpha_{-1}^\star=\frac{\phi}{\phi^2+1}=\alpha_{+1}^\star$

So finally, what we got here is

$EL(X_{\tau}\vert X_{\tau-2}, X_{\tau-1})=\phi X_{\tau-1}$

$EL(X_{\tau}\vert X_{\tau+1}, X_{\tau+2})=\phi X_{\tau+1}$

and

$EL(X_{\tau}\vert X_{\tau-1}, X_{\tau+1})=\frac{\phi}{\phi^2+1}\left(X_{\tau-1}+X_{\tau+1}\right)$

The mean squared errors for each of those estimates are obtained computing

$\mathbb{E}\left([X_\tau-EL(X_{\tau}\vert\boldsymbol{X})]^2\right)$

I guess I should probably stop here… that’s a detailed hint actually.