ACF and PACF of White Noise and IID Noise

Last edited: 2023-01-08 16:06:09

In time series sample data there is always noise. But there exist different types of noise. In this post we will run through regular white noise and iid noise and how to identify them in your residuals with ACF/PACF plots and the Ljung-Box test after for example applying an ARMA or GARCH model. First we need to define weak stationarity before we define white noise.

What is Weak Stationarity?

A time series $X$ is weakly stationary if

• $\mathbb{E}[X_t] = \mu = \text{const}$ for all $t \in \mathbb{Z}$
• $\text{Var}[X_t] = \sigma^2 = \text{const} < \infty$ for all $t \in \mathbb{Z}$
• $\text{Cov}[X_r, X_s] = \text{Cov}[X_{r+h}, X_{s+h}]$ for all $s,r,h \in \mathbb{Z}$

Defintion of White Noise

A time series $X$ is said to be white noise if

• $X$ is weakly stationary
• $\gamma(h) = \text{Cov}[X_t, X_{t+h}] = 0$ where $h \neq 0$

Definition of IID Noise

What is iid noise? A time series $X$ is said to be iid noise if it meets the conditions for regular white noise above but also $X_r$ and $X_s$ ($r \neq s$) is independent of each other and identically distributed for all $r,s \in \mathbb{Z}$, that is why it is called iid noise where iid stands for independent and identically distributed.

Before going through the tricks to identifying the two different kind of noises we need to go through the definitions of the autocorrelation function (ACF) and the partial autocorrelation function (PACF).

The Autocorrelation Function (ACF)

The autocorrelation function for the time series $X$ is defined as:

$$\rho_X(h) = \frac{\gamma(h)}{\gamma(0)} = \frac{\text{Cov}[X_t, X_{t+h}]}{\text{Var}[X_t]}.$$

The Partial Autocorrelation Function (ACF)

The autocorrelation function $\alpha(h)$ for the time series $X$ is defined by $\alpha(0) = 1$ and $\alpha(h) = \phi_{hh}$ for $h \geq 1$ where $\phi_{hh}$ is the last component of

$$\phi_h = ([\gamma(i-j)]_{i,j=1}^{h})^{-1} [\gamma(1), ... , \gamma(h)]'.$$

ACF Plot Interpretation - How to Identify White Noise

The first simple thing you could do to see if your data is just white noise is if it looks like it has no structure. The second thing is looking at the sample ACF and PACF graphs for the data. Lets look at the definition of white noise and the ACF. We know that $\gamma(h) = 0$ for $h \geq 1$ for white noise which means $\rho_X(h) = 0$ for $h \geq 1$. So all lags greater than zero should be within the red striped confidence band ($\pm 1.96/\sqrt{n}$, where $n$ is the sample size) of the ACF in the plot below. So the following sample ACF plot indicates white noise:

The same can be said for the PACF. If $X$ is white noise then the vector $[\gamma(1), ... , \gamma(h)]'$ will just be filled with zeros and thus $\alpha(h) = 0$ for $h \geq 1$.

How to Identify IID Noise

To identify iid noise we will have to look at the sample ACF and PACF of the data squared, $X^2$. Lets start with the ACF. We have:

$$\gamma_{X^2}(h) = \text{Cov}[X^2_t, X^2_{t+h}]$$ $$= \mathbb{E}[(X_t^2 - \mathbb{E}[X_t^2])(X_{t+h}^2 - \mathbb{E}[X_{t+h}^2])]$$

and now we use the property of iid noise that $X_t$ and $X_{t+h}$ is independent of each other so we get the following:

$$(\mathbb{E}[X_t^2 - \mathbb{E}[X_t^2]]) (\mathbb{E}[X_{t+h}^2 - \mathbb{E}[X_{t+h}^2]])$$ $$= (\mathbb{E}[X_t^2] - \mathbb{E}[\mathbb{E}[X_t^2]]) (\mathbb{E}[X_{t+h}^2] - \mathbb{E}[\mathbb{E}[X_{t+h}^2]]).$$

Lastly we use the general property that $\mathbb{E}[\mathbb{E}[Y]] = \mathbb{E}[Y]$ so we get:

$$(\mathbb{E}[X_t^2] - \mathbb{E}[X_t^2]) (\mathbb{E}[X_{t+h}^2] - \mathbb{E}[X_{t+h}^2]) = 0.$$

So the sample ACF for $X^2$ should be in the confidence band for all lags greater than zero. For the PACF we can see that the vector $[\gamma_{X^2}(1), ... , \gamma_{X^2}(h)]'$ would be zero for all $h$. Therefore the PACF should also be within the confidence band for lags greater than zero if $X$ were to be iid noise which we can see in the following plot:

So to conclude, for iid noise $X$ and $X^2$'s sample ACF and PACF should all have to be within the confidence band for lags greater than zero.

Ljung-Box Test for IID Noise

Lastly one can do a Ljung-Box test to determine whether the data is independently distributed. The test is defined by the following hypothesis:

$$\begin{aligned} & H_0 : X \sim \text{IID}(\mu, \sigma^2) \\ & H_1 : X \not\sim \text{IID}(\mu, \sigma^2) \end{aligned}$$

The Ljung-Box test has a test statistic which is:

$$\lambda = n (n + 2) \sum_{i=1}^h \frac{\hat{\rho}(i)^2}{n-i},$$

where $\hat{\rho}(i)$ is the sample ACF at lag $i$, $n$ is the sample size and $h$ is the number of lags being tested. The null-hypothesis, $H_0$, is rejected at the $\alpha$-level, $\alpha \in (0,1)$, if $\lambda > \chi_{1-\alpha, h}^2$, where $\chi_{1-\alpha, h}^2$ denotes the $1 - \alpha$-quantile of the $\chi^2$-distribution with $h$ degrees of freedom.