Time Series Analysis

The following notes come from the course Time Series Analysis, taught at ETH Zurich by Professor Fadoua Balabdaoui.

Feel free to reach out to me with any comments, clarifications, or corrections.

Introduction to Time Series

A time series is a stochastic process indexed by time $t\in T\subseteq \mathbb{R}$.

Examples (they are all continuous!):

• $T=[0,1]$: Brownian bridge from $(0,0)$ to $(1,0)$, where the pair notation denotes $(\text{time},\text{value})=(t,X_t)$
• $T=[0,+\infty ]$: Brownian motion starting at $(0,0)$
• $T=\mathbb{R}$: two-sided Brownian motion
• $T={\mathbb{N}}_0$, or $T=\mathbb{Z}$: discrete random walk.

We use the notation $\{X_t{\}}_{t\in T}$ to denote a random variable and the notation $\{x_t{\}}_{t\in T}$ to denote realizations of that variable.

A time series model is either the specification of:

• all joint distributions $f(\{X_t{\}}_{t\in T})$ (i.e., the model provides $\mathrm{Pr}(X_{t_1}\le x_{t_1},\dots ,X_{t_k}\le x_{t_k})$ for all $k\in \mathbb{N}$, $t_1,\dots ,t_k$, and $x_1,\dots ,x_k\in \mathbb{R}$)
• all their means and covariances $(\mathbb{E}(X_t),\text{Cov}(X_t,X_p))$ for all $t,p\in T$

The i.i.d. noise $\{X_t{\}}_{t\in T}$ is a sequence of independent and identically distributed random variables such that $\mathbb{E}(X_t)=0$ for all $t$.
In general, we also assume that $\mathbb{E}[X_t^2]=\text{Var}(X_t)<\infty$ for all $t$. Note that in this case $\mathbb{E}[X_t^2]$ does not depend on $t$.

Consider $\{X_t{\}}_{t\in T}$ such that $\mathrm{Pr}(X_t=-1)=\mathrm{Pr}(X_t=1)=1/2$ for all $t$.

If $\{B_t{\}}_t{\sim }_{\mathrm{iid}}\text{Bernoulli}(1/2)$, then:

• $\{X_t{\}}_t=2\{B_t{\}}_t-1$
• $\text{Var}(X_t)=4\text{Var}(B_t)=1$ (note that $X_t^2=1$ and $\mathbb{E}(X_t)=0$)

${X_t}_t$ is a sequence of centered and uncorrelated random variables. More specifically, $\mathbb{E}(X_t)=0$, $\mathbb{E}(X_t^2)<\infty$, and $\text{Cov}(X_t,X_s)=0$ for all $t\ne s$.

• Note that i.i.d. noise such that $\mathbb{E}(X_t^2)<\infty$ is also white noise.
• The term "white" is used by analogy with white light. Indeed, white noise with constant variance $v^2$ has a flat spectral density given by $f(\lambda )=v^2/2\pi$, with $\lambda \in [-\pi ,\pi ]$ (this means that all time-series frequencies are present).

Let ${X_t}_t$ be i.i.d. noise such that $\mathbb{E}(X_t^2)<\infty$ for all $t\in \mathbb{N}$.

${S_t}_{t\in {\mathbb{N}}_0}$ is a random walk if $S_0=0$ and $S_t={\sum }_{i=1}^t{X}_i$ for all $t\ge 1$.

If ${X_t}_{t\in {\mathbb{N}}_0}$ is a Rademacher process, then ${S_t}_{t\in {\mathbb{N}}_0}$ is called a symmetric random walk, with $\mathbb{E}(S_t)={\sum }_{i=1}^t\mathbb{E}(X_i)=0$.

Let ${Y_t}_t$ be a zero-mean time series and ${m_t}_t$ be a deterministic (non-random) sequence.
Consider $X_t=Y_t+m_t$ for all $t\in T$.

Then $\mathbb{E}[X_t]=m_t$ is called the trend of ${X_t}_t$.

Given a zero-mean time series $\{Y_t{\}}_t$ and a function $t\mapsto p_t$ such that $p_t$ is periodic with known period $d$ (i.e., $p_{t+d}=p_t$).

Consider the time series $X_t=Y_t+p_t$ for all $t\in T$. Then ${p_t}_t$ is called the seasonal component of ${Y_t}_t$.

For example, we could have the seasonal component in the following forms:

with known integers $k\ge 1$, ${\alpha }_j,{\beta }_j>0$, and unknown $a_0,a_j,b_j$ for all $1\le j\le k$.

$X_t=m_t+p_t+Y_t,\forall t\in T$

with ${m_t}_t$ the trend, ${p_t}_t$ the seasonal component, and ${Y_t}_t$ a zero-mean time series.

Stationarity and auto-covariance function (ACVF)

A time series $\{X_t{\}}_t$ is said to be strictly (strongly) stationary if, for all $n\ge 1$ and $h\in \mathbb{Z}$, it holds that:

i.e., the stochastic behavior of $\{X_t{\}}_t$ remains unchanged for any time shift.

Let $\{X_t{\}}_t$ be a time series.

• The mean function of $\{X_t{\}}_t$ is defined as:

• The covariance function of $\{X_t{\}}_t$ is defined as:

provided that $\mathbb{E}[|X_r{X}_p|]<\infty$ and $\mathbb{E}[X_r]<\infty$ for all $r,p$.

On the other hand, $\{X_t{\}}_t$ is said to be (weakly) stationary if $\mathbb{E}[X_t^2]<\infty$ for all $t$, and:

• ${\mu }_x(t)$ is costant in $t$
• ${\gamma }_x(t,t+h)$ depends only on $h$