Home

Credit: Rob J Hyndman and George Athanasopoulos
Full Text Using R: https://otexts.org/fpp2/

Elementary Methods

We list the following methods for general knowledge and leave it to the reader to investigate their uses

Average

Future values = Average historical values

Naive

Future Values = Last actual values

Seasonal Naive

Future Value = Last actual value from the same season
For example a forecast for the coming january would be equal to last january's value

Drift

A slight variation on the Naive adjusting for an increase/decrease over time. This adjustment would be indicated by the slope of the line between the some two points in the past. This incorporates the historical average rate of change into future values. $$ \begin{equation} \label{eq1} \begin{split} \hat{y}_{T+h|T} & = y_T + \frac{h}{T-1} \sum_{t=2}^{T} (y_t - y_{t-1}) \\ & = y_T + h (\frac{y_T - y1}{T-1}) \end{split} \end{equation} $$

Decomposition Methods

Time series decomposition works by determining the time factors influencing the response. Some examples might include an increase in sales for a retail store during the months leading up to christmas.

Underlying Models

Let t = time then the model options are $$ y_t = S_t + T_t + R_t \text{additive decomposition}$$ $$ y_t = S_t * T_t * R_t \text{multiplicative decomposition}$$ $$ log(y_t) = log(S_t) * log(T_t) * log(R_t) \text{multiplicative alternative}$$

where the capital letters represent the three components

  • S=seasonal
  • T=trend
  • R=remainder.

Which one to choose?
Additive is better when the level of the time series causes little fluctuation in

  • the magnitude of the seasonal fluctuations
  • or the variation around the trend-cycle Multiplicative is generally better when either of the above is proportional to the time series level.
In [1]:
import pandas as pd
import numpy as np
import matplotlib.pylab as plt
%matplotlib inline
from matplotlib.pylab import rcParams
import seaborn as sns

Trend Methods

Trend is defined by the moving average. There are several different methods for computing the moving average.

Centered Moving Avg

A centered moving avg of order m, m odd, as defined by $$ \hat{T}_t = \frac{1}{m} \sum_{j=-k}^{k} y_{t+j} \text{ where } m = 2k + 1 $$ Which is simply the centered mean of the m periods nearest t.

In [2]:
rcParams['figure.figsize'] = 20, 6
data = pd.read_csv('./Data/AirPassengers.csv')
    #convert month to date
data['Date'] = pd.to_datetime(data['Month'], format='%Y-%m')
#print(data.dtypes)
#print(data.head())
    # do a quick line plot
plt.plot(data['Date'],data['#Passengers'], label='Orig' )

# Add a moving average trend - The higher the m the straighter the trend becomes
data['MA5'] = data['#Passengers'].rolling(min_periods=2,window=5, center=True).mean()
plt.plot(data['Date'], data['MA5'], label='CMA-5' )
Out[2]:
[<matplotlib.lines.Line2D at 0x1b5ed6bf400>]

MA x MA(Weighted MA)

The moving average of a moving average can be used to centre an even-order MA. Making it symmetric about t. It is also useful in extracting a trend-cycle from a seasonal cycle. What's most interesting about this method is that it effectively weights each period. So a 2x4,8,12... MA works well for data containing quarterly periods. A 2x12,24,etc would best apply to say monthly data. This also has the added advantage of placing a lower weight to values as they enter and leave the calculation window.

A 2m-MA is equivalent to a weighted moving average of order m+1 where all observations take the weight
1/m, except for the first and last terms which take weights (1/(2m)

Side-note: A seasonal cycle is a fixed time-driven cycle like the weather (but it doesn't have to be yearly). A Trend-cycle is anything outside that
Example:
Seasonal = a spike in purchases every 12 months
Trend = the economy has peaks and troughs, but the length of time b/w is not fixed.

In [3]:
plt.plot(data['Date'],data['#Passengers'], label='Orig' )

# Add a 2x4MA to get a centred 4MA
data['MA4'] = data['#Passengers'].rolling(min_periods=2,window=4, center=True).mean()
data['MA2x4'] = data['MA4'].rolling(min_periods=2,window=2, center=True).mean()
plt.plot(data['Date'], data['MA2x4'], label='MA2x4' )

data['MA8'] = data['#Passengers'].rolling(min_periods=2,window=8, center=True).mean()
data['MA2x8'] = data['MA8'].rolling(min_periods=2,window=2, center=True).mean()
plt.plot(data['Date'], data['MA2x8'], label='MA2x8' )

plt.legend()
Out[3]:
<matplotlib.legend.Legend at 0x1b5ed7524e0>

Modeling Algos

Additive Model

  1. Choose m integer
  2. For m even, compute the trend cycle $\hat{T_t}$ using a 2xM-MA
    For m odd, Compute $\hat{T_t}$ using a m-MA
  3. Calculate the Trend adjusted Series $y_t - \hat{T_t}$
  4. Estimate the seasonal component $\hat{S_t}$ by taking the average for each of the seasons ie for monthly take the average over the same month in each year
  5. Finally compute the remainder $\hat{R_t} = y_t - \hat{T_t} - \hat{S_t}$

Multiplicative Model

The process is similar to the additive but we replace the subtraction with division Trend adjusted Series becomes $y_t / \hat{T_t}$
Remainder becomes $\hat{R_t} = y_t / (\hat{T_t} * \hat{S_t})$

We could use an example, let's use the Stats_models

In [4]:
import statsmodels.api as sm
dta = data['#Passengers']
dta.index = pd.DatetimeIndex(data['Date'])
#dta.head()

# If you have missing values: .interpolate(inplace=True)

# model='additive'
res = sm.tsa.seasonal_decompose(dta,model='multiplicative')

#results = pd.DataFrame(data[['Date','#Passengers']],res.trend,res.seasonal,res.resid)
#results.head()
resplot = res.plot()

# As we can see below Statsmodels can effciently pluck out the needed components. 
# Most likely better than we can code our own solutions

C:\ProgramData\Anaconda3\lib\site-packages\statsmodels\compat\pandas.py:56: FutureWarning: The pandas.core.datetools module is deprecated and will be removed in a future version. Please use the pandas.tseries module instead.
  from pandas.core import datetools

Summary Notes

Classical decomposition is still used but should be avoided if it can be. It makes a lot of assumptions which may not play out in the real world. It also over-smooths the data, and is error prone. Suppose a strike had happened during the timeframe of our data, then we would see an outlier event that would throw things out of whack. There are far better/robust methods out as we shall soon see.

X11 Method

(http://www.abs.gov.au/websitedbs/d3310114.nsf/4a256353001af3ed4b2562bb00121564/c890aa8e65957397ca256ce10018c9d8!OpenDocument)

The X11 method is an iterative classical decomposition. It involves a classical decomposition with one, or more, added iterative/adjustment steps that try to eliminate the effect of outliers.

The process can be described as follows:

  1. Estimate the trend using a moving average
  2. Remove the trend leaving the seasonal & irregular
  3. Estimate the seasonal using a moving average to smooth out the irregular

Algorithm

Here is a basic overview:

Let $O_t$ be the original time series

Step 1 Estimate the Trend $\hat{T_t}$ $$ \frac{O_t}{\hat{T_t}} = \frac{(T_t * S_t * I_t)}{\hat{T_t}} \approx T_t*I_t $$

Step 2 Seasonal Estimate 1
Apply a weighted 5-term moving average ($S_{3x3}$) to the series $S_t I_t$ for each month separately. Missing values at the end of the seasonal component can be replace with the values from the previous year.

Step 3 Adjusted Estimate 1 $$ \frac{O_t}{\hat{S_t}} = \frac{(T_t * S_t * I_t)}{\hat{S_t}} \approx T_t*I_t $$

Step 4 Estimate the trend 2
Repeat step 1 on the Seasonally adjusted date

Step 5 Seasonal - Final Estimate Repeat step 2 to get final seasonal component

Step 6 Adjusted - Final estimate
Repeat step 3 using $\hat{S_t}$ from step 5

Step 7 Trend - Final Estimate
Repeat step 4

Step 8 Irregular - Final and Only Estimate $$ \frac{(T_t * I_t)}{\hat{T_t}} \approx I_t $$

Seats Decomposition

Seats, Seasonal Extraction in Arima Time Series, works with quarterly & monthly data.

We won't go into this method as it is a specific type of ARIMA which will be discussed later.

STL Decomposition

Seasonal and Trend using Loess. Where Loess is a method for estimating nonlinear relationships. STL is a filtering procedure for decomposing a time series into three components.

Links

  1. http://www.nniiem.ru/file/news/2016/stl-statistical-model.pdf
  2. http://www.gardner.fyi/blog/STL-Part-II/

STL has several advantages over the previous methods

  • It can handle any type of seasonality
  • Seasonal component can change over time as well as have a rate of change
  • Smoothness of the trend-cycle can be controlled
  • Extreme values and outliers do not affect the trend and seasonal components much. Their effect is limited to the remainder

It has a few disadvantages as well

  • It cannot handle trading day or calendar variations without user intervention
  • Only does additive decomposition

This second point can be negated by taking the log of the data then back transforming the data. Another trick is to use the Box-Cox transformation with $0 < \lambda < 1$. When $\lambda = 0$ you get the multiplicative decomposition. When $\lambda = 1$ you get the additive.

Recall: 1-param Box-Cox Transformation is defined as $$ \begin{cases} \frac{y^\lambda_i - 1}{\lambda} \text{ if } \lambda \neq 0 \\ ln(y_i) \text{ if } \lambda = 0 \end{cases} $$ If you use the transformation then you'll of course need the inverse functions to get to the actual forecast.

Unfortunately python does not have a forecast package that has an stl decomposition. However, there is an easy to follow and free piece of code available at https://github.com/jrmontag/STLDecompose