At first, I had very little interest in time series. Within finance, predicting future prices has made many managers underperform a simple basket of stocks like the SPY, which tracks the S&P 500 index. The chart below shows the performances of active vs passive managers and their performance over their respective indexes.
That all changed with the introduction of a stock market prediction coding challenge that I had to face. As I begrudgingly approached the subject, I started finding applicable ways to implement the techniques to trading. One of the insights I came across, was to adjust investing size based on confidence in signal, with more money being invested when several strong signals are flashing buy. The second insight comes from the application of these techniques to more than just stock prices. For example, time series works great in weather forecasting, earthquake prediction, and in general science and engineering.
We’re going to over a couple of different topics:
- Correlation vs Autocorrelation
- White Noise vs Random Walk
- Autoregressive, Moving Average, and ARIMA models
- AIC and BIC for testing
- Different Plots
Correlation vs Autocorrelation
Correlation is calculating the relationship between two models. Measured between -1 and 1: two models with a correlation of 1 means they have a perfect linear relationship. When one model moves, we expect the similar move by the other model. Autocorrelation is a measure of a lagged version of a time series model vs itself in successive time intervals. A simple way of looking at this comparing the price of a TSLA vs a moving average of TSLA prices. The lag is how many prices we’ll take into account for the moving average. The autocorrelation is also measured between -1 and 1, with an autocorrelation of -1 leading to a proportionate decrease in the other time series.
The reason we use autocorrelation is because there are sometimes jumps that occur in stock prices, such as when breaking news comes out about company earnings, product, or FED intervention. Whether the stock mean reverts or continues on a new trend can be seen as a negative autocorrelation or a positive correlation respectively.
#find correlation with .corr()
correlation = series_a.corr(series_b)
#plot correlations
plt.scatter(series_a, series_b)We find very little in patterns when looking at just a scatter plot of prices. Taking a percentage change of the prices leads to better insight.
We use the autocorrelation function to find any non-zero correlation. A result of around 0 suggests a very weak relationship between the lag and its current prices. A negative value shows that prices are reverting to the mean. Even though there is a minuscule autocorrelation value, there may still be a nonlinear relationship between a time series and a lagged version of itself. When plotting the autocorrelation prediction, we’ll use a confidence interval as a band around our prediction to show our certainty.
Preparing The Inputs
# we need to add a column of ones to calculate OLS intercept
df = sm.add_constant(df)
# must get dates into datetime object to years
df.index = pd.to_date_time(df.index, how = '%Y')
# how to change to weekly data
df.index.resample(rule = 'W', how = 'last')
# use pct_change() or .diff() for changes
df['PercentChange'] = df.CLOSE.pct_change()
#compute autocorrelation returns
# shows results for just one lag (pct_change())
autocorrelation = df['PercentChange'].autocorr()
print(autocorrelation)
plot_acf(df.PercentageChage, lags = 20, alpha = .5)
Seasonality
One of the things to watch out for is patterns of seasonality. Seasonality is the patterns that show up at regular intervals (daily at 2pm, 1st quarter of the year, every 4 years). Using the code below, we plot the returns of H&R block.
# To find out if there's seasonality in data, use ACF
from statsmodels.tsa.stattools import acf
from statsmodels.graphics.tsaplots import plot_acf
# Compute the autocorrelation function array of H&R Block
acf_array = acf(HRB)
print(acf_array)
# Plot the acf function
plot_acf(HRB, alpha=1)
plt.show()
In the plot we see, a pattern occurring ever 4 periods. In our case, this is quarterly data so we can explain the revenue as coming in from tax time. We will see how the
Seeing if ACF is statistically significant. Even if we get a value, how should we view the value? Are we confident in that answer? How will we know that the answer is good enough to go by? By testing the significance level.
# find autocorrelation of prices
autocorrelation = returns['Adj Close'].autocorr()
print(autocorrelation)
# -.16
# find number of observations
nobs = len(returns)
# compute confidence interval
conf = 1.96/sqrt(nobs)
# .12
#plot
plot_acf(returns, alpha=.05, lags=20)
plt.show()
The lags are the value of how long the lag is from current prices. In the plot above we can see a value of 1 for 0 lags (present time). The 1st lag shows that we are significant (above the confidence interval). From the 2nd to the 20th lag, we don’t see any significance with their autocorrelation results. The alpha shows how wide the confidence interval is. The CI bands are wider if Alpha is lower. An alpha level of .5 shows that there is 5% chance that it is true will fall outside the blue band.
Final Notes:
– A lag1 autocorrelation means that we include a one day lag in prices.
– Negative autocorrelation: mean reversion
– Positive autocorrelation: trend following
– pd.to_datetime() to convert strings or objects into datetime series
– autocorr() to get autocorrelation results
– autocorr() only works on series, not dataframes
– .resample() to get weekly prices
– autocorrelation: how it manages vs it’s previous changes in the past( must use pct_change() or .diff() )
White Noise
Constant mean, constant variance, 0 autocorrelations at all lags. Returns of the stock market fall within white noise. If we were to create an autocorrelation with stock market returns, nothing over the lag of 0 would be statistically significant. The past will not do a good job of predicting the future. Another way to view white noise is a sequence of uncorrelated random variables that are identically distributed.
Random Walk
Today’s price is yesterday’s price plus noise. The change in price of random walk is white noise. If stock prices follow random walk, then stock returns are white noise. In random walk with drift, prices on average change by mu. To test whether something follows random walk: regress current price to lag price, slope coefficient is significantly less than 1, then we reject. Dickey Fuller test is similar to testing for random walk, but instead of current price we use the difference in price on to the lag price. We then test weather the slope coefficient is 0. Augmented Dickey Fuller (adfuller() on statsmodels) is if we have more lag prices on right-hand side. Then we test weather p-value is less than 5%.
Because random walk can be negative, we need to look at multipicative noise, not just additive model. Prices can go positive or negative
#change to the difference in prices
AMZN_ret = AMZN.pct_change()
#drop the missing values
AMZN_ret = AMZN_ret.dropna()
#augmented dickey fuller test
adfuller(AMZN_ret['Adj Close']
#notes
'''
- Add 1 to the random steps
- Set first element to 1
results = adfuller(AMZN['Adj Close'])
'''Stationarity
Stationarity means that the statistical properties of a time series do not change over time. While the time series might change over time, the way it changes does not change. The stringent version says that joint distributions are not dependent on time. While the weaker version says that mean, variance, and autocorrelation are time invariant. If model is not stationary, it will be difficult to model. Most statistical forecasting methods are based on the assumption that time series can be rendered stationary.
Some examples of nonstationarity:
– Random walk (variance grows with time)
– Seasonal series (diet searches spike up after holidays)
– white noise (if mean increases over time)
How to turn random walk to stationary: take the 1st difference and take plot the autocorrelation. If there is seasonality, we’ll see it in the lag plots (plot_acf(HRBsa)). If we see seasonality, we can get rid of it by setting periods to that amount. For example, we see that HR Block has higher revenue on the 4th, 8th, 12th, and 16th lag. We simply set the periods to 4 to reflect that seasonality.
Autoregressive Model
When AR parameter (phi) is 1, the model is a random walk. When phi is 0, then it is white noise. Phi is between -1 and 1. With negative meaning mean reversion and positive showing momentum. The lags of autocorrelation is simply the phi raised to the number of lag. For example, an AR parameter of .8 in the first lag is .8, whie the 2nd lag is .9**2. For negative values, we simply alternate the sign but end up with the same value.
# in order to create an array with an ar parameter of .9, we need to input it into array as -.9
ar1 = np.array([1,-.9])
ma= np.array([1])
AR_object = ArmaProcess(ar1, ma1)
simulated_data = AR_object1.generate_sample(nsample =1000)
#plot_acf for simulated Data
plot_acf(simulated_data_1, alpha=1, lags= 20)
plt.show
# if we see seasonality, seasonally adjust quarterly earnings
HRBsa = HRB.diff(periods = 4)
# Estimating an AR Model
from statsmodels.tsa.arima_model import ARMA
mod = ARMA(simulated_data, order = (1,0))
result = mod.fit()
# Forecasting an AR Model
mod =ARMA(simulated_data, order = (1,0))
res = mod.fit()
res.plot_predict(start = '2016-07-01', end = '2017-06-01')
plt.show()
# example
mod = ARMA(interest_rate_data, order = (1,0))
res = mod.fit()
# create plot of our predictions for the future
res.plot_predict(start=0, end = '2022')
plt.show()
# to show two figures side by side
fig, axes = plt.subplots(2,1)
fig = plot_acf(interest_rate_data, alpha = 1, lags = 12, ax = axes[0])
fig = plot_acf(simulated_data, alpha=1, lags=12, ax= axes[1])
plt.show()Identifying the right model: PACF and AIC/BIC
Partial Autocorrelation Function (PACF) measures incremental benefit of adding another lag. Usually, more information is better, so including more lags is better. PACF issues a penalty with more lags included.
Information criteria are ways to adjust for the goodness of fit (AIC and BIC).
– Akaike Information Criterion and Bayesian Information Criterion (AIC and BIC)
– How should they be measured?
– choose p with lowest BIC .
#simulate AR(2) with phi1 = .6, phi2 = .3
ma = np.array([1])
ar = np.array([1, -.6, -.3])
AR_object = ArmaProcess(ar, ma)
simulated_data_2 = AR_object.generate_sample(nsample=5000)
#when plotting we see that there are 3 lags that should be looked into because they explain a lot of the data
# information criteria
# create empty array
BIC = np.zeros(7)
#create loop to find BIC of each AIC at different values
for p in range(7):
mod = ARMA(simulated_data_2, order=(p,0))
res = mod.fit()
BIC[p] = res.bic
#we look at plot and see where the smallest BIC for p level
plt.plot(range(1,7), BIC[1:7], marker = 'o')
plt.xlabel('Order of AR Model')
plt.ylabel('Bayesian Information Criterion')
plt.show()Moving Average Parameter
MA(1) parameter is simply moving average with a lag of 1. We only focus on the most recent lag and it a price move 2 or more away does not have any effect on our model. 0 autocorrelation for MA(1) beyond lag 1. We must include a zero-lag coefficient of 1 like AR, but now we don’t switch the sign of the coefficient.
– bid/ask
– simulate MA (moving average)
– MA is theta:
– higher order MA models: how do we pick?
– ma= np.array([1,.5])
– why do we put a lag coefficient:
ex:
plt.subplot(2,1,1)
ar1 = np.array([1])
ma1 = np.array([1, -0.9])
MA_object1 = ArmaProcess(ar1,ma1)
simulated_data_1 = MA_object1.generate_sample(nsample=1000)
plt.plot(simulated_data_1)MA(1) no autocorrelation beyond lag 1 (moving average: beyond the range doesn’t compute)
-estamating MA like AR: the order is (0,1) because we are estimating the MA side, not the AR
-Forecasting
mod = ARMA(simulated_data, order = (0,1))
res = mod.fit()
res.plot_predict(start=990, end=1010)
plt.show()
ARMA Models:
A combination of AR(1) and MA(1)
Cleaning Data:
intraday = intraday.reindex(range(391), method = ‘ffill’)
intraday.index = pd.date_range(start= ‘2017-09-01 9:30’, end = ‘2017-09-01 16:00′, freq=’1min’)
intraday.plot(grid=True)
Calculating ARMA Example
ma = [(.8**i) for i in range(30)]
ar = np.array([1])
AR_object = ArmaProcess(ar, ma)
simulated_data = AR_object.generate_sample(nsample=5000)
plot_acf(simulated_data, lags = 30)
plt.show()
Create Lag Character:
– use of lag to create corrrelations between what we can expect in the future
– We care about trading signals instead of raw correlations
–
On creating strategies:
– have a sliding scale of when to apply correlations with time
– sometimes we’ll have a change in fundamentals that will render strategy obsolete
– The most important is the strength of the signal, not the direction of correlations
– model should be relatively interpretable
–
Feed forward deep learning model (DNN)
– use the trained scaler from training set
–
Add or Stack models together
–
Mean Encoding
– df.groupby(‘item_id’)[‘sold’].transform(‘mean’).astype(np.float16)