Kalman Filters for Dynamic Hedge Ratios: Tracking a Beta That Refuses to Sit Still

EWA (Australia) and EWC (Canada) are the canonical cointegration pair — two commodity-heavy developed markets whose daily returns correlate at 0.83. Regress EWC on EWA over the full sample and you get one hedge ratio for fifteen years: $\mathbf{\beta }=1.57$, with an ADF p-value of 0.038 on the residual spread — cointegrated, by the book (Engle–Granger: regress one price on the other, then unit-root-test the residual). The problem is the word one. A hedge ratio is not a constant of nature; it is a snapshot of an economic relationship — commodity mix, currency betas, index composition — and every one of those drifted between 2010 and 2024. The standard fix is a rolling window, and it limps for a reason worth understanding: every observation inside the window carries equal weight, so year-old data moves today's estimate exactly as much as yesterday's does — and the day an old observation falls out of the window, the estimate jerks for no economic reason at all. That artefact has a name: the window cliff.

The Kalman filter starts from a different premise, and it is worth sitting with for a moment: the “true” hedge ratio is something you can never observe directly — you only see prices, which are noisy evidence about it. So treat the regression coefficients as unobserved statesdrifting through time, and treat each day's prices as one more noisy measurement of where they are. (A GPS does exactly this with your position; here the hidden position is β.) Two equations define the model. The state equation says the hedge ratio and intercept follow a random walk — $[{\mathbf{\beta }}_{\mathbf{t}},{\mathbf{\alpha }}_{\mathbf{t}}]=[{\mathbf{\beta }}_{\mathbf{t}-1},{\mathbf{\alpha }}_{\mathbf{t}-1}]+{\mathbf{\omega }}_{\mathbf{t}}$— tomorrow's relationship is today's, plus noise. The observation equation says ${\mathrm{EWC}}_{\mathbf{t}}={\mathbf{\beta }}_{\mathbf{t}}{\mathrm{EWA}}_{\mathbf{t}}+{\mathbf{\alpha }}_{\mathbf{t}}+{\mathbf{\epsilon }}_{\mathbf{t}}$. Two variances close the model: the state noise $\mathbf{Q}=\frac{\mathbf{\delta }}{1-\mathbf{\delta }}\mathbf{I}$ with $\mathbf{\delta }=10^{-5}$ (the standard parameterization from Chan), and observation noise $\mathbf{R}=10^{-3}$.

Delta is the single real knob, and it replaces the window size entirely: it is a forgetting rate. Larger δ lets the states wander faster (adaptive but noisy); smaller δ pins them down (smooth but laggy); δ = 0 collapses to recursive least squares — a static beta refined forever. Both values here are textbook defaults, deliberately not tuned on this sample: tune δ to the backtest and you are optimizing the strategy through the back door.

No library, no black box — the whole filter is a predict step and an update step, looped over the sample. Predict: with a random-walk transition the state estimate is unchanged and its covariance grows by Q — a day passes, so the filter becomes a little less sure of where β is. Update: compare the observed EWC to the prediction, and shift the states toward the error in proportion to the Kalman gain — the ratio of state uncertainty to total uncertainty. Big surprise while unsure of yourself: move a lot. Big surprise while confident: blame measurement noise, barely move.

def kalman_hedge(x, y, delta=1e-5, r_obs=1e-3):
    q = (delta / (1 - delta)) * np.eye(2)     # trans_cov
    state, p_cov = np.zeros(2), np.eye(2)     # [beta, alpha], diffuse start
    betas, alphas = np.zeros(len(x)), np.zeros(len(x))
    for t in range(len(x)):
        h = np.array([x[t], 1.0])             # observation map
        p_cov = p_cov + q                     # predict   (F = I)
        e = y[t] - h @ state                  # innovation
        s = h @ p_cov @ h + r_obs             # innovation variance
        k = p_cov @ h / s                     # Kalman gain
        state = state + k * e                 # update
        p_cov = p_cov - np.outer(k, h @ p_cov)
        betas[t], alphas[t] = state
    return betas, alphas

The gain is the elegance. When the filter is uncertain (large p_cov), it learns aggressively from each observation; once confident, new data barely moves it — unless Q keeps injecting doubt, which is exactly what lets β keep adapting forever. An exponentially-weighted regression, derived from first principles rather than picked from a menu of window sizes.

Slice off the first year (rolling-OLS warm-up, Kalman burn-in) and compare the three estimators of the same quantity. The static line says the answer is 1.57, forever. The rolling OLS swings between 0.19 and 2.75 — whipping around every regime change a full window late. The Kalman path stays in a far narrower band, moving early and smoothly: no cliff, because no window.

Note what the rolling estimator does around 2020–2021: the COVID shock enters the window, distorts the regression for exactly 252 trading days, then falls out and the estimate jumps again — two artefacts from one event. The filter digests the same shock in weeks and moves on.

Now make the estimator earn its living — a better β is only worth money if it makes a better spread. The spread is ${\mathrm{EWC}}_{\mathbf{t}}-{\mathbf{\beta }}_{\mathbf{t}}{\mathrm{EWA}}_{\mathbf{t}}-{\mathbf{\alpha }}_{\mathbf{t}}$, z-scored on a trailing 60-day window. Rules, identical for both variants: enter long the spread (long EWC, short β·EWA) when $\mathbf{z}<-2$, short when $\mathbf{z}>+2$, exit when z crosses zero. Positions are sized to $1 gross notional at entry, the Kalman variant re-hedges the EWA leg to the current β daily, and every unit of traded notional pays 10 bp. Signals use the close and P&L starts the next day — no lookahead in the rule. The static beta itself, of course, is one giant lookahead: it was fit on all fifteen years, including the future of every trade it takes.

spread = ewc - beta * ewa - alpha              # beta_t, alpha_t from the filter
z = (spread - spread.rolling(60).mean()) / spread.rolling(60).std()

# enter |z| > 2, exit when z crosses 0, hold in between
if p == 0:
    p = 1 if z[t] < -2 else (-1 if z[t] > 2 else 0)
elif p == 1 and z[t] >= 0: p = 0               # long leg reverted
elif p == -1 and z[t] <= 0: p = 0              # short leg reverted

Here is the honest scoreboard, and it is more interesting than a clean win. Gross of costs, the Kalman spread is the better signal on every risk-adjusted axis — more Sharpe, half the volatility, half the drawdown. But the filtered spread mean-reverts fast, so it trades 152round trips to the static variant's 57 — and at 10 bp per unit of traded notional, that turnover consumes the entire edge and then some.

So did the Kalman beta “improve” the strategy? As an estimator, unambiguously — better gross Sharpe, half the drawdown, and no lookahead, against a static baseline that was handed the answer key. As a net P&L line at retail costs, no: the same adaptivity that tracks the relationship also generates signals faster than 10 bp round trips can pay for. Halve the cost and the gap halves; at institutional frictions of 1–2 bp the Kalman variant pulls level and ahead. Estimation quality and implementability are different axes, and a backtest that reports only one is hiding the other.