Kalshi Weather Derivatives

Overview

Prediction markets do not reward accurate forecasts in isolation — they reward identifying when market-implied probabilities are miscalibrated.

This project builds a quantitative trading system for Kalshi's daily high temperature markets. Rather than predicting a single temperature, the system estimates a full probability distribution over outcomes, compares it against the market's implied distribution, and generates trading decisions based on expected value and risk constraints. The system launched in a single market and scaled to seven live U.S. metro markets — New York, Los Angeles, Chicago, Miami, Denver, Atlanta, and Las Vegas — within about two months, using a registry pattern where onboarding a new city is a configuration change, not a code change.

The forecasting models are only one layer of the system. The harder engineering problem was building the plumbing around them: a shared contract that lets three independently developed models operate as one ensemble, a data layer that reconciles six different upstream weather sources on different update cadences, and an execution layer disciplined enough to run unattended, across seven simultaneous markets, with real capital behind it.

Research

Before building models, we studied how Kalshi weather markets behave across the trading window. Early market prices tend to anchor to broad public forecast distributions while underweighting higher-resolution meteorological data that becomes available closer to settlement — creating windows where the market's implied distribution is measurably less precise than what the available data supports.

We also identified a structural mismatch in common forecasting approaches: Kalshi contracts settle on a single daily high temperature, yet most weather models are trained to minimize error across all 24 hourly readings. That means the standard loss function optimizes the wrong target relative to the actual payoff structure. Reframing the problem around the settlement event itself — the single daily maximum, observed at a specific station, under a specific betting cutoff — became one of the largest performance improvements in the system.

As part of this research, we also experimented with re-anchoring one model to predict residuals against a physical NWP baseline (HRRR forecast error) rather than absolute temperature — an attempt to remove a systematic cold bias at the source rather than correcting for it downstream in the ensemble.

Edge Generation

The system decomposes edge into two independent components.

Point Forecast Edge — the ensemble improves estimation of the expected daily high by directly targeting the settlement outcome rather than full-day averages.

Distributional Edge — each model produces a calibrated probability distribution over outcomes (p10 / p50 / p90). These are compared against Kalshi's implied probabilities to identify mispricing in the shape and variance of the distribution, not just the mean.

A key insight is that profitability does not require a superior point forecast; a correctly calibrated distribution alone can generate edge if the market misprices uncertainty.

System Architecture

The system combines three independent forecasting models, each built and owned by a different team member, communicating through one shared prediction schema:

• Temporal Fusion Transformer (TFT) — PyTorch-based sequence model using temporal attention over hourly observation and forecast history, with feature engineering that aggregates raw hourly data into the lag/rolling-window features the model needs while strictly respecting the betting cutoff. Runs once per day as a next-day forecast; it does not have same-day refresh data available, so it is intentionally treated as a frozen morning prior for the rest of the day.
• XGBoost — gradient-boosted trees for nonlinear residual correction and feature interaction learning; retrained from scratch on every inference run (~5–10 minutes) rather than relying on a static checkpoint, trading extra compute time for freshness.
• N-BEATS/N-HITS — deep time-series basis expansion (via the neuralforecast library), using both historical exogenous features (numerical weather model error terms, rolling temperatures) and future exogenous features (forecast data, cyclical time encodings).

Each model outputs a full probabilistic forecast distribution rather than a point estimate, validated against a shared contract so any of the three can be added, removed, or replaced without touching the other two or the ensemble layer.

An ensemble layer combines these outputs using dynamically updated weights derived from each model's recent rolling forecast error — models that degrade in recent performance are automatically down-weighted without manual intervention. Two independent guards run before weighting: one drops any model whose forecast diverges sharply from cross-model consensus, and a second drops any model whose forecast diverges sharply from an independent physical NWP baseline — so a bad forecast can't hide just because another model happens to agree with it. A floor on the reported uncertainty band also prevents any single model from making the ensemble overconfident.

Two problems that only showed up once the system was live drove further refinement of the weighting logic:

• Cold start for new markets — a freshly-onboarded city has too little prediction history for its rolling error estimate to be trustworthy, and using it anyway tends to penalize the actually-strongest model. New markets run on fixed, empirically-chosen weights until enough clean history accumulates, then transition automatically to error-based weighting. This is what let five new markets be onboarded together without re-tuning the ensemble by hand.
• Intraday freshness mismatch — a midday refresh re-runs the models that can meaningfully update on same-day data, while the frozen TFT prior is down-weighted relative to its morning-run confidence rather than kept at full strength, since a stale forecast shouldn't carry the same weight as a fresh one.

The resulting aggregated distribution feeds directly into a trading decision engine that evaluates expected value across every Kalshi contract in each market.

Risk Management & Decision Layer

Forecasting and capital allocation are treated as separate problems. The execution layer runs as a scheduled, serverless process (invoked automatically at fixed intervals during each market's active trading window) and advances through a persistent state machine — from initial market scanning, to position entry, to incremental slicing, to a final hold-only phase near settlement — so that partially-built positions survive across many independent invocations throughout the day, independently for each of the seven markets, rather than depending on one continuous process.

Independent risk controls layered on top of the raw edge signal:

• Divergence filters — models that deviate significantly from both ensemble consensus and external baselines are temporarily excluded from that day's decision.
• Kelly-based sizing — position sizing derived from expected edge, subject to strict capital caps.
• Market-implied volatility gate — trading is reduced or skipped when market prices already imply a sufficiently tight distribution (i.e., there's no mispricing left to exploit).
• Cost and liquidity constraints — orders are not placed into unfavorable market structures (wide spreads, thin books).
• Fallback sizing — reduced exposure when contract structure limits optimal positioning.

Exact thresholds and calibration constants are intentionally not disclosed.

Validation & Backtesting

A central engineering requirement was strict point-in-time correctness. All backtests are walk-forward: models only access data available before the trading cutoff for each simulated day, and every historical evaluation replays the exact temporal boundary the live system would have seen.

During development, I identified a feature leakage issue in the Temporal Fusion Transformer pipeline where engineered features inadvertently included post-cutoff information. After correcting it, we observed a significant divergence between the (inflated) pre-fix and corrected performance metrics — a concrete, quantified reminder of why temporal isolation has to be enforced structurally, not just assumed.

The validation approach also includes:

• Temporally disjoint validation sets (no random splits)
• Regime-aware error analysis across distinct weather conditions
• Continuous post-hoc performance monitoring against realized outcomes
• Live paper trading before any capital was deployed

Production Infrastructure (AWS)

The system runs as a fully automated production pipeline, not a research notebook. It includes:

• Automated ingestion and reconciliation of weather data from six independent upstream sources (surface observations, two different NWP model families, a blended-model product, MOS bulletins, and upper-air soundings), each on a different update cadence and latency, normalized to one unit standard and partitioned per-city across all seven live markets.
• A pre-run freshness gate that checks each data source's age before models are allowed to run, distinguishing mandatory sources (which halt inference and page the team if stale) from optional ones (which degrade gracefully).
• A daily forecast run and a separate midday refresh run, each dispatched onto a freshly-launched, purpose-built machine image rather than a long-lived server — the instance boots, runs the pipeline, uploads its logs, and terminates itself. The trading-day run deliberately uses on-demand rather than spot capacity, since a spot interruption silently skipping a day's forecast was judged unacceptable given the fixed trading-window deadline; weekly model retraining (which tolerates interruption) uses spot capacity instead.
• A serverless execution layer (AWS Lambda) that is fully decoupled from the research/pipeline environment — it reads only the finished ensemble output and has its own narrowly-scoped IAM role, so a bug in model code cannot touch trading credentials or vice versa.
• Per-account credential isolation via Secrets Manager, trade state and audit history in DynamoDB, and automated alerting (SNS → email) for pipeline failures, stale data, and accounts that failed to trade during a session.

This separation between research code and production infrastructure lets the team iterate quickly on models without risking the stability of the live trading process.

Results

A key finding was that optimizing for hourly temperature prediction significantly underperformed relative to optimizing directly for the settlement target. Reframing the problem to predict the daily high directly — rather than as a byproduct of a full-day forecast — produced the largest single improvement in system performance.

The system has passed backtesting and paper-trading validation and is currently deployed with real capital across seven live markets. Live performance is being withheld until a statistically meaningful sample size is accumulated.

My Contributions

As Integration Lead, I focused on turning independently developed models into a unified production system. Key contributions:

• Designed the ensemble architecture and shared probabilistic forecast schema
• Built the Temporal Fusion Transformer forecasting pipeline
• Developed interfaces enabling three independent models to operate as a single system
• Implemented rolling-error-based dynamic ensemble weighting, including the cold-start and intraday-freshness logic that let five new markets onboard without manual re-tuning
• Discovered and fixed a critical feature leakage issue in the TFT pipeline
• Led the scale-out from a single market to seven live U.S. metro markets via a registry-based onboarding pattern
• Helped transition the system from research prototype to automated production deployment

Technologies

Machine Learning

• Temporal Fusion Transformer
• XGBoost
• N-BEATS / N-HITS (neuralforecast)
• Probabilistic forecasting
• Ensemble learning
• Calibration and uncertainty estimation

Infrastructure

• AWS EC2 (on-demand and spot, purpose-built AMIs)
• AWS Lambda
• AWS Secrets Manager
• AWS DynamoDB
• AWS SNS (alerting)
• Docker
• Automated scheduling / cron-based orchestration

Methods

• Time-series forecasting
• Walk-forward validation
• Probability calibration
• Quantitative risk management
• Feature engineering for spatiotemporal data