August 10, 2026

Context-Aware Distributionally Robust Deep Q-Learning — a robust trading agent that hedges hard when uncertain and commits when confident.

Robust trading agents hedge against the worst plausible market move — safe, but on the S&P 500 they finish at ~2–3× while buy-and-hold earns 9.5×. CARDQN makes that hedge context-aware: it labels the market regime, scores how reliably its model predicts it, and tightens the safety margin only where that score is high. The result: 3.40× terminal wealth — more than double the robust baseline — at roughly 2× the Sharpe and Sortino.

Terminal wealth: CARDQN 3.40× vs RDQN 1.59× vs S&P 9.52× — End of training, 5 seeds: CARDQN **3.40×** vs RDQN **1.59×** vs S&P 500 buy & hold **9.52×**.

Risk metrics: Sharpe, Sortino, volatility, max drawdown — CARDQN roughly doubles RDQN's Sharpe and Sortino, at comparable volatility and drawdown.

How it works

Three pieces, all computed from past data only: a regime tag τ labels the market (trend, volatility, position — 27 regimes); a fidelity score φ measures out-of-sample predictability per regime; an adaptive radius ε̃ and reference P̃ tighten the ambiguity ball where φ is high, relax it where it’s low.

CARDQN pipeline — State → regime tag → fidelity → adaptive radius & reference → robust Bellman update.

Links

📄 Paper (PDF) · 💻 Code (GitHub) · ✍️ Explainer

FAQ

Does it beat buy & hold?

No — 3.4× vs 9.5×. All agents are trained on a simulator and hit a sim-to-real ceiling; CARDQN’s claim is over the robust baseline (RDQN), not over the market.

What does “distributionally robust” mean?

Don’t trust one model — consider all nearby models and plan for the worst. You trade as if the nastiest plausible market were the true one.

What is the context tag?

A past-only label with three components (trend, volatility, position), giving 27 regimes total. Computed entirely from recent returns, with no look-ahead.

What is the fidelity score?

An out-of-sample measure of how reliably the model predicts each regime. It’s high only when the regime is well-sampled and genuinely predictable.

What is the proposed Bellman-target blend?

A rule that de-hedges toward the risk-neutral objective in favorable regimes, by a capped weight that never becomes optimistic. Proposed in the paper, not yet validated.

How is the system trained?

On a signature-MMD market simulator, then evaluated out-of-sample on the real S&P 500 (1995–2024, 0.05% transaction costs). Each configuration runs across 5 random seeds.

How long does training take?

~10–14 minutes per episode on an RTX 2080 Ti. A full 10-episode run takes about 2 hours; a 5-seed campaign runs overnight.

What markets does CARDQN trade?

The S&P 500 (1995–2024), single-asset, daily rebalancing with proportional transaction costs. The framework generalizes to any liquid asset with sufficient price history.

Can CARDQN be applied to other domains?

Yes — the context-aware ambiguity idea is domain-agnostic and applies to robotics, control, and supply-chain optimization.

How does CARDQN compare to other strategies, and can it do high-frequency trading?

CARDQN is a daily-rebalancing, position-level robust strategy — not HFT. It sits in the systematic/quantitative category alongside momentum and risk-parity, but is unique in its distributionally-robust formulation. Inference is a single forward pass through the Q-network (milliseconds), fast enough for daily or intraday use. Microsecond-scale HFT would require a different execution layer.

What license?

CC BY-NC-SA 4.0.

Cite

@misc{golinelli2026cardqn,
  author = {Giulio Golinelli},
  title  = {Context-Aware Distributionally Robust Deep {Q}-Learning ({CARDQN})},
  year   = {2026},
  url    = {https://giuliocsr.github.io/papers/cardqn.pdf},
  note   = {Code: https://github.com/giuliocsr/CARDQN}
}

Giulio Golinelli — giuliocsr