Overfitting
Why a good backtest is not the same as a good strategy.
Overfitting (curve fitting) is tuning a strategy so tightly to past data that it captures noise instead of a real edge. An overfit strategy looks brilliant in the backtest and falls apart live. The more parameter combinations you try, the easier it is to find one that fit the past by pure luck.
This is the central risk in all backtesting, so the platform is built to catch it.
Why it happens#
Every extra parameter, every "let me just try 20 and 30 and 40", multiplies the number of ways a result can look good by accident. Keep the best of 100 variations and its headline Sharpe is inflated by the search itself, not by any edge that will repeat.
The checks#
The report ships several independent checks, each attacking the result from a different angle:
- Seeded Monte Carlo resampling with permutation testing: reshuffled trade orders and permuted returns show how much of the result is path luck.
- Out of sample splits: performance on data the strategy was not tuned on.
- Walk forward optimization: parameters refit on rolling in sample windows, judged only on the forward window that follows.
- Parameter sensitivity sweeps: whether the result survives nearby parameter values or lives on an isolated spike.
- Multi asset robustness: the same logic run on other markets.
And the overfitting aware statistics, Deflated Sharpe, Overfit Probability (PBO), Haircut Sharpe, discount the headline numbers for how many configurations you tried. See the full table in the Robustness reference.
Fit on one slice, judge on another you never touched. If the edge only exists on the data you tuned it on, it isn't an edge. It's a memory of the past.
A strategy that passed every check on this page can still lose money live. Backtests are hypothetical, benefit from hindsight, and exclude live latency and venue specific costs. Past performance does not guarantee future results.
Next: run the validation workflow on one of your own strategies.