# Overfitting

> Why a good backtest is not the same as a good strategy, and the checks that tell them apart.

Source: https://docs.texttoquant.com/concepts/overfitting

---

Overfitting (curve fitting) is tuning a strategy so tightly to past data that it captures *noise*
instead of a real edge. An overfit strategy looks brilliant in the backtest and falls apart live.
The more parameter combinations you try, the easier it is to find one that fit the past by pure luck.

This is the central risk in all backtesting, so the platform is built to catch it.

## Why it happens

Every extra parameter, every "let me just try 20 and 30 and 40", multiplies the number of ways a
result can look good by accident. Keep the best of 100 variations and its headline Sharpe is inflated
by the *search itself*, not by any edge that will repeat.

## The checks

The report ships several independent checks, each attacking the result from a different angle:

- **Seeded Monte Carlo resampling** with permutation testing: reshuffled trade orders and permuted
  returns show how much of the result is path luck.
- **Out of sample splits**: performance on data the strategy was not tuned on.
- **Walk forward optimization**: parameters refit on rolling in sample windows, judged only on the
  forward window that follows.
- **Parameter sensitivity sweeps**: whether the result survives nearby parameter values or lives on
  an isolated spike.
- **Multi asset robustness**: the same logic run on other markets.

And the overfitting aware statistics, Deflated Sharpe, Overfit Probability (PBO), Haircut Sharpe,
discount the headline numbers for how many configurations you tried. See the full table in the
[Robustness reference](/docs/reference/robustness).

<Callout variant="danger" title="The honest test">
  Fit on one slice, judge on another you never touched. If the edge only exists on the data you tuned
  it on, it isn't an edge. It's a memory of the past.
</Callout>

<Callout variant="warning" title="Even a survivor can fail">
  A strategy that passed every check on this page can still lose money live. Backtests are
  hypothetical, benefit from hindsight, and exclude live latency and venue specific costs. Past
  performance does not guarantee future results.
</Callout>

Next: run the [validation workflow](/docs/guides/validate-an-idea) on one of your own strategies.