# Validate an idea

> Pressure test an edge with out of sample, robustness, sweeps and Monte Carlo before you trust it.

Source: https://docs.texttoquant.com/guides/validate-an-idea

---

A good backtest is a *hypothesis*, not a conclusion. This guide is the workflow that separates a real
edge from a lucky fit. Run it on any strategy before you believe the headline number.

## 1. Does it hold out of sample?

<Steps>
  <Step n={1} title="Read the out of sample verdict">
    When a hold out split exists, the header shows an always on OOS verdict: performance on data the
    strategy was never fit on. If it collapses out of sample, stop here: the edge was in the fitting.
  </Step>
  <Step n={2} title="Run walk forward">
    Walk forward refits parameters on rolling in sample windows and grades them only on the forward
    window that follows: the strictest test, and the closest to how you'd retune a live strategy.
  </Step>
</Steps>

## 2. Is it robust, or a lucky spike?

- **Parameter sensitivity sweeps**: does the result survive nearby parameter values, or does it live
  on an isolated spike that a slightly different period would miss?
- **Multi asset robustness**: does the same logic work on other markets, or only the one you picked?
- **Regime context**: does it work across bull, bear and chop, or only one regime?

See the [Robustness reference](/docs/reference/robustness) for what each check means.

## 3. How much was luck?

Run **Monte Carlo** to resample the trade order thousands of times. It shows the range of outcomes you
could have had and how deep a drawdown to expect. If the 5th percentile path would have wiped you
out, the strategy is riskier than the single equity curve suggests.

## 4. Discount for the search

If you tried many configurations, read the overfitting aware statistics. A **Deflated Sharpe** that
drops far below the raw Sharpe, or an **Overfit Probability (PBO)** at or above 50%, means the result
is mostly selection luck.

<Callout variant="success" title="A survivor is a candidate, not a guarantee">
  Passing every check makes an edge *plausible*, not certain. Backtests exclude live latency and
  venue costs, and past performance never guarantees future results. Size accordingly.
</Callout>

Background reading: [why overfitting matters](/docs/concepts/overfitting).