Metrics
Every performance metric TextToQuant reports, what it means, and how to read it.
Every backtest returns the same set of metrics so you can compare strategies on equal footing. This page is the definitive reference: what each number measures, and, just as important, how to read it without fooling yourself.
Metrics fall into four groups: headline performance, risk adjusted return, trade quality, and the overfitting aware statistics that make a result trustworthy.
Headline metrics#
These six appear on every results card. They answer "did it make money, and how?"
| Metric | What it measures | How to read it |
|---|---|---|
| Total return | Net profit or loss over the test window, as a percentage of starting capital | The top line result, but never judge it alone; a big return with a huge drawdown is fragile |
| Win rate | Share of trades that closed profitable | High win rate ≠ profitable. A 40% win rate can beat a 70% one if winners are bigger |
| Profit factor | Gross profit ÷ gross loss | Above 1.0 is profitable; 1.5+ is healthy; be suspicious of very high values on few trades |
| Sharpe ratio | Return per unit of total volatility | >1 is good, >2 is strong, but see the overfitting section before trusting it |
| Max drawdown | Largest peak to trough equity drop | The pain metric. Ask yourself: could you sit through this loss live? |
| Total trades | Number of closed trades | Sample size. Under ~30 trades, treat every other metric as noisy |
A strategy is a trade off between return, risk, and reliability. Total return tells you the reward, max drawdown tells you the risk, and total trades tells you how much to trust either.
Return over time & exposure#
Two runs of different lengths, or with very different time in the market, aren't directly comparable on raw return. These normalise for that.
| Metric | Meaning |
|---|---|
| Annualized return (CAGR) | Total return expressed as a yearly rate, so a 6-month and a 3-year run compare on equal footing |
| Exposure / time in market | Share of bars you actually held a position. A strategy in the market 5% of the time carries very different risk from one always in |
| Avg / median bars held | Typical holding time per trade |
Risk adjusted return#
Raw return ignores how much risk you took to earn it. These metrics divide reward by risk, so a calm strategy and a wild one become comparable.
| Metric | Divides return by | Use it when |
|---|---|---|
| Sharpe ratio | Total volatility (up and down) | The default risk adjusted score |
| Sortino ratio | Downside volatility only | You don't want to be penalised for big winning months |
| Calmar ratio | Max drawdown | You care most about surviving the worst stretch |
Because Sharpe penalises upside and downside equally, a strategy with occasional large gains can look worse than it is. That's when Sortino tells the truer story. Calmar is the one to watch if your real constraint is "how deep a hole can I tolerate?"
More risk-adjusted & sizing scores#
| Metric | Meaning |
|---|---|
| Omega ratio | Probability-weighted gains ÷ losses around a threshold, captures return skew the Sharpe misses |
| Information ratio | Return above a passive buy-&-hold, divided by the volatility of that excess. Differs from Sharpe, which compares to a flat zero |
| SQN (System Quality Number) | Trade expectancy scaled by √trades (Van Tharp), one number for how tradable the system is |
| Kelly fraction | The bet size the edge implies. A ceiling to respect, not a target, full Kelly is famously wild |
Trade quality#
Headline numbers hide how the money was made. These read the trade ledger directly.
| Metric | Meaning |
|---|---|
| Expectancy (R) | Average profit per trade, expressed in units of risk (R). Positive expectancy is the whole game |
| Average win / loss | Mean profit of winners vs mean loss of losers |
| Payoff ratio | Average win ÷ average loss, how much bigger winners are than losers |
| R multiple | Each trade's result as a multiple of the risk taken to enter it |
| MAE / MFE | Average worst drawdown (adverse) and best unrealised gain (favourable) reached inside a trade, how much heat you sat through before the exit |
| Success rate | Share of trades that closed on one of your exit rules (stop, target, signal) rather than being force-closed at a horizon or the test's end |
A profitable strategy needs win rate × payoff to clear 1. A low win rate is fine if the payoff
is high (trend following), and a low payoff is fine if the win rate is high (mean reversion).
Win rate asks whether a trade made money. Success rate asks whether it exited the way you designed (on a stop, target or signal) versus being closed because it ran out of time or the test ended. A trade can be a win that never hit your target, or a loss that cleanly hit its stop. Read the two together: a high win rate with a low success rate means your exits aren't doing the work.
Long vs short#
The ledger is also split by direction, so you can tell a genuinely two-sided edge from one that only works one way.
| Metric | Meaning |
|---|---|
| Long / short win rate | Win rate computed separately for long and short trades |
| Long / short profit factor, expectancy, avg R | The same trade-quality read, per direction |
A strategy that's only profitable long in a bull market is a very different bet from one that works both ways, this is where you catch it before it costs you.
Overfitting aware statistics#
A raw Sharpe is inflated by how many configurations you tried. Once a parameter search records the trials, these statistics appear on the metric cards, an honest read no other natural language tool ships.
| Metric | Corrects for | How to read it |
|---|---|---|
| Probabilistic Sharpe (PSR) | Short samples & fat tails | Higher = more confident the Sharpe beats 0 |
| Deflated Sharpe (DSR) | How many configs you tried | ≥95% = survives the search; a big drop vs PSR = search luck |
| Overfit Probability (PBO) | In sample best failing out of sample (grid / joint sweep only) | Low is good; ≥50% = likely overfit |
| Haircut Sharpe | Bonferroni correction for T trials | The Sharpe you can still claim after the search |
| Min. backtest length | Sample too short for the search | Warns when history can't support that many trials |
Your query compiles deterministically to a fixed strategy spec, then a fixed engine scores it. The AI never sees, ranks, or tunes the numbers, so it can't steer toward a good looking result. Same query ⇒ same spec fingerprint ⇒ same test. Signals use only closed bar data (no look ahead). The results header shows a Reproducible chip and, when a hold out split exists, an always on out of sample verdict.
How grading uses these#
The letter grade you see on each run is not a single metric. It's a blend across four pillars, so a strategy can't earn an A by maximising return while ignoring risk.
| Pillar | Driven mainly by |
|---|---|
| Profit | Total return, profit factor, expectancy |
| Risk | Max drawdown, Sharpe / Sortino |
| Consistency | Win rate, equity curve smoothness, robustness checks |
| Edge | Sample size and the overfitting aware statistics above |
Use the grade to triage which runs deserve a closer look, then validate the survivors with out of sample and Monte Carlo before you trust an edge.
Ready to see them live? Run a strategy and read its report end to end.
Buy ETH when the 50 EMA crosses above the 200 EMA, exit on the opposite cross, on the 1dRun in terminal