The idea
Design the strategy on one part of history, then test it on another part it has never seen. If the rules capture something real, they should still work, perhaps less well. If they were fitted to noise, the out-of-sample results fall apart. See Overfitting in investing.
Ways to split
| Method | How |
|---|---|
| Simple split | Design on 2005 to 2017, test on 2018 to 2025 |
| Walk-forward | Design on a rolling window, test on the next year, move forward, repeat |
| Different markets | Design on US stocks, test on international |
| Live paper trading | Run the rules forward in real time without money |
Rules for honest testing
- Look once. If you tweak the rules after seeing out-of-sample results and test again, that data is no longer out-of-sample.
- Include different conditions. The test period should contain a downturn.
- Expect some decay. Out-of-sample results are usually weaker. A strategy that loses half its edge may still be worth using; one that loses all of it is not.
Time is the final test
The only truly unseen data is the future. Strategies with real, published track records deserve more trust than any backtest.
See also
- Overfitting in investing What overfitting is, why strategies tuned to past data fail in live trading, what a large study of community strategies found, and how to measure it.
- What is backtesting? What backtesting a strategy means, how a backtest is built, the common biases that make backtests look better than reality, and how to use results wisely.
- Signal mining and data snooping What signal mining is, how testing millions of rules produces convincing but false results, and how to tell a real fundamental signal from data-mining luck.
Pages that link here: If-then signals, Systematic investing
Last updated September 30, 2026. Education only, not investment advice.