Backtesting
The flight simulator for strategies — priceless for training, lethal if you confuse it with flying. This lesson: what a backtest really does, and the four ways it lies to you.
8.1The flight simulator
A is a time machine with strict rules: take a strategy, hand it years of old candles one bar at a time (it can never peek ahead), let it place pretend trades with pretend money, charge it realistic , and count the results. You've already read one of its report cards in Lesson 2 — win rate, profit factor, drawdown, Sharpe.
Why bother? Because it's the only way to ask "would this rule have worked?" a thousand times for free. A pilot who crashes the simulator two hundred times has learnt something priceless without a scratch. That's what the simulator is for: rejecting bad ideas cheaply.
Beginners use backtests to confirm their idea works. Professionals use backtests to try to kill their idea — and only trust what survives the assassination attempts. Everything below is an assassination technique.
8.2Lie #1 — overfitting, the suit tailored to yesterday
is the big one. Imagine tailoring a suit so precisely to how you stood yesterday — mid-stride, arm half-raised — that it only fits that exact pose. Magnificent in the fitting room; unwearable on the street. A strategy with enough rules and tuned numbers can "fit" the past perfectly — because it has memorised the past's accidents, not learnt the market's habits.
⚙ Overfitting lab — add rules, watch the future leak away
Notice the gap: past performance always improves as you add rules — that's the trap. The future peaks early, then every extra rule is tailoring to yesterday's pose. The cure is Lesson 9.
8.3Lie #2 — peeking, the open answer sheet
is the exam-answers lie: the strategy accidentally sees information it couldn't have had at the time — today's close before today ends, tomorrow's news today. Even one line of sloppy code does it, and the backtest becomes a student marking their own exam with the answer sheet open. The results look like brilliance; they're just cheating that nobody noticed, including the cheater.
8.4Lie #3 — free lunches, the skipped toll
Every real trade pays the spread toll, occasional , and swap fees for positions held overnight. A backtest that skips them flatters every strategy — and it flatters the seagulls most of all, because small bites pay the toll most often:
--spread, --commission), with a full transaction-cost model behind it.8.5Lie #4 — the highlight reel
Test a trend-follower only on a year of lovely trends and it looks like genius; run the same strategy across a choppy year and it looks mortal. One golden window is a movie trailer, not the movie:
8.6Reading a backtest like a skeptic
Profit factor above ~1.3 after costs? Below that, the edge may be noise.
Enough trades? 30 trades prove nothing; 300 start to. Small samples are the highlight reel in disguise.
Max drawdown you could actually live through? A 40% drawdown on paper is a strategy you'd abandon in real life — which makes its backtest irrelevant.
Simple enough to explain to a friend? Every rule you can't justify is probably tailoring. The wiggly line always has a story for why it needed one more bend.
Your backtest engine runs from one command: python -m python.backtest.run_backtest --symbol EURUSD --timeframe H1 --strategy ma_crossover. It replays history bar by bar, charges costs, and writes the full report card (trade log, metrics, equity chart) to results/. There's also a Dash UI for browsing runs — and a regression suite that re-runs standard backtests to make sure code changes never silently change results.