Every trader who’s backtested a strategy has experienced the high: that moment when the equity curve goes up and to the right, the Sharpe ratio looks fantastic, and you’re mentally calculating your annual returns.
Then you go live and it falls apart within two weeks.
We build and validate trading systems for a living. Here are the five mistakes we see most often — and they’re not the ones you think you already know about.
1. Overfitting (You Probably Have More Parameters Than You Think)
Everyone knows overfitting is bad. Fewer people recognize when they’re doing it.
Here’s the test: count every decision you made while developing the strategy. Not just the explicit parameters — the lookback period, the threshold values — but also the implicit ones. Did you choose a 15-minute timeframe after trying 5, 15, and 60? That’s a parameter. Did you filter out Mondays because they looked worse? Parameter. Did you set your stop at 2 ATR instead of 1.5 or 2.5? Parameter.
A strategy with 3 explicit inputs but 8 implicit decisions made during development has 11 degrees of freedom. You just can’t see most of them because they’re baked into the design.
The practical fix: track every decision in a development journal. When you’re done, ask yourself: with this many choices, how surprising is it that something worked on this specific dataset? If you tested 50 combinations and one of them has a great equity curve, that’s not a discovery — that’s statistics.
We run walk-forward analysis on every strategy we build (it’s part of our development process). The strategy is optimized on a training window, then tested on unseen data, then the window rolls forward. If performance holds across multiple out-of-sample periods, you might have something real. If it doesn’t, you saved yourself from a very expensive lesson.
2. Lookahead Bias (Sneakier Than You Think)
The textbook version of lookahead bias is using tomorrow’s close to make today’s decision. Nobody does that on purpose. The real-world version is much more subtle.
Example 1: Indicator calculation timing. You’re using a VWAP indicator that resets daily. In your backtest, the final VWAP value for the day is calculated using the full day’s data. But your strategy makes a decision at 2:00 PM. At 2:00 PM, VWAP doesn’t include the data from 2:01 PM to 4:00 PM. Does your backtesting engine know that? Some do. Many don’t.
Example 2: Corporate event data. You’re filtering stocks based on earnings dates. In your historical data, you know company X reported earnings on March 15th. But on March 10th — when your strategy would have made a trade decision — was that date publicly known? Earnings dates get moved. Some databases only store the final reported date, not when it was announced.
Example 3: Index composition. You’re backtesting a strategy on current S&P 500 components going back 10 years. But the S&P 500 in 2016 had different stocks than it does today. Companies that went bankrupt, got acquired, or were removed are missing from your universe. You’re only looking at survivors. (More on this in mistake #4.)
The practical fix: for every data point your strategy uses, ask: “at the moment of the trading decision, would I have actually had this exact value?” If you can’t answer confidently, you have a potential lookahead problem.
3. Ignoring Slippage and Commissions (The Silent Killer)
This one is almost embarrassingly simple, yet it kills more strategies than any other mistake.
I’ll give you a real example. A trader came to us with a scalping strategy on ES futures. Backtested beautifully — 60% win rate, 1.5:1 reward-to-risk, traded about 12 times per day. On paper, $400/day net profit.
We added realistic execution costs:
One tick on ES is $12.50. So each round trip actually cost $4.50 (commission) + $25.00 (1 tick slippage × 2 sides) = $29.50.
At 12 trades per day: $354 in daily execution costs.
His $400/day strategy was actually a $46/day strategy. Before taxes. Some days, after wider slippage during volatile periods, it was negative.
The scaling problem: slippage isn’t fixed — it gets worse with size. Trading 1 contract, you might get 1 tick of slippage. Trading 10 contracts on a moderately liquid instrument? Expect 2-3 ticks on the aggressive side. Your backtest doesn’t know this.
The practical fix: always model conservative execution costs. For liquid futures (ES, NQ, CL), assume at least 1 tick slippage per side. For less liquid instruments or small-cap stocks, double or triple that. If your strategy is only profitable with zero slippage, it’s not profitable.
On platforms like NinjaTrader and QuantConnect, you can configure slippage models directly in the backtester. Use them. On Pine Script, you’re limited to the `strategy()` function’s `slippage` parameter — set it to something nonzero.
4. Survivorship Bias (The One Nobody Checks)
If you’re backtesting an equity strategy on a current list of stocks, you’re only looking at companies that survived. The ones that went bankrupt, got delisted, or crashed 90% aren’t in your data. This makes every historical analysis look better than reality.
How much better? Studies put it at 1-2% annual return inflation for broad equity strategies. For strategies that specifically buy distressed or small-cap stocks, the bias can be significantly larger.
A concrete example: you’re testing a momentum strategy that buys stocks making new 52-week highs. In your backtest on current S&P 500 stocks going back to 2018, the strategy looks great. But Bed Bath & Beyond was in the index in 2018. So was Xerox before it was removed. Your backtest never bought these stocks because they’re not in your current universe — but a live strategy in 2018 might have.
The practical fix: use a survivorship-bias-free database. QuantConnect’s Lean engine handles this natively with point-in-time data — it’s one of the platform’s genuine strengths. For NinjaTrader and other platforms, you’ll need to source delisted stock data separately. Norgate and similar providers offer survivorship-bias-free datasets, but they cost money. It’s worth it if you’re trading equities.
For futures traders: survivorship bias is less of an issue since you’re trading the same contracts (ES, NQ, etc.), but you still need to handle contract rollovers correctly. Front-month backtesting with improper roll logic creates its own class of data problems.
5. Curve-Fitting with Too Many Parameters
This is overfitting’s close cousin, but it’s specific enough to warrant its own section.
A strategy with 2-3 parameters is testable. You can visualize performance across a reasonable parameter space and see if there’s a stable plateau. A strategy with 12 parameters is unfalsifiable — there will always be some combination that works on any historical dataset, purely by chance.
The rule of thumb we use: you need roughly 10 trades per parameter per optimization window to have any statistical confidence. If your strategy trades 200 times per year and has 10 parameters, you need 10 years of data just for the training set. That’s before reserving anything for out-of-sample validation.
Most traders we talk to have 2-3 years of data and 8-15 parameters. The math doesn’t work.
What to do about it:
1. Reduce parameters aggressively. Can two of your inputs be combined? Can a threshold be derived from market data (like ATR) instead of being a fixed number? Every parameter you eliminate makes the remaining ones more trustworthy.
2. Use walk-forward analysis, not single-pass optimization. Optimize on 2020-2021, test on 2022. Optimize on 2021-2022, test on 2023. If the strategy consistently works on the out-of-sample windows, the parameters are capturing something real.
3. Check parameter sensitivity. If your strategy works at a lookback of 14 but fails at 13 and 15, that’s not a robust parameter — it’s noise. A good parameter has a neighborhood of values that all perform reasonably well.
When to Get Professional Help
If you’ve read this far and you’re thinking “I might have some of these problems,” you probably do. Most self-backtested strategies have at least two of these issues.
That’s not a knock on your abilities. Proper strategy validation is tedious, requires specific tooling, and benefits from having seen hundreds of strategies — both the ones that worked and the ones that didn’t.
Here’s when it makes sense to bring in a professional:
We build strategies with walk-forward analysis, realistic execution modeling, and out-of-sample validation as standard practice — not optional add-ons. You can see how that works on our process page.
Ready to Automate Your Trading Strategy?
Whether you’re looking to build a new algorithm from scratch or optimize an existing strategy, our team of experienced quant developers can help. Book a free 30-minute consultation to discuss your project and get a custom quote.