The Problem With Backtesting

My next few posts will cover the common mistakes people make with backtesting.

No, wait, that should read "the mistakes I made with backtesting".

Backtesting, I think, is essential. You've probably heard over and over again that past performance does not necessarily reflect future results. And that's very true - essentially the future prices would need to be exactly the same as the ones you tested it on for you to guarantee the system's performance. Which won't happen. Probably.

But surely you wouldn't trade with real money on a system you knew to be a dog for the last couple of years?

So I guess backtesting lets you filter your list of systems to a just a couple that have been performing well up to now.

This should all tie in with your Equity Curve Analysis. You only trade real money on the systems that meet your criteria for allowing real money to be traded. If you're just starting off, then the only way to know if it meets you criteria is to backtest and found out.

Or you could paper trade every system you have thought of in real time, until one or two of them meet your critiera. But I don't think anyone has the patience for that. So it's off to backtest town for us.

But, and here's the thing, you can do backtesting the wrong way. And when that happens you're in possession of faulty results, and you'll be making your decision to trade a system on faulty information.

Here's a tip - whenever you create a system that has mind-boggling results - results which would make you rich beyond your dreams in about 6 months, and leaves you kind of giddy with excitement - you've got a bug in your backtesting process. For sure. It's happened twice to me. The good part is, after you get over the depression brought about from realising you're not going to be rich in 6 months, you've found your bug and your backtesting process has become that little bit better.

So, the next few posts will be about the mistakes that I made, and hopefully you won't need to make them yourself.