When Backtesting Will Differ From Reality

There are some trading systems settings that can't actually be backtested.
 

Holidays

It's because all you know about the price movement in any given bar is the Open, High, Low, and Close. That's it. You have a zero percent chance of knowing exactly what the price did between those extreme points.

So let's say you are using Hourly bars for backtesting. And you have a holiday that goes from 12:15am to 12:45am, because you want to stop trading during a major announcement scheduled for 12:30am.

Auto-trading can handle this - it just shuts off at 12:15am, and starts again at 12:45am. However, during backtesting, ATM would have to say "exit when the time becomes 12:15am". Easy. But what was the price at which it exited? i.e. what was the price at 12:15am? No idea.

ATM knows what the price was at 12:00am (the Open), and at 12:59:59am (the Close), and it knows the extreme points it reached between those two times (the High and Low), but it has no chance of knowing what the price was at a particular point between those times. And so backtesting can't take holidays into consideration.

However, there is the "Dates and Times" custom indicator, which you might be able to use instead of holidays, and which will work the same in backtesting as it will in auto-trading.
 

Frequency

If you think about it, telling a system to start trading at particular times is just the same but different as telling it to stop trading at particular times. So the frequency settings suffer from the same problem as holidays above.

Again, you would have to try to emulate your desired start times by using the "Dates and Times" custom indicator.
 
See also:

• Pessimistic backtesting results