Checklists

The files are available at http://freeserv.dukascopy.com/exp/. Also www.dukascopy.com/swiss/english/data_feed/csv_data_export/.

But you can't start the downloading of all the files and go and do other things - you have to export the data you want one-by-one. Scroll about 4/5 of the way down to get to the currencies available. They have |f| next to them. You can choose any format you like, but make sure you choose "Windows" in the "End line" field.

The good part is that they have data going back to 1986 for some currencies.

DukasCopy provides indicative data, which has no spread. During backtesting, make sure you add on a few pips in the "Commission" field to simulate the spread, otherwise your backtesting results will be overly optimistic.

The Currency Secrets website has covered the issue of indicative data in some depth, so I point you to their pages:

• Beware Of indicative data
• Is indicative data really *that* bad?

p.s. I think their conclusion is that as long as you add the spread, it should be ok.

I guess it's always better to use the prices from the broker you are going to use, if possible.

Once they are finished downloading, follow the rest of the instructions to load text files. DukasCopy historical prices are already in price bar format though. So you don't need to do the steps to create price bars, unless you want to create other price bars not provided by DukasCopy.
 

The method described below works the same for any symbol/interval/broker. We will use the example of creating EUR/USD Hourly and Daily price bars from historical tick prices provided by Gain Capital.

These steps assume you have already downloaded the text files containing the historical prices to your computer. See here for instructions for:

• Gain Capital
• Oanda
• DukasCopy
   

Create the Data Manager entries

Here's a good explanation of what Data Manager entries do.
 

1. Create a Data Manager entry to store the tick prices

1a. In ATM, click "Price Data" > "Price data configuration"
1b. Click "Add"
1c. In the top box, choose EUR/USD, Tick, Gain Capital
1d. In the bottom box, choose the bottom option - "Just set up (or keep)..."
1e. Click OK.
 

2. Create a Data Manager entry to store the Hourly bars

2a. (Should still have the Data Manager Search window open).  Click "Add"
2b. In the top box, choose EUR/USD, Hourly, Gain Capital
2c. In the bottom box, choose the second option - "...shorter time-period..."
2d. Choose "EUR/USD, Tick, Gain Capital" from the list.
2e. Click OK.
 

3. Create a Data Manager entry to store the Daily bars

3a. (Should still have the Data Manager Search window open).  Click "Add"
3b. In the top box, choose EUR/USD, Daily, Gain Capital
3c. In the bottom box, choose the second option - "...shorter time-period..."
3d. Now you have a choice - you can either choose "EUR/USD, Tick, Gain Capital" from the list, just like you did with the Hourly bars. That would mean the Daily bars are also made from the tick prices. Or, you can choose "EUR/USD, Hourly, Gain Capital", to have the Daily bars made from the Hourly. It's up to your preference, but the second option would be far quicker.
3e. Click OK.
 

Side note #1:

Each Data Manager entry creates a file on your computer which will store the prices. If you look in your file system you should now see 3 new files: gain_32_tk.tsphd, gain_32_1h.tsphd, and gain_32_1d.tsphd, respectively.
 

Side note #2:

The number 32 is what we call the "FIP ID", or "Financial Product ID". It's a number only useful within the Thinking Stuff world. 32 equates to the currency with symbol "EUR/USD".
 
Now to get the price history from the text files into the ATM files. To do this, we first create a "file load task", then process the task.
 

Load the files

 
4. Create the file load task

4a. Click "Price Data" > "Price history file loader"
4b. Files to load - browse for and select the file(s) to load. Recommend to just choose 1 file if this is your first time.
4c. Fill in the "File format" and "Dates in the file(s)" fields as per these standard settings for your broker.
4d. Price data to be loaded - EUR/USD, Tick, Gain Capital
4e. Click "Add To List"
 

5. Process the task

5a. Still on the File Loader window, go to "The Task List" tab
5b. Click "Start Loading"
5c. Wait.
 

6. Confirm the task worked

6a. Go to the "Completed List" tab.
6b. Scroll all the way to the right to see the result - "Success" is good. Anything in the "Error Text" field is bad.
6c. Back to the Data Manager Search window.
6d. Right-click on the "EUR/USD, Tick, Gain Capital" entry. A little pop-up menu appears. Choose "Table Information (row count, etc)"
6e. The message that shows up should display a count greater than zero.
 
So now we have the tick prices. What comes next is creating the Hourly and Daily price bars. This time we create "interval creation tasks", then process the tasks.
 

Create the price bars

 
7. Create the interval creation tasks

7a. (Should still have the Data Manager Search window open). Click (normal left-click) on the "EUR/USD, Hourly, Gain Capital" entry to highlight it.
7b. At the bottom of that window, go to the "Bulk Price Bar Creation" tab.
7c. Click "Add To List"
7d. Click on "EUR/USD, Daily, Gain Capital" to highlight it.
7e. Click "Add To List"
 

8. Process the tasks

8a. Still on the Data Manager Search window, go to "The Task List" tab
8b. Click "Start Creating Price Bars"
8c. Wait.
 

9. Confirm the tasks worked

9a. Go to the "Completed List" tab.
9b. Scroll all the way to the right to see the result - "Success" is good. Anything in the "Error Text" field is bad.
9c. Back to the "Price Data Configurations" tab.
9d. Right-click on the "EUR/USD, Hourly, Gain Capital" entry. A little pop-up menu appears. Choose "Table Information (row count, etc)"
9e. The message that shows up should display a count greater than zero. Same for the Daily one.
 

Cleanse the prices

Not all text files are created equal. Some have minor errors which can throw out your backtesting results. The easiest way to check for these errors is to view the chart. Ridiculous prices are immediately obvious.
 

10. View the chart

10a. Click "Price Data" > "View charts"
10b. Fill in the search criteria.
10c. Click "Chart"
 

11. Cleanse the data

11a. If the chart looks fine then you're all done.
11b. If there are extreme spikes in the chart then it's possible there is some erroneous data in the text files. Follow these steps to perform the cleanse.
 

[Note: these instructions for Oanda were written a couple of years ago. If you have more recent experience of obtaining their historical prices we would appreciate an update].

Oanda, through the FXTicks tool in their FXLabs section, offers tick data going back to January 1st, 2004. This is for five currencies only though - EUR/USD, GBP/USD, USD/CAD, USD/CHF, USD/JPY.

From the website:

In order to qualify for this service, you must have an FXTrade account with a balance of 1000 USD or more. Data may be ordered as far back as January 1, 2004, and as recently as two (2) months ago.

Steps to obtain the files:

1. Go to Oanda to order the files.
2. Fill in the form on that webpage. ATM can handle ASCII Text, CSV, and Pipe-delimited.
3. Repeat step 2 for all currencies that you want the data for.
4. Wait up to 2 weeks for Oanda to process your request. When they notify you, they will provide you with a website link to click.
5. Click the link and download the file. Whether you use downTHEMall! or not is up to you.

Now you have the files downloaded to your computer. But they are huge! Seriously. They are going to be more manageable if you split them into smaller files. That's where Text File Cleaver comes in.

Start Text File Cleaver, select the file to cleave, and select the directory to put the small files in. Click the "Go!" button. Repeat that for each big file you downloaded.

Then follow the rest of the instructions to load text files and create price bars. But on the File Loader window, select the smaller files to load instead of the big file.
 

There are many different combinations which make up a "complete" trading system. The most basic trading system is to have one entry rule, one entry value, and one initial stop loss value.

While that is technically "complete", it's not much of a system. Generally your trading system will fall into one of two categories:

1. Where you set both a stop loss and a take profit, and wait for one or the other to be hit; or
2. Where you set only a stop loss, but you gradually move that stop loss to lock in profit. Your stop loss will be taken out at some point, but you hope that by that time it would have moved above your initial entry price.

In keeping with that theory, here is what makes up a complete trading system. Either:

1. Entry rules, entry values, initial stop loss values, initial take profit values; or
2. Entry rules, entry values, initial stop loss values, stop loss management rules, stop loss management values.

With both of those, you are free to add any other settings you like. Of course you can combine the two methods of making money by having both a trailing stop loss and take profit set. But the above are the minimum requirements for orders to be placed, and to give at least a chance of making a profit.

Systems which only go long, or which only go short, are completely acceptable.
 

The first step is to make sure you understand about rules and values.

the second step is to make sure your trading system has the minimum required rules and values.

Now, most likely you are reading this article because even though you have read and implemented the above, your trading system refuses to make any trades in your account. So now we will focus on the possible causes. The first step is to diagnose where the problem exactly is.

And this is actually quite simple. Just backtest over a decent number of bars - enough so that at least a few trades should be taken. More is better. The backtest results will fall into one of 3 scenarios:

1. No trades are made at all.
2. Trades are made initially, but then stops.
3. Trades are made often, as you expect.

For the first 2 scenarios, the things to check are listed in the corresponding entry for getting a trading system to trade - backtesting.

Once you trading system does exactly as you expect while backtesting, come back here and finish this checklist.
 

Trades normally

If your backtest results show trades taken as you expected, then the problem is not with your trading system, but with some other setting related to connecting to your broker. Check these:

• Do you have an account yet? If no then read this.
• Is the trading system's status set to "Enabled"?
• Are all other settings on the "Auto-trading" tab set correctly? And the account balance is above the minimum setting there?
• With the account(s) you chose to trade in:
  • Make sure your account balance is enough to purchase the number of units that the system wants to buy.
  • Is the account number entered correctly?
  • Is the password for the user who owns that account entered correctly?
  • Have you installed the API for that broker?
• Is there a holiday attached that's preventing new orders?
• Are the settings on the "Frequency" tab set correctly?
• Have you set the frequency to stop after a certain number of checks, and that number has been reached?
• Are you sure that the entry rules have been true at some point after starting auto-trading?
• Scheduled processes on the Command Centre:
  • Have you started the price download and/or creation process? Is it working and new price bars are coming in? You can check this by using the Chart window or Price History Search window. If no then read this.
  • Have you started the auto-trading process?
  • Are the auto-trading tasks being processed without error? You can check this on the "Automated Trading Tasks" >> "Completed List" tab. Scroll to the right-side of the result grid. Anything in the "Error Text" field is bad.
• Did you accidentally set the trading system's "last bars used" dates or "last checked" date into the future? This will pause trading until those dates are reached.
• Is it possible that trades are being opened, but just ATM's trade and order information is out of date? Check your account through your broker's platform instead. If trading is taking place, then it's all good and read this.
   

Graphical representation of the checklist

Here's what it kind of looks like graphically:

If it still doesn't work

If your trading system still refuses to trade, you will need to seek advice in the forum. Give as much information as you can. An exported version of your trading system would be ideal. If you are worried about your privacy, then make a copy of your trading system. Edit the copy, delete the private stuff, and maybe make a generic version of your system that still displays the same problem but doesn't give away your special sauce. Export that one instead.

Screenshots are also good. We've found that glancing through these screenshots, it takes about 20 seconds to diagnose the problem. Without the screenshots or the exported system it takes a lot longer.
 

The first step is to make sure you understand about rules and values.

the second step is to make sure your trading system has the minimum required rules and values.

Now, most likely you are reading this article because even though you have read and implemented the above, your trading system is not opening trades during the backtest. So now we will focus on the possible causes.

Firstly, are you sure that the test has enough price bars in it for the entry rules to be checked? If you have a moving average of period 200 in the rules, then you need at least 200 price bars to backtest with.

Secondly, are you sure that the entry rules can be true at some point during the test?

With that out of the way, now we focus on the results of the backtest. How the system behaved during the backtest will fall into one of 3 scenarios:

1. No trades are made at all.
2. Trades are made initially, but then stops.
3. Trades are made often, as you expect.
    

1. No trades

This is an indication that your trading system is not set up correctly. Try these:

• Are you sure that you have corresponding values for all the rules?
• Are you sure that you have the minimum required rules and values?
• Do you have any rules which will never be true together? E.g. one entry rule says to enter when the close is above the moving average, while another entry says to enter when the close is below the moving average.
• Long trading - make sure the stop loss will work out to be lower than the entry price.
• Long trading - make sure the take profit (if set) works out to be higher than the entry price.
• Short trading - make sure the stop loss will work out to be higher than the entry price.
• Short trading - make sure the take profit (if set) works out to be lower than the entry price.
• Make sure your account balance is enough to purchase the number of units that the system wants to buy.

Once you have fixed any problems, run the backtest again and start the checklist from the beginning.
 

2. Trades initially, but then stops

The first thing to check is your account balance. If the trades that were made ended up in your balance falling and falling, then eventually there won't be enough balance to make any more trades. This is expected behaviour and in fact the only thing that's wrong is that your system is a dud :-)

If that is not the case, then most likely the cause is related to the "No Trading" settings. These settings can interfere with each other so that after a few trades no further trades are possible. Here's an example:

• Allow up to and including 2 long trades in a row.
• There must be 3 long trades in a row before allowing short trades or orders.

Taken together, it means there will be 2 long trades and then nothing more. After those 2 long trades, the first rule disallows any further long trades until a short. And the second rule is still waiting on another long trade before it allows any shorts.

Once you have fixed any problems, run the backtest again and start the checklist from the beginning.
 

3. Trades normally

If your backtest results show trades taken as you expected, then that is problem solved if all you wanted to do was backtest. If however you are reading this article because you are unsuccessfully trying to auto-trade, head to the corresponding entry for getting a trading system to trade - autotrading.
 

If it still doesn't work

If your trading system still refuses to trade, you will need to seek advice in the forum. Give as much information as you can. An exported version of your trading system would be ideal. If you are worried about your privacy, then make a copy of your trading system. Edit the copy, delete the private stuff, and maybe make a generic version of your system that still displays the same problem but doesn't give away your special sauce. Export that one instead.

Screenshots are also good. We've found that glancing through these screenshots, it takes about 20 seconds to diagnose the problem. Without the screenshots or the exported system it takes a lot longer.
 

To download price data from a broker, you need four things:

1. A user set up for the broker you are trying to download prices from. The password must be entered correctly.
2. At least one account set up for that user. The account number must entered correctly.
3. Data Manager entries, where you have specified to download the prices, and choose the user you created in the step above.
4. The API for your broker properly installed on your computer.

You wouldn't have been able to save the Data Manager entry without a user selected. But you could save it without an Account for that user. Just remember that in ATM, each user needs a corresponding account set up for it before the user will actually be used for anything. It's just that one user could have many accounts, so we had to split them up.
 

Gain Capital files are available to all - no account required.

1. Download the files from http://ratedata.gaincapital.com/ to your computer.
2. They are zip files, so unzip them.

Then follow the rest of the instructions to load text files and create price bars.
 

An easier way to download

Gain Capital gives one file per week per currency. About 20 currencies equals about 80 files per month and about 960 files per year. That's a lot of clicks. downTHEMAll! will make the downloading much easier than doing it one-by-one.

1. Browse to http://ratedata.gaincapital.com using Firefox.
2. Click through to the month of data that you want to download.
3. Click "Tools" > "downTHEMAll!" > "downTHEMall!..." from the main menu (in Firefox).
4. On the downTHEMall! window:
   1. In the Filters area at the bottom, check the "Archives (zip, rar...)" checkbox.
   2. Click the Browse button to select the directory you want to download the files to.
   3. Click the "Start Downloads!" button.