Gain Capital

This tutorial starts from a brand new settings file, loads a text file of historical tick prices, creates price bars from those ticks, and charts the price bars. The file in question was obtained from Gain Capital, but the theory is the same for text files from any source.

Note, however, that the file we used in this particular case was an older one. Gain Capital's newer files are slightly different. See the settings to use on the File Loader window.

This tutorial assumes:

1. You started ATM, created a new settings file, opened it, and ATM is sitting there ready to go.
2. You downloaded some text files containing historical prices.

This tutorial does these things:

1. Loads the historical tick prices from the files.
2. Creates 10-Minute bars from the ticks.
3. Creates Hourly bars from the 10-Minute.
4. Charts the 10-Minute bars.

A very similar process is described (in words) in this post.

Click here to watch the video, and use the browser's back button to come back. Or click here to open in a new window.
 

See also:

• Video: Backtesting

The File format and Dates in the file(s) fields on the File Loader window have standard settings based on where you got the files from. (You could also just look in the file).

By "file formats", I mean how the price data is arranged in the text file. As an example, here's how Gain Capital price data looks like:

277248713,EUR/USD,2006-11-05 17:00:02,1.271600,1.271900,D

Comparing that with one from an Oanda text file, you can see they are completely different:

13/02/04 08:36:34,1.280600,1.280800
 

Gain Capital

 
Older files up to and including 2009/05 May/Week3

Order of fields - ID, Currency, Data, Bid, Ask, "D"
Delimiter - Comma
Ignore first row - No
Date format - yyyy-MM-dd HH:mm:ss
Timezone - New York (GMT-5/-4)
 

Files from 2009/05 May/Week4 up to and including 2009/11 November/Week3

Order of fields - ID, Currency, Data, Bid, Ask, "D"
Delimiter - Comma
Ignore first row - Yes
Date format - yyyy-MM-dd HH:mm:ss
Timezone - New York (GMT-5/-4)
 

Newer files from 2009/11 November/Week4 to present

Order of fields - ID, "D", Currency, Data, Bid, Ask
Delimiter - Comma
Ignore first row - Yes
Date format - yyyy-MM-dd HH:mm:ss
Timezone - New York (GMT-5/-4)
 

Oanda

Order of fields - Date, Bid, Ask
Delimiter - depends on what you requested from Oanda. XML files cannot be used.
Ignore first row - check the file
Date format - dd/MM/yy HH:mm:ss (note only 2-digit year)
Timezone - Greenwich Mean Time (GMT)
 

DukasCopy

Order of fields - Date, Time, Volume, Open, Close, Min, Max
Delimiter - depends on what you requested from DukasCopy
Ignore first row - check the file
Date format - can't remember - you should check the file
Timezone - Greenwich Mean Time (GMT)

Something to be aware of with these files is that there is no spread included. Therefore ensure you use a few pips of commission when backtesting, to try and mimick the spread. If you fail to do so, you're backtesting results will be overly fantastic.
 

ATM price history files (*.tsphd)

Order of fields - ATM Price History File (*.tsphd)
Delimiter - choose any
Date format - choose any
Timezone - choose any

The File Loader has some mandatory fields that aren't actually mandatory in this case. Just choose any value to get past the mandatory field check.

As per this wiki entry, the price history files have very specific names, e.g. fxcm_23_1d.tsphd. The ones being used by ATM (i.e. the ones which correspond to Data Manager entries) cannot be renamed at all. But if you are wanting to merge a price history file with another, most likely it means you got one from somewhere or someone else. This file you can rename, if you are going to use the File Loader window to merge it. But make sure you keep the file prefix intact. That is, add another dot and type after it, e.g. fxcm_23_1d.temp.tsphd or fxcm_23_1d.copy.tsphd.
 

Text files exported by ATM v3+

Order of fields:

• for files containing tick prices - Date, Utc, FIP ID, Symbol, Bid, Ask, Volume
• for files containing price bars - Date, Utc, FIP ID, Symbol, Bid OHLC, Ask OHLC, Volume

Delimiter - depends on what the setting was when the file was exported.
Ignore first row - check the file
Date format - yyyy-MM-dd HH:mm:ss
Timezone - choose any

As the date and Utc offset is written into these files, the timezone field is not actually required. So just choose any value to get past the mandatory field check.
 

Text files exported by ATM pre-v3

Order of fields:

• for files containing tick prices - Date, Currency, Bid, Ask
• for files containing price bars - Date, Currency, Bid OHLC, Ask OHLC

Delimiter - depends on what the setting was when the file was exported.
Ignore first row - check the file
Date format - yyyy-MM-dd HH:mm:ss
Timezone - Greenwich Mean Time (GMT)
 

Other

ATM supports just about any date format, and any 1-character delimiter. So if you have a text file from somewhere else, all you have to do is somehow get the order of fields to match one of the available options.

If that's beyond your technical prowess, or you know of another popular file format that you think we should support, you can add a post to the feature request forum.
 

When you create an account in ATM you have to enter an account number. Read below for what to enter, as it differs by broker.
 

FXCM

To find the account number, first log in to the FXCM trading platform. You should be able to see an "Accounts" section, with all your accounts listed there. For a demo account, there's probably only one account listed. Anyway, in that "Accounts" section, there should be an "Account" column. The value in that field is close to, but not the value you want to type into ATM.

First, if the account number has letters at the start, don't type them.

Second, if the account number has zeroes at the start, don't type them.

Third, if the number is longer than 8 digits, only type the last 8 digits.

So an account number like:

• 0012346 would be entered into ATM as 123456
• ABC000123456789 would be entered as 23456789
   

Oanda

To find the account number, first log in to the Oanda trading platform. Then from the main menu of that trading platform, click "Account", then choose "Change Account". You should now be able to see a list of your accounts. The number to enter in this field is the number in brackets. E.g. if it says something like "Primary (1234567)", then enter 1234567 in this field in ATM.
 

Gain Capital

Doesn't give out account numbers, so just enter 0 (zero).
 

If you're going to download any of the text files containing historical price data from the brokers, there is some free software that is going to make your life a whole lot easier.
 

downTHEMall!

Especially with Gain Capital, there are *a lot* of files to download. If you do one at a time you'll probably go insane. I used to do it that way until I found a better method. Using a combination of the free Firefox web browser (an Internet Explorer replacement), and a free Firefox extension called downTHEMall!, you can download a month's worth of files with a couple of clicks.

Here are the steps:

1. Download Firefox.
2. Install Firefox.
3. Run Firefox.
4. Using Firefox, browse to the downTHEMall! extension page.
5. Click on the link that says to "Install Now for Windows".
6. When that finishes, close down Firefox.
7. Restart Firefox.

Now you're ready to get the historical price data.

Using downTHEMall! also ensures that if you are disconnected for any reason, the download can pick up from where it left off (and not have to re-download the file from the beginning).
 

Text File Cleaver

Some of the files you download, especially those from Oanda, are huge. The Oanda ones contain around 20 million rows of tick price data, and ATM will probably not be able to load a big file like that in one go because of the database timeout setting on the Options window.

Instead, please use a file-splitting utility such as Text File Cleaver. Using the default options, Text File Cleaver splits the one big file into a number of smaller files, each containing 65,000 rows. ATM can easily process those files individually within the timeout period. For the Oanda file I tested with, 299 smaller files were produced.

The reason for recommending Text File Cleaver (aside from that it's free), is that it keeps each row intact. Other file splitters just split the file after a particular number of characters, which often falls in the middle of a row of price data.

Simply follow this link, scroll down to find the "Text File Cleaver" section, download and install.
 

Gain Capital files are available to all - no account required.

1. Download the files from http://ratedata.gaincapital.com/ to your computer.
2. They are zip files, so unzip them.

Then follow the rest of the instructions to load text files and create price bars.
 

An easier way to download

Gain Capital gives one file per week per currency. About 20 currencies equals about 80 files per month and about 960 files per year. That's a lot of clicks. downTHEMAll! will make the downloading much easier than doing it one-by-one.

1. Browse to http://ratedata.gaincapital.com using Firefox.
2. Click through to the month of data that you want to download.
3. Click "Tools" > "downTHEMAll!" > "downTHEMall!..." from the main menu (in Firefox).
4. On the downTHEMall! window:
   1. In the Filters area at the bottom, check the "Archives (zip, rar...)" checkbox.
   2. Click the Browse button to select the directory you want to download the files to.
   3. Click the "Start Downloads!" button.