Price history

"Data Manager entries" and "Price data configuration entries" are used interchangeably.

You create these to tell ATM (a) what prices you want to gather; and (b) how to gather them.

ATM will create one .tsphd file on your computer for every Data Manager entry. And that's where the prices are physically stored on your computer. But that's the end of it as far as Data Manager entries are concerned. They just do the setup - they don't do any gathering.

Gathering is done on the Command Centre window (downloading), File Loader window (loading from text files), or the "Bulk Price Bar Creation" tab on the Data Manager Search window (creating price bars from other prices you have already downloaded or loaded from file).
 

Location of the .tsphd files

Is set by you when you first create the settings file. It can be viewed and changed on the Options window.
 

Deleting Data Manager entries

You are given the choice of whether or not you also want to delete the .tsphd file. So you may have more .tsphd files than you do Data Manager entries. That's fine.
 

Adding a Data Manager entry when there is already a .tsphd file

So you delete a Data Manager entry but choose not to delete the .tsphd file. Then you re-create that same Data Manager entry. The .tsphd file is already there. ATM doesn't overwrite it - it just connects to the existing one.

What this means is, someone can give you their .tsphd file, and you copy it to your price history directory (as set on the Options window). Then you create the matching Data Manager entry, and all that price history will immediately become available for use.
 
See also:

• To transfer data from one ATM file to another
• ATM file extensions

The File format and Dates in the file(s) fields on the File Loader window have standard settings based on where you got the files from. (You could also just look in the file).

By "file formats", I mean how the price data is arranged in the text file. As an example, here's how Gain Capital price data looks like:

277248713,EUR/USD,2006-11-05 17:00:02,1.271600,1.271900,D

Comparing that with one from an Oanda text file, you can see they are completely different:

13/02/04 08:36:34,1.280600,1.280800
 

Gain Capital

 
Older files up to and including 2009/05 May/Week3

Order of fields - ID, Currency, Data, Bid, Ask, "D"
Delimiter - Comma
Ignore first row - No
Date format - yyyy-MM-dd HH:mm:ss
Timezone - New York (GMT-5/-4)
 

Files from 2009/05 May/Week4 up to and including 2009/11 November/Week3

Order of fields - ID, Currency, Data, Bid, Ask, "D"
Delimiter - Comma
Ignore first row - Yes
Date format - yyyy-MM-dd HH:mm:ss
Timezone - New York (GMT-5/-4)
 

Newer files from 2009/11 November/Week4 to present

Order of fields - ID, "D", Currency, Data, Bid, Ask
Delimiter - Comma
Ignore first row - Yes
Date format - yyyy-MM-dd HH:mm:ss
Timezone - New York (GMT-5/-4)
 

Oanda

Order of fields - Date, Bid, Ask
Delimiter - depends on what you requested from Oanda. XML files cannot be used.
Ignore first row - check the file
Date format - dd/MM/yy HH:mm:ss (note only 2-digit year)
Timezone - Greenwich Mean Time (GMT)
 

DukasCopy

Order of fields - Date, Time, Volume, Open, Close, Min, Max
Delimiter - depends on what you requested from DukasCopy
Ignore first row - check the file
Date format - can't remember - you should check the file
Timezone - Greenwich Mean Time (GMT)

Something to be aware of with these files is that there is no spread included. Therefore ensure you use a few pips of commission when backtesting, to try and mimick the spread. If you fail to do so, you're backtesting results will be overly fantastic.
 

ATM price history files (*.tsphd)

Order of fields - ATM Price History File (*.tsphd)
Delimiter - choose any
Date format - choose any
Timezone - choose any

The File Loader has some mandatory fields that aren't actually mandatory in this case. Just choose any value to get past the mandatory field check.

As per this wiki entry, the price history files have very specific names, e.g. fxcm_23_1d.tsphd. The ones being used by ATM (i.e. the ones which correspond to Data Manager entries) cannot be renamed at all. But if you are wanting to merge a price history file with another, most likely it means you got one from somewhere or someone else. This file you can rename, if you are going to use the File Loader window to merge it. But make sure you keep the file prefix intact. That is, add another dot and type after it, e.g. fxcm_23_1d.temp.tsphd or fxcm_23_1d.copy.tsphd.
 

Text files exported by ATM v3+

Order of fields:

• for files containing tick prices - Date, Utc, FIP ID, Symbol, Bid, Ask, Volume
• for files containing price bars - Date, Utc, FIP ID, Symbol, Bid OHLC, Ask OHLC, Volume

Delimiter - depends on what the setting was when the file was exported.
Ignore first row - check the file
Date format - yyyy-MM-dd HH:mm:ss
Timezone - choose any

As the date and Utc offset is written into these files, the timezone field is not actually required. So just choose any value to get past the mandatory field check.
 

Text files exported by ATM pre-v3

Order of fields:

• for files containing tick prices - Date, Currency, Bid, Ask
• for files containing price bars - Date, Currency, Bid OHLC, Ask OHLC

Delimiter - depends on what the setting was when the file was exported.
Ignore first row - check the file
Date format - yyyy-MM-dd HH:mm:ss
Timezone - Greenwich Mean Time (GMT)
 

Other

ATM supports just about any date format, and any 1-character delimiter. So if you have a text file from somewhere else, all you have to do is somehow get the order of fields to match one of the available options.

If that's beyond your technical prowess, or you know of another popular file format that you think we should support, you can add a post to the feature request forum.
 

The files that ATM creates are normal files that you can copy, move, rename, backup, and so on.
 

Two Very Important Points

Firstly, make sure that ATM is closed before moving any files. Secondly, price history files (the ones ending in .tsphd) have very specific names. You can copy/move them to different folders, but you can't rename them.
 

Copying an entire setup

The files to copy include 1 settings file, and then zero-to-many price history files. The folder that the price history files are in is shown on the Options tab, in case you forgot where you put them. Full file paths are also shown on the Data Manager Search window.

Copy all those files to the new computer. Start ATM. Go immediately to the Options window. Change the price history file location if necessary (to match where the files are located on the new computer). If you did change it then re-start ATM.
 

Merging settings files

You can export trading systems, alerts, custom indicators, and charts. The exported files are just text files. Copy them to your other computer and then import. All other settings have to be re-entered manually.
 

Merging price history files

While you can export price history to text files and then import on the other computer, there are better ways. In the examples below, we will say that we are copying the file containing prices for EURUSD/Hourly/FXCM. This corresponds to a file with name fxcm_32_1h.tsphd:

1. If on the second computer you don't have a Data Manager entry for that yet, just copy the file straight to the directory where all the other price history files are. Start ATM, make the Data Manager entry for EURUSD/Hourly/FXCM, and ATM will immediately recognise all of the price history in that copied file.
2. If on the second computer you already have a Data Manager entry for that, but haven't gathered any or much price history for it, just delete the Data Manager entry and follow the steps in paragraph #1.
3. If on the second computer you already have a Data Manager entry for that, and have already started gathering price history for it (enough to make you feel sad to just delete it and follow the instructions in paragraph #1), then you can use the File Loader window to merge the two .tsphd files. It will go faster if you put the larger file in the directory with all the others, and merge the smaller file.

   On the File Loader window, select the file to load, set "order of fields" to "ATM Price History file (*.tsphd)", and choose "EURUSD/Hourly/FXCM" as the price data to be loaded. The File Loader will say that the rest of the fields are mandatory, but they're not in this case and you can choose any value just to get past the mandatory fields check.

Just to repeat, when copying files to or from the computers, make sure ATM is closed.
 

When you create a Data Manager entry, you specify how you want to gather the prices - by downloading, creating, or none. On the Command Centre, the scheduled processes that do price downloads do so for all the Data Manager entries where you said to download them. All good so far.

But there may come a time when you want to stop downloading a particular symbol, but continue to download all the rest. You can delete the Data Manager entry completely, for sure. But that's probably overkill.

Instead, edit the Data Manager entry in question. Choose the very bottom option - "just set up...". Click OK or Apply.

In this way all of the prices you have already obtained will still be available, but none more will be downloaded. To restart downloading, just revert to the original settings.
 

To download price data from a broker, you need four things:

1. A user set up for the broker you are trying to download prices from. The password must be entered correctly.
2. At least one account set up for that user. The account number must entered correctly.
3. Data Manager entries, where you have specified to download the prices, and choose the user you created in the step above.
4. The API for your broker properly installed on your computer.

You wouldn't have been able to save the Data Manager entry without a user selected. But you could save it without an Account for that user. Just remember that in ATM, each user needs a corresponding account set up for it before the user will actually be used for anything. It's just that one user could have many accounts, so we had to split them up.
 

Gain Capital files are available to all - no account required.

1. Download the files from http://ratedata.gaincapital.com/ to your computer.
2. They are zip files, so unzip them.

Then follow the rest of the instructions to load text files and create price bars.
 

An easier way to download

Gain Capital gives one file per week per currency. About 20 currencies equals about 80 files per month and about 960 files per year. That's a lot of clicks. downTHEMAll! will make the downloading much easier than doing it one-by-one.

1. Browse to http://ratedata.gaincapital.com using Firefox.
2. Click through to the month of data that you want to download.
3. Click "Tools" > "downTHEMAll!" > "downTHEMall!..." from the main menu (in Firefox).
4. On the downTHEMall! window:
   1. In the Filters area at the bottom, check the "Archives (zip, rar...)" checkbox.
   2. Click the Browse button to select the directory you want to download the files to.
   3. Click the "Start Downloads!" button.