Loading Text Files And Creating Price Bars

The method described below works the same for any symbol/interval/broker. We will use the example of creating EUR/USD Hourly and Daily price bars from historical tick prices provided by Gain Capital.

These steps assume you have already downloaded the text files containing the historical prices to your computer. See here for instructions for:

• Gain Capital
• Oanda
• DukasCopy
   

Create the Data Manager entries

Here's a good explanation of what Data Manager entries do.
 

1. Create a Data Manager entry to store the tick prices

1a. In ATM, click "Price Data" > "Price data configuration"
1b. Click "Add"
1c. In the top box, choose EUR/USD, Tick, Gain Capital
1d. In the bottom box, choose the bottom option - "Just set up (or keep)..."
1e. Click OK.
 

2. Create a Data Manager entry to store the Hourly bars

2a. (Should still have the Data Manager Search window open).  Click "Add"
2b. In the top box, choose EUR/USD, Hourly, Gain Capital
2c. In the bottom box, choose the second option - "...shorter time-period..."
2d. Choose "EUR/USD, Tick, Gain Capital" from the list.
2e. Click OK.
 

3. Create a Data Manager entry to store the Daily bars

3a. (Should still have the Data Manager Search window open).  Click "Add"
3b. In the top box, choose EUR/USD, Daily, Gain Capital
3c. In the bottom box, choose the second option - "...shorter time-period..."
3d. Now you have a choice - you can either choose "EUR/USD, Tick, Gain Capital" from the list, just like you did with the Hourly bars. That would mean the Daily bars are also made from the tick prices. Or, you can choose "EUR/USD, Hourly, Gain Capital", to have the Daily bars made from the Hourly. It's up to your preference, but the second option would be far quicker.
3e. Click OK.
 

Side note #1:

Each Data Manager entry creates a file on your computer which will store the prices. If you look in your file system you should now see 3 new files: gain_32_tk.tsphd, gain_32_1h.tsphd, and gain_32_1d.tsphd, respectively.
 

Side note #2:

The number 32 is what we call the "FIP ID", or "Financial Product ID". It's a number only useful within the Thinking Stuff world. 32 equates to the currency with symbol "EUR/USD".
 
Now to get the price history from the text files into the ATM files. To do this, we first create a "file load task", then process the task.
 

Load the files

 
4. Create the file load task

4a. Click "Price Data" > "Price history file loader"
4b. Files to load - browse for and select the file(s) to load. Recommend to just choose 1 file if this is your first time.
4c. Fill in the "File format" and "Dates in the file(s)" fields as per these standard settings for your broker.
4d. Price data to be loaded - EUR/USD, Tick, Gain Capital
4e. Click "Add To List"
 

5. Process the task

5a. Still on the File Loader window, go to "The Task List" tab
5b. Click "Start Loading"
5c. Wait.
 

6. Confirm the task worked

6a. Go to the "Completed List" tab.
6b. Scroll all the way to the right to see the result - "Success" is good. Anything in the "Error Text" field is bad.
6c. Back to the Data Manager Search window.
6d. Right-click on the "EUR/USD, Tick, Gain Capital" entry. A little pop-up menu appears. Choose "Table Information (row count, etc)"
6e. The message that shows up should display a count greater than zero.
 
So now we have the tick prices. What comes next is creating the Hourly and Daily price bars. This time we create "interval creation tasks", then process the tasks.
 

Create the price bars

 
7. Create the interval creation tasks

7a. (Should still have the Data Manager Search window open). Click (normal left-click) on the "EUR/USD, Hourly, Gain Capital" entry to highlight it.
7b. At the bottom of that window, go to the "Bulk Price Bar Creation" tab.
7c. Click "Add To List"
7d. Click on "EUR/USD, Daily, Gain Capital" to highlight it.
7e. Click "Add To List"
 

8. Process the tasks

8a. Still on the Data Manager Search window, go to "The Task List" tab
8b. Click "Start Creating Price Bars"
8c. Wait.
 

9. Confirm the tasks worked

9a. Go to the "Completed List" tab.
9b. Scroll all the way to the right to see the result - "Success" is good. Anything in the "Error Text" field is bad.
9c. Back to the "Price Data Configurations" tab.
9d. Right-click on the "EUR/USD, Hourly, Gain Capital" entry. A little pop-up menu appears. Choose "Table Information (row count, etc)"
9e. The message that shows up should display a count greater than zero. Same for the Daily one.
 

Cleanse the prices

Not all text files are created equal. Some have minor errors which can throw out your backtesting results. The easiest way to check for these errors is to view the chart. Ridiculous prices are immediately obvious.
 

10. View the chart

10a. Click "Price Data" > "View charts"
10b. Fill in the search criteria.
10c. Click "Chart"
 

11. Cleanse the data

11a. If the chart looks fine then you're all done.
11b. If there are extreme spikes in the chart then it's possible there is some erroneous data in the text files. Follow these steps to perform the cleanse.