Automated Trading Machine (ATM) makes it simple to remove fear and greed from your trading. Automated trading is no longer just for the rich or nerdy. Our revolutionary software runs on your computer, using your trading rules, but none of your emotions. There's just one requirement - you know how to use a mouse. Learn more...
First Time?
Then you should read:
Navigation
Site Feed
User login
Sponsors
Bug: loading Gain Capital text files doesn't work
- Login to post comments
Sun, 03/14/2010 - 00:30
A customer followed the instructions to load text files made available by Gain Capital but kept receiving a message "input string was not in correct format".
It turns out all our testing had been with older files. Gain have changed the ordering of fields. It used to be like this:
978047,"AUD/USD",2005-01-07 18:09:32,.756800,.757200,"D"
Now like this:
1075796183,D,AUD/CAD,2010-02-28 17:01:42,.944700,.945600
The "D" moved. "D" stands for "dealable", which means a trade actually took place there (i.e. it's not just a quoted price). However, and I'm happy to be proven wrong, but I don't remember ever seeing a line that didn't have a "D".
Thanks Manasa
Mon, 03/15/2010 - 12:24
#2
No, the date format field
No, the date format field should take care of that.
What I hope is happening is simply the file has column headers, and you have not put a tick in the "ignore first row" field.
Hopefully it's that simple. Here are the default settings for the File Loader window.
Mon, 03/15/2010 - 23:45
#3
Have tried that, with the old
Have tried that, with the old and new field format etc, The message I'm receiving is still the same and problem still there.
Tue, 03/16/2010 - 02:08
#4
Alrighty then... you'll have
Alrighty then... you'll have to send me the file so I can have a look. Or point me to the file on the Gain website.
Tue, 03/16/2010 - 09:31
#5
Try this file....
Try the XAG/USD price data ..I had downloaded the XAG/USD price data for the whole of 2009 and was trying to laod price data for june only (wk 1) 2009 as a trial...but that's how I got the problems menyioned above.
Wed, 03/17/2010 - 01:56
#6
I could reproduce this error.
I could reproduce this error. Just once. But not anymore.
I downloaded all the Week1 files for EUR/USD, XAG/USD, and XAU/USD from November 2008 to March 2010.
In 2 of the months there was no Week1 file so I used Week2.
XAU/USD was made available starting March 2009. XAG/USD started in May 2009.
They started putting column headers in June 2009. They moved the position of the "D" in December 2009.
The first time I loaded all these files, everything worked fine. I tried again, but during processing I changed my computer's regional settings from English (Australia) to English (United States). At the very time I clicked OK to change the settings, that's when I got a "String not recognise as a valid date time" error.
Obviously I'm not handling when the regional settings are set to US-style (i.e. dates are written month-day-year). At least that's the conclusion I came to when I received this error, even though I'm pretty sure the code I wrote should handle that case. But here you are talking about this error, and here I am receiving the same error when using US settings.
So this time I tried loading all the files again with my regional settings set to English (United States). It worked - even the same file that gave me the error before.
I got all the XAG/USD files for June 2009 and loaded them. It worked.
Huh...?
I got the error before at the exact same time I had changed my regional settings. I have to put that error down to the sudden change in regional settings rather than a problem in the code.
As for what's happening with you... short answer is I don't know yet. We have to play around a bit more to find cases when it works and cases when it doesn't. That will help us pinpoint what's going wrong.
If anyone else reading this thread, if you could let us know if it works or doesn't work for you. Either way it's valuable information.
Mantise:
- Do you have the right date format selected - yyyy-MM-dd HH:mm:ss?
- What exactly are your regional settings?
- If you go to the Control Panel, Regional and Language Options, and change your settings to English (Australia), does the file load now work?
- Still in Regional and Language Options - do you have the option to change it to English (United Kingdom)? Because that's the culture I use to parse the dates when the date format is set as above.
The regional settings, except that last point, should not actually have any effect. My code is written to be Regional Settings-agnostic. But if you do change it and try again, whether you get the same error or not gives us more information.
Wed, 03/17/2010 - 07:20
#7
I'm still having problems.
I'm still having problems. I have checked and I have the right date format yyyy-MM-dd HH:mm:ss
I have tried the regional settings you suggested viz Aust. UK, even NZ. The original problem on date time has dissapeared. But with all of them, I have this recurring error message :
- arithmetic exception, numeric overflow,or string truncation - .
Wed, 03/17/2010 - 08:17
#8
Geez mate, you're having a
Geez mate, you're having a rough time of it. Unfortunately I don't get any errors at all. Works like a charm. So it's pretty tricky to diagnose.
I guess the best I can do is put a better error message in there, so you can actually see which line/field is causing the problems.
But a new release is probably a week away at earliest, and then as I say, all you'll get is a better error message. If you want an actual fix then you'd have to do some (more) investigative work yourself, due to it only giving these errors on your machine. Have any files worked at all? Is it possible for you to narrow down the entries (single lines in the files) and show me 1 line that works and 1 that doesn't?
If you put your regional settings back to whatever it was originally, are you getting the original error, or still this new one?
You're ignoring the first row? You've set comma as the delimiter? You're using the order of fields with the "D" at the end for pre-December 2009 files, and with the "D" as the second field for December 2009 and beyond? You've rubbed your rabbit's foot and thrown salt over your shoulder? (This last one is not written in the documentation, as I thought people just naturally do it).
Sat, 03/20/2010 - 01:18
#9
Summary so far: a bug was
Summary so far: a bug was reported to us that the Gain Capital files weren't loading. We found that Gain had changed the ordering of the fields in their files - a "D" that used to be at the end of each line was now in the second position. In v3.0.1 we added a new file format that reflected the new Gain style. Problem solved...
Except that the customer still had problems, and was in fact trying to load files that pre-dated the change to the position of the "D". So we had fixed a problem, only it wasn't related to what this guy was doing.
Unfortunately we could not reproduce the error they were getting. So in v3.0.2 we changed the error message to give precise details on the line of text that causes the error, along with the delimiter used to split that line of text into separate fields, and what each field looks like.
While we took away "[Fixed in v3.0.1]" from the header of this post, we don't get any errors and can now only wait until we get further information.
(N.b. it turns out Gain has 3 different file formats. See here for the default settings to use on the File Loader window for each).
Wed, 04/14/2010 - 08:07
#10
Here's an Idea
Mark, it might be good to know that for some reason Gain Capital likes to randomly create their text files in ANSI and UNICODE.
If you hit a UNICODE file then I would expect you get an error.
I know this from trying to copy/paste mixed ANSI and UNICODE into an ANSI text file and getting bizarre results.
It's best to spend the time and edit every text file in Notepad and remove the header line and save it as ANSI.
Also a question, please advise if your generated time bars are the Spot price (average of bid/ask) as I forgot from way back.
Please email me the answer. I'll need a custom bot from you this year.
Thanks
Jeff
Fri, 04/16/2010 - 05:43
#11
What ATM does is calculate
What ATM does is calculate the open, high, low, close for both the bid and ask prices. A full price bar is therefore made up of the following information:
- timestamp
- volume (not for currency though)
- openbid, openask
- highbid, highask
- lowbid, lowask
- closebid, closeask
Indicators and other calculations then work out if they need to use the bid or ask price based on what the desired action might be. Opening a long trade is done at the ask, closing a long trade is done at the bid. Opening a short trade is done at the bid, closing a short trade is done at the ask.
So as an example, if you wanted a long order placed at an SMA plus 50 pips then the ask prices are used to calculate the SMA. If you wanted the long stop loss to be put at the SMA minus 50 pips then the bid prices are used to calculate the SMA.
Sun, 05/23/2010 - 15:01
#12
Same bug here
I have the same bug here when I try to load January EUR USD data from Gain Capital:
error: "arithmetic exception, numeric overflow, or string truncation".
My local settings are Chinese ones (Windows XP, english version).
Any help?
Mon, 05/24/2010 - 00:00
#13
Which version?
Which version are you using? If it's v3.0.2 then it should have given you a much more detailed error message, so can you please post that here. (Search for it using the Messages window).
If you are not using v3.0.2 then please upgrade, try again, and post the long error message here if the problem persists. The new error message contains information such as the exact line it fails on, and what the individual fields look like when split using the chosen delimiter.
Recent blog posts
- New Release: v3.0.3 - Everything Except Autotrading
- Mid July 2010 Update
- Start of June 2010 Update
- New Release: v3.0.2 - Copy, better error message, more options
- New Release: v3.0.1 - The Stabilise-ening
- Ah, The First Bug [Fixed in v3.0.1]
- New Release: v3.0.0 - The Rewrite
- Start of February 2010 Update
- End Of 2009 Update
- New Release: v2.0.10
Recent comments
- good choice
21 weeks 2 days ago - Hello and welcome. I will
36 weeks 3 days ago - Of course...
42 weeks 1 day ago - Easy one (hopefully). You
42 weeks 1 day ago - Thanks for that. I learnt
48 weeks 5 days ago - Oops!
50 weeks 12 hours ago - Changing the windows date
1 year 4 weeks ago - Interesting problem you have
1 year 4 weeks ago - You want to use the Value of
1 year 7 weeks ago - A tick backtester will do it
1 year 12 weeks ago
Still it doesnt work. Error message reads "File exited with error.String not recognise as a valid date time." Is this because time stamp in the "order fields" in "File Format" of the "Price Histroy loader" is missing which should be included??