v0.6.7 - Interval Data Creation

Nothing like a paying customer to induce some effort.

v0.6.7 is now available for download:
http://www.thinkingstuff.com/download.aspx

You can see what was fixed in the Change Log:
http://www.thinkingstuff.com/change-log.aspx

And let's not forget:
http://www.thinkingstuff.com/purchase.aspx

Box of Shark

First, I came up with this independently, but if you know someone else has already published it or something very similar, please let me know so I can give the appropriate credit.

Second, if you publish it then give me the appropriate credit. A link to this page or website certainly couldn't hurt.

Third, I've given you this indicator and how to calculate it for free. In the spirit of "you get back what you give out". If you come up with a decent way to trade with it, or a way to change it to make it better, you must leave a comment on this page detailing it. There's no being a Scrooge. In fact, it doesn't make sense - more people following your system will result in more people buying when you are buying, which should theoretically mean better results.

Fourth, comments and criticisms are welcomed.

---

Background

I came up with this indicator after reading about "Market Profile".

Essentially, when a new bar has closed, that close price fits into price area. For example, a USDJPY price of 114.23 fits in the price area of 114.20 to 114.24.

114.23 also fits in the price area of 114.20 to 114.29, 114.00 to 114.49, 114.00 to 114.99, etc. It all depends on the number of pips you decided upon for each price area.

A number of other close prices also fit within that price range. You add up the total number of "hits" that each price range gets, and that number is what you chart.

I don't think there's anything new yet.

Here's an example of a Box of Shark:
114.00 to 114.04: 1
114.05 to 114.09: 0
114.10 to 114.14: 0
114.15 to 114.19: 2
114.20 to 114.24: 2
114.25 to 114.29: 4
114.30 to 114.34: 8
114.35 to 114.39: 2
114.40 to 114.44: 0
114.45 to 114.49: 1

In total, the above example plots 20 close prices. That's the period for this Box of Shark. And the box size is 5 pips.


Theory & Interpretation

When you chart this kind of thing, the chart has 2 dimensions.

When all the prices are grouped together in a small range (i.e. congestion), the number of price ranges used becomes small, and the average becomes big. In the example above there are 10 price ranges, with 20 close prices used, so the average is 2. But if the last 20 close prices fell between 114.10 and 114.19, then there would be just 2 price ranges, with the average becoming 10.

The number of price ranges, or the average, could therefore be used as some kind of warning - either the number of price ranges shrinks (and average grows) and therefore we are in congestion, or the number of price ranges grows (and average shrinks) and therefore the prices have become quite spread-out and volatile.

The other dimension is the total number of close prices that fall in a price range. In the example above, 8 is the maximum, and 2 is the average. This could be used as a replacement for support and resistance lines - any price range containing more than the average (or more than a multiple of the average) could be considered as an area of support or resistance.


Trading Ideas

You can use a rising average as signs of congestion, which might keep you out of a trade you might have otherwise entered.

Conversely, you might want a declining average, as a sign of increasing volatility, before you enter a trade.

In either case, I don't think those two signals could be used independently to make trading decisions - more likely they would be used as a further confirmation before entering a trade based on your other rules.

---

As I mentioned, the number of hits in each price range could be used as an alternative to support and resistance lines. In our example, the price range of 114.30 to 114.34 has 8 hits, so if the last close price fell in the price range of 114.25 to 114.29, and you were looking to go long, perhaps you wouldn't.


Box of Shark Histogram

I was just looking at the rules I've already coded into Thinking Stuff to handle the Box of Shark. And I saw these two - "Box of Shark Histogram positive" and "Box of Shark Histogram negative". It was so long ago that I made this indicator up, and I had completely forgotten where the Histogram part came from.

Well, seems like I caught a MACD craze. The Box of Shark Histogram is calculated by taking a slow and fast average, of the average, and subtracting the slow from the fast.

Let's say the box size is 5 pips, and the period (number of close prices to chart) is 20, slow average has a period of 20, and the fast average has a period of 10.

Thus:

1. Construct the Box of Shark, with period 20 and box size of 5 pips.
2. Work out the average, which is the period divided by the number of price ranges that you end up with.
3. For each new close price, you will get a new average. So, do the above 2 steps until you have 20 average values.
4. Add up the last 10 averages, and divide by 10. That's your fast average.
5. Add up the 20 averages, and divide by 20. That's your slow average.
6. Box of Shark Histogram = Fast average - Slow average.

The Box of Shark Histogram is going to be positive when the Fast average is higher than the slow average.

What this means is the most recent averages are getting bigger. And averages getting bigger means the price is heading into congestion. All in all, not too much different to above, but perhaps a better signal than just using "one average is higher than another", because we are basing our judgement over many averages.


Naturalised Box of Shark

I also have rules such as "Last X B.O.S. Naturalised Average(s) Falling". A naturalised average just means rounding the average to a whole number. The average changing from 8 to 7, say, is a stronger signal than the average changing from 7.8 to 7.7.


Summary

Grouping the prices into boxes lets you step back a bit from the absolute detail of the price chart. This kind of thing is nothing new. It's used in Market Profile, and also P&F Charts, etc.

It's unknown whether the techniques above are going to be useful or not. Only backtesting will tell. At the least, they give objective ways to determine times of congestion and volatility, and of support and resistance levels.

Subscriber #1

Thinking Stuff has its first paying subscriber!

And, he lives on the opposite side of the world to me. Thanks goes to the magic of the Internet.

Here's to a successful partnership.

v0.6.6 - Loading Data From Text Files

v0.6.6 is now available for download:
http://www.thinkingstuff.com/download.aspx

You can see what was fixed in the Change Log:
http://www.thinkingstuff.com/change-log.aspx

And let's not forget:
http://www.thinkingstuff.com/purchase.aspx

If you are upgrading from v0.6.5, unfortunately you will either need to
- create a totally new database; or
- delete all the Data Manager entries that are for Tick data, and make them again.

This was because of a problem with how the tick data tables were created. They were created with a unique constraint on the date column, meaning no 2 rows could have the same date. But, many ticks can happen in the same second. This error meant the text files containing the historical data from ratedata.gaincapital.com could not be loaded.

I guess that's lucky, because it means nobody would have spent too much time loading the data from those text files. So, what you need to do is re-create those tables, and as I said, that either means you start over with a brand new database, or you go and delete those Data Manager entries and make them again.

Also, it is possible to install v0.6.6 without first uninstalling v0.6.5. However, if you do this, and then uninstall v0.6.6 at some point in the future, not all the files are going to be removed.

Without uninstalling v0.6.5 first, and then uninstalling v0.6.6 sometime in the future, the Windows uninstaller thinks some of those files still belong v0.6.5. And because you're uninstalling v0.6.6, the files that "belong" to v0.6.5 will not be removed.

If that happens, you can simply delete the Thinking Stuff folder. Or, you can uninstall v0.6.5 before installing v0.6.6. It's up to you. Like I said, it's not going to affect your database(s).

If there are any problems, please let me know. I think I even got that Download link working properly, but now you're downloading a ZIP file instead of an MSI file.

You might also notice the website looks a little different. It ain't just yellow no more :-)

Have a look at the sitemap to see what's available.

And here's a special offer for people who read to the end of my blog entries. As soon as someone pays to use the software, I can start claiming any software and website-related costs from my tax.

While the software is still a beta release, the Backtesting functionality is well worth the money.

So, who wants to be first? To the first subscriber, I offer 50% off everything for the life of the product. Please get in touch with me by the Contact Us page.

The Path

Having outlined The Goal, this page will explain The Path to get there.

But first, let me outline the path I took to get where I am right now.

The Path To Here

I've been to two seminars about trading. One was about trading options on the ASX. The other was about trading currency on the forex markets.

Both were very informative, although attending them has not made me a successful trader. Following the rules I bought from these people only seemed to make me do exactly the wrong thing.

Many of you are scoffing now - of course the rules don't work, because if they did, those guys wouldn't need to give seminars. They'd be rich already.

I decided the reason I could not get these rules to work for me, was the inherent subjectiveness in some of the rules. (The 2nd seminar was far more objective than the first, but there was still some subjectiveness in there). I have big problems with trend lines, and support-resistance (SR) lines. I have problems trying to work out if an indicator is "diverging". Oh sure, on a chart of historical data it's as clear as day. But during real-time?

Anyway, I needed to find a trading system (bunch of rules which tell me when to buy and sell) which is completely, 100% objective. IF such-and-such a condition is true THEN buy. IF the price is near an SR line THEN don't buy looks on the face of it as objective, but because the placement of that SR line is subjective, the rule becomes subjective.

So my new requirement for a trading system is that it must be so objective that a computer could do it. No emotion. Just rule says yes, so do it. By the way, both of the seminar dudes recommended trading diaries - to keep track of how you were feeling while trading. Did you buy because the rules said to, or because you just really wanted to trade something? Then at the end of the month, read through your diary and try to remove the emotion from your trading. My take on it is that if the rules were 100% objective, there is no need for such a diary.

And I thought, well, if my requirement is that a computer could trade the system, then a computer should trade the system. There's no need for me to sit and stare at the computer screen all day. And that's something I did glean from the 2nd seminar - he said he had a business partner who had made a trading-bot based on his rules. Before hearing that, I didn't think such a thing was possible. As soon as I heard it was, my mind kicked into gear on how to go about making my own.

So for the past... forever... I've been working pretty hard on my software which will interact with the forex broker, and automatically trade on its own based on the system that I've given it. So I could be getting a suntan at the beach (unlikely - more like drinking at the pub), and yet making money at the same time. Well, trading at least, making money if I'm lucky.

I guess a quick and nasty version would have been up by now. But I wanted to make it special. It does backtesting now, so I can test my systems before making them "live". And the way I've set it up is so that rules can be added to a system by a few clicks of the mouse.

There is similar software available, but they are generally in the thousands of dollars range, and all require the learning of a cut-down version of computer programming language. You code your system, each system, and plug it into the software. It doesn't appeal to me. Where there is coding there is chance of bugs. And if you want to change/add/delete a rule from the system, there is modification of code. And with modification of code there is chance of bugs.

So my software is all mouse clicks. Want a system based on the Directional Movement System (based on the interplay between three calculated lines)? Click click click. Done. A couple more clicks and it's backtested. Ok, it didn't work. Want to change it from buying when +D is above -D, to buying when ADX is below +D and also below -D? Click click click. A couple more clicks and it's backtested. Ah, much better. But let's change the period from 20 to 30. Click click click... You get the picture.

You can set up different accounts to trade with. You can use a different company's data feed to make trading decisions with. You can tell each system how much money it has to trade with, and what percent of that money to risk on each trade. You can even tell it to stop trading until the last X number of trades have a certain Profit Factor.

And, to be honest, making that software kind of made me lose my way. The goal, after all, is to make money from trading. Now I'm spending all my time making software.

The Path From Here

The first step is to complete the circle, and use the software for its originally intended purpose - to find a purely objective trading system that I could trade profitably.

To experiment with different trading rules, first I need historical data to backtest with. So, I've downloaded all the files freely available from ratedata.gaincapital.com. And they are being loaded into my Thinking Stuff database.

Millions of rows of tick data take a very long time to load...

Once that's finished, I create trading systems and backtest them. And use the power of my software to experiment with adding new rules, and removing, or changing settings on existing rules. If those changes provide better results, they stay. If not, revert back to how it was before.

And when I say "the power of my software to experiment", I really mean that. Once you get set up, you will be amazed how fantastically simple experimentation is.

So I have my seemingly profitable trading system or systems, and then I use the instructions in "How Should I Decrease Risk?" to go from there.

With regard the automated trading, I am going to use my free Gain Capital demo API account initially. When I have proved to myself that the trading system(s) are genuinely good, I am going to purchase the Oanda API for US$600, and use that instead.

All credit to Gain Capital for providing their API and historical data for free. This is going to help a lot of people get started with very little, if any, start-up costs. And should translate into brokerage fees going to Gain Capital when people go live.

However, I'm all about decreasing risk. And unfortunately, sticking with Gain Capital to do live trading would not be decreasing my risk (in my humble opinion, but please, please decide this for yourself).

The "Gain Capital Quirks" article outlines why I believe this. In particular, the live API is slightly different to the demo. And, most importantly, they do not allow API trading in their mini accounts. They made this decision consciously and deliberately, as this is a feature that they used to offer, but took away.

Further, the Oanda API seems to be a little better (from a coding standpoint), and being able to trade any number of units is a great feature. Ok, sure, it's US$600 to get started, but I'm guessing the tech support team at Oanda actually answer emails.

That's the plan.

To R Or Not To R

A couple of the trading blogs I read use R to describe their profit/loss on any particular trade. R is the amount they risk on the trade. So if they are risking $5 and the profit from the trade is $5, they say their profit was +1R. Or if they profit $7.50, then their profit was +1.5R.

Similarly, if they lose $5 on that trade, their loss was -1R. Due to slippage, it is possible to lose more than -1R.

Anyway, some people don't like reading about R. It doesn't mean anything to them. R could be $5 or $50,000. Just saying "I got stopped out for -1R" is meaningless. They say.

You could also say that talking about a profit of $5 is equally meaningless. Risking $100 to win $5 is a lot different to risking $1 to win $5.

So, just R figures or just $ figures doesn't seem to say much. Perhaps both should be given.

However, let's say a really fantastic trader doesn't have any money to trade with. It happens. We have phone bills, petrol to buy, mortgages on our fabulous mansions. If he/she told you that they profited +10R on trades day in, day out, those fantastic results may be overshadowed by the fact that he/she was only risking $10 at a time.

People are very quick to scoff when it comes to trading. People are going to lose the benefits of reading whatever that trader had to say, just because (for whatever reason) he had no trading capital to begin with.

Further, even though people write blogs about their trades, they are still allowed some privacy. They shouldn't need to tell everyone how they won/lost $10,000 last month.

So you might lean towards using a standard starting figure. Let's say $100,000. All figures are then based on if that person had exactly $100,000 in their trading bank at the start of each month, or year. It provides privacy and the required information.

But does it? If you think about it, using a standard starting figure tells you nothing more than telling you about R.

So to conclude, the Rs have it. If you really want $ figures, work them out for yourself based on your own account balance.

The Goal

[This story also found at the Thinking Stuff website under Misc. > The Goal]

The goal is to live full lives (which has different meanings for everyone), paid for from the profits of trading.

Sounds good, doesn't it? Work your own hours, be your own boss, take days off whenever you want.

However, trading is not exactly passive income. Day-traders need to stay at their computers for at least a few hours each trading day. I've personally spent entire days looking at the screen.

That was part of the reason to create the Thinking Stuff software. Not only does automated trading software (or a "trading-bot") remove fear and greed from the trades, it also gives you freedom from staring at the charts yourself.

In fact, you could be at the beach drinking beers. In the meantime, your trading-bot is (hopefully) making the money for you.

But there are some problems with having a computer trade for you. Electricity can be switched off. Pets can pull plugs. Internet connections can be cut. If you're at the beach, there's no way to know if any of these have occurred.

Okay, maybe you have the software send you an email every 30 minutes. And if you don't receive the email, then you know something is wrong. But email is not a perfect communication medium. Sure, most of the time email arrives seconds after being sent. But occassionally it takes a day or more. This late email makes you rush home from the beach to find your trading-bot running nicely.

Or what if you're hours away from your house? Or even in a different country? Maybe the Internet connection drops out the second you put your foot on the plane. You want your trading-bot to be make you money even while you're on holiday. That was the point of the trading-bot.

Now, while your automated trading career is just beginning, you'll probably want to install it on your computer, and keep a close eye on it. It means you'll be restricted to your house, just like a day-trader. The difference is that you'll be free to do anything else in your house, rather than stare at charts. Further, the advantages of the trading-bot, such as removing fear and greed, will still apply.

Not right away, but eventually, after you have faith in your trading system, faith in your trading-bot, and you are making enough money from trading to cover the expenses, you will probably want to hire a dedicated server somewhere. There are many companies around that provide 24/7 uptime. You want to employ the services of one of these companies, get your trading-bot installed on that machine, and have them ensure it's always running.

That's the goal - to have a trading-bot do the trading, and someone else looking after the trading-bot. That's passive income. Then you're free to do whatever else you like. Even work.