Thinking Stuff Is In Beta

uhhhh.... Happy Valentine's Day?

It sets up its own PostgreSQL database.

It has its own End User Licence Agreement ("EULA") which no-one will read (but really should).

It's obfuscated.

It's packaged up into an Microsoft Installer (msi) file.

It's uploaded to somewhere on my website for my beta testers to obtain.

Thinking Stuff is in beta.

So... a week later... if there's nothing majorly wrong with it (that we know about)... I'll make the pre-release version available to those who want to start using it as soon as possible. A couple of such people have already contacted me and are ready and just itching to get started.

And then, later, (remembering that I really need a break from anything computer-related, and my girlfriend is coming from Japan in 3 weeks for a 3 week holiday), the proper release version will be available.

But, I really think you should jump on board as soon as possible :-) Free trial version... Backtesting just US$200 per year... Auto-trading free with backtesting until it's mostly stable...

Installing PostgreSQL

PostgreSQL is the commercial-quality database that I chose to use for my software. You can read why I chose it at this post (just forget anything I say there about making it compatible with many other types of databases - PostgreSQL has since become the one and only).

Installing PostgreSQL onto your computer follows exactly the same path as installing any other kind of software. Download the install file, run it. That's it.

But, if you go to their download page, you'll see all kinds of files on offer. The link I gave in the previous post, to install PostgreSQL v8.1.3, is to the file that I downloaded.

It's a zip file. Unzip that file and you get access to the installation file. Run the installation file, and it takes you through the installation wizard where you can set various options.

I used all the default options, except for the page that has Locale and Encoding options. These were the only options I changed. Locale went to Australia (because that's where I am), and the encoding was set to UTF-8.

Now, in my code where I create the Thinking Stuff database, I explicitly set the encoding to UTF-8, so you probably don't need to set it here, but I would anyway.

As well as the database software, that installation file also installs pgAdmin III, the Admin program which you may or may not want to use later on to view the data in the database. You don't need to download that separately.

Potential Down Time

Due to the location of my parents' farm, 12km outside a small country town, unfortunately we can't get a broadband Internet connection. But, we can get a fresh install of outdated technology - ISDN. Luckily there is a sub-exchange just across the road from us. Unfortunately, we are the first of the properties nearby to get ISDN installed, and that little sub-exchange was wired all wrong. So, yesterday and until lunchtime today we were without phones and therefore dial-in Internet while the tech guy was trying to work out the problem. He never did, and had to order new parts just so he can determine where the problem lies.

However, during that time I was able to finalise the way that the PosgreSQL (yes, it works, forget MySql with its nasty commercial licence and go install PostgreSQL v8.1.3) database will be created and populated. I won't bore you with details - I could have decided upon an incremental upgrade approach, where the user (you), to upgrade from v1.0 to v2.0 would have to download and install upgrade scripts v1.0 to v1.5, and v1.5 to v2.0, in that order. But, luckily, I thought of a way to make everything very easy for everyone. You download the Thinking Stuff exe, and it takes you through the creation, and upgrade (if necessary) of the (PostgreSQL) database it needs to function with a click of a single button.

With that done, there's pretty much nothing left to do. It's the installation of the SSL certificate that is now holding us back. Mine was made a couple of days ago, and the support crew of my new webhost told me to forward it on to a particular email address. I did that. 2 days later (today) I ask their live support about the status. Oh, I was supposed to send that to a different email address to the one I was told about earlier. Ok. So that's installed now, but I have to change my name servers over to them (the new webhost), in order for it to work. That can take a couple of days. And that's why there's a possibility of down time. And that's all we are waiting on for Thinking Stuff to go beta.

I don't even have a job and I need a holiday.

Late Release Update 19-Feb-06

Still no word back from the obfuscator people. 5 minutes to type out an email and they could have had a $200 sale. But I found another one, more expensive, however it (seemed to) work straight off the bat.

Obtaining my own SSL certificate took longer than I thought - it wasn't a case at all of putting in the request before going to bed, and then waking up and all would be done. No, it's still being installed. But I've tested the webservice without going through https and all seems fine, therefore it shouldn't (emphasis on shouldn't) take long to get up and running once I get the certificate.

Yesterday I spent all day going through the dump of the MySQL database, and changing it to PostgreSQL syntax. That task continued through to lunch time today. It was fun, in the same way as pulling your own teeth out might be fun. I don't want to say definitely that PostgreSQL will be the winner yet, until I know for sure that Thinking Stuff works properly with it. But PostgreSQL is certainly the front-runner, as it is completely free, and probably has more features and more support behind it from the developer community than SQLite (which has other good features of its own). And, PostgreSQL downloaded and installed very easily.

I now have a PostgreSQL database fully loaded with Thinking Stuff schema and data, and this afternoon I will change the code in Thinking Stuff to connect to that instead of the MySQL database. And hopefully, everything will work fine (emphasis on hopefully...)

Late Release Update 16-Feb-06

Because I promised my beta testers something to beta test on Valentine's Day, and because now it's 2 days after that, I thought I'd give short, regular updates on what's happening. As if you were my project manager. Breathing down my neck like all good project managers do :-)

I spent most of a day researching obfuscators. For those non-techies, essentially creating a program in Java or .NET allows anyone with freely-available software to view the actual code that you wrote. I did it. It was scarily easy. So you need to buy something that makes that code incredibly hard, but not impossible, to read. Those things are called obfuscators.

The first few obfuscators I tried did a good job, but I could still tell what I was looking at. And if I could tell, then, sure, it would take a bit longer, but anyone else could work it out as well. Copy the code, minus the subscription checks, and they've got a year's worth of free software development.

So, after most of a day had passed, I decided I would use one of the obfuscators that creates a WIN32 executable, rather than one that makes the code hard to read but retrains the .NET compatability. Found one, cheaper than the others I had already tried, and spent most of another day trying to get it to work. Sent an email to their support asking for a reply asap, and am yet to have any kind of reply. Have downloaded a different one which I will try soon.

Deployed the webservice to the server, and it works fine. Through http. But I want to connect to it through https. It's another Intellectual Property protection technique, so people don't use other freely-available software to modify the reply from the webservice and give themselves free access to everything. I did it. It was scarily easy.

Sent an email to my webhost's support crew, and the word back just now is that it's because they don't support webservices under their shared ssl certificate. So I have to purchase my own ssl certificate. I'll arrange that right now and look to deploy and get that sorted when I wake up in the morning.

Doing the little things that would take longer to explain than to fix has not been started. But that should take just a couple of hours.

The executable to set up and load the MySQL database needed for Thinking Stuff, has not been started. But, the recent comment about SQL-Lite got me thinking about my whole database strategy of making it available for many database types.

I might ditch MySQL altogether, and make Thinking Stuff available solely for the free ones. That would get around the licence fee MySQL want for commercial software. To start with, I'm thinking of using SQL-Lite or PostgreSQL. In the future I might also make it compatible with MS Access, so people can use a trial version with little hassle. Unfortunately I can't see a way around doubling the amount of code I already have for each type of database I make it compatible with. This doesn't please me. And makes my plans for making it compatible with each and every database type I could think of as rather... far-fetched. As a project manager, I'm sure you are happy to hear that. (Hey, if this was a real project, we would have had 10 meetings to discuss this with all the stake-holders and decide upon the best strategy to move forward).

Did I say short updates?

If you click this link, you can see a simple web page detailing the currently known bugs in Thinking Stuff.

What you should remember, is that this is still a beta, i.e. pre-release, version. And beta, i.e. pre-release, versions always have lots of bugs. That's why they are pre-release.

Valentine's Day

Happy Valentine's Day everybody. I hope you got presents from someone, because unfortunately you won't be getting a present from me. I had really hoped to get the first version to my beta testers today, but alas the planets were not in alignment.

I had a couple of items on my TODO list which I did not take into account that actually I had no experience in creating or deploying such things. Specifically, the webservice that Thinking Stuff uses. And also Thinking Stuff needs to be obfuscated before release, otherwise someone is going to come along and reverse-engineer my code in about 2 seconds.

Sure, reverse-engineering is theft, just the same as if they had broken into my house and stolen my computer. But hackers don't care. They don't care that some guy in Australia that spent more than a year developing some software on his own is going to miss out on a few dollars. They just care that they can do it.

So, I have to spend a minimum of US$400 to buy a licence for an obfuscator. Add that to the US$700 I paid for the commercial version of the charting module, and you'll see that I need about 6 paying subscribers just to break even. Oh, and then the US$50,000 or so that used to be my salary, and then you'll see that I need more than 250 paying subscribers to break even. Lucky I made this software for my own use first, and then decided to make it a commercial venture second :-)

Anyway, I had no experience in either of those 2 things I was talking about, but yet treated them just like the other items on the TODO list. So I struck some hurdles and wasn't able to overcome them by the due date I had given myself.

Soooooo, now the excuses are done, here's what's left:

• Buy obfuscator, and obfuscate.


• Deploy webservice to server.


• Go through and fix all the tiny things that would take longer to explain than to fix.


• Make webpage detailing all other known bugs.


• Create exe to set up the database Thinking Stuff uses.

What does this mean for the release date? Well, beta testers can expect something in a couple of days (and I'm using the vagueness of "a couple of days" on purpose). And then a week after the beta testers get something, if nothing's majorly wrong, I'll release the beta version, as in pre-release version, as in bugs-still-there version, to the public that wants to get started as soon as possible.

But, because I'm not clear yet on the specifics of the MySQL commercial licence, I won't be accepting any money for anything. So, downloading data will be free, and will continue to be free. Auto-trading will be free, to get a lot of people using it to hammer out any bugs. Backtesting will not be available. It's just the corner I painted myself into by coding everything with MySQL first, and them wanting US$500 per copy sold commercially.

If I was reading this as a customer, I don't think I would fully understand what was going on, so please feel free to ask questions through comments or email.

Average Pullback Histogram

Okay, so here are the rules of using my (hopefully new) indicator.

First, I came up with this independently, but if you know someone else has already published it or something very similar, please let me know so I can give the appropriate credit.

Second, if you publish it then give me the appropriate credit. A link to this page or website certainly couldn't hurt.

Third, I've given you this indicator and how to calculate it for free. In the spirit of "you get back what you give out". If you come up with a decent way to trade with it, or a way to change it to make it better, you must leave a comment on this page detailing it. There's no being a Scrooge. In fact, it doesn't make sense - more people following your system will result in more people buying when you are buying, which should theoretically mean better results.

Fourth, comments and criticisms are welcomed. Geez, the guy over at Akumulate Pips (great site - go read it) released an indicator without even giving the details of how to calculate it, and he got all kinds of gratitude. I did this for me and not the gratitude, but a simple "thanks" would light me up for the rest of the day.

---

Background

I was looking into using some aspect of the previous bar's range as a stop loss. And then I thought I should calculate exactly how far the price normally dips back into the previous bar on an upward march.

On an upward march, the price normally does dip back into the previous bar's range, doesn't it? And that's perfectly normal behaviour on an upward march, so you don't want to bring your stops too close.

And sometimes the price actually goes lower than the low of the previous bar, only to continue its upward march. So blindly using the low of the previous bar as a moving stop, while still a solid rule, could potentially be made better. Or just more complicated - that's where your backtesting comes in to decide if these extra calculations are worthwhile.

But, sometimes the price goes lower than the low of the previous bar, and then continues going down. That's the turning point of the trend.

Anyway, I think I discovered that the amount that the price dips into the previous bar's range while continuing an existing trend jumps around all over the shop.

But, what I stumbled across, was an indicator based on how far the price "pulls back" into the previous bar's range.


Theory

A pullback into a previous bar's range depends on whether this is an uptrend or downtrend.

In an uptrend, the pullback is determined by how far the low of the current bar went below the high of the previous bar.

In a downtrend, the pullback is determined by how far the high of the current bar went above the low of the previous bar.

Make sense? Look at a chart. I've been a bit lazy and not provided one.

So we don't have to work out if we are in an uptrend or downtrend, we calculate both.

Ok, so first get:
1. Current High
2. Current Low
3. Previous High
4. Previous Low

Then calculate "Upward" as equal to Previous High minus Current Low. That's how far the low of today pulled back into the range of the previous bar.
5. Upward = Previous High - Current Low

This number will be positive if in fact the low did indeed pull back into the previous bar's range. And the more it pulled back, the bigger the number.

And calculate "Downward" as equal to Previous Low minus Current High, and then multiply that result by -1. This is for how far the high of today pulled back into the range of the previous bar.
6. Downward = (Previous Low - Current High) * -1

This number will be positive if in fact the high did indeed pull back into the previous bar's range. And the more it pulled back, the bigger the number.

You want these numbers in terms of pips, so if it's a currency with 4 decimal places like EURUSD, then multiply both of those numbers by 10000, and if it's a currency with 2 decimal places like USDJPY, then multiply both of those numbers by 100.

Now, one calculation of Upward or Downward really doesn't give us much to go with. As I said, they'll be all over the shop. So get a whole bunch of such calculations, and take an average. An average of period 20 sounds good enough. Or 10. Or whatever.

7. So the average of Upward becomes UpAverage.
8. And the average of Downward becomes DownAverage.

And here's where things become kind of hazy. I made this so long ago (but haven't actually used it, instead focusing on finishing the software) that all I can remember is that charts of those averages were very bumpy indeed. They needed to be smoothed.

So smooth them, by taking an average of each of the averages. Again, the period could be 10 or 20 or something.

I probably lost a lot of people just then :-) "Averages of averages!? Why I never". If it works then who cares?

9. The average of UpAverage becomes SmoothedUp.
10. And the average of DownAverage becomes SmoothedDown.

Then, we calculate what I will call the Main Line, and this equals SmoothedUp minus SmoothedDown. That's where the "histogram" part of the name comes in. Which could be incorrect terminology, but again I'm not much of a caring sort.
11. Main Line = SmoothedUp - SmoothedDown

But, here's where I lose some more people, because that line was again too rough. So *gulp* smooth it by taking an average of the Main Line.

12. The average of the Main Line becomes the Average Pullback Histogram.

These calculations are very roughly similar to the MACD Histogram, only the MACD starts off with taking averages of the closing price instead of averages of the pullback amount, and I have one more smoothing step than the MACD.

All that gives us a single line which rises above, and falls below zero (the top chart is the actual price).

Interpretation

If you remember, the bigger the pullback, the bigger the Upward and/or Downward calculation becomes.

And also, if you remember, we calculate the Main Line by subtracting SmoothedDown from SmoothedUp. So if SmoothedUp was very big, and SmoothedDown small, then the Main Line would be a positive number. And hopefully you can see, because this is the tricky part, that a positive number means we are in a downtrend.

Because the bigger the pullback, the bigger Upward, therefore the bigger SmoothedUp, therefore the bigger the Main Line value. And a big pullback into the upward direction means the price is heading down.

And in reverse, if SmoothedDown was much larger than SmoothedUp, the Main Line will be negative, and it means we are in an uptrend.


Trading Ideas

The basic idea is that when the Upwards pullback overtakes the Downwards pullback, the trend has changed to a downtrend (you've wrapped your head around the fact that Upwards pullback means going down, right?).

The following are just ideas, and the charts just examples. On the charts provided you can see that some ideas work, and some don't. But, that's just for that particular currency and timeframe. You would need to experiment yourself with a stop-loss technique, and with your financial instrument and time-frame of choice.

---

The point at which we can say that one pullback has overtaken another in strength is where the chart equals zero. So, one trading idea is to go long when the Average Pullback Histogram (AVPBH) goes from above zero to below zero. Go short when the AVPBH goes from below zero to above.

---

Instead of zero, you could say go long when the AVPBH falls below a certain point on the chart. And go short when the AVPBH goes above a certain point on the chart.

---

Instead of going long when the AVPBH goes from positive to negative, just allow a long entry whenever the AVPBH is below zero. And allow short entries whenever the AVPBH is above zero. And then use some other entry rule to decide the precise time of entry.

---

Instead of rising and falling above zero, it could also be argued that one pullback is overtaking another in strength when a peak or trough is formed.

Go long when a peak is formed, short when a trough is formed.

You can see that the lines are just a little the right of the peaks and troughs. It's because you don't know if it's a peak or trough until the peak or trough has actually formed.

---

Not just a peak, but go long when a peak is formed and the AVPBH is negative. And go short when a trough is formed and the AVPBH is positive.

---

Or, go long when a peak is formed and the AVPBH is positive. And go short when a trough is formed and the AVPBH is negative. Yes, the opposite of above! Who cares! Backtest to see which one works!

---

Or, go long when a lower peak is formed. Go short when a higher trough is formed. [Edit: one "higher trough" line is missing at the very start of this chart]

Summary

While a little convoluted, this indicator will hopefully show when a trend has turned in a new direction. This either signals to get in on the new trend, or get out of trades you have in the opposite direction.

Don't forget the rules of use at the top of this post!

Enjoy.

Thinking Stuff Update 05-Feb-06

The software I've been working on for the past... forever... is getting very close to something very good. That is to say, very soon I will be giving copies to my beta testers, and hopefully very soon after that it will be available commercially.

The beta testing spots are filled. Thanks to the guys who have volunteered their time. However, a couple of people have asked if they can pay to use a pre-release version. And I have absolutely no problem with that at all :-)

What I told them is that I will get it to my beta testers around Valentine's Day. Then give them a week or so with it. If there is nothing majorly wrong, I will make available the website which lists all of the currently known bugs.

If they read through the currently known bugs and can live with them, then they are free to use the software. After payment. Well, there is a "trial version" mode of the software which you can use for free.

I guess in this respect it is no different to purchasing the software after it's officially released - there will always be bugs in any software, and it's up to you to read through the known bugs and decide if you still want to purchase it.

So, if you also want to use the pre-release version, just send me an email.

---

I sometimes get emails asking for details of my API, or if by using my software can they get around the $600 charge from Oanda.

Actually my software merely provides a user interface for the APIs offered by Gain Capital (and soon Oanda). This means that there is no API that I provide that you need to code stuff for. That's the point of my software - that there is no coding required on your part. Just selecting options from pull-down menus.

It also means that it is up to you to create your accounts with those companies, and obtain access to their API. It's like you are a developer wanting to use their API, only you are getting someone else to make the software for you.

That means you have to pay Oanda the $600 before you can use my software with them.

Gain Capital's API is free, and it's pretty good, but until they allow API trading in their mini accounts, they will always just be a stepping stone before moving on to Oanda. In my opinion.

They have stated publicly in their forum that if your system is right, and tested in their demo account, then there should be no problem in jumping straight in to full contract positions. But this is nonsense.

Trading, for me, is about minimising risk. First backtest the system. If it works, then let it run in the demo account (where no real money is used). If it works, then start if off small in a mini account. As confidence in the system and/or auto-trading software builds, gradually raise the stakes until *eventually* you are trading with full contracts.

To leave out the step involving the mini account is not minimising risk.

So, I think the steps would be: Backtest the system. When you find a good one, and that might take months, then auto-trade it in the Gain Capital demo account. When you are satisifed that your system and the auto-trading software works in real-time, and that might take months, then you purchase the $600 API from Oanda and switch to them. And then buy tiny amounts until you build more confidence, and gradually grow.

Of course, you can jump straight into Oanda (when it's coded), but the method above would delay the $600 payment until you know you have a decent system (and software) to trade.

Of course, by that time Gain may have changed their mind on their mini accounts. Who knows?

---

To make up for my recent silence, I will soon release the details of an indicator which I came up with independently (but there is a high chance someone else already published something very similar), which I call the Average Pullback Histogram.