Having quite a background in Q-learning (my masters thesis), I understand its beautiful ability to map a state space to an expected reward, elegantly and without any models. Doesn't trading in general boil down to state-reward mapping? For sure!

I have tried different implementations of reinforcement learning without much success. But this weekend I managed to combine some ideas from pattern recognition and RL to an implementation that could work.

Some people are probably wondering by now: ' Here we go with the AI bullshit again!'. I'd like to call it probability mapping. Also, probably many of the advanced quantitative traders are already using it. Its all about estimating the chances of a bet at a set of certain conditions.

First, I define the conditions, called 'feature 1' and 'feature 2'. Two features means a two dimensional feature space, nothing keeps us from making it more (or less) dimensional, but since my monitor can best visualize two dimensions I choose that number. In my case, both features are oscillators based on cumulative returns over past x days. Feature 1 uses 3 days averaging and feature 2 20 days. Any other measure could be used (RSI, Stochastics, etc).

In the figure above the ratio between XLE and XOM is plotted along with two oscillators. It is clear that a high oscillator value correlates with the subsequent drop in the ratio.

Normally you could start applying threshold conditions from here based on oscillator levels, but i want to go a couple of steps further.

So now I plot my state map for all values of feature 1 and 2 along with the corresponding future XLE/XOM ratio after 5 days. A green dot represents increase and red dot decrease of the XLE/XOM ratio.

From this map an estimation can be made how likely it is for the ratio to go up or down for each combination of the features. Let's call it 'Sharpe surface'. I define it similar to the sharpe ratio:

*mean(20 nearest neighbors)/std(20 nearest neighbors).*

Scanning the feature space gives me the nice plot as above (note that the vertical axis is flipped over compared to the previous figure).

Again, normally we would use just one of the dimensions and put thresholds somewhere around 20 and 80. But with this probability mapping we can go for cherry picking!

Files: probability_mapping.zip

Any remarks about the code are very welcome.

Hi, could you please send me your code

ReplyDeleteWhat programming language are you using? R? Matlab?

ReplyDelete@Maverick: I'll post a package of the code on the blog.

ReplyDelete@Gigi: Matlab. I've been using it professionally for years. Haven't tried R yet and honestly I am so satisfied with Matlab that I don't feel the need.

ReplyDeleteHI, I stumbled across your site from quantivity. I like some of your ideas.

ReplyDeleteHave you looked at the feature map evolve over time? And have you found any stability or relationship between out of sample sets?

@intelligent trading: Yes, in the past I've taken a look at the feature map evolving over time. I did it then for a directional strategy. The map tended to be quite unstable. I'm planning to do the same for the market-neutral strategies, expecting much more stability.

ReplyDelete