Monday, January 24, 2011

Nearest neighbour classification

I've been following trading the odds blog  for some time now. It surprised me how well  Frank was able to predict the short-term movement of the S&P500. At first I was quite sceptical about the predictability of the directional stock movement, but had to admit now that there is some sense to it.
Previously I've been playing around with some techniques from pattern recognition field, especially the 'nearest neigbours' approach. Just as one could get an impression of somebody by meeting his friends, the same can be said about stocks. By finding some 'examples' most related to an 'new' situation, a statistical picture could be made of the things to come.
To provide a simple illustration, I've taken the intraday SPY and TICK data for 2010. Some traders are using the first hour of trading to decide what kind of strategy they need to implement for that situation. But is the first hour of trading representative for the rest of the day? It turns out it is. Take a look at the chart below.
Here I've classified the intraday SPY data based on the first hour of the TICK reading. For this I've used the mean and standard deviation of the 30-sec TICK-NYSE data.  Green lines in the chart represent ten days with minimal mean and minimal std of the TICK in the first hour (closest to [0,0]). Blue lines are days with large positive mean of the tick and  low std ([max, 0]).  It turns out that days with large positive bias int the first hour of trading have a quite strong negative bias for the rest of the day, who would have thought?
Predicting the market regime for the rest of the trading day is not very profitable by itself, but can be very well used for choosing the right strategy.


  1. Interesting discovery.

    Have you conducted the hypothesis over a longer time series (2009, 2008, 2007)?

    What are the results of large negative mean "mornings"?


  2. @Anonymous: unfortunately my intraday data goes only one year back.