Saturday, August 11, 2012

The case of the mysterious distracting type class

I had a magical insight recently about something that was bugging me almost two years. The question was: why the hell was I so unproductive creating type classes!

So here is the deal: when you develop, you combine invariants and transformation properties of your  different pieces to make something new. If you are someone like me, you live your code, and that means that the relations between the different concepts throughout the code drive me "emotionaly". Yesterday, it occured to me that what is special with type classes is that they are "open".  They are open because their properties are not always obvious from the definitions. Worse, new type classes often have unambiguous properties.

I say open because by contrast if I avoid type classes by explictly defining type specific functions or if I create encapsulating types to replace the functional polymorphism by data variants, then I am in effect "closing" the design. With a "closed" design I have the whole design in my head and I "just know" what is best to do next. With an open design I find myself worrying about peripheral design issues which are mostly not so important,  yet I still worry about then... and that really hurts my productivity.

I will therefore adopt a much stricter approach to creating new type classes. The basic rule is NOT to create new type clases. Yes, I love to think about them, but I can't afford to. Then, when I have mentally built all the invariants with the explicit function calls or encapsulating types, and more importantly when I have finished what I was orginally trying to do, I can imagine refactoring and bringing in new type classes.

Friday, August 10, 2012

monadic folds

I have been rewriting my type inference framework over late evening sessions this week. I would not do it just for the refactoring, I want to be able to do more with it. Although I will not tell you why, and I can tell you how.

This is F# code, and my first version was monadic, modular, but flat. In some sense I was using the monadic abstraction as a comfort zone but without pushing it. Now I have "internalized" the concept and "objectified" it to the point where I am comfortable to program mostly without the "do" notation, composing higher order monadic constructions. And this brings me to my favorite pattern of the moment: the monadic fold.

Here is how it works:
  • the fold pattern is "joining" an operator accross a slice of data. In effect it is a universal "block" but which is "projected" against existing data, see each fold as an new inner hierarchy in your code.
  • the monadic environment supports the "environmental" needs of the operators
Building your specialized library of environments and operators is no easy. Yet the good news is that it easier with time.  The good news is that monads are like "single dimensional objects", and that is to say that if you have packed things in objects and you know how to be selective, then packing single concepts into monads is not too hard.

Friday, August 03, 2012

Easy to make money, hard to keep money, assymetric systems

A market making and brokerage firm, Knight Capital Group, lost all their money (~400M$) when their automated trading system went crazy and it seems repeatedly bought high and sold low, as fast as it could, across many instruments.
My guess is that they turned on a test app on the live market. This would explain why they were not feeding the trades into their portfolios.
An algo to buy the spread makes sense if it is used to act as a counterparty to test other algos such as a market making application.
Then again, a test app normally has hard coded detections of the live market that turn it off. So who knows?!

This event is interesting in a few different ways:
  • What fail-safe code do you want to include in your algo system?
  • What fail-safe development management process do you implement to try to guarantee that those failsafe coding standards are met?
  • Trading is really about keeping your profits, not about making them!
  • Organic systems are assymetric, their behaviors change with the change of direction of flows!
My fail-safe coding rules are the following:
  • Do hard code constants! For example, test applicaiton includes production IP addresses and will refuse to run on these.
  • Too much of a good thing is a bad thing: automatically disable strategies  that have triggered "too many times".
  • Four eyes: have important code reviewed by someone else. 
Fail-safe development rules are much harder to implement:
  • Keep developer that worry about overall consistency. Developers that forget things are expensive.
  • Encourage your developer to expose and maintain a clean stateful design for your algo system.
(to be continued)