Easy to make money, hard to keep money, assymetric systems

A market making and brokerage firm, Knight Capital Group, lost all their money (~400M$) when their automated trading system went crazy and it seems repeatedly bought high and sold low, as fast as it could, across many instruments.
My guess is that they turned on a test app on the live market. This would explain why they were not feeding the trades into their portfolios.

An algo to buy the spread makes sense if it is used to act as a counterparty to test other algos such as a market making application.
Then again, a test app normally has hard coded detections of the live market that turn it off. So who knows?!

This event is interesting in a few different ways:

• What fail-safe code do you want to include in your algo system?
• What fail-safe development management process do you implement to try to guarantee that those failsafe coding standards are met?
• Trading is really about keeping your profits, not about making them!
• Organic systems are assymetric, their behaviors change with the change of direction of flows!

My fail-safe coding rules are the following:

• Do hard code constants! For example, test applicaiton includes production IP addresses and will refuse to run on these.
• Too much of a good thing is a bad thing: automatically disable strategies  that have triggered "too many times".
• Four eyes: have important code reviewed by someone else. 

Fail-safe development rules are much harder to implement:

• Keep developer that worry about overall consistency. Developers that forget things are expensive.
• Encourage your developer to expose and maintain a clean stateful design for your algo system.

2021: the wikipedia page above confirms:

The incident happened after a technician forgot to copy the new Retail Liquidity Program (RLP) code to one of the eight SMARS computer servers, which was Knight's automated routing system for equity orders. RLP code repurposed a flag that was formerly used to activate an old function known as 'Power Peg'. Power Peg was designed to move stock prices higher and lower in order to verify the behavior of trading algorithms in a controlled environment.