Saturday, December 28, 2019

Beware of the language trap in software development

Consistent design of multiple views of distributed state

The notion of "state" is key to software development. For example, we talk about stateful and stateless designs. Still, there are multiple notions of state. For example, there is the stored state, the communicated state, the received state, the processed state. Therefor when we make a statement like "stateful", we are always just referring to one "notion" of state and not the others. This "abuse of stateful view" was historically not an major issue. Yet with the rise of processing speeds and truly distributed systems, these multiple views of states started more and more to exist "simultaneously" and this caused problems. For example, we might model a trading system has four different states: the state of trading action messages received by the exchange, the current state of the exchange, the state of confirmations received from the exchange, and the current state of our trading system. A question is then: in our concrete code, from which state views should we build our design, and how to do it consistently? We now know this is the wrong question. A trading system is a distributed system, all views of states are real and have business significance. Therefore the question is not which state, but how to design all state views/models as one coded design in a cohesive manner?

We now know we can use higher order types to do this, to capture the multiple views of states "as one model" within a distributed computation. It took me a long time to understand this. One reason being that only C++14 had the explicit template specialization needed to do this purely in C++. And while I had experimented with Prolog to generate complementary C++ code models in the mid-1990s (and was not convinced). And in early 2000, I tried to use generative technics with dynamically typed functional programming (I wrote about a generative programming conference in Dr. Dobb's at the time). It was only when I picked up statically typed FP (initially F# in 2006), that I understood how it could be done (e.g. with monad-comonad adjunctive joins as hinted here pre-Elevence).

Business success

The reason I bring "managing distributed states" in this posting, is that I was retrospecting on the challenge to make a success out of Elevence (acquired by Digital Asset in 2016).  I had already "solved" the distributed state problem before I co-founded Elevence (as mentioned above). And Elevence had already developed late 2015 the language that became DAML (see BearingPoint's announcement about this in 2016). This made me think back to my experience in developing a market making system. (I wrote about Actant in a recent posting).

Early 2000 market and regulatory pressures meant that many new derivative exchanges where being created. This resulted in many "easy" new business opportunities to sell an algo trading and market making system, as one "just" needed to connect a product to these new exchanges. However, what we did not foresee, was that this "lively" exchange business development would also have the exchanges competing "lively" among themselves and be updating their trading and quoting APIs at an unprecedented rate (e.g. every quarter). In parallel, we consistently had mismatches in our different view of distributed state, for the reason mentioned above, but also because we were exposing different state views that were easy on our end-users, but which broke distributed consistency. The result was, we spent most of our development resources maintaining high-performance exchange connectivity and managing a hard to manage "mismatch" of state models, with little resources left over to be strategic.

The language trap

One of my Actant partners once said: C++! It was C++ that hurt us the most. (Again Actant mentioned here). By that he meant "stick to C", but C is a subset of C++, right? So how can C++ be an issue?

Here is a timeline that will help us:
  1. Pre-C++98 (templates): we used macros, generative and model driven programming to write consistent models that ran distributed programs as multiple and consistent views of state.
  2. Post-C++98: we used templates, type unsafe models, and redundant code to write distributed programs with often inconsistent views of state.
  3. Post-C++14 (explicit template specialization): we used templates and explicit template specialization to write consistent models that run distributed programs as multiple and consistent views of state.
Because we chose to adopt the rules of C++, because we did not understand that by doing so we could not code a single consistent model of multiple views of state that we needed for distributed computing, we got "caught" in the 16 year gap of formal inconsistency that C++98 introduced! 

I write "formal inconsistency" because nothing in C++98 says that you couldn't continue to use macros, generative and model driven programming to get around the limitations of templates. The thing is "we do not know what we do not know", so we did not know that it would have been best to ignore the "formal" template solution and stick with our old technics. And that is example of a language trap.

A language trap is when developers choose to adhere to limiting coding language semantics without understanding that they are "shooting themselves in the foot" because they are no longer able to "get things right" when adhering to these limiting language rules. In some sense, a language trap is a technical form of semantic barrier.

Unfortunately, again "we do not know what we do not know". So we may or not be in a language trap, we do not usually know if this is the case. In early 2000, we did not realise how much the choice of purist C++ approach had been a bad choice.

My learning from that period was: never let a language nor language purism restrict you. Because you do not know how that it harming you until it is too late. The safer and future resistant approach is to deconstruct "a single language", and be purist in how you formally compose multiple language constructions.  An advantage of this approach is that it may also be applied consistently across multiple languages.

All original content copyright James Litsios, 2019.

Sunday, November 10, 2019

The Soul of an New Old Machine: my Lenovo x220

Let it be known that when I significantly change jobs I buy a new laptop.
Which is why I find myself typing this on the Lenovo x220 I purchased for 120CHF at the local used hardware shop here in Oerlikon. Once I have finished to set it up (it already has new SSD, RAM and battery pack for an extra 200CHF), my 13" MacBook will be mostly my work computer (for my technical advisor job).

Here is a picture of it. If you care why I chose an x220 and the switch back to linux, then do read on.

The thinking goes as follows:

  • Leaving Apple: In 2015 I chose MacBooks as standard laptop for engineers at Elevence, really because when you get acquired, you need to be already halfway enterprise. Yet the choice became a pain because of the extra effort needed to get development tools running on Mac OS. Then in 2016, Apple stopped being developer friendly by killing touch typing with the Touch Bar, and worse by removing the Escape key. (And yes, of course you can reassign it, but that is not the same as a native, well placed key).
  • Not a new laptop: I use to buy new generation laptops for speed, features, and security. Speed is no longer needed. Features are always lacking as product managers have learned their lesson and avoid perfect products that no one replaces. And true security seems to be gone for good now, as network security, and installed OS and application choice is now more important than hardware choice for security. 
  • Linux: "What else?" as Mr. Clooney would say. Not only for the developer tools and window manager friendliness, but also because it is cool to type fdisk and to be brought back to my early 90's Sun Microsystems experience (I was a SunOS sysadmin for ~3years).
  • Lenovo x220: The x220 is the last laptop with a deeper key action, and I love to touch type. As I also love well placed control keys (ctrl, alt, super, ...) to "drive" window managers like i3 or XMonad and editors like vi and emacs (10+ years under DOS (Desmet C, Zortech C++) and then Emacs and vi on SunOS and Linux for another 10+ years will do that to you!).

For your info I am programming in Haskell on this machine.
Actually, it is worse than that, I am programming in Haskell and much with profunctors.
Those that know why are few. Some might guess why. Hopefully this will all be open sourced decently quickly down the road.

PS.1: "The Soul of a New Machine" is a book published in 1981. My title is a homage to the book as there are only so many good narratives on hardware and software developments. The book has aged a bit: who cares about 32bit computers in 2020? Yet it captures that "soul" aspect, where we engineers look at the world inside out, from technology outwards.  Of course, over the years I have learned to see the world in many other ways, for example, from a the product manager's client using the product view, or from head of HR orchestrated people perspective, yet the engineer, in his/her deeply mechanized and often deeply theoretical ether, is a special place I still do like to revisit.

PS.2: My cranky side I am not happy to program in Haskell. I am not happy that most of the Haskell libraries are "old" because only written with a "single dimensional view', and I am not happy that most developers cannot deal with Haskell, and that Haskell cannot easily "thread" multiple dimensions of type polarities without tons of type level programming. But I know Haskell, and Haskell tools are better than any other higher order FP language, so Haskell it is!

All original content copyright James Litsios, 2019.


Saturday, November 02, 2019

QT Software, the Troika, to insights on co-founding Elevence

Mattias Jansson, Lukas Lüthy, James Litsios

In summer 1996, I met Mattias Jansson and Lukas Lüthy (left and Luki middle above, I am at the right), as well as Adrian Lucas (left below) and in what seemed to be less than 15m, they were inviting me to join them set up a software company to develop a non-proprietary derivative market making system. Another key partner in this story is Stig Hubertsson (not shown). Therefore, we found ourselves co-founding QT Software, which later was renamed Actant.


I bring up this past in this blog to cheer excellent individuals and a great team, and give you insights into how QT and Actant led me to co-found Elevence and develop a unique smart contract language (then acquired by Digital Asset).

First the cheering: Still now Actant software is known to be the fastest non-proprietary derivative trading software. And while I don't have the numbers, I could well believe that they compete well with proprietary derivative trading software on classical hardware (meaning CPUs and not FPGAs).
And we might mention golden years of early 2000's when two of the top three European derivative market making firms were relying on Actant's product to do their business.

Mattias, Luki, and I were the core the company's engineering. Like minded, we worked much as one,  relying on voting to resolve the sometimes disagreement. Did we always make the best choice? I think not, yet we did shape a unique company with its specific market niche and customers.

That first picture above was taken in early 2018. Two hours later we were out of the house watching the fireman putting out the fire of the gas barbecue grill. Luckily no one was hurt, the gas tank did not explode, and house suffered only minimal damage. We then cooked our steaks in the oven and enjoyed an evening of great friendship.

The second picture is LIFFE Connect advertisement in the Financial Times!

In 1998, QT software was running on LIFFE Connect and Eurex. For the LIFFE Connect opening, which was first electronic access to the LIFFE exchange, we were asked to help publicise the new exchange offering. Adrian is to the left of that picture, and I on the right. Funny story, we showed up for the photoshoot at the exchange. They offered to walk down to the trading floor, but then proceeding to disallow Adrian's brown shoes, as only black shoes were allowed there!

One insight I gained in market making was how to model distributed systems with "adjoint" relations. The date was October 2008, a difficult period because of the impact of the 2007 crisis on our customers. I was lying in bed at our company's yearly executive get-together at the Palace Hotel Lucerne, and it occurred to me that one could model the market maker as some form of adjoint dual to the market of buyers and sellers. Years later I used that insight to co-found Elevence Digital Finance where we developed a language of rights and obligations sustained by a dual adjunction like relation to a blockchain ledger (see BearingPoint's announcement when Elevence was acquired by Digital Asset).

All original content copyright James Litsios, 2019.