Sunday, May 24, 2020

On growth of digital ledger and blockchain technology (spring 2020)

Digital Ledger Technology (DLT) and blockchain technology has very much suffered a slower growth over the last year and a half. Here are a few reasons:
  1. Integration of DLT is expensive because the integration needs to be deep to be robust to failure.
  2. Given the depth of integration, the DLT owns much of the future, not the companies that integrate the DLT. This happens also with cloud and AI migrations, yet with blockchain this dependency is much more absolute. To alleviate this, DLT users may become partners in the DLT system, yet with additional risks and costs.
  3. Cryptocurrencies are killing the growth of broader DLT technology. In effect the equivalent of >200 billion $ of cryptocurrencies are digging their feet in the ground, and all saying "not without me", with almost an infinite marketing budget to fight other forms of tokens that would cut them out, such as tokens of trustful processes.
  4. The cycle of innovation in blockchain is extremely fast because it is mostly just math, and math will progress, ensuring a quick obsolescence of each generation of blockchain technology. While obsolescence of technology is normal, blockchain is very much a space where theory progresses faster than implementation. And that is a tricky space to invest in.
Interestingly, in 2015, I founded what was probably the most sophisticated DLT technology company (Elevence, then acquired by Digital Asset). I wanted to build a very formal, "pure thought" like, software company, with the goal of developing a smart contract product, and was lucky to hire a great team that delivered a blockchain based smart contract language in less than six months. The project then was a bit like the legendary planet Magrathea, in the Hitchhiker's Guide to the Galaxy, where the Magratheans go into hibernation, awaiting for the rest of the galaxy to catch up with them. This can be a challenge with new technology, where sometimes your users need a lot of time to deal the changes brought by your innovation. The game is then to hold out, while shaping your product with chosen first movers, at risk of even more sophisticated technology rendering you obsolete.

All original content copyright James Litsios, 2020.

Sunday, May 10, 2020

Search and Vision for Systematic Innovation

How to be productive in innovation?

The simple answer is "work hard", "be systematic", "believe in a larger vision".

Best to illustrate this:
I am writing a python program to do something new. I know that to be innovative I need to be "one level above" what was previously done. So I include a construction which I know enables this "newness". The problem is that this construction does not fit with "the normal code".  I notice this only after my first attempt to put things together, when on the way to a design, I cannot find a way to bring back in the requirements I have dropped. Let's call this first attempt design A
When you fail, you try again. I pivot my design, now to focus on B. Same result, design fails to bring in all requirements. I try again, now focusing on C. Same result, the design does not fit. A this point I start worrying. I step back. I explain to my daughter: "tricky, but I have got it! What we need is to go a level above and dynamically bring in A, B, and C concepts in a pre-sorted meta-design that depends on the input data". This new design, let's call it sorted ABC is not easy... and it also fails! Now I am seriously worried.  
I try again. Yet this time, knowing that A, B, C have failed, as did sorted ABC, I step back further. I go all the way to the "one level above" concept, which I take out, and put back in, in a complementary way. In two days, I have the design, and in a week I have all the requirements in it.
Innovation is just a "search algorithm". Edison knew this. The pharmaceutical industry has scaled this to the extreme. What is a little bit tricky in computer science, and similar technology domains, is that innovation is a mental game. The result is that failure within a search effort impacts moral and motivation.  The fact is, if you have not prepared yourself, it is not possible to invest yourself, and not feel the pain of failure.

The effort I described above took me more than a month. I might mention that I was getting seriously upset with my repeated failures. Yet the trick is that failures that are progress within a bigger vision are acceptable. They are painful, they make you lose time, but they are still ok if they bring you closer to your vision. Therefore the simple rule in innovation is that your need that larger vision before you start! And in fact no work should be done without that larger vision, and that larger vision needs to be a real concept of delivery of a real product with real innovative advantages.

A key ingredient to this process is that failure must reduce the search space towards your innovative vision. As a leader or in advisory roles, I find myself repeating and emphasising this more than anything else. Because this is the "crux" of productivity in innovation. Again: FAILURES MUST BRING YOU CLOSER TO YOUR VISION!

Finally, if your failures are not bringing you closer to your vision, then you have placed the bar "too high" and you need to diminish your expectations, and reduce the scope of your vision.

All original content copyright James Litsios, 2020.

Sunday, April 12, 2020

Thinking versus brute force "doing" in software design in 2020

About thinking versus doing

(I put a subset of this on YouTube as:

Is it ok to program by "brute force" and put aside your brain?

A few years ago, I wrote on the pros and cons of programming with explicit and implicit type declarations. In 2020, I feel that this question is almost irrelevant. Therefore to ask the question:
Should we think about software design when we program? 
There is a trend in software development which states "developing software products is not about thinking but about doing".  I know many venture people who believe very much in this statement. Yet the reality is that software is a process of thinking (in part at least). Still, from a pure seed-venture-money game, the rule of "it is not about thinking" is true"! Because the number one rule is "be fast" and thinking too much will very much slow you down.

Is it then both true and untrue, that we should both use and not use our brains when we write software? Yes, and yes! And it is exactly this subtle challenge that makes software management in innovation a tricky game.

Thinking can be done different ways, different approaches can be taken. Some are efficient, some are a waste of time. Some will help your software grow, others will block you. Some will help you meet your deadline, others will invariably lead to failure. The game is then only to think in the ways that help you!

Some of my colleagues have heard me condense these concepts into one statement:
It is about thinking backwards!
There are many ways to do this. I will cover just a few here, there are in fact as many more.

Play to win above all else

In recent blitz chess games, FM Lefong Hua repeats "it is not about calculating the end's win game", (because that is not a guarantee win), instead is about about "flagging your opponent". By which he means, winning is not about thinking within the classical rules of chess, it is about meeting the "win conditions", which in blitz chess games is much about not running out of time (being flagged).

Transposed into a software development, that means it is not about meeting the classical rules of software design, it is about thinking through how to always meet your goals and deliver results. Or to be more specific, it is about thinking backward, to find a path back from your "runnable software" goals, to your current status that guarantees your development success.

The heart of your software comes first

I have mentioned that "One of the little understood properties of programming is that at the lowest level things tend to be single dimensional".  A good software design builds a "break" or "separation" in that unique dimension "just about where your main product abstraction is", to achieve a sandwich like design where new product abstractions are added "by reassembly" of the two separated parts.

There are few ways to do this, domain specific languages (DSL) and functional programming (FP) being my favoured approaches.  While all my major software developments had DSLs, it was only much later in my career that I understood that what was working for me was not the DSLs but the effort to maintain separation between interpreter and interpreted abstractions. This separation is naturally achieved with a DSL, yet can also be achieved with adjunctive higher order FP. (The "ultimate tool" is then to combine the two concepts as adjunctive FP based DSLs, which was the base of Elevence's contract language).

Be functional and agile

Agile development is a lot about prioritizing your requirements and implementing them in work iterations. That means that the first level of your design, what might be seen as the first draft of primary keys of your data base, are directly tied to the most important business concept (taken from OO vs FP vs agile). The trick is then not to see this as a database of data, where keys are data, but a database of code, where keys are functional properties such as APIs with specific invariants (e.g. somewhat like class types in Haskell). The extended trick is then also to make this a normalized design (just like a normalized classical DB schema, but in a the "functional space"), for example by using linear types.

Brute force yes, but with the right pieces

Let us put this all together, our first conclusion is when building new software:
Use brute force and minimal brains to assemble data and call external APIs 
Importantly, keep your program structure as flat as possible when you do this. Which pretty much means to shun OO patterns, and to use as many array and container like constructions as you want.

Then, and this is the tricky part:
Incrementally invest effort to separate interpreter and interpreted abstractions in your code.
Always make sure that these abstractions fit within the plan to your deliverables. That means you are able to think backwards from your deliverable goal, and work out the path to your current development state, the path which you will follow to build your code. An agile process and a good team will make this an easier job. Also, all those arrays and containers will need to fit into a single unified concept (e.g. such as indexed tensors).

It's complicated

Sometimes things are just too complicated. Many developers do not know how to "separate interpreter and interpreted abstractions".  Here I will be honest, it is not easy. And just to make this clear, the last DSL I wrote took me multiple tries until "I got it right".  Also, to mention that embedding product abstraction in higher order FP is even harder than writing DSLs.

Worse, the process is "hack brute force" to assemble data and use external APIs, and only then think about how you can slice your interpretive abstractions. These means that initially, software development is about doing, not thinking. Tricky...

Welcome to 2020

I have been writing this blog for almost 15 years. Interestingly, a major change over these years is that there is so much well integrated open source software (OSS) out there that it is cheaper to "just try" than to "think" (when using OSS and third party APIs). And in part it was this reality that led me to write this blog today. Enjoy, and stay safe!

All original content copyright James Litsios, 2020.