What does the US government teach us about managing software projects?

Most of us have probably worked all night and all weekend sessions in order to finish a project "on time". And we all know that is the sign of "bad management", and can also be the sign of an impossible task. Therefore, looking back at the US government working hard a few days before the deadline to get the debt ceiling raised, can be seen as a sign that the US government is badly managed, but can also be seen as a sign that it had an something impossible to do.

You might say, a government and politicians is not the same as a software development organization. Governments are fragmented, many politicians only have a job because of these differences of opinions, politicians are continuously trying to shift blame, etc. While the goal of all members of a software development team is to finish a project on time and be successful.  Or is it?...Hmmm...  In fact, that is not always clear!

The tricky nature of people, is that if they do not feel enough belonging into a  common cause, they can very easily switch to supporting its "anti-cause". Politicians know this, and much "political noise" is the game of making sure that not everyone agrees with your oponents. A development team can suffer much damage when not everybody is just "happy" to work towards the common goal of finishing the project on time. This "bad behavior" is often subconscious, yet can also be a very conscious feeling, and even sometimes an open secret, referred to, for example, with flippant remarks. What happens is that dissension can feed upon itself and simply suck away much precious work time. The team may recover under the stress of the approaching deadline, but often that pressure simply comes in too late.

Luckily, difficulties in managing teams is not new, and good guidelines exist. The first recommendation is to use an agile process:

• The "short" duration of the each iterative step ensures an optimal common focus driven by the "stress" of the deadline. And by the way, that is how you decide how long your agile iterations should be: short for a less mature team, longer for a more mature team. 
• The independence of product management (e.g. product owner role in scrum), ensures that no dissension exists in the business goals presented to the team.
• Roles like the scrum master, help focus on value to all, above the value of individuals.

Yet the agile process does not keep dissension happening on "long term" concepts. For example, on design and architecture, or product goals. Usually, technical dissensions are more problematic than business dissensions. The question is often: "How to make sure that people work together productively on long term technical goals"?

My answer is:

Get every one to agree that common long term technical goals exist only to constrain the team; New concepts can be brought in by the team's individuals, but only within the short term "iterative" process. 

For many, the idea of only be able to bring in new concepts within a short time frame is just not acceptable. They argue: we need more time! But too much time would mean too much uncertainty, and uncertainty is an area in which dissension can strive. Another argument is that vital new concepts will never be brought in if there is no long term goal, and as a result the team may fail. This is true. But the consequence is that people need slack time: unmanaged time outside the process in which they can mature new ideas (see delivery focus versus creative slack). It is only when ideas and new concepts have matured that an attempt can be made to bring them in to a project within a solution. And the criteria of maturity is "needs to be able to deliver within the short term iterative part of your agile process".

People may still argue that a bleak constraining, almost pesimistic, long term technical vision does not provide value, is "no fun", and possibly even not worth working for! These are not unreasonable worries.  Yet these remarks are all based on experiences where short term efforts fail to provide enough value, or fail to provide enough satisfaction.They are also come out of cultures that do not understand the critical need of a "side" process in which ideas and techniques are selected, grown, and matured, until either they are rejected or they are brought into production.

US politics, like most political systems, have this concept of the parallel process that prepares material before bringing it into the short term process of getting decisions made.This is a system of influence, of key players, committees and lobbyists. It would be wrong to deny that development organization are not subject to similar games of power, yet the big difference is that a development organization should only get one source of funding (from business). While a democratic system gets multiple sources of funding with often contradicting goals.

I cannot make this posting too long, therefore to summarize:

• Run a tight development process: only bring in ideas that are mature enough to succeed withing your "short term" production cycles.
• Encourage independent new long term ideas but outside your production process: Accept that there is some political process in how your team brings in new ideas into their production effort; Make sure that new concepts are mature enough that they can be brought into production with good certainty, within the normal "short term" development cycles.
• Encourage "constraining" long term team goals within your production process: It is OK to debate as a team on longer terms plans that simplify things and restrict usage of new technology. I say this because technical entropy, the fact that things "grow more complicated with time" will kill you if you do stay united as a team to fight it.
• Make sure that your team has enough slack time: And make sure also that they have the culture to use it wisely, for the benefit of the project (and not for each developer's ego).
• Have only one external driver per development team: The same for "normal" short term development process as for the long term "creative process". This will keep the "political" debate healthy. And that is again a reason why agile processes have only one business representative per team.
• Drive software with deliverable features, business with revenue streams: It is this separation and its "double layering" within the overall development process (e.g. agile development teams versus one product management team) that keeps the political "game" sane! Technology does not generate money, only features do! (Well, within a bigger model it can, but that is why you might consider having a CTO outside the development team process).

 This brings us back to US politics: they are like a very, very, bad development team. They apply none of the "good behaviors" above. Not a very good role model for their fellow citizens!

All original content copyright James Litsios, 2013.

OO design patterns from a functional programming perspective

Here my attempt to match up design patterns with functional programming constructions:
Chain of responsibility
Fold request across the chain of handlers, at each step returning either (using Either type) request to continue fold (possibly with transformed request), or returning handled request indicating that fold is finished.

Command
Request is held in data (e.g. as discriminant union)

Interpreter
Language is available as typed data structure (e.g. discriminant union) and is interpretable with functions calls.
Alternative:Language is defined in monadic form (e.g. as function calls)  and is interpreted as part of the monad's interpretation.

Iterator
Exposed traversal functions hide data structure of containers. Functions typically work on zipper focus adapted to the containers structure.
 
Mediator
Exposed functions stay at top/root of exposed data structures.

Memento
Use immutability and exposure of intermediate states to allow recovery, restart or alternate computation from previous intermediate states.

Observer
Associate container of data with container of queries (e.g filter plus callbacks). Changes to data container (insert, update, removal) triggers matching query callbacks, change to queries may trigger callbacks. Callbacks are either in a monadic structure or pass around a state context.

State
Data context position is indexed (e.g. by one or more keys), associated state is stored in corresponding state monad where individual states are found with index. 

Strategy
Single function signature implemented in multiple ways.

Template Method
Use type classes to refine implementation by type variant.

Visitor
Break down algorithm into sub-functions; Provide these sub-functions in argument to algo; Use them within algo; Provide different ones if you want to change behavior of algo.

Adapter
Adapter functions that takes one or more functions with given signatures and returns one or more functions with adapted functions.

Bridge
Clients accesses a bridge data type that holds functions.

Composite
Use recursive data types.

Decorator
Data has a dynamic map of functional operators.

Facade
API presents no internal types and is as "flat" as possible. This may mean that certain functional constructions have been "collapsed" into a purely data centric view in order to limit the "depth" of the API.

Flyweight
Flyweight data structure are often provided as "first" arguments of functions. These functions can then curry their first argument so that the flyweight shared data is provided but does not need to be "passed around" everywhere.

Proxy
Data type that "hides" another type. Possibly relies on using an index/key reference and a map like container to store original data.

Abstract Factory
Factory hides the finer type of created data and functions. Losing the inner type can be tricky because the  exposed signatures must still need to be powerful enough to do everything you want with the exposed API (as we assume that no dynamic type casting is allowed). This may mean that you need powerful type features to make the abstract factory concept work (e.g. GADT).

Builder
Define functions to construct data. This is in fact the "normal" way to write functional style.

Factory Method
Use polymorphism, types are determined by inner types of arguments.

Prototype
(Immutable data does not need to be copied).

Singleton
Encapsulate the access to the singleton as a mutable variable within a function; Use this function to access singleton. In immutable setting, store singleton within a  state monad.

(I'll revisit this one if I have the time to try to put side by side an OO definition of the pattern with a functional definition of design pattern. Also it is a bit minimalistic).

All original content copyright James Litsios, 2013.

Silhouettes: understanding unique talent

My favorite skateboarders these days are Rodney Mullen, Ben Raybourn, Chris Haslam, and Chris Cole. There is an interview of Rodney Mullen where he says he is looking for a special "silhouette" in skaters. In fact, he is saying the he is looking for uniquely talented skaters that have internalized their skating into their cognitive growth process. To give you a mental image of what I am saying:
We all need things we can lean on to grow. What I mean by "leaning on something", is that we use it to support us, we use it as dependable reference. When we are young we lean on our parents. Later, most of us lean on our social/economic "fabric". Yet others lean on their own selves. If you are unlucky, this self "leaning" may make you a unpleasant narcissistic person, yet if you are more lucky you do not lean on a "synthetic" self created social reference, like a narcissist person, but on a self created non-social reference. And for the people above, on a self created skateboard cognitive reference.
This all sounds complicated, yet in fact it is not, Rodney Mullen has already "said it all": look at the silhouette of these skaters, are they connecting to you like others, do you see them like others? In fact, neither: they are not connecting to you like others, and you do you see them like others. There are connecting to themeselves, and we perceive this as a combination of a unique "character" or style of skating, plus a slight social disconnect.
You probably need to have been a skateboarder to "see" what I mean when I talk about these people. But if I say: Steve Jobs, David Bowie, Cat Stevens, Stevie Wonder, Eric Schmidt, maybe you get a feeling of what I am talking about.

(Say thanks, to Splat for having ask me to explain this)
All original content copyright James Litsios, 2013.