Sunday, October 27, 2013

What does the US government teach us about managing software projects?

Most of us have probably worked all night and all weekend sessions in order to finish a project "on time". And we all know that is the sign of "bad management", and can also be the sign of an impossible task. Therefore, looking back at the US government working hard a few days before the deadline to get the debt ceiling raised, can be seen as a sign that the US government is badly managed, but can also be seen as a sign that it had an something impossible to do.

You might say, a government and politicians is not the same as a software development organization. Governments are fragmented, many politicians only have a job because of these differences of opinions, politicians are continuously trying to shift blame, etc. While the goal of all members of a software development team is to finish a project on time and be successful.  Or is it?...Hmmm...  In fact, that is not always clear!

The tricky nature of people, is that if they do not feel enough belonging into a  common cause, they can very easily switch to supporting its "anti-cause". Politicians know this, and much "political noise" is the game of making sure that not everyone agrees with your oponents. A development team can suffer much damage when not everybody is just "happy" to work towards the common goal of finishing the project on time. This "bad behavior" is often subconscious, yet can also be a very conscious feeling, and even sometimes an open secret, referred to, for example, with flippant remarks. What happens is that dissension can feed upon itself and simply suck away much precious work time. The team may recover under the stress of the approaching deadline, but often that pressure simply comes in too late.

Luckily, difficulties in managing teams is not new, and good guidelines exist. The first recommendation is to use an agile process:
  • The "short" duration of the each iterative step ensures an optimal common focus driven by the "stress" of the deadline. And by the way, that is how you decide how long your agile iterations should be: short for a less mature team, longer for a more mature team. 
  • The independence of product management (e.g. product owner role in scrum), ensures that no dissension exists in the business goals presented to the team.
  • Roles like the scrum master, help focus on value to all, above the value of individuals.
Yet the agile process does not keep dissension happening on "long term" concepts. For example, on design and architecture, or product goals. Usually, technical dissensions are more problematic than business dissensions. The question is often: "How to make sure that people work together productively on long term technical goals"?

My answer is:
Get every one to agree that common long term technical goals exist only to constrain the team; New concepts can be brought in by the team's individuals, but only within the short term "iterative" process. 

For many, the idea of only be able to bring in new concepts within a short time frame is just not acceptable. They argue: we need more time! But too much time would mean too much uncertainty, and uncertainty is an area in which dissension can strive. Another argument is that vital new concepts will never be brought in if there is no long term goal, and as a result the team may fail. This is true. But the consequence is that people need slack time: unmanaged time outside the process in which they can mature new ideas (see delivery focus versus creative slack). It is only when ideas and new concepts have matured that an attempt can be made to bring them in to a project within a solution. And the criteria of maturity is "needs to be able to deliver within the short term iterative part of your agile process".

People may still argue that a bleak constraining, almost pesimistic, long term technical vision does not provide value, is "no fun", and possibly even not worth working for! These are not unreasonable worries.  Yet these remarks are all based on experiences where short term efforts fail to provide enough value, or fail to provide enough satisfaction.They are also come out of cultures that do not understand the critical need of a "side" process in which ideas and techniques are selected, grown, and matured, until either they are rejected or they are brought into production.

US politics, like most political systems, have this concept of the parallel process that prepares material before bringing it into the short term process of getting decisions made.This is a system of influence, of key players, committees and lobbyists. It would be wrong to deny that development organization are not subject to similar games of power, yet the big difference is that a development organization should only get one source of funding (from business). While a democratic system gets multiple sources of funding with often contradicting goals.

I cannot make this posting too long, therefore to summarize:
  • Run a tight development process: only bring in ideas that are mature enough to succeed withing your "short term" production cycles.
  • Encourage independent new long term ideas but outside your production process: Accept that there is some political process in how your team brings in new ideas into their production effort; Make sure that new concepts are mature enough that they can be brought into production with good certainty, within the normal "short term" development cycles.
  • Encourage "constraining" long term team goals within your production process: It is OK to debate as a team on longer terms plans that simplify things and restrict usage of new technology. I say this because technical entropy, the fact that things "grow more complicated with time" will kill you if you do stay united as a team to fight it.
  • Make sure that your team has enough slack time: And make sure also that they have the culture to use it wisely, for the benefit of the project (and not for each developer's ego).
  • Have only one external driver per development team: The same for "normal" short term development process as for the long term "creative process". This will keep the "political" debate healthy. And that is again a reason why agile processes have only one business representative per team.
  • Drive software with deliverable features, business with revenue streams: It is this separation and its "double layering" within the overall development process (e.g. agile development teams versus one product management team) that keeps the political "game" sane! Technology does not generate money, only features do! (Well, within a bigger model it can, but that is why you might consider having a CTO outside the development team process).
 This brings us back to US politics: they are like a very, very, bad development team. They apply none of the "good behaviors" above. Not a very good role model for their fellow citizens!

All original content copyright James Litsios, 2013.

Sunday, October 13, 2013

How to become an expert developer

I am very decent touch typist: I never look at the keyboard. If I cannot remember where a  key is, I "search" for it "blind", until I find it. The reason I do this, is that I know that if I look, I will learn to "look", not to "type". This is an approach that I use over and over, and can be summarized as follows:
To learn, you need to experience learning within the same context as usage.
The thing is, the mind will learn with everything you give it. If you want to learn efficiently, give the mind only what it will normally get. You need to give yourself an environment that is not "too" different from the environment that you will experience when you actually need to use the learning.

Now all of us may sometime "learn wrong". That is learn within the wrong context, and then be obliged to bring this context in when we need our learning. For example, the top two lines of your keyboard are typically "badly learned.  We make the effort to learn to touch type the alphabet, possibly the numbers, but not all the symbols, and definitely not the function keys (f1...f12). When you have learned "wrong", you must relearn. And to do that you must force yourself to stop using the "bad environment" (using your eyes) until you have learned enough to use the "good environment" (only use fingers), which is the one that matches you productive environment. I had such a hard time "unlearning" looking at my top keys, that I purchased a blank keyboard many years ago.  Having used this blank keyboard at work for a few years, I learned that "looking down" was of no use. Now I use a normal keyboard, and yet I still do not look down. First because my first reaction is not too, but also because the feeling of touch typing without help is a great feeling, it is like running or riding a bicycle, and as your fingers move freely you can concentrate on more important things. Also, looking down is breaking that great feeling.

This brings me to programming. For example, you might ask, how do I teach myself to use a new library? My first answer is:
Never use code completion!
If I want to learn a new library, first I read the documentation. Or read code that uses the library. Then I usually print out the header files and "study them" a bit. I keep these printouts nearby (yes, sorry for the trees). Then when I start programming. Usually, I will type in example code, which I will change to try out different features of the library. I will always try to guess the right name and usage patterns to use the library. If I feel that I am really not sure enough, I will "physically" make the effort to look at my header printout, or example code. What is important is that this use of external help be somewhat inefficient.  Also important, is that you make the effort to remember the names and signature of things (I'll come back to that below).

Now I have noticed a few things. By printing the header files, or main code usages, I am giving my mind a physical reference of what needs to be remembered. And even though I may not use this reference, I will learn better because I know that this reference is there and that its physicality gives it an immutable nature. Then the fact that the printout has a structure, that the definitions follow a certain order, or are on one page or the other is important for the memory process. This brings me back to code completion. Code completion provides little reference for you memory to work with. Yes, you will remember things on the long run, but not as quickly and not as well, and that will make you a less good developer.

Now we can get to the core of the learning process:
Make it personal!
If you want to be a good developer, you need to accept that you approach your craft like a chess grandmaster. It is a life long learning process, where you are building layers and layers of learning. And each of these layers of learning are working off each other, in a manner that generates feelings. It is these feelings that cruise you along. Your thinking process works one level above making big decisions on where you are going. Therefore, when you learn a library, you really want to invest the time to learn its signatures, not only the names that provide the different features. This is because these signature will support the feeling that you have when you use the libraries. Without those feelings you will be wasting your time thinking about little details, which is really not what you want to do.

In reality, you do not learn each signature of each type, function, object, module, etc. as a separate learning. Each piece of code is written with a certain style, following styles of others, and following patterns of design and coding. It would be way too hard to learn everything "in separation", so the mind will try to learn things "as a grand scheme". BUT... this only works if you have started along this path in the first place. You need to make the effort from the start to remember the little patterns and names that you meet along your programming lifetime. Of course it is never too late to start later along this way of learning.  But if you never approach it this way, then you will never become a master developer.

I am not sure it is much more complicated. The only thing I would add, is to take your time.  The thing is, if a piece of code/library is written in a manner that is too far away from your current learning. You will not be able to remember it. Possibly even not be able to understand it. That is when you need to be strategic. You need to have a few long term goals of "areas" to develop. Then spend a lot of your personal time programming "towards" those areas. Finally, when you are close enough, you will find that you can read code, and remember code, that you previously could not.