Managing agile from technology to business

 This how I tend to look at management ownership in agile development (see previous blog entry for more on the alternate views of agile):

Where:

• The CTO owns technology and most often engineering teams.
• The Engineering Manager may be an inwards looking CTO and also own technology and engineering teams, but is often less owning technology and more owning agile processes and engineering teams.
• The Product Owner owns the team's product vision and possibly the customer strategy. She/he is a member of the development team or has a representative that is part of the development team.
• The Technical Product Owner owns the customer view of technology, may be a full fledged product owner, or may be a support member of a larger product owner team.

Individuals become more and more key enablers as your product and technology base become more advanced. A Servant Leader is then a manager (e.g. engineering manager) that helps individuals contribute in independence yet within orchestrated flows. This figure illustrates where the servant leader provides critical guidance (the red corners!):

Here, from the central diagonal outwards:

• The team iterates a technology and product development and review cycle, again, and again.
• In parallel, the team learns and explores productivity and innovation. Product does the same with their customers and market segment.
• Productivity issues are monitored. In a physical model of agile processes 'high momentum' is high productivity. Servant Leaders help monitor for issues such as unsustainability, diverging focus, low or over capacity, and help teams and product avoid or resolve these issues.

 My experience is:

• Product Owner and CTO have too much "skin in the game" to be effective servant leaders. Therefore engineering managers and technical product owners (when they have only partial ownership) have a critical servant leader role to play to help all deliver better.

All original content copyright James Litsios, 2022.

Agile from business to technology

This is how I scale agile towards business and towards technology.:

Where:

• In the center we have an agile process that blends team process and product process.
• To the right we have a business process that ties into the product process.
• To the left we have a technology process (expertise process to be general) that ties into the team process.

To be honest, this is my minimal theoretical business to technology agile process. In practice, a specific agile process is chosen, with process steps and process artefacts. People like to say that agile is about people, not process. Yet people need to communicate, and the common vocabulary for that is given by a process.

Deepening in to the product business view (right side above):

Where:

• The core agile process (blue) delivers features and deliverables that are part of a product and business goals.
• These deliverables are picked-up by customers
• The features and deliverable sustain (or not) the customers competitivity within its market segment
• The product owner (in scrum terminology) is the expert that guides the product with regards to the customer needs.

We can do that also for technology/engineering (left above):

Where:

• The core agile process (blue) includes experts within the team.
• The experts enable/produce the technical/engineering needs of the features and deliverables.
• The expert individuals may be experts in different domains.
• Coordinating technology/engineering frameworks allow experts to work both as individuals and in coordination (within their team).

Many domains of technical expertise are possible. Above are the ones closer to my experience.

Note:

• The complete process above needs to coordinate at least SIX streams.
  A minimal agile process only presents three streams (team vs product vs customer). 
• Each of these (six) streams will naturally be driven at their own different rate!
• Your job, as CTO / engineering lead is to work with individuals, teams and product to pace work within these different streams to achieve an optimal throughput!
•  Think laminar flow! Productive is as fast as you can go before things become turbulent!

I wrote this up as a (much longer) slide deck, and am always happy to share my experiences!

All original content copyright James Litsios, 2022. 

Energize your team: analogy between agile teams and lasers

Electricity, magnetism and time are tightly intertwined. You can try to work separately with them but that will not give you the whole picture. While if you consider the dynamic relation between all three, you can do magical things. For example, you can “bounces” electrical energy back and forth into magnetic energy resulting in alternative current, which magically almost avoiding the need to actually carry “real current” (which is expensive). Or you can “juggle” these energies while moving forward, and thereby creating photons and light.

In project management, tasks, resources and time are also tightly intertwined. You can try to work with them separately but again at risk of being inefficient.  It is a bit of a stretch to compare project management with electromagnetism, yet it is intuitively pleasing. In electromagnetism, work is created by “applying” electrical currents and magnetic fields on each other. In project management, work is creating by “applying” tasks and resources on each other.
I could argue how a waterfall process is comparable to an electrical motor with a rigid physical setup to maintain a well-calibrated cross product between the magnetic field and the flow of currents, but instead I would like to focus on the photonic aspect of light and how it can be compared to an agile process.

Here is the idea: a photon of light is a “self-encapsulation” (wavelet) of oscillating electrical and magnetic fields. (I am not a physicist, so please excuse an abuse of interpretation here). Comparing this to a work process, the photons can be seen as agile work cycles that are “bouncing” between defining the tasks and applying the resources on them. Every cycle of an agile process (e.g. Scrum), can be seen as the photons running through a single frequency cycle.

Agility is the ability to “bounce” your energy between “defining” tasks and “doing” tasks. Stretching the analogy a bit, you might say that the repetition of each agile iteration (or sprint) and with its generation of client deliverables and new future tasks, is a bit like a laser where photons “hit” energized states and generate new photons. The beauty of this analogy is that we can all “feel” how the coherence of light produced by a laser is somewhat similar to an agile team efficiently bouncing past results and knowledge into the definition of new tasks, and then efficiently executing these tasks within an agile iteration.

The laser analogy goes even one step further: lasers produce light that usually goes all in the same direction with the same frequency. If you are very lucky and your product backlog is very much aligned with what the customer needs and your past work always matches up with your new needs, then yes, you too can have a “laser” agile team that goes straight and coherently over your tasks to your goals. But, if your backlog needs to undergo changes (e.g. because there is important new information to interpret), or if you have made mistakes (and we all do!), then this focus on coherence and “straightness” is actually a great risk: a “laser” like team may well end up with the wrong product and possibly with the wrong technology. Again, the issue is that if you try too much to make your new tasks match up with your previous work, you end up going straight, but not necessarily in the right direction. And although the product owner may realize that something is amiss, it is not easy to understand what. One reason is that the team does not understand that it could turn away from an obvious “coherent and straight line” future, and therefore may never indicates to the product owner that the future could be different. Another reason to by wary, is that being a “laser team” feels good: it is great to feel tasks fitting together with marvelous coherence, it is great to have a fantastic history of “little refactoring”. Yet that great feeling is also blocking the team from questioning its actions, because that “would not feel as good”. Without enough self-questioning the team is too much influenced by its previous actions and may end up elsewhere than desired.

So what should a team do about this? The thing is, if you are aligned with your long-term goals, then following the feelings of coherence and alignment is the right thing to do. It is only if you are not aligned with your goals that you are fooling yourself. The product owner is therefore key here: he or she must work hard so that when the team feels that everything is fitting together, the team is also going in the right direction. Said differently, the product owner’s or product manager’s primary job is not to check that the team is aligned, but to ensure that when the team feels “right” it is also going in the right direction. To use the laser image again, the product owner wants to assure that work will be aligned with the overall goals at the creation of each photon. He/she cannot do so by physically controlling each creation and execution of task because that would simply not scale. Instead the product owner strives to provide a “higher level” guidance that ensures this alignment. In scrum, this is why the stories are not tasks. In practice good product owners evangelize their teams so that they “see the goals” and therefore produce tasks that are aligned.

Lasers work because the materials are in “energized states”. Pushing our analogy all the way, (and using scrum), we can say that the product owner energizes the team in such a manner that when the team creates and implements tasks, in a coherent and aligned manner, these tasks are also directed towards the product goals.

Who wants to work with clunky electrical motors when we can all have high-powered laser devices?

Greenfield project development, or "How I learned to love scrum and good product owners"



There is nothing special to greenfield projects yet teams that can tackle greenfield project have something special! It is not the projects that matter as much as the approach to resolve unknowns when they appear in the projects that matters.  My goal here is to give you a few insights on how do this.

Greenfield projects, or greenfield areas of non-greenfield project, is really about hitting the limits of what you know, it is about meeting unknowns. When that happens, and it does happen decently often, you should not apply a different process, but continue with the one you have, one that already has those “off road tires” that allow you to move in greenfield pastures.  This concept of “one process” fits them all is really the hardest thing to get across to teams that have not really managed to deepen their agility. Yet processes like Scrum works equally well on greenfield projects than on non-greenfield projects.
 Best explain thing in action, here is how it works. We know that software development is a combination of different tasks. I’ll simplify it to the following:

• Requirements
• Design
• Code
• Test
• Deliverable

The most efficient way for a team to “develop” is follow the flow of dependencies between these tasks.  With this approach, you start by writing requirements, then you think about the design, then you code, you test and finally you deliver a software package. You can do this the agile way, or the waterfall way.  You can start one phase before ending the previous, but you still follow the dependency order. This works well because although there are cyclic constraints between the different development phases, they are manageable. For example, requirements (stories) are analyzed by the team before the sprint, then during the sprint team members have their independence to pick up work as suites them best. Another example, the team gets together and debates design, when all are aligned each can continue with their code development.

A “non-greenfield” project has "manageable" interdependencies between the different development tasks. By manageable I mean predictable, and by predictable I mean that the team has enough common knowledge that they can organize a process around their unknowns and resolve them together (e.g. as in preparing your next scrum). While, to the contrary, greenfield projects and greenfield areas of a project are, before all else, situations where it is simply harder for the team to efficiently pick up the development tasks. And the number one reason for this breakdown in team productivity is that the tasks no longer are supported by sufficiently clear dependencies, and this results in the team no longer have clear enough objectives when working on the development tasks, and finally too many tasks "just do not go somewhere fast enough"!

You may say: “Ok, so yes sometime we do not know, that is when we can take the time to learn!” But if you do think that, do read on because you are wrong, projects are not the right place to "take time"! Here is the reasoning:

• It is hard to notice what you do not know.
• It is hard to learn what you do not know.
• Having extra time to “observe” and learn will only help you so much

Let’s start with the first one, which leads me to greenfield project rule #1: 

Do not expect to recognize what you do not understand

What you do not understand is what is missing, what is unknown, and unknowns are mostly invisible and therefore hard to notice. That is because they are unknown! (Duh… )

Many of us feel anxious when confronted with unknowns and things we do not understand, yet in a team these feelings are hard to express and hard to use as drivers that can be shared. In fact, in a team negative feelings may distance people, which is the last thing you want. As anxious feelings do not support a team well, we need a surer and easier way to detect the presence of “unknowns” that are affecting development , we want an “unknown detector”.  And we want some automatic way of using this detection tool, so as not “miss” reacting to the team's lack of understanding. And we are in luck, because  nothing is better than an agile iterative process to serve as your unknown detector!  When you fail your development iteration, when your sprint does not deliver, that is when you did not know what you thought you did!  It could be that you quickly understood what you did not know before, for example that a sick team member is the cause of your troubles, but more often, failed deliverable are an indication that you have unknowns and that your team as entered “the greenfield zone”… (add here appropriate music).

Rule #2, for any development, is therefore very important (in scrum language):

Track velocity, review your sprints truthfully 

Velocity tracking and sprint reviews are your team’s unknown detector. The more newness and unknowns you have the more precious they are in alerting the whole team of unknown dangers. The next question is “what should you do when the knowledge alarm bell rings”?  As I forewarned before, the last thing to do is to “go back to school”. It is not the learning that is wrong, it is what you learn, how quickly you learn, how productively you learn that is most often the problem. Or to put it another way, with every development task you can be expected to learn something, the more you learn the better, as long as you get your task done! In a non-greenfield project, there are still lots of unknowns, but these get resolved because your team has a process to conquer these unknowns. In a greenfield project, unknowns accumulate, and when the team fails a sprint, it is in fact the team’s unknown resolution process that is failing. Trying to fix a process by reading up on things rarely does the job. First because, reading is a personal thing, not a group process, then because learning from reading is simply not going to happen fast enough. To keep your team going you need your learning to be part of the team process, not external to it. This bring us to rule #3:

Drive learning within your process 

Here is the idea, you are ending your sprint, things have not happened the way you wanted them to happen, you have delivered half what you hoped you could. Obviously something is wrong! The easy action is to do nothing about it, just push the failed tasks into the next sprint and move on. Yet that is absolutely the wrong thing to do! Each failure needs to bring a learning.  The next level of easy action is to talk about the failure in a sprint retrospective and add future action of improvement list. Beep… Failed again. That list will not be consumed fast enough, and no learning will happen.  Now we are ready for rule #4: 

Realign your backlog to avoid your unknowns but still touch them

I’ll start by saying: “yes, I realize this sound a little bit crazy”.
Here is the reasoning:
One or more unknowns are keeping your team from achieving its highest productivity, specific aspects of the sprint’s tasks have failed.  Your mission, as a team, is to understand what changes can be made to the backlog that will bring your productivity back up AND move your project forwards as much as possible in the right direction. The whole team’s goal is to move around these unknowns that have been blocking you and yet to get as close as possible to them because in many cases that is where the projects added value will be. The added value will be learning but also new deliverables that will make you think differently.

How then to extend your process to tackle these unknowns? The situation is that you have a scrum process, you are normally able to tackle the deeper unknowns in the pre-sprint story grooming and design discussion phases, but now you find yourself starting your sprints neither with good stories, nor good design, nor a clear idea of how you want to develop. Again you are in the greenfield territory. This is when you have team members that make suggestions like: "we should use domain specific languages", or "can we use the GPU for hardware acceleration"? This type of suggestions are signs that your team is “uneasy”, that they are out of their comfort zone and are looking for ways to feel better. As I mentioned above, detecting that subtlety of emotions in a team is not a process, but noting your sprint failure and drop in velocity is. The team needs then to take action, but given their current state of "failure" something different needs to be done. Yet I have already told you what you should do, you should revisit your backlog and rework it. The trick is how to make it systematic! This leads to rule #6:

In case of doubt, the product owner bridges the gaps, not the team

Again, you do not change your scrum process, neither do you change your product owner and his/her role. Yet the product owner's tactics are now somewhat reversed. Normally, it is the team that tries to meet the expectations of the product owner. In greenfield territory, the tables are turned, it is the product owner's task to meet the team's capabilities. The process is actually pretty simple: the product owner asks the team "can you deliver this" over and over, with different variants, until he has rebuilt the best of possible backlogs. Again, it is not the team that decides that it needs to learn, yet in their answer to the product owner they may say: "yes, we can deliver that, we will need to do some learning, yet we are pretty sure we can deliver what you are asking". This is scrum, so the double checking process is known to all: if the team's independent estimations (sizing) of a task are similar enough (and small enough!), then all have a good confidence that they can deliver that task.

In non-greenfield, the team is charge of delivering what it masters. In greenfield projects, it is the product owner responsibility to maintain a backlog that the team can master. Often this means that the product owner needs to be creative in formulating his/her desires to meet up with the teams capabilities. Here a few examples how such a process works, to make it obvious I'll imagine the product owner wants to build a car, but for software development, the process is no different:

• Basic features
• Example: Deliver a motor connected to wheels
• Limits, boundaries
• Example: Work out what is the minimum, maximum, and average weight, power, length, height, etc. of the 50 most popular cars on the market
• Out of order
• Example: Design a test track
• Disconnected, but related
• Example: Deliver the driver's seat design, the interior sizing, and the design of the windows

In each of these example, the product owner needed to pull back from his/her initial goal of getting a full car, each time he/she chooses a new team goal that moves the team in the right direction, delivering something of real value, but while staying in the comfort zone of the team. You may note that the out of order example takes the Test Driven Development approach, but out of order with a good product owner can be much more flexible than TDD, especially when the team has no idea of how to hook in the tests into an nonexistent design. For example, let's say the the team needs to develop a cloud app. They will need to use some new APIs, project's often have given technical constraints, in such a case, the product owner owner can focus on asking for the shell of an app that uses those APIs in order to at least tick off the boxes "code calls API", and "deliver code that works in the cloud". Nevertheless, the product owner should always ask that that the app does something which is line with his/her business. This is an implicit rule: always deliver some form of business value! Also note that the product owner, and the scrum process, continue to strive to deliver deliverables with enough size so that much of the whole team contributes. Fragmentation of knowledge, and new learning is a high risk in new domains and all should work together to keep the knowledge flowing. 

Inquiry driven process to complement your agile teams

Helping agile team with "out of team" work is always a challenge, they naturally tend to rely only on their own resources.  It is really a tricky management problem, and I have had my fair share of failures around teams that do not know that they would be better off with help. Yet my current thinking is that it is possible to set up parallel intra-team processes, but for these to work they need to include a large amount of inquiry activity from the teams; By inquiry I mean that people have questions for which they want answers.  So when team members have questions, and these do not find answers within the team, that is when supporting them with intra-team processes works.

In "out of the book" scrum, the product owner and the scrum of scrums are meant to be the "only" intra-team process. But the bigger the domain, the more both the product owner and the scrum of scrum become bottlenecks. The insight is then that you can add more interfaces into your teams but they must be PULL oriented: The team members must be asking for learning.

There is catch, like with any form "leveraging", any misalignment between what is learned and what you are trying to achieve is going to cause to harm. With scrum, it is then the job of the product owner to play politics and if needed "nudge" the providers of the external processes to provide input that keeps the team aligned.