I decided my life needed a little bit more math, and so to decided to add a few differential equations in some F# code. Math in code means applied math and lots of arrays, but arrays are a pain functional style. So heres my appoach.
I have written many numerial applications, mostly in C++, yet also some in Fortran. I mention Fortran because it has the notion of equivalence, where you can map arrays on to each other. Equivalence is like C unions for array and there to reminds us is that arrays do not need to be managed as dynamic memory.
A functional style of programming is a lot about preserving your invariants with immutability, yet immutable vectors are not a solution (yet). Still, numerical code has its own invariants, and therefore some immutability is possible, and should be exploited to write robust code. In my case, I have vectors and matrices; Their sizes are invariant, and so is the structure of the matrices, they have lots of zeros, they are sparse. The approach is then to specialize a state monad to maintain size, and structure in an immutable manner, while allocating mutable vectors only within the core of the numerical engine.
Long ago I wrote an matrix assembly algorithm. You can see it as a sparse matrix structure inference code. It was really pretty simple: the final matrix is an assembly of many small sparse matrices, finding the final structure can be seen as solving a set of matrix structure equations, the algorithm defines an order of traversal for all this matrix points and then proceeds to "crawl" across all matrices at once, working out all the local structures. In a functional style of programming, I can use the same newVar like concept you would find in a constraint or inference solver, extend it with an equivalent newEq, to build a monadic environment to support my numerical code. The result is that "application code" just uses variable and defines equations, these are then assembled and only later does the code allocate just a few very large arrays to store the real data associated to the problem
This approach is "easy" because all invariants are built first and then used. Much modern applied math depends on hieararchical and adaptive grids. Hierarchy is probably easy to bring in to a functional style, adaptive grids maybe less obvious.
Tuesday, October 09, 2012
Wednesday, October 03, 2012
Greenfield projects, or greenfield areas of non-greenfield project, is really about hitting the limits of what you know, it is about meeting unknowns. When that happens, and it does happen decently often, you should not apply a different process, but continue with the one you have, one that already has those “off road tires” that allow you to move in greenfield pastures. This concept of “one process” fits them all is really the hardest thing to get across to teams that have not really managed to deepen their agility. Yet processes like Scrum works equally well on greenfield projects than on non-greenfield projects.
Best explain thing in action, here is how it works. We know that software development is a combination of different tasks. I’ll simplify it to the following:
- It is hard to notice what you do not know.
- It is hard to learn what you do not know.
- Having extra time to “observe” and learn will only help you so much
Do not expect to recognize what you do not understandWhat you do not understand is what is missing, what is unknown, and unknowns are mostly invisible and therefore hard to notice. That is because they are unknown! (Duh… )
Velocity tracking and sprint reviews are your team’s unknown detector. The more newness and unknowns you have the more precious they are in alerting the whole team of unknown dangers. The next question is “what should you do when the knowledge alarm bell rings”? As I forewarned before, the last thing to do is to “go back to school”. It is not the learning that is wrong, it is what you learn, how quickly you learn, how productively you learn that is most often the problem. Or to put it another way, with every development task you can be expected to learn something, the more you learn the better, as long as you get your task done! In a non-greenfield project, there are still lots of unknowns, but these get resolved because your team has a process to conquer these unknowns. In a greenfield project, unknowns accumulate, and when the team fails a sprint, it is in fact the team’s unknown resolution process that is failing. Trying to fix a process by reading up on things rarely does the job. First because, reading is a personal thing, not a group process, then because learning from reading is simply not going to happen fast enough. To keep your team going you need your learning to be part of the team process, not external to it. This bring us to rule #3:Track velocity, review your sprints truthfully
Drive learning within your processHere is the idea, you are ending your sprint, things have not happened the way you wanted them to happen, you have delivered half what you hoped you could. Obviously something is wrong! The easy action is to do nothing about it, just push the failed tasks into the next sprint and move on. Yet that is absolutely the wrong thing to do! Each failure needs to bring a learning. The next level of easy action is to talk about the failure in a sprint retrospective and add future action of improvement list. Beep… Failed again. That list will not be consumed fast enough, and no learning will happen. Now we are ready for rule #4:
Realign your backlog to avoid your unknowns but still touch themI’ll start by saying: “yes, I realize this sound a little bit crazy”.
Here is the reasoning:
One or more unknowns are keeping your team from achieving its highest productivity, specific aspects of the sprint’s tasks have failed. Your mission, as a team, is to understand what changes can be made to the backlog that will bring your productivity back up AND move your project forwards as much as possible in the right direction. The whole team’s goal is to move around these unknowns that have been blocking you and yet to get as close as possible to them because in many cases that is where the projects added value will be. The added value will be learning but also new deliverables that will make you think differently.
How then to extend your process to tackle these unknowns? The situation is that you have a scrum process, you are normally able to tackle the deeper unknowns in the pre-sprint story grooming and design discussion phases, but now you find yourself starting your sprints neither with good stories, nor good design, nor a clear idea of how you want to develop. Again you are in the greenfield territory. This is when you have team members that make suggestions like: "we should use domain specific languages", or "can we use the GPU for hardware acceleration"? This type of suggestions are signs that your team is “uneasy”, that they are out of their comfort zone and are looking for ways to feel better. As I mentioned above, detecting that subtlety of emotions in a team is not a process, but noting your sprint failure and drop in velocity is. The team needs then to take action, but given their current state of "failure" something different needs to be done. Yet I have already told you what you should do, you should revisit your backlog and rework it. The trick is how to make it systematic! This leads to rule #6:
In case of doubt, the product owner bridges the gaps, not the teamAgain, you do not change your scrum process, neither do you change your product owner and his role. Yet the product owner's tactics are now somewhat reversed. Normally, it is the team that tries to meet the expectations of the product owner. In greenfield territory, the tables are turned, it is the product owner's task to meet the team's capabilities. The process is actually pretty simple: the product owner asks the team "can you deliver this" over and over, with different variants, until he has rebuilt the best of possible backlogs. Again, it is not the team that decides that it needs to learn, yet in their answer to the product owner they may say: "yes, we can deliver that, we will need to do some learning, yet we are pretty sure we can deliver what you are asking". This is scrum, so the double checking process is known to all: if the team's independent estimations (sizing) of a task are similar enough (and small enough!), then all have a good confidence that they can deliver that task.
In non-greenfield, the team is charge of delivering what it masters. In greenfield projects, it is the product owner responsibility to maintain a backlog that the team can master. Often this means that the product owner needs to be creative in formulating his desires to meet up with the teams capabilities. Here a few examples how such a process works, to make it obvious I'll imagine the product owner wants to build a car, but for software development, the process is no different:
- Basic features
- Example: Deliver a motor connected to wheels
- Limits, boundaries
- Example: Work out what is the minimum, maximum, and average weight, power, length, height, etc. of the 50 most popular cars on the market
- Out of order
- Example: Design a test track
- Disconnected, but related
- Example: Deliver the driver's seat design, the interior sizing, and the design of the windows