Saturday, April 23, 2011

OO versus functional programming versus agile

Have you ever asked yourself the question of why database tables do not have two primary keys? Or maybe you have started a spreadsheet with a choice of columns and rows to only change them later?

One of the little understood properties of programming is that at the lowest level things tend to be single dimensional. So for example, the balanced tree has just one key, and so on. There is theoretical reason for this, but the general fact is that it is much easier to design a structure or algorithm to do one thing than to do two or more.

Object orientation starts with the fundamental idea that objects can do many things. And this naturally leads to the expectation that they can do them equality well. We know from the previous paragraph that this is not true, and yet many do not know this or forget it. You would think that this mismatch between expectation and reality is not really a big problem, yet sadly it is. OO programmers are constantly trying to fit multiple dimensions of abstractions into single objects without understanding that they will never be able do it well without giving more weight to some dimensions over others. Not prioritizing things, not having a clear idea of hierarchy of concepts, leads to a random choice of hierarchy at the deeper implementation level, and leads to redundancies that do not support each other well. The end result is that OO software tend often both to grow and age badly.

Functional programming approaches things differently. A function has only one entry point: from the top. Primitive data structures are all simple (one dimensional) and must be composed to build anything real. The consequence is that with functional programming you need to decide "on construction" how you will interleave the different dimensions of your problem. Unlike OO, you are immediately forced to care about how you will implement things. This is both a good thing and a bad thing. The good thing is that in the end, there is no escape that the deep down implementation of your structures needs a hierarchy, so you might as well take responsibility and choose how you will build your solution. By doing so you can keep things consistent, and consistent does not always mean the same, it may mean that your choice of order of composition is different for different parts of your program but that the unrolling of one part matches the "rolling-up" of another part of the your program. The bad thing about needing to choose early how you split and hierarchize things is that you need to have a lot of experience to do it right (think monads for example). This is why although OO programs tend to not grow well, they are still cheaper to produce as you need senior guys to work functional style.

But now I would like to bring in something different and yet very relevant. Agile development is a lot about prioritizing your requirements and implementing them in iterations. That means that the first level of your design, what might be seen as the first draft of primary keys of your data base, are directly tied to the most important business concept. If you think of it, that is not a bad choice! So to observe that agile development helps structure your design when you do not know better.

Sunday, April 17, 2011

Should you use F# in your projects?

A few years back I read that a guy killed a child with his Ferrari: he missed the turn after a strong acceleration (as he left a Ferrari gathering!). And then more recently I read the same type of story, again with a Ferrari and an uncontrolled turn at an intersection.

Well, F# is like a Ferrari (maybe that is what the F stands for ;-) ). It is incredibly more productive than other .net languages; see it as a language with an incredible acceleration. But the caveat is that programming in a functional language is much more of a balancing act, and therefore beware the slippery turns and “intellectual crashes”. This is especially a management and team issue, a language like F# allows you to blend in the same code concepts that can be dealt by your “standard” developer and concepts that take years to master. (It can be even worse than C++ template meta-programming.)

If you were going to set up a taxi company would you buy a fleet of Ferraris if they were available at the same price? Well that is a little bit the same type of question you need to ask yourself when you choose a language like F# or Scala for your team. And it is a tricky question and I would argue that Microsoft is wise not to over market F#.You choose it because you know what it will bring you and you are "old enough" to understand the risks you are taking!

Friday, April 15, 2011

The big picture and the touchy feely side of technology

I have a big directory in which I have been accumulating articles and downloads from the web. I think I have mentioned that I do not believe you can be good at what you do and stick to the nine to five focus of your job; You need to add a lot of private time, to write code if you are a developer, and to read up on research and development in the different fields that support you business activities.

I'll admit that I rarely revisit any of these documents. Yet by giving them an independent place that I own, I am making them part of my life, and it allows me to grow with them. I believe that the brain can be seen as a big optimization process that grows like a plant. In that metaphor my download stash can be see as a little part of the soil I grow on.

I can't share the documents with you, mostly for copyright reasons. So my second best is to tell you what I am interested in. I include here a raw dump of the top level of my directory structure. Arguably it needs clean up work, but it is good fun already in this form.

Update: do ask me to blog about any of these subjects. I'd love to do it, I just need an audience!

Here is the list:

3dLibraries
AbstractInterpretation
AbstractStateMachines
ACE
ActiveDB
ADD
ADT
Agents
AHEAD
AI
AlgebraicSystems
AlgorithmicTrading
AlgoTrade
AnalogCircuits
Antler
ANTLR
APIGen
apowell
approximateSorting
approximation
architecture
ArtOfAssembly
ASN1
aspect
AssemblyLanguage
Assert
auction
AutomatedReasoning
AutomatedSolver
AutomaticDerivative
AutomaticDifferentiation
berglas
BernsteinBase
BinaryMagic
blackice
CacheOblivious
CAS
CASE
CellProcessor
CGAL
ChangeManagement
CLI
cluster
cminusminus
CMP
Coach
codetransformation
Combinatorial optimization
Communication
compiler
Component
constraint
Constraint propagation
ConstraintSolver
ContinuedFraction
ContinuousQueries
Contract
Control
ConvexHull
CPP
CPPPreProcessor
CTO
CUDA
Databases
Dataflow
DataGrid
DataModel
DecisionTree
Declarative
Delanauye
DeltaVolatility
DerivativePricing
Design
diophantine
dll
dotnet
DSL
duality
ElectronicMarkets
EmbeddedLanguages
Emulators
Erlang
EventDriven
excel
Exchange
ExpertSystems
failsafe
FaultTolerant
Financial
FiniteAutomata
FiniteStateMachine
FIX
FlashCrash
fog
Formal
FPGA
FPGL
FpML
fsa
fsharp
Functional
functionalCompiler
Fuzy
GADT
GameTheory
GarbageCollection
Generative
GeneticLanguage
google
GPU
Graph
graphics
Greeks
Grid
GTTSE
hamiltonian
Hardware
Haskell
Hedging
HighLevelLanguages
Hilbert
html
indexing
InMemory
InMemoryDB
Interpolation
IntersectionTypes
Interval search
Interviewing
java
jira
KnowledgeAndDataEngineering
KnowledgeModeling
LambdaCalculus
lean
Library
lisp
LLVM
Locking
LogicProgramming
malign
Management
MarketMakers
MarketMaking
Markets
MemoryAllocation
Mesh
metacpp
MetaInterpretation
MetaProgramming
Military
modeling
Movie
mp3
MPI
NonBlocking
NonLinearProgramming
NSDE
NumericalMethods
ocaml
ODBC
OfficeAutomation
Oleg
OnLineAPI
Optical
OS
Parallelization
ParityGame
Parser
particleFilter
particleSimulation
Patents
PDE
physics
pictures
Polyhedra
PolynomialChaos
PolynomialContinuation
Porting
pqueue
prescheme
Process
ProductManagement
Products
ProgramAnalysis
ProgrammingLanguages
ProgramTransfomation
projectmanagement
Prolog
psycho
pure
python
QueryCompilation
QueryOptmization
quotientApproximation
Randomized
RandomizedAlgorithm
randomNumbers
Reactive
RealTime
recursive
Reengineering
Reflective
regexp
RegRel
RemoteCommunication
RendezVous
Rewriting
RuleCompilation
rules
Sampling
SAT
scala
Scheduler
scheme
scilab
scrum
SDE
SDI
SearchEngines
Security
sforce
shedskin-0.0.4
shmem
SkipList
SkipListJava
SoftEng
software
SoftwarePatterns
SoftwareProcess
SoundLib
SourceControl
Spanish
sparsegrid
SpatialDatabases
speed
SplayTree
srcml
StateMachines
StoryGeneration
strategic
string
SVD
SwissCompanies
SymbolicSimplification
TCAD
TemporalData
TheoremProver
TheoreticalComp
Threads
tomography
trace
Trading
TransactionalMemory
TransactionalModel
TransactionCost
translation
Transport
Travel
treap
tree
TreeMatching
TreeRewriting
Trees
Tropical
typeinference
TypeSystems
UML
UnscentedKalmanFilter
Valuation
ValueAtRisk
Values61
VaR
Velara
Volatility
Voronoi
vpn
Wavelet
WordTemplate
wpf
www
Zurich

Tuesday, April 05, 2011

Google recognizes the "F#" keyword!

Finally! Hopefully not because they fear being broken up for monopolistic behavior, Google search finally recognizes the F# key work and is not just search for "F".