Thinking about parallel and distributed computation
A classical way to look at parallel / distributed systems is in terms of: Work-Efficiency Parallelism Locality This is one of those inconsistent triads (or trilemma ), where only two among three properties can be chosen. For example, a system can have both work-efficiency and parallelism, or parallelism and locality but cannot be fully work-efficient, parallel and local. This is not the only inconsistent triads to think about when designing scalable systems. Here are more: Velocity Resources Work Momentum Quality Traffic And of course the CAP theorem's: Consistency Availability Partition tolerance And the important: Fast Good Cheap All of these are ultimately important, yet some of these are more practically useful. Especially when you realise that you can weaken the inconsistent of these triads by introducing additional complementary properties. To illustrate this with examples: Two people must work together vs an engineer and a sales person must work together Two tim...