Symmetrical, Multi-Pass, Trace-Driven Compiler Pipeline in Python in two days

In 2011, after fourteen years developing derivative market making software, I took some time for myself, and wrote a prototypal compiler for a reactive trading language. I bring this up because a reactive trading language is a DSL, and I wanted to write something about DSLs in the context of my recent writing about High-Performance Software Engineering. Therefore I asked Gemini to rebuild something similar as my 2011 effort, with four new twists: 

  1. the introduction of profunctors (I had only monads and comonads back then), 
  2. the use of combinators to not only to parse the DSL's syntax, but also it's traced execution semantics, 
  3. a JIT approach based on working with traces,
  4. This is Python. It was F# in 2011.

What you have below is Gemini's description of two days of work. And to highlight the still very toy language stage of this language. The goal is show how advanced computer science can be done very quickly in 2026. From a language design perspective, what we have is a Symmetrical, Multi-Pass, Trace-Driven Compiler Pipeline. From a developer perspective, we have what Gemini describes as "complete separation of concerns": 

  • The team writing the interpreter (Phase 1) doesn't need to know anything about the type system.
  • The team writing the type system (Phase 2) doesn't need to write complex environment-simulation logic—they just read the facts straight out of the trace. 
  • Adding a new static analysis pass just means writing a new tree-walker.

Ten Major Architectural Pivots

Below is the evolutionary path I took. To be honest, I did not immediately remember all the tricks I used for my trading DSL. For example, it was only when this effort resulted in what I call sandwich like trees where expressions and type node alternate, that I remembered that I had such a structure in my original 2011 effort. Importantly, none of the architectural steps below were suggested by the LLM. If you do not know what you want, the LLM does not give it to you!

The ten major pivots (as narrated by Gemini with minor edits by me):

  1. From Monolithic Visitors to Modular Combinators: We abandoned the traditional "God-class" AST visitor (which centralizes all logic and is hard to extend) in favor of small, self-contained combinators. Each combinator knows exactly how to handle one specific grammatical or semantic shape.
  2. Desugaring over AST Bloat: Instead of forcing the core execution engine to understand complex structures like multi-argument functions, we introduced syntactic embedding. Complex structures are rewritten into fundamental primitives (nested single-argument closures) before execution, keeping the core engine incredibly lean.
  3. The Synthetic Trace Tree (History over State): Instead of an interpreter that just spits out a final value and discards the steps, we built an engine that records its exact execution history. It builds an immutable TraceTree, permanently linking every computed value to the exact lexical environment that produced it.
  4. "Semantics in Semantics" (Parsing the Trace): We realized that type inference shouldn't involve guessing at a static AST. Instead, we built a second set of combinators that treat the execution TraceTree as their input alphabet. The type checker effectively "parses" the execution history to retrospectively prove the code's soundness.
  5. The Constraint Manifold (Isolating Dimensions): Instead of throwing all semantic rules (types, constraints, effects) into one giant list, we created a stratified ConstraintManifold. This allows the engine to track distinct sets of rules perfectly isolated from one another, preventing complexity explosions.
  6. Upward-Sifting Resolution (Scope-Bound Solving): We avoided the performance death-trap of feeding all constraints into a massive global solver. By exploiting the AST's topology, constraints bubble up the tree and are solved locally the exact moment their variables come into scope.
  7. Profunctorial Fusion (Parsing = Engine Generation): We erased the boundary between building the syntax tree and building the execution logic. As the parser recognizes the text, it simultaneously instantiates the exact object-graph required to execute it. The parser doesn't just return data; it returns a fully wired engine. (Note: steps 1 to 6 are all monad/comonad based).
  8. Object-Oriented Functors over Deep Closures: We hit the limits of Python's functional closures (memory bloat, impossible debugging) and pivoted to explicit callable objects (Functors) utilizing __slots__. This gave us the purity of functional programming with the memory profile and debuggability of systems-level OOP.
  9. Context-Driven Memoization (O(1) Execution): By hashing both the AST nodes and the immutable execution environments, we created a deterministic footprint for every operation. We injected a dynamic caching layer that intercepts duplicate computations, turning recursive execution into an instant O(1) lookup.
  10. Lazy Projections (Infinite Extensibility): Because the TraceTree perfectly suspends the execution state, we realized we don't have to evaluate all semantics at once. We can pass the exact same trace to a Type Solver, a Linearity Solver, or a Cost Solver independently. The architecture is infinitely extensible without touching the core runtime.
Warning: Gemini is overly generous. Some of the "can" above, should be "could".

All original content copyright James Litsios, 2026.