(picture By Klem - This vector image was created with Inkscape by Klem, and then manually edited by Mnmazur, Public Domain, https://commons.wikimedia.org/w/index.php?curid=3213322 )
Dynamic typing of duck typing
Two months ago I drafted a Python tracer library with Gemini, then I rewrote it from
scratch (here). It is however "clunky". Yet I do not worry, as Python is a computer scientist dream language, and in Python one can quickly implement something meta and probably slow, but then use a cache or two to make it fast enough. That is what I did last week: I sat in front of my screen, stared at it for 30m thinking through my options, and then asked Gemini to draft the concept. And while the result is missing some logic, it is however doing what I want: it dynamically wraps "types" around duck-typed operation. I could try to explain what it does... but I do not need to. Instead, I share what I told Gemini. The key desires are the following:
- let's make a cached wrapper factory, that when given an object, first is looks in the cache, and returns the instantiated wrapper with the object if found, otherwise it uses the above greedy cover, where given a list of mixin classes, it dynamically creates a "best covering wrapper" that is remembered in the cache, and used to wrap the object.
- Ok, that works. Now we want to extend the concept to include some __init__ specify logic. First we need to make sure the __init__ is special and not part of the covering logic. Then we will assume that all mixins will get the target_obj as the first positional argument, and other mixin arguments will come through kwargs arguments.
- Nice. Now a last little bit: A list of additional "context specific" mixins can be given to create_wrapper, these will be will be added as parents in addition to the coverage mixins. Now the caching needs to be extended, as a same target object might be wrapped with different additional context mixins.
(the full set of prompts is given below)
The idea is the following: Python naturally favors a duck-typing style of programming, yet duck typing is hard to scale without boilerplate code. Therefore one approach is to "find" what types support the duck typing. The types (classes in Python), can then be used to scale the program's design. In effect, we are dynamically typing the duck typing. Caching is then used to ensure performance.
Open vs closed sets in software design
Traditional designs ask the developer to choose a closed set of types, and then enables an open set of data to be created. Trace oriented designs work the other way around: we design a closed set of data traces, from these we derive an open set of types. This is the first level of Yin and Yang of software design. Something you might when you want to optimize high-performance code, such as an array heavy ML system. This technique is explained from a slightly different angle
here.
The Gemini prompt strategy
I did really think about the design for a half an hour before starting on the prompt session below. As mentioned before (
the-art-of-prompt and
master-class-techniques-for-llm), you really cannot just randomly chat with an LLM to get world class output, you need to think through your strategy, and the key part is that you are really trying to achieve a "one shot" chat with no corrections needed. The reality is that the billions of facts known by the LLM will only collapse into to the desired solutions if you pull it carefully, almost like a glass blower will shape molten class. This comparison is especially true: you only have a limited number prompts before the quality of the LLM fails you!
The chat start with pedantic info... this is important! You really must ensure that the base concepts are well aligned with end goal of the task!
Note my very chatty style: don't forget: the LLM is trained on human text, you want to resonate with that humanity to pull out quality output.
- What are the rules of double inherentence in python? supposing I have A and B both deriving from X, and now I create Z deriving from A and B, is is a special case for Python, or does it just take it as a normal class hierarchy and simply takes Z of A when there is an ambiguity (supposing that A comes before B in the parent list).
- Ok, clear. What about the __init__, and multiple inherentence, is it correct that I need to explicity call the parent __init__ , What about X, who calls it?
- Ok, but then what arugments? Am I pretty much foced to use kvargs so that the parents can pick and choose their arguments?
- Finally, how do I inspect an object to gets its class hierarchy?
- Great. What would be the name of programming style where I rely heavily on the MRO? In effect I take all the MRO, and build a dynamic logic that depends on their interdependent structures?
- so maybe what I want to do is "dynamic mixin" as their will be only dynamic assemblies of interdependent classes.
- Right, and to be double shure, I can provide any base class list I want to that type constructor, python will work out the MRO that ensures the proper class parent hierarchy order, right?
- Does itertool have a "generic" tree iterator? I want to get all classes of a module, ideally I want the recursive call to have already been written, and I feed it functions that depend on inspect. Can I do that?
- And how about: given an object, and a list of classes, use a greedy search to find the subset of classes that "covers" the methods of the given object. The greedy search always tries to first cover with classes with a larger number of methods. Note however that it is not allow to cover using a class with a method that is not present in the given object.
- Ok, so we follow up on the wrapper: let's make a cached wrapper factory, that when given an object, first is looks in the cache, and returns the instantiated wrapper with the object if found, otherwise it uses the above greedy cover, where given a list of mixin classes, it dynamically creates a "best covering wrapper" that is remembered in the cache, and used to wrap the object.
- Ok, that works. Now we want to extend the concept to include some __init__ specify logic. First we need to make sure the __init__ is special and not part of the covering logic. Then we will assume that all mixins will get the target_obj as the first positional argument, and other mixin arguments will come through kwargs arguments.
- Nice. Now a last little bit: A list of additional "context specific" mixins can be given to create_wrapper, these will be will be added as parents in addition to the coverage mixins. Now the caching needs to be extended, as a same target object might be wrapped with different additional context mixins.
- Right, last steps is to remove the underscore filter, as the mixins are specifically designed for dunder logic.
- Right, but LengthMixin does not need to test for hasattr __len__ as it will only be brought in dynamically.
- Et voila. If you remember, we wrote a first wrapper version about two month ago. The issue with that first approach without this dynamic mixin logic is tha the wrapper logic becomes clunky with special cases. This double mixin approach allows the wrapper logic to be added very generically, although at the cost of a more complex and slower execution.
- Let's add an MIT license with my name James Litsios and year 2025
- You can add a comment under the license where you share the credit of the work!
What next?
I will rewrite my tracing library for the fourth time using the above!
All original content copyright James Litsios, 2025.
