Working Notes: my commonplace notebook for recording & exploring ideas.
Home. Site Map. Subscribe. More at expLog.
Panopticon is a python library I released several years ago to quickly understand and explore programs – even distributed programs (by logging traces to a central store).
There are some good ideas I was able to explore with this: traces that relied on sys.settrace and sys.setprofile, documented here for capturing behavior and trace through async programs.
More interestingly – and usefully – I also implemented probes – the idea being that often enough I just want to understand how a specific function is triggered, how it behaves, what arguments it’s called with and the values it returns; without actually collecting a full trace that captured everything. I used these regularly at work to explore a new codebase.
These days, I’m exploring the idea of a dye that I could apply to an object and capture traces anytime anything interacts with it – as a way to trace how a configuration may be used in python, and then see how it affects changes downstream. This seems fairly tricky to implement, so I need to see if I can pull it off.
There are also a lot of new promising profiling abilities available: particularly through sys.monitoring since Python 3.12.
The other libraries I’m excited about are usdt and pyusdt to put bpf tracepoints in python, that can then be enabled and traced with bpf tools.
Finally, there’s a lot of promise in just cleaning up the code I already have and implementing it in C instead for speed and reducing the overhead, particularly for probes.
— Kunal