Working Notes: a commonplace notebook for recording & exploring ideas.
Home. Site Map. Subscribe. More at expLog.
I spent most of this week hacking on termdex and learning new things from the process: I hope to combine all of that into a new setup to publish this website (and bring over my previous slipboxes) within the next few weeks.
I've started using it to collect notes, tasks, bookmarks, and anything else that catches my fancy: and to write this week's letter, I have it open in a split window to see things I explored (trying out remote-pdb, python-manhole, etc.).
In time I'm excited to throw an LLM at this, and also to have it automatically figure out when I wrote the same note / explored the same resource last time. There's also a lot of functionality to build out to make it easier to create notes with values prefilled: one of the pieces of functionality I really liked in FocalBoard -- that was almost perfectly implemented but still had some issues.
Some of the new tricks I learned this week include:
WITH RECURSIVE
split_path(path, remainder, moved) AS (
SELECT * FROM (SELECT "" as path, path AS remainder, 0 FROM markdown_files WHERE basename LIKE '%.md' LIMIT 2)
UNION
SELECT
path || substr(remainder, 1, instr(substr(remainder, 2), '/')) as path,
substr(remainder, instr(substr(remainder, 2), '/') + 1) as remainder,
moved + 1 as moved
FROM
split_path
WHERE
instr(substr(remainder, 2), '/') != 0
)
SELECT SUBSTR("............", 1, 2*moved) || remainder FROM split_path LIMIT 20
;
Tmux has a popup-window command that can be very convenient to make a surprisingly fancy fzf setup.
FZF has a reload command that lets you reload the contents without restarting FZF; but it needs setting FZF_DEFAULT_COMMAND to enable this -- and it quickly becomes incredibly finicky. I find it much better to become( ... original command ...) instead.
I even ended up implementing a Pomodoro implementation to track my time; I can probably add something to visualize the pomodoros over time when I start the command.
Having an easy query interface over flat files is astonishingly valuable in building features quickly; I just wish there were more tools to construct queries live.
I plan to explore sqlite views to define custom views of tables that are local to the folder the extension is used in. This makes sense as a way to layer abstractions: termdex the application doesn't know what front matter is specific to a given markdown folder, so I can define it in the folder itself and use it easily.
YAML parsing with C is not as hard as I'd expected; it emits tokens / events and that lends itself particularly well to deal with the expected inputs.
Hacking on TermDex is somewhat addictive; I find myself as addicted to this as Factorio or similar games; and it's a little bit more satisfying. (Trigger warning: blasphemy -- It's ever so slightly useful compared to constantly growing a virtual factory).
The other interesting project this week was trying to listen for cuda context initialization using LD_PRELOAD and subscribing to CUPTI callbacks. I have a tiny implementation working, but need to see if I can get good cross programming language stack traces.
While playing with PTYs to get unbuffered output (mainly for improving torchrun I tried implementing a quick program to tag stdout and stderr separately by running a subprocess with 2 pty pipes attached). This worked surprisingly well, though I think this interferes with the terminal size estimations -- at least when done through basic python.
Having something that explicitly displays stderr / stdout separately is pretty valuable, so I plan to make this a tiny zig utility in the near future.
— Kunal