Working Notes: a commonplace notebook for recording & exploring ideas.
Home. Site Map. Subscribe. More at expLog.
This week's letter starts with a face lift, combining some of the pieces I've been writing about: a color palette inspired by Dieter Rams and a simple site builder written in Hy. Almost everything is typeset in Inter; though at some point I'll swap in Berkeley Mono for monospace fonts and maybe the headings.
(tl;dr; for the rest of this week: I've been playing with Hy.)
There are a remarkable amount of excellent notes and explorations on Transformers out there: this week I stumbled upon Neel Nanda's blog from a Zulip discussion from RC. With sufficient Googling I'm beginning to realize that Transformers may be this decade's Monad -- and I probably shouldn't write yet another post on how to implement my own. Though I almost certainly will do one for a Transformer in Hy.
Neel's blog took me to
Arena
which hand-holds you through building a transformer. I'm taking a
slightly different approach from the recommended set up and working in
Hy, but maintaining everything else. Instead of copy-pasting set up
code directly, I'm manually transforming it to Hy and doing my own set
up, but so far that approach seems to be working. Learning about
einops and jaxtyping has also been interesting -- working with
Hy I can see myself writing a couple of macros to make all of this
significantly more ergonomic.
I finally ended up making a working application with Textual, using Hy as my actual language: a minimalistic version of TensorBoard that renders scalars as sparklines. It was surprisingly satisfying, particularly after I could enable sparklines.
Message Queues are probably my favorite mechanism to make sure the main thread in a program is unblocked; and once I wrapped my head around Textual's APIs (particularly that every widget has access to the message queue -- and the thread safety of certain functions) things became much easier. Enough to be able to set up auto-refresh and manual refresh, with Toasts popping up as data quietly updates. Textual also seems to be handling mounting new elements much more simply.
Hy keeps growing on me: while looking at old Hacker News posts on Hy, I also came across a post on using hy that resonates a bit too much.
I slowly find myself thinking about common lisp's approach to
compilation: and how Python is also reaching there:
Triton compiles code live and
creates Cuda kernels; torch.compile is the same idea. With a good
REPL I can easily seem myself having a similar workflow with Hy.
There are some tools that I miss at the moment: JedHy is a bit out of date, the highlighting is slightly broken and autocomplete is accordingly somewhat minimalistic. On the other hand, the language itself simplifies code enough to make these trivial annoyances.
My favorite recent snippet is for generating HTML:
(defn tag [name #* children #** attrs]
(let [child-tags (.join "" children)
attr-str (.join " " (gfor #(key val) (attrs.items)
f"{key}=\"{val}\""))]
f"<{name} {attr-str}>{child-tags}</{name}>"))
(tag "html"
(tag "body"
(tag "h1" :style "color: red"
"Hello World!")))
It's delightfully minimal, handles the HTML as strings but still
gives me a remarkable amount of flexibility. In Python, I would
probably end up using with, but with the right notation I can just
write it out directly in Hy.
I'll rewrite it to be slightly friendlier with a macro, potentially
allowing for #< instead of tag.
My biggest set of expenses for Black Friday have been books, of course. I've picked up 3 books so far (that I will almost definitely read):
— Kunal