Working Notes: a commonplace notebook for recording & exploring ideas.
Home. Site Map. Subscribe. More at expLog.
Partially in California this week, and happened to be in town for the LLaMa hackathon. I'm very interested in seeing what people make, and writing this from the event.
There were several interesting projects, and it was fun to see so many people use something I'd helped out with. The project that fascinated me most was one that basically undid LLaMa's fine tuning by adjusting weights: I have to wonder what else we can achieve with their mechanism, and what the actual process was (waiting to see the repository).
At the same time, there wasn't anything that did something completely out of left field: a lot of practical application of LLMs today beyond an omniscient and occasionally loopy chat bot seem to be a bit farther from the maturity level I would hope for. That doesn't mean we can't use them, we'll need to be significantly more creative in how and where we apply them.
The finetuned LLM I'd love to build is a CLI helper: everything is text anyways and it has amazing amounts of context available; I'd like it to quickly complete my commands or let me simply ask for things and to translate them into explicit machine instructions.
Started playing with implementing TF-IDF to identify outlier logs in HPC jobs. The idea seems so obvious I'm sure someone has to have implemented it, but perhaps I'm just missing something obvious. Some ideas seem obvious enough I can't imagine it hasn't been done yet.
Asking Claude for recommendations on papers and approaches generally lead to hallucination, but I did find that one of the papers was real (just from a different year and with different authors): RNN Attention Mechanisms for System Log Anomaly Detection that also has a healthy number of downstream references, including literature surveys.
Go is being surprisingly pleasant to work with, though I still don't quite know how to write idiomatic go. The performance is a boon after getting used to Python.
I've also been making progress on my index card application, leaning into FZF and bash scripting to fill in gaps that I'll actually implement with Go later. It's interesting just how much flexibility and speed is available using Bash scripting: and at some point, the complexity just goes up to make the script unmaintainable. Bash's inherent global state matches my intuitions around debuggability (that I've tweeted about in the past), but I still don't quite know how to demonstrate this feeling better.
Another idea that's started wondering through my head is the implementation of a modern shell scripting language: something POSIXy but not quite; perhaps an extension of SkyLark that is just as good at handling stderr/stdout/nesting commands but has significantly cleaner language semantics and more modern lexical scoping and language constructs? I'm surprised this doesn't seem to exist yet.
Through bitter experience I also learned about the cost of constantly appending to a bytestring in Python when I profiled my code and realized it was only allocating memory (Py-Spy is a gift!). Which lead me to a rabbit hole to avoid copying bytes on slicing, and for easier manipulation: I ended up testing out ByteArrays and memoryviews which are useful tools to have: they helped me turn a 30 minute long script to something that ran in seconds.
— Kunal