Working Notes: a commonplace notebook for recording & exploring ideas.
Home. Site Map. Subscribe. More at expLog.
A mixed week as I adjust to working remotely with a large time zone difference.
I've been reading -- and trying to also compile and run at the same time -- Understanding Software Dynamics. While I could get sample code running for playing with the cost of CPU utilization and arithmetic operators, I haven't been able to just copy paste and run code memory or disk utilization. The book is both fascinating and a little bit overwhelming given the sheer amount of complexity that can affect actual runtimes of a program -- and doesn't actually include GPUs.
My plan so far had been to carefully work through and play with each component of the book, potentially writing some reusable benchmarking scripts that I could compile and use everywhere. At this point, I'm planning to just make it through the book while taking notes, and then following up with some code to make sure I have some understanding -- and then extending the same principles to GPUs.
Reading through the chapters on memory and disk access, there are so many potential sources of noise and simplify artifacts of the shape of memory that I'm surprised the author still manages to construct experiments on it and reason about it; trying to isolate memory patterns and figuring out if the results match the actual hardware bandwidth is fascinating. It's basically about constructing and validating experiments.
The trick used to measure disk performance is to read/write a large block of memory: for reading, check when disk block offsets in the memory get updated (indicating that block was read in) or for writing constantly updating the block being written so the time of writing can be timestamped. This only works because of order of magnitude time differences in updating memory vs disk, but is very clever. I'm also not quite sure how I'd actually implement it, and will continue reading the book / plaiyng to find out more.
The other long standing bugbear I've been thinking of is how to have easily movable conda environments; PyTorch does rpath patches to have a relative path to the conda environment's library. Conda pack works well, but it needs the library to be mutated, and once unpacked libraries can't be moved again because the update is somewhat destructive.
I've been wondering if I can update conda-pack to do non-destructive updates -- allowing the same environment to be moved repeatedly without thought, and then also having it fix itself every time it's activated through the activate scripts.
Looking through the issues on conda-pack I also stumbled across constructor
which seems to be a slightly fancier conda env to declaratively create and install
conda environments. This is still not as flexible as I would like because some
packages like NVidia's Apex must be
installed from source and cannot be installed from PyPI -- installing from PyPI
actually drops in some other random package.
I've been meaning to write about a complex system with rich visualizations for several months now, and observable framework seems like the perfect tool. After spending some time exploring, I've generally come to appreciate the default design choices and generally smooth experience.
Funnily enough, I've been generating plotly graphs from relatively expensive data sources and caching them recently -- so observable's dataloader approach that statically generates the data once and then loads it resonates particularly well. I need to check if it's smart enough to be able to only partially load data from parquet files, because that is still one of the shortcomings of this approach -- if you have too much data to load up front, things will become extremely slow and blow up on the client anyways.
— Kunal