Working Notes: a commonplace notebook for recording & exploring ideas.
Home. Site Map. Subscribe. More at expLog.
Developing with asyncio https://docs.python.org/3/library/asyncio-dev.html
=PYTHONASYNCIODEBUG= = 1
=-X dev=
=asyncio.run(debug=True)=
=loop.set_debug()=
Enable =ResourceWarning=
logging.getLogger("asyncio").setLevel(...)
AOSA Book: Web crawler with Asyncio Coroutines [2020-05-09 Sat]
http://www.aosabook.org/en/500L/a-web-crawler-with-asyncio-coroutines.html
TODO Understand how non-blocking sockets actually work
TODO Dig into the different event loops
Understand how the different event loops sockets actually work at the
kernel / C level, and visualize it using ebpf.
TODO Check out the selectors module in python
TODO Read through the new asyncio modules, read about =Event=
Notes
"Stack Ripping": Loss of context because of the broken event loop.
Coroutine takes ~3k of memory, compared to ~50k per thread
"Note a nice feature of async programming with callbacks: we need no mutex around changes to shared data"
=PyEval_EvalFrameEx= takes a frame object and evaluates bytecode in the context of that frame.
=inspect= module also looks pretty good for looking at where I am
Python stack frames are allocated in the heap!
How generators work:
Coroutines
This would be much harder to read if I hadn't extensively used coroutines in Advent of Code.
Python's =yield from=
Handles exceptions correctly as well!
To callers, advancing a generator is the same as advancing an iterator
"The real framework addresses zero-copy I/O, fair scheduling, exception handling, etc"
"Real" AsyncIO
Generator cancel:
Coroutines are only interruptible at yield points
The crawler deals with redirects by stepping through them to avoid redownloading the same pages again and again.
A =Task= is a =Future= that /resolves itself/
=yield from= --> =await=
Asyncio & Music
TODO Check out UVLoop Understand how nodejs's event loop works.
TODO Write about /how/ to digest a technical document/book
TODO Write about /why/ to digest a technical document/book
— Kunal