Working Notes: a commonplace notebook for recording & exploring ideas.
Home. Site Map. Subscribe. More at expLog.

Multi Gpu Programming: Nccl, Nvshmem

steps

implementations

pairwise nccl send recv

shmem avoids synchronizaiton at all costs

nvshmem pointer acess only works within an nvlink domain

Kunal