neuralgorithm/Research — comment on commit 22d0832e552d12d4ea06b340ebe4da328833a2d2
File:  line 0
By jHackTheRipper:

Hi,

I'm working with CUDA too so I was very interested in your global synchronization technique.
I have tested it and it froze my machine. Indeed, when you think about it, this just can't work when you have many blocks. The CUDA hardware dispatches blocks on the available SM, however, a block can't be preempted. Thus, the representative thread will eternally wait for the barrier to be equal to the number of blocks, because some blocks have not been run yet.

Your approach should work with a small number of blocks though.
Hope to talk about this and other GPU-related things with you soon.

Regards