neuralgorithm/Research — comment on commit 22d0832e552d12d4ea06b340ebe4da328833a2d2
File:  line 0
By neuralgorithm:

Hi, jH@CKtheRipper,

Thanks for the message. Yeah, I have noticed that the code does not work
correctly, but I have left it because of just my laziness. I also have
noticed that if we run ic1 first and then run ic7, the code works! I have
no idea on what's going on, but maybe due to some cache mechanisms.

I agree with your thought that this method would work with a small number
of blocks.

Best Regards,
Neuralgorithm

jH@CKtheRipper wrote:

> Hi,
>
> I'm working with CUDA too so I was very interested in your global synchronization technique.
> I have tested it and it froze my machine. Indeed, when you think about it, this just can't work when you have many blocks. The CUDA hardware dispatches blocks on the available SM, however, a block can't be preempted. Thus, the representative thread will eternally wait for the barrier to be equal to the number of blocks, because some blocks have not been run yet.
>
> Your approach should work with a small number of blocks though.
> Hope to talk about this and other GPU-related things with you soon.
>
> Regards
>
> ---
> Reply to this email directly or view it on GitHub:
> https://github.com/neuralgorithm/Research/commit/22d0832e552d12d4ea06b340ebe4da328833a2d2#commitcomment-781497