In many cases the weights do not fit in constant memory, so let's leave
them in global memory.