I think I have discovered a bug in the nvcc compiler. When you write the following for loop ...

for (uint32_t i = 0; i < BENCHMARK_ITERATIONS; i++)
{
    mul_loc(c_loc, a_loc, b_loc)
}

... and compile the code to ptx, you will see that some instructions in the asm statements I have written in mul_loc() are not in the ptx. The compiler is skipping them. Therefore, we get incorrect results.

But if you write multiple mul_loc() instructions in a row to mimic the for loop behaviour, all asm instructions handcoded in the mul_loc() function are kept in the ptx, and the code execution yields correct results.

The generated ptx (cleaned a bit) is in "nvcc bug/" and the missing instructions are marked. Have to discuss with Nvidia for this issue.