Buffer do not pass through cuda_make_buf() on the normal path so we need to tag them there also.