Fix bug that modify device memory in host code