Fix the interact user interaction when inverted