Prevent X changes over SSH

SSH connections are usually to headless machines.