Revert "Move .Xauthority away too"

This reverts commit 78e392794790b56c5fb8110ba472710f149a78cd.