Only use X-Real-IP when coming from localhost

Closes #520