Set the correct headers for browser and crawler requests