application/xhtml+xml should be type html not xml
otherwise we don't end up spidering the links