Rewrites robots.txt concern into a mixin for WebProcessorSession.