Why is Yahoo bot hitting a page when my robots.txt file is configured to disallow all bots?
Channel: Roel Van de Paar
Date: 2021-10-28

Webmasters: Why is Yahoo bot hitting a page when my robots.txt file is configured to disallow all bots?

Helpful?  Please support me on Patreon: https://www.patreon.com/roelvandepaar

With thanks & praise to God, and with thanks to the many people who have made this project possible! | Content (except music & images) licensed under CC BY-SA https://meta.stackexchange.com/help/licensing | Music: https://www.bensound.com/licensing | Images: https://stocksnap.io/license & others | With thanks to user Zistoloen (webmasters.stackexchange.com/users/20122), user Stephen Ostermiller (webmasters.stackexchange.com/users/14543), user joshuahedlund (webmasters.stackexchange.com/users/10576), user Eric Belair (webmasters.stackexchange.com/users/332), and the Stack Exchange Network (webmasters.stackexchange.com/questions/53036). Trademarks are property of their respective owners. Disclaimer: All information is provided "AS IS" without warranty of any kind. You are responsible for your own actions. Please contact me if anything is amiss at Roel D.OT VandePaar A.T gmail.com