DENIED URLS DATA ADDED AS "ROBOTS DENIED"
THIS FUNCTIONALITY ADDDED