Implemented BFS to fetch different MIME types of WebPages, following Robots Exclusion Protocols, parsing HTML/XML URL in Python.