Add extra safety around the static dictionary
Remove pyPdf library usage and corresponding functions
Add Timeout exception handling for requests.get
Add strem arg to request_html func`
Generalize the way href is reaad instead of hardcoding the path

Used lxml.html methods such as clean and iterlinks to get the href
element and added support to remove urls which have the same reponse
object and added comments