updated crawler for ignoring .ico files on url prefix