added exclude option to crawler