new crawler that uses OS dependent line separator when storing text file #422
fix for line ending for both working & non working urls