Code - Crawler has been rewritten and works fine. Unicode needs to be sorted out
Code - Unicode has been fixed. Crawler needs to be fixed so it only crawls in the specified domain