remove repetitions and scruffy stuff