chomp stuff, check for duplicate sigs, optomise some sql