for lmf specific script remove duplicates and annotate output file.