Implemented duplicate first thing first filtering