Refactor and fix bug in deduplication performance improvement

1) Moved deduplication performance improvement into removeDuplicateRows, rather than having intersect deal with de-duplication itself

2) Fixed bug in hash based deduplication where the tempValue was not being considered when deduplicating