added flushing buffer after deduplication