[nemo-qml-plugin-contacts] Fix de-duplication error

Numbers that have no possibility of being duplicates are wrongly
omitted from the result set.
Merge pull request #111 from matthewvogt/fix-omission

[nemo-qml-plugin-contacts] Fix de-duplication error