Yes, you can un-match with someone if you don't like them after a conversation or if you have second thoughts. You can go to the toolkit option and simply "un-match" the other person for any reason.