Fix conda build for binaries

It still needs to be tested in Python 3.
Add conda clean to the changelog