WELD-1746 Don't destroy ended conversations outside of synchronized block

This fix should be applied to 2.2 branch as well.