Fix destroy method by storing references to bound listeners
Fixes #236