Simplify SampleBuffer dtor
Don't open plugins twice when adding to chain

Caused resource leak
Fix some memory leaks in the test suite
Don't abort the test if timing assertion fails

This is necessary to properly profile the tests with valgrind.