This changes the behavior here non-trivally (We now destruct/cleanup the buffer without the lock). @jmlvanre look okay to do this?