If destroying a buffer, destroy it's lcl.