We will always clearmatches when leaving a buffer