Clear the surface observer when blurring

Otherwise it won't detect the cursor being restored to the exact
same place.

Change-Id: Ida027ba7bbb2eda9119ba4e00a2296c591f81627