Only use the click event workaround when it's actually a click

Change-Id: I1fba41744488a5a96159e339e524abe9e823b3d6