Okey, I just tried this:
if ($(event.target).closest('li, a, input, button').length == 0) {

For me it's working as a charm, but I could have missed something :-)