fix regression in dlsym: rejection of special RTLD_* handles as invalid