Only override the target architecture on -m32 and friends if it is
actually different. Fixes a surprising link error with nodejs on rpi,
where armv6-netbsd-eabihf turned into armv5e-netbsd-eabihf, which
doesn't lacks the necessary VFP support.