fix failure of open to read variadic mode argument for O_TMPFILE
fix invalid access by openat to possibly-missing variadic mode argument

the mode argument is only required to be present when the O_CREAT or
O_TMPFILE flag is used.