Add more restrictive oflag checks to sys_open

Also make sure and error out with EISDIR if the named file is a
directory and oflag includes O_WRONLY or O_RDWR. This has only been
added to the vfs.