Let open test pass on OS X even without O_APPEND

OSXFUSE and/or darwin vnode limitations mean O_APPEND is never visible
to the FUSE server.
Let truncateWithOpen test pass whether or not it has a Handle

Used to trigger this on OS X:

    fuse_test.go:1104: got Valid = "SetattrSize+SetattrHandle"; want "SetattrSize"
Test fsyncDir now passes on OS X

Expected values are hardcoded in an ugly way. To be cleaned up later.
syscallx.Getxattr size probing now works on OS X

It used to pass a pointer to a phony memory location, and not a NULL,
and just size==0 is not enough to trigger OS X getxattr(2) size
probing. Move syscall wrapper from []byte to *byte to have control
over sending NULL.
Test code cleanup: use nil not 0-len slice in getxattrSize test
Test code cleanup: use nil not 0-len slice in listxattrSize test
Xattr operations now return ENOTSUP if a Node does not implement them

Used to return ENOSYS, but that makes osxfuse disable xattr support
for the whole filesystem from there on.

Note, this wouldn't trigger with just

    go test ./fs -test.v -test.run=TestFuse -fuserun=getxattr

but only on full test runs.
Test code cleanup: simplify type switch

Thanks to James Tucker (raggi) for pointing this out.