set: Print an error when setting `umask` to a bad value

Repurpose the ENV_INVALID return value for env_set(), which wasn't
currently used by anything. When a bad value is passed for the 'umask'
key, return ENV_INVALID to signal this and print a good error message
from the `set` builtin.

This makes `set umask foo` properly produce an error.