Make sure what we were passed is actually a directory