Revert "Support empty Time inputs"

See #901 for further information as to why this was reverted. Will merge in on green build.