Handle numeric time format

Also move tests around to be more consistent.
Addresses #87