BUG: to_timedelta not properly converting some units (GH6855)

closes #6855