Fix issue #4476 by adding fill values for all possible datetime dtypes