Make we sure we handle doubles/longs correctly, for class cast
exception.