Ensure late calls return correct results

.. the cache expiration would reset to using +0 as
the faketime ignoring the FAKETIME env var.
Prevent cache expiring when ft_mode is FT_FREEZE

.. as it's a waste of time to recalculate the same
thing.