Learning to Tell (A Life) Time