Keep a list of recent observations