Fix another corner case with skipping before timeline has started