Premier-TACO: Pretraining Multitask Representation via Temporal
  Action-Driven Contrastive Loss
Authors: name: Ruijie Zheng; name: Yongyuan Liang; name: Xiyao Wang; name: Shuang Ma; name: Hal Daumé III; name: Huazhe Xu; name: John Langford; name: Praveen Palanisamy; name: Kalyan Shankar Basu; name: Furong Huang
Published: 2024-02-09 05:04:40

Abstract: We present Premier-TACO, a multitask feature representation learning approach
designed to improve few-shot policy learning efficiency in sequential
decision-making tasks. Premier-TACO leverages a subset of multitask offline
datasets for pretraining a general feature representation, which captures
critical environmental dynamics and is fine-tuned using minimal expert
demonstrations. It advances the temporal action contrastive learning (TACO)
objective, known for state-of-the-art results in visual control tasks, by
incorporating a novel negative example sampling strategy. This strategy is
crucial in significantly boosting TACO's computational efficiency, making
large-scale multitask offline pretraining feasible. Our extensive empirical
evaluation in a diverse set of continuous control benchmarks including Deepmind
Control Suite, MetaWorld, and LIBERO demonstrate Premier-TACO's effectiveness
in pretraining visual representations, significantly enhancing few-shot
imitation learning of novel tasks. Our code, pretraining data, as well as
pretrained model checkpoints will be released at
https://github.com/PremierTACO/premier-taco.