# roberta-temporal-predictor

A RoBERTa-base model that is fine-tuned on the The New York Times Annotated Corpus to predict temporal precedence of two events. This is used as the temporality prediction'' component in our ROCK framework for reasoning about commonsense causality. See our paper for more details.

# Usage

You can directly use this model for filling-mask tasks, as shown in the example widget. However, for better temporal inference, it is recommended to symmetrize the outputs as $P\left({E}_{1}\prec {E}_{2}\right)=\frac{1}{2}\left(f\left({E}_{1},{E}_{2}\right)+f\left({E}_{2},{E}_{1}\right)\right)P\left(E_1 \prec E_2\right) = \frac\left\{1\right\}\left\{2\right\} \left(f\left(E_1,E_2\right) + f\left(E_2,E_1\right)\right)$ where f(E_1,E_2) denotes the predicted probability for E_1 to occur preceding E_2. For simplicity, we implement the following TempPredictor class that incorporate this symmetrization automatically. Below is an example usage for the TempPredictor class:

from transformers import (RobertaForMaskedLM, RobertaTokenizer)
from src.temp_predictor import TempPredictor

TORCH_DEV = "cuda:0" # change as needed

tp_roberta_ft = src.TempPredictor(
tokenizer=RobertaTokenizer.from_pretrained("CogComp/roberta-temporal-predictor"),
device=TORCH_DEV
)

E1 = "The man turned on the faucet."
E2 = "Water flows out."
t12 = tp_roberta_ft(E1, E2, top_k=5)
print(f"P('{E1}' before '{E2}'): {t12}")


# BibTeX entry and citation info

@misc{zhang2022causal,
title={Causal Inference Principles for Reasoning about Commonsense Causality},
author={Jiayao Zhang and Hongming Zhang and Dan Roth and Weijie J. Su},
year={2022},
eprint={2202.00436},
archivePrefix={arXiv},
primaryClass={cs.CL}
}

Mask token: <mask>