  In this work, we test the performance of two bidirectional transformer-based
language models, BERT and SpanBERT, on predicting directionality in causal
pairs in the textual content. Our preliminary results show that predicting
direction for inter-sentence and implicit causal relations is more challenging.
And, SpanBERT performs better than BERT on causal samples with longer span
length. We also introduce CREST which is a framework for unifying a collection
of scattered datasets of causal relations.
