Large Content And Behavior Models To Understand, Simulate, And Optimize
  Content And Behavior
Authors: name: Ashmit Khandelwal; name: Aditya Agrawal; name: Aanisha Bhattacharyya; name: Yaman K Singla; name: Somesh Singh; name: Uttaran Bhattacharya; name: Ishita Dasgupta; name: Stefano Petrangeli; name: Rajiv Ratn Shah; name: Changyou Chen; name: Balaji Krishnamurthy
Published: 2023-09-01 09:34:49

Abstract: Shannon, in his seminal paper introducing information theory, divided the
communication into three levels: technical, semantic, and effectivenss. While
the technical level is concerned with accurate reconstruction of transmitted
symbols, the semantic and effectiveness levels deal with the inferred meaning
and its effect on the receiver. Thanks to telecommunications, the first level
problem has produced great advances like the internet. Large Language Models
(LLMs) make some progress towards the second goal, but the third level still
remains largely untouched. The third problem deals with predicting and
optimizing communication for desired receiver behavior. LLMs, while showing
wide generalization capabilities across a wide range of tasks, are unable to
solve for this. One reason for the underperformance could be a lack of
"behavior tokens" in LLMs' training corpora. Behavior tokens define receiver
behavior over a communication, such as shares, likes, clicks, purchases,
retweets, etc. While preprocessing data for LLM training, behavior tokens are
often removed from the corpora as noise. Therefore, in this paper, we make some
initial progress towards reintroducing behavior tokens in LLM training. The
trained models, other than showing similar performance to LLMs on content
understanding tasks, show generalization capabilities on behavior simulation,
content simulation, behavior understanding, and behavior domain adaptation.
Using a wide range of tasks on two corpora, we show results on all these
capabilities. We call these models Large Content and Behavior Models (LCBMs).
Further, to spur more research on LCBMs, we release our new Content Behavior
Corpus (CBC), a repository containing communicator, message, and corresponding
receiver behavior.