Paper: Large Content And Behavior Models To Understand, Simulate, And Optimize Content And Behavior (ICLR-2024)

Reviewer comment (Experiments):
Is BFT training using CBC data? Is the test benchmark also using CBC data? If both are using CBC data, is there a train/test split within CBC so a different split is used for training and testing?

Author response:
Regarding your point (1) on Behavior Finetuning:

We utilize the train subset of the CBC data for Behavior Finetuning. Here is the breakdown for different tasks using various subsets of the CBC data:

1. Behavior Simulation:

    a. YouTube Videos: 2027 videos were selected without temporal overlap with the train set, encompassing diverse brands, industries, dates, and receiver behavior.

    b. Twitter Tweets (new dataset):
        
    * Brand Separated: 1557 tweets from diverse brands, industries, dates, and receiver behavior.

    * Time Separated: 782 tweets without temporal overlap with the train set from diverse brands, industries, dates, and receiver behavior.

2. Content Simulation:

    a. YouTube Videos: 2027 videos chosen without temporal overlap with the train set.

    b. Twitter Tweets (new dataset):

    * Brand Separated: 1557 tweets sampled from diverse brands, industries, dates, and receiver behavior.
        
    * Time Separated: 782 tweets without temporal overlap with the train set from diverse brands, industries, dates, and receiver behavior.

3. Behavior Domain Adaptation Tasks:

    a. Emails Corpus: 3000 email-segment pairs without temporal overlap between train and test sets, using emails until July 2022 for training and testing on emails from August 2022 onwards.

    b. LVU Benchmark [1]: Zero-shot testing on the entire dataset without prior training.

4. Content Understanding Tasks:

    a. Advertisement Understanding Corpus (Hussain et al, 2017): Zero-shot testing on the entire dataset without training.

    b. Persuasion Strategy Corpus (Bhattacharyya et al, 2023): Zero-shot testing on the entire dataset without training.

5. Behavior Understanding Tasks:
    
    a. Replay Behavior Understanding: No training, 200 videos randomly sampled from the behavior simulation test set.

    b. Comment Sentiment Understanding: No training, 200 videos randomly sampled from the behavior simulation test set.

Additionally, subsequent to paper submission, we've compiled a large-scale content behavior dataset comprising images and corresponding human behavior extracted from 168 million tweets across 10135 enterprise Twitter accounts from 2007 to 2023. This dataset includes account names, tweet text, associated media verbalizations (including image captions, keywords, colors, and tones), tweet timestamps, and like counts. We plan to release both the YouTube and Twitter corpora. Our experiments with behavior simulation, domain adaptation, and content simulation on this corpus yield results consistent with those reported in the paper:

**Behavior Simulation. Give content, channel, time, predict behavior (High, Low). Reported values are 2-way classification accuracies**

(Brand Separated means that the train and test set don't have any overlap in terms of brands, Time Separated means that the test set starts after the last tweet in the train set)

| **Model** | **Behavior Training** | **Behavior Datasets Included In Training** | **Brand Separated** | **Time Separated** |
|---|---|---|---|---|
| GPT-3.5 (few-shot) | In-context-learning |  | 58.84 | 64.19 |
| LCBM | Finetuning | Twitter (77.3k tweets) | 74.3 | **97.69** |
| LCBM | Finetuning | Twitter (77.3k tweets) + Youtube (40k) data | **76.87** | 92.19 |




**Content Simulation Experiment. Given behavior, channel, time, tweet media caption as prompt, predict content (tweet text)**

| Model | Behavior Training | Behavior Datasets Included In Training | Test-split | Bleu-1 | Bleu-2 | Bleu-3 | Bleu-4 | Rouge-l |
|---|---|---|---|---|---|---|---|---|
| GPT-3.5 (few-shot) | In-context-learning |  | Brand Separated | 53.95 | 42.36 | 31.84 | 24.28 | 15.24 |
|  | In-context-learning |  | Time Separated | 57.69 | 45.11 | 33.67 | 25.52 | 15.27 |
| LCBM | Finetuning | Twitter (77.3k tweets) | Brand Separated | 62.29 | 46.59 | 33.98 | 25.64 | 14.44 |
|  | Finetuning | Twitter (77.3k tweets) | Time Separated | 70 | 54.4 | 41.43 | 32.48 | 17.38 |
| LCBM | Finetuning | Twitter (77.3k tweets) + Youtube (40k) data | Brand Separated | **64.28** | **48.1** | **35.17** | **26.63** | **14.83** |
|  | Finetuning | Twitter (77.3k tweets) + Youtube (40k) data | Time Separated | **70.23** | **54.54** | **41.52** | **32.54** | **17.45** |

Reviewer comment (Novelty):
The authors claimed that the paper studies the problem of predicting and optimizing communication for desired receiver behaviors and intents, i.e., the effectiveness level of communication, but what the authors actually did in this paper is simply exploring the Youtube and email datasets. As far as I understand, the key to improve the performance of behavior/content prediction and understanding is to improve the general reasoning ability of LLMs, rather than finetuning LLMs on specific datasets. From this perspective, the contribution of this paper is very limited.

Author response:
Thank you for your review and interesting questions 🙂. 

Regarding (1), although we acknowledge the potential of enhancing the general reasoning ability of Language Models (LLMs) to address the issue of communication effectiveness, our study presents compelling evidence that directly exposing LLMs to behavior tokens alongside content tokens can be a more effective method to instill behavioral understanding. Our approach and underlying rationale for fine-tuning diverge significantly from established methodologies. Specifically:

1. While GPT-3.5 and GPT-4 have demonstrated superior general comprehension across various benchmarks [1,2,3], our research illustrates that training an LLM on behavior tokens enables a 13B model to surpass much larger models—despite their stronger general understanding—across all tasks. This performance extends to both tasks specifically fine-tuned and those that were not originally targeted for fine-tuning.

2. In-context learning, although impressively successful in other domains, proves ineffective in the behavior domain. Large in-context-finetuned language models (GPT-3.5, GPT-4, etc.) exhibit performance comparable to a random baseline, indicating their lack of exposure to behavior tokens. 

3. Incorporating behavior tokens in content and behavior simulations not only enhances behavior and content simulation (as anticipated) but also unexpectedly improves behavior domain-adaptation (Table-6), behavior-understanding (Table 4, Figs. 7, 8), and content understanding (Table-5). Notably, LCBM exhibits enhancements in these tasks despite not being explicitly trained for them. This aligns with how LLMs, trained for next word prediction, showcase proficiency in various tasks such as question answering, summarization, and natural language inference. We demonstrate these capabilities across three behavior domain datasets (YouTube, Emails, and LVU) and five tasks spanning two content domain datasets.

4. Presently, large language models focus on training with content tokens (text, vision, videos, etc.) while disregarding behavior tokens (Biderman et al 2023, Penedo et al 2023). Consequently, LLMs lack behavior simulation (Table 1, 2), content simulation (generation) (Table 3), and behavior understanding (Table 4) capabilities.

Further, we show our results on four datasets beyond Youtube:
1. Long Video Understanding (LVU) Benchmark
2. Emails dataset
3. Two Advertisement Understanding Datasets
4. Additionally, subsequent to paper submission, we've compiled a large-scale content behavior dataset comprising images and corresponding human behavior extracted from 168 million tweets across 10135 enterprise Twitter accounts from 2007 to 2023. This dataset includes account names, tweet text, associated media verbalizations (including image captions, keywords, colors, and tones), tweet timestamps, and like counts. We plan to release both the YouTube and Twitter corpora. Our experiments with behavior simulation, domain adaptation, and content simulation on this corpus yield results consistent with those reported in the paper:


**Behavior Simulation. Give content, channel, time, predict behavior (High, Low). Reported values are 2-way classification accuracies**
(Brand Separated means that the train and test set don't have any overlap in terms of brands, Time Separated means that the test set starts after the last tweet in the train set)

| **Model** | **Behavior Training** | **Behavior Datasets Included In Training** | **Brand Separated** | **Time Separated** |
|---|---|---|---|---|
| GPT-3.5 (few-shot) | In-context-learning |  | 58.84 | 64.19 |
| LCBM | Finetuning | Twitter (77.3k tweets) | 74.3 | **97.69** |
| LCBM | Finetuning | Twitter (77.3k tweets) + Youtube (40k) data | **76.87** | 92.19 |




**Content Simulation Experiment. Given behavior, channel, time, tweet media caption as prompt, predict content (tweet text)**

| Model | Behavior Training | Behavior Datasets Included In Training | Test-split | Bleu-1 | Bleu-2 | Bleu-3 | Bleu-4 | Rouge-l |
|---|---|---|---|---|---|---|---|---|
| GPT-3.5 (few-shot) | In-context-learning |  | Brand Separated | 53.95 | 42.36 | 31.84 | 24.28 | 15.24 |
|  | In-context-learning |  | Time Separated | 57.69 | 45.11 | 33.67 | 25.52 | 15.27 |
| LCBM | Finetuning | Twitter (77.3k tweets) | Brand Separated | 62.29 | 46.59 | 33.98 | 25.64 | 14.44 |
|  | Finetuning | Twitter (77.3k tweets) | Time Separated | 70 | 54.4 | 41.43 | 32.48 | 17.38 |
| LCBM | Finetuning | Twitter (77.3k tweets) + Youtube (40k) data | Brand Separated | **64.28** | **48.1** | **35.17** | **26.63** | **14.83** |
|  | Finetuning | Twitter (77.3k tweets) + Youtube (40k) data | Time Separated | **70.23** | **54.54** | **41.52** | **32.54** | **17.45** |

Reviewer comment (Presentation):
In terms of presentation, I think the authors make things too complicated by discussing the effectiveness and behavior issues, which make the readers hard to quickly get the main idea and contribution of this paper.

Author response:
Regarding (2) (presentation), we regret any confusion caused. Could you kindly specify the specific figures or sections that posed challenges for clarity in the upcoming version?

Reviewer comment (Novelty):
The behavior simulation task sounds like a data mining problem, e.g. predicting whether a user will watch a video. LLMs are not quite suitable for such tasks because they are too slow and they cannot make use of massive user-item interaction data.

Author response:
Re (3), while LLMs might be slow for certain specific tasks, we believe they present a more general framework, which has empirically shown much better content understanding and generation capabilities than any other systems historically. This motivates us to use LLMs (as compared to other competitive technologies) for behavior understanding (the next step and the third level of communication). 

- To enable LLMs to learn behavior and content, following the T5 methodology, we convert behavior learning task to a “text-to-text” problem. We show that a model trained on ~2 trillion tokens (Llama), can be taught behavior with a few billion additional tokens. This shows that an adequately trained LLM might be taught a behavior as a new modality relatively easily. 

- A similar evidence is also provided by LLMs trying to learn the vision modality, such as Llava, VideoLlama, Minigpt-4, and others [4,5,6], where they tune an LLM for vision modality with a few billion vision tokens (similar scale as us).

Reviewer comment (Novelty):
One of the main concerns is the technique novelty. From my understanding, the proposed model is mostly based on InstructBLIP[1] and only the GMHRA is newly introduced to aggregate the time dimension in video frames.

Author response:
Thank you for your insightful review! 🙂

About (1) technique novelty, we agree with your review. Other than showing how to do behavior finetuning (using content and behavior instruction tokens), we do not intend to go for any other technique novelty in this paper because we find existing techniques are good enough to train our model well. Rather, we focus on the following contributions:

1. Incorporating behavior tokens in content and behavior simulations not only enhances behavior and content simulation (as anticipated) but also unexpectedly improves behavior domain-adaptation (Table-6), behavior-understanding (Table 4, Figs. 7, 8), and content understanding (Table-5). Notably, LCBM exhibits enhancements in these tasks despite not being explicitly trained for them. This aligns with how LLMs, trained for next word prediction, showcase proficiency in various tasks such as question answering, summarization, and natural language inference. We demonstrate these capabilities across three behavior domain datasets (YouTube, Emails, and LVU) and five tasks spanning two content domain datasets.

2. Presently, large language models focus on training with content tokens (text, vision, videos, etc.) while disregarding behavior tokens (Biderman et al 2023, Penedo et al 2023). Consequently, LLMs lack behavior simulation (Table 1, 2), content simulation (generation) (Table 3), and behavior understanding (Table 4) capabilities.

3. In-context learning, although successful in other domains, proves ineffective in the behavior domain. Large in-context-finetuned language models (GPT-3.5, GPT-4, etc.) exhibit performance comparable to a random baseline, indicating their lack of exposure to behavior tokens.

4. Next, to spur research on the topic of content and behavior models, we release behavior and content tokens collected over 40,000 public domain YouTube videos containing a variety of behavior tokens. 

5. Additionally, subsequent to paper submission, we've compiled a large-scale content behavior dataset comprising images and corresponding human behavior extracted from 168 million tweets across 10135 enterprise Twitter accounts from 2007 to 2023. This dataset includes account names, tweet text, associated media verbalizations (including image captions, keywords, colors, and tones), tweet timestamps, and like counts. We plan to release both the YouTube and Twitter corpora. Our experiments with behavior simulation, domain adaptation, and content simulation on this corpus yield results consistent with those reported in the paper:

**Behavior Simulation. Give content, channel, time, predict behavior (High, Low). Reported values are 2-way classification accuracies**

(Brand Separated means that the train and test set don't have any overlap in terms of brands, Time Separated means that the test set starts after the last tweet in the train set)

| **Model** | **Behavior Training** | **Behavior Datasets Included In Training** | **Brand Separated** | **Time Separated** |
|---|---|---|---|---|
| GPT-3.5 (few-shot) | In-context-learning |  | 58.84 | 64.19 |
| LCBM | Finetuning | Twitter (77.3k tweets) | 74.3 | **97.69** |
| LCBM | Finetuning | Twitter (77.3k tweets) + Youtube (40k) data | **76.87** | 92.19 |




**Content Simulation Experiment. Given behavior, channel, time, tweet media caption as prompt, predict content (tweet text)**

| Model | Behavior Training | Behavior Datasets Included In Training | Test-split | Bleu-1 | Bleu-2 | Bleu-3 | Bleu-4 | Rouge-l |
|---|---|---|---|---|---|---|---|---|
| GPT-3.5 (few-shot) | In-context-learning |  | Brand Separated | 53.95 | 42.36 | 31.84 | 24.28 | 15.24 |
|  | In-context-learning |  | Time Separated | 57.69 | 45.11 | 33.67 | 25.52 | 15.27 |
| LCBM | Finetuning | Twitter (77.3k tweets) | Brand Separated | 62.29 | 46.59 | 33.98 | 25.64 | 14.44 |
|  | Finetuning | Twitter (77.3k tweets) | Time Separated | 70 | 54.4 | 41.43 | 32.48 | 17.38 |
| LCBM | Finetuning | Twitter (77.3k tweets) + Youtube (40k) data | Brand Separated | **64.28** | **48.1** | **35.17** | **26.63** | **14.83** |
|  | Finetuning | Twitter (77.3k tweets) + Youtube (40k) data | Time Separated | **70.23** | **54.54** | **41.52** | **32.54** | **17.45** |

Reviewer comment (Reproducibility):
Please provide more detailed information on how the visual encoder handles video input.

Author response:
The question about detailed information on how the visual encoder handles video input. We apologize for not having a clear enough coverage of this topic. We will make sure to cover this in the next version. The details are: 
The pipeline uses the pre-trained EVA-ViT-G with the Global Multi-Head Relation Aggregator (GMHRA) which is a temporal modeling module.  For token projection, we emply a pretrained QFormer with extra linear projection to incorporate additional query tokens to include video context for training the LLM. While training, the projected video encodings are used to represent video context in the training corpus.