Repository: facebookresearch/ParlAI

Issue:
Fail to train any agent with conell_movie dataset
For example, when I train transformer/generator with cornell_movie dataset by command:
`
parlai train_model -t cornell_movie -m transformer/generator -mf ./data/models/transformer_generator/model_cornell_movie_lr1 -gpu 0 -bs 64 -eps 11 -sval True -veps 1 -lr 1
`

The following error occurred:
`
17:54:42 | Current ParlAI commit: 2426d74b93184689be5067bdbf99f1ba96748f7b
17:54:43 | creating task(s): cornell_movie
17:54:43 | loading fbdialog data: /data4/lulc/ben/util/ParlAI/data/CornellMovie/train.txt
17:54:44 | training...
Traceback (most recent call last):  
  File "/data4/lulc/anaconda3/envs/BEN/bin/parlai", line 33, in <module>
    sys.exit(load_entry_point('parlai', 'console_scripts', 'parlai')())
  File "/data4/lulc/ben/util/ParlAI/parlai/__main__.py", line 14, in main    
    superscript_main()
  File "/data4/lulc/ben/util/ParlAI/parlai/core/script.py", line 306, in superscript_main
    return SCRIPT_REGISTRY[cmd].klass._run_from_parser_and_opt(opt, parser)  
  File "/data4/lulc/ben/util/ParlAI/parlai/core/script.py", line 89, in _run_from_parser_and_opt
    return script.run()
  File "/data4/lulc/ben/util/ParlAI/parlai/scripts/train_model.py", line 935, in run    
    return self.train_loop.train()
  File "/data4/lulc/ben/util/ParlAI/parlai/scripts/train_model.py", line 899, in train    
    for _train_log in self.train_steps():
  File "/data4/lulc/ben/util/ParlAI/parlai/scripts/train_model.py", line 802, in train_steps    
    world.parley()
  File "/data4/lulc/ben/util/ParlAI/parlai/core/worlds.py", line 865, in parley
    batch_act = self.batch_act(agent_idx, batch_observations[agent_idx])  
  File "/data4/lulc/ben/util/ParlAI/parlai/core/worlds.py", line 833, in batch_act
    batch_actions = a.batch_act(batch_observation)
  File "/data4/lulc/ben/util/ParlAI/parlai/core/torch_agent.py", line 2131, in batch_act    
    batch = self.batchify(observations)
  File "/data4/lulc/ben/util/ParlAI/parlai/core/torch_generator_agent.py", line 658, in batchify
    batch = super().batchify(obs_batch, sort=sort)  
  File "/data4/lulc/ben/util/ParlAI/parlai/core/torch_agent.py", line 1690, in batchify
    [ex.get('label_truncated_length') for ex in exs]
TypeError: an integer is required (got type NoneType)
`

Actually, any attempts on command line parameters have failed, like '--truncate', '--text-truncate', or '--label-truncate'.
And any agents training with this dataset have failed either.
So, what is the correct way to train cornell_movie dataset?
Thanks~


Discussion:

Resolving patch:
diff --git a/parlai/core/torch_agent.py b/parlai/core/torch_agent.py
--- a/parlai/core/torch_agent.py
+++ b/parlai/core/torch_agent.py
@@ -1687,7 +1687,7 @@ def batchify(self, obs_batch, sort=False):
                 )
             if any('label_truncated_length' in ex for ex in exs):
                 label_truncated_lengths = torch.LongTensor(
-                    [ex.get('label_truncated_length') for ex in exs]
+                    [ex.get('label_truncated_length', 0) for ex in exs]
                 )
             field = 'labels' if labels_avail else 'eval_labels'
 
diff --git a/parlai/tasks/cornell_movie/agents.py b/parlai/tasks/cornell_movie/agents.py
--- a/parlai/tasks/cornell_movie/agents.py
+++ b/parlai/tasks/cornell_movie/agents.py
@@ -4,44 +4,67 @@
 # This source code is licensed under the MIT license found in the
 # LICENSE file in the root directory of this source tree.
 
-from parlai.core.teachers import FbDeprecatedDialogTeacher
+from parlai.core.teachers import DialogTeacher
 from .build import build
 from parlai.utils.data import DatatypeHelper
 
 import copy
 import os
+import codecs
 
 
-def _path(opt, filtered):
-    # Build the data if it doesn't exist.
-    build(opt)
-    dt = opt['datatype'].split(':')[0]
-    return os.path.join(opt['datapath'], 'CornellMovie', dt + filtered + '.txt')
+def _path(opt, *additions):
+    return os.path.join(opt['datapath'], 'CornellMovie', *additions)
 
 
-class DefaultTeacher(FbDeprecatedDialogTeacher):
+class DefaultTeacher(DialogTeacher):
+    DOUBLE = False
+
     def __init__(self, opt, shared=None):
         opt = copy.deepcopy(opt)
-        opt['datafile'] = _path(opt, '')
-        opt['cands_datafile'] = opt['datafile']
         self.fold = DatatypeHelper.fold(opt['datatype'])
+        build(opt)
+        opt['datafile'] = _path(opt, self.fold + '.txt')
         super().__init__(opt, shared)
 
-    def num_examples(self):
-        if self.fold == 'train':
-            return 133125
-        elif self.fold == 'valid':
-            return 16759
-        elif self.fold == 'test':
-            return 16611
+    def setup_data(self, datafile):
+        lines_file = _path(self.opt, 'cornell movie-dialogs corpus', 'movie_lines.txt')
+        convo_file = _path(
+            self.opt, 'cornell movie-dialogs corpus', 'movie_conversations.txt'
+        )
+
+        lines = {}
+
+        codecs.register_error('strict', codecs.ignore_errors)
+        with codecs.open(lines_file, 'r') as f:
+            for line in f:
+                l = line.split(' +++$+++ ')
+                lines[l[0]] = ' '.join(l[4:]).strip('\n').replace('\t', ' ')
+
+        cnt = 0
+        with codecs.open(convo_file, 'r') as f:
+            for cnt, line in enumerate(f, 1):
+                l = line.split(' ')
+                convo = ' '.join(l[6:]).strip('\n').strip('[').strip(']')
+                c = convo.replace("'", '').replace(' ', '').split(',')
+
+                texts = [lines[l] for l in c]
+
+                if (cnt % 10 == 0) and self.fold != 'test':
+                    continue
+                elif (cnt % 10 == 1) and self.fold != 'valid':
+                    continue
+                elif (cnt % 10 > 1) and self.fold != 'train':
+                    continue
+
+                for i, (prompt, response) in enumerate(zip(texts[::2], texts[1::2])):
+                    yield {'text': prompt, 'label': response}, i == 0
 
-    def num_episodes(self):
-        if self.fold == 'train':
-            return 66478
-        elif self.fold == 'valid':
-            return 8310
-        elif self.fold == 'test':
-            return 8309
+                if self.DOUBLE:
+                    for i, (prompt, response) in enumerate(
+                        zip(texts[1::2], texts[2::2])
+                    ):
+                        yield {'text': prompt, 'label': response}, i == 0
 
 
 class DoubleTeacher(DefaultTeacher):
@@ -49,64 +72,4 @@ class DoubleTeacher(DefaultTeacher):
     This version creates text-label pairs from the perspective of both speakers.
     """
 
-    def num_examples(self):
-        if self.fold == 'train':
-            return 176975
-        elif self.fold == 'valid':
-            return 22349
-        elif self.fold == 'test':
-            return 22013
-
-    def num_episodes(self):
-        if self.fold == 'train':
-            return 102401
-        elif self.fold == 'valid':
-            return 12806
-        elif self.fold == 'test':
-            return 12790
-
-    def _rebuild(self, entries):
-        new_list = []
-        if len(entries) > 0:
-            # add all ( y_t => x_(t+1) ) pairs
-            new_list.extend(
-                [
-                    (entries[i][1][0], [entries[i + 1][0]])
-                    for i in range(len(entries) - 1)
-                ]
-            )
-        return new_list
-
-    def _is_valid(self, entry):
-        if entry[0] == '' or entry[1] is None:
-            return False
-        return True
-
-    def setup_data(self, path):
-        """
-        Adds additional perspectives. For example, in the conversation:
-
-        x1 y1
-        x2 y2
-        x3
-
-        Creates the additional dialog:
-
-        y1 x2
-        y2 x3
-        """
-        # this shows conversations in both directions
-        alternate = []
-        for entry, new in super().setup_data(path):
-            if new:
-                for i, e in enumerate(self._rebuild(alternate)):
-                    if self._is_valid(e):
-                        yield e, i == 0
-                alternate.clear()
-            alternate.append(entry)
-            if self._is_valid(entry):
-                yield entry, new
-        if alternate:
-            for i, e in enumerate(self._rebuild(alternate)):
-                if self._is_valid(e):
-                    yield e, i == 0
+    DOUBLE = True
diff --git a/parlai/tasks/cornell_movie/build.py b/parlai/tasks/cornell_movie/build.py
--- a/parlai/tasks/cornell_movie/build.py
+++ b/parlai/tasks/cornell_movie/build.py
@@ -20,55 +20,11 @@
 ]
 
 
-def create_fb_format(lines_file, convo_file, outpath):
-    print('[building fbformat]')
-    with PathManager.open(
-        os.path.join(outpath, 'train.txt'), 'w'
-    ) as ftrain, PathManager.open(
-        os.path.join(outpath, 'valid.txt'), 'w'
-    ) as fvalid, PathManager.open(
-        os.path.join(outpath, 'test.txt'), 'w'
-    ) as ftest:
-        lines = {}
-
-        codecs.register_error('strict', codecs.ignore_errors)
-        with codecs.open(lines_file, 'r') as f:
-            for line in f:
-                l = line.split(' +++$+++ ')
-                lines[l[0]] = ' '.join(l[4:]).strip('\n').replace('\t', ' ')
-
-        cnt = 0
-        with codecs.open(convo_file, 'r') as f:
-            for line in f:
-                l = line.split(' ')
-                convo = ' '.join(l[6:]).strip('\n').strip('[').strip(']')
-                c = convo.replace("'", '').replace(' ', '').split(',')
-
-                # forward conversation
-                s = ''
-                index = 0
-                for i in range(0, len(c), 2):
-                    index += 1
-                    s += str(index) + ' ' + lines[c[i]]
-                    if len(c) > i + 1:
-                        s += '\t' + lines[c[i + 1]]
-                    s += '\n'
-
-                cnt = cnt + 1
-                handle = ftrain
-                if (cnt % 10) == 0:
-                    handle = ftest
-                if (cnt % 10) == 1:
-                    handle = fvalid
-                handle.write(s + '\n')
-
-
 def build(opt):
     dpath = os.path.join(opt['datapath'], 'CornellMovie')
-    version = None
+    version = 'v1.01'
 
     if not build_data.built(dpath, version_string=version):
-        print('[building data: ' + dpath + ']')
         if build_data.built(dpath):
             # An older version exists, so remove these outdated files.
             build_data.remove_dir(dpath)
@@ -78,12 +34,5 @@ def build(opt):
         for downloadable_file in RESOURCES:
             downloadable_file.download_file(dpath)
 
-        dpext = os.path.join(dpath, 'cornell movie-dialogs corpus')
-        create_fb_format(
-            os.path.join(dpext, 'movie_lines.txt'),
-            os.path.join(dpext, 'movie_conversations.txt'),
-            dpath,
-        )
-
         # Mark the data as built.
         build_data.mark_done(dpath, version_string=version)


Test patch:
diff --git a/parlai/tasks/cornell_movie/test/cornell_movie_double_test.yml b/parlai/tasks/cornell_movie/test/cornell_movie_double_test.yml
--- a/parlai/tasks/cornell_movie/test/cornell_movie_double_test.yml
+++ b/parlai/tasks/cornell_movie/test/cornell_movie_double_test.yml
@@ -3,152 +3,28 @@ acts:
     eval_labels:
     - You're sweet.
     id: cornell_movie:double
-    label_candidates:
-    - ' And he agreed?'
-    - ' The only one who understands what this me-...'
-    - '"...unsure whether or not Enemy Action..."'
-    - '"A Reason to Love."'
-    - '"A people is a detour of nature to get 6 or 7 great men - Yes, and then to
-      get around them..." Nietzsche said that.'
-    - '"Cora" is my part. You''ve got to tell Lloyd it''s for me.'
-    - '"Debbie Does Dallas"...  Hell, it''s in Russian.  I can''t read it...'
-    - '"Deserve" don''t mean shit, Little Bill.'
-    - '"Deutchland, Deutchland ... "'
-    - '"Do chickens give milk?"'
-    - '[Your brother wants to talk to you.]'
-    - '`Cause you know I can do other stuff. I mean,  if you wanted me to talk or...'
-    - and HERBERT exchange a glance. HONORA smiles at Juliet.
-    - frowns at Henry.
-    - how much weight have you lost?
-    - kissed a lotta tadpoles.  Listen, I been thinking about your problem. I'm not
-      the guy to sponsor you. It would be unethical. But, there is something I could
-      do for you.  Putt-putt golf.
-    - ooohhhhhh no.
-    - xxxxxx
-    - yeah, I had a bad night.
-    - yeah.
-    reward: 0
     text: You have my word.  As a gentleman
 - - episode_done: false
     eval_labels:
     - What crap?
     id: cornell_movie:double
-    label_candidates:
-    - ' And he agreed?'
-    - ' The only one who understands what this me-...'
-    - '"...unsure whether or not Enemy Action..."'
-    - '"A Reason to Love."'
-    - '"A people is a detour of nature to get 6 or 7 great men - Yes, and then to
-      get around them..." Nietzsche said that.'
-    - '"Cora" is my part. You''ve got to tell Lloyd it''s for me.'
-    - '"Debbie Does Dallas"...  Hell, it''s in Russian.  I can''t read it...'
-    - '"Deserve" don''t mean shit, Little Bill.'
-    - '"Deutchland, Deutchland ... "'
-    - '"Do chickens give milk?"'
-    - '[Your brother wants to talk to you.]'
-    - '`Cause you know I can do other stuff. I mean,  if you wanted me to talk or...'
-    - and HERBERT exchange a glance. HONORA smiles at Juliet.
-    - frowns at Henry.
-    - how much weight have you lost?
-    - kissed a lotta tadpoles.  Listen, I been thinking about your problem. I'm not
-      the guy to sponsor you. It would be unethical. But, there is something I could
-      do for you.  Putt-putt golf.
-    - ooohhhhhh no.
-    - xxxxxx
-    - yeah, I had a bad night.
-    - yeah.
-    reward: 0
     text: do you listen to this crap?
 - - episode_done: true
     eval_labels:
     - Thank God!  If I had to hear one more story about your coiffure...
     id: cornell_movie:double
-    label_candidates:
-    - ' And he agreed?'
-    - ' The only one who understands what this me-...'
-    - '"...unsure whether or not Enemy Action..."'
-    - '"A Reason to Love."'
-    - '"A people is a detour of nature to get 6 or 7 great men - Yes, and then to
-      get around them..." Nietzsche said that.'
-    - '"Cora" is my part. You''ve got to tell Lloyd it''s for me.'
-    - '"Debbie Does Dallas"...  Hell, it''s in Russian.  I can''t read it...'
-    - '"Deserve" don''t mean shit, Little Bill.'
-    - '"Deutchland, Deutchland ... "'
-    - '"Do chickens give milk?"'
-    - '[Your brother wants to talk to you.]'
-    - '`Cause you know I can do other stuff. I mean,  if you wanted me to talk or...'
-    - and HERBERT exchange a glance. HONORA smiles at Juliet.
-    - frowns at Henry.
-    - how much weight have you lost?
-    - kissed a lotta tadpoles.  Listen, I been thinking about your problem. I'm not
-      the guy to sponsor you. It would be unethical. But, there is something I could
-      do for you.  Putt-putt golf.
-    - ooohhhhhh no.
-    - xxxxxx
-    - yeah, I had a bad night.
-    - yeah.
-    reward: 0
     text: Me.  This endless ...blonde babble. I'm like, boring myself.
 - - episode_done: true
     eval_labels:
     - Me.  This endless ...blonde babble. I'm like, boring myself.
     id: cornell_movie:double
-    label_candidates:
-    - ' And he agreed?'
-    - ' The only one who understands what this me-...'
-    - '"...unsure whether or not Enemy Action..."'
-    - '"A Reason to Love."'
-    - '"A people is a detour of nature to get 6 or 7 great men - Yes, and then to
-      get around them..." Nietzsche said that.'
-    - '"Cora" is my part. You''ve got to tell Lloyd it''s for me.'
-    - '"Debbie Does Dallas"...  Hell, it''s in Russian.  I can''t read it...'
-    - '"Deserve" don''t mean shit, Little Bill.'
-    - '"Deutchland, Deutchland ... "'
-    - '"Do chickens give milk?"'
-    - '[Your brother wants to talk to you.]'
-    - '`Cause you know I can do other stuff. I mean,  if you wanted me to talk or...'
-    - and HERBERT exchange a glance. HONORA smiles at Juliet.
-    - frowns at Henry.
-    - how much weight have you lost?
-    - kissed a lotta tadpoles.  Listen, I been thinking about your problem. I'm not
-      the guy to sponsor you. It would be unethical. But, there is something I could
-      do for you.  Putt-putt golf.
-    - ooohhhhhh no.
-    - xxxxxx
-    - yeah, I had a bad night.
-    - yeah.
     text: What crap?
 - - episode_done: true
     eval_labels:
     - Sometimes I wonder if the guys we're supposed to want to go out with are the
       ones we actually want to go out with, you know?
     id: cornell_movie:double
-    label_candidates:
-    - ' And he agreed?'
-    - ' The only one who understands what this me-...'
-    - '"...unsure whether or not Enemy Action..."'
-    - '"A Reason to Love."'
-    - '"A people is a detour of nature to get 6 or 7 great men - Yes, and then to
-      get around them..." Nietzsche said that.'
-    - '"Cora" is my part. You''ve got to tell Lloyd it''s for me.'
-    - '"Debbie Does Dallas"...  Hell, it''s in Russian.  I can''t read it...'
-    - '"Deserve" don''t mean shit, Little Bill.'
-    - '"Deutchland, Deutchland ... "'
-    - '"Do chickens give milk?"'
-    - '[Your brother wants to talk to you.]'
-    - '`Cause you know I can do other stuff. I mean,  if you wanted me to talk or...'
-    - and HERBERT exchange a glance. HONORA smiles at Juliet.
-    - frowns at Henry.
-    - how much weight have you lost?
-    - kissed a lotta tadpoles.  Listen, I been thinking about your problem. I'm not
-      the guy to sponsor you. It would be unethical. But, there is something I could
-      do for you.  Putt-putt golf.
-    - ooohhhhhh no.
-    - xxxxxx
-    - yeah, I had a bad night.
-    - yeah.
-    reward: 0
     text: Bianca, I don't think the highlights of dating Joey Dorsey are going to
       include door-opening and coat-holding.
-num_episodes: 12790
-num_examples: 22013
+num_episodes: 8309
+num_examples: 13725
diff --git a/parlai/tasks/cornell_movie/test/cornell_movie_double_train.yml b/parlai/tasks/cornell_movie/test/cornell_movie_double_train.yml
--- a/parlai/tasks/cornell_movie/test/cornell_movie_double_train.yml
+++ b/parlai/tasks/cornell_movie/test/cornell_movie_double_train.yml
@@ -1,145 +1,31 @@
 acts:
 - - episode_done: true
     id: cornell_movie:double
-    label_candidates:
-    - '   We have twenty-one minutes. Captain,    we can still save V''ger... and
-      ourselves.'
-    - '  DELETE IN CUTTING'
-    - '  I''ve got the real thing!'
-    - '  Oh, we''re screwed, Night...'
-    - ' ...Mason.  I can barely hear you.'
-    - ' ...checked.'
-    - ' ...or it''ll ignite.'
-    - ' ...shit!'
-    - ' ...the night is a tunnel... a hole into tomorrow... if we''re to have a tomorrow...'
-    - ' All right.  Hey.'
-    - xxxxxx
-    - y-y-yes--
-    - yeah, yes, hi, hello.
-    - yeah.
-    - yeah?
-    - yet cannot love nor write it.
-    - you'll need one more hit.
-    - '{suddenly unctious) Oh, but of course ...'
-    - '|      It''s not working. You''re not doing it properly.  Ni!'
-    - ~Don't shoot, G-Men..."
     labels:
     - Forget it.
-    reward: 0
     text: You're asking me out.  That's so cute. What's your name again?
 - - episode_done: false
     id: cornell_movie:double
-    label_candidates:
-    - '   We have twenty-one minutes. Captain,    we can still save V''ger... and
-      ourselves.'
-    - '  DELETE IN CUTTING'
-    - '  I''ve got the real thing!'
-    - '  Oh, we''re screwed, Night...'
-    - ' ...Mason.  I can barely hear you.'
-    - ' ...checked.'
-    - ' ...or it''ll ignite.'
-    - ' ...shit!'
-    - ' ...the night is a tunnel... a hole into tomorrow... if we''re to have a tomorrow...'
-    - ' All right.  Hey.'
-    - xxxxxx
-    - y-y-yes--
-    - yeah, yes, hi, hello.
-    - yeah.
-    - yeah?
-    - yet cannot love nor write it.
-    - you'll need one more hit.
-    - '{suddenly unctious) Oh, but of course ...'
-    - '|      It''s not working. You''re not doing it properly.  Ni!'
-    - ~Don't shoot, G-Men..."
     labels:
     - Cameron.
-    reward: 0
     text: No, no, it's my fault -- we didn't have a proper introduction ---
 - - episode_done: true
     id: cornell_movie:double
-    label_candidates:
-    - '   We have twenty-one minutes. Captain,    we can still save V''ger... and
-      ourselves.'
-    - '  DELETE IN CUTTING'
-    - '  I''ve got the real thing!'
-    - '  Oh, we''re screwed, Night...'
-    - ' ...Mason.  I can barely hear you.'
-    - ' ...checked.'
-    - ' ...or it''ll ignite.'
-    - ' ...shit!'
-    - ' ...the night is a tunnel... a hole into tomorrow... if we''re to have a tomorrow...'
-    - ' All right.  Hey.'
-    - xxxxxx
-    - y-y-yes--
-    - yeah, yes, hi, hello.
-    - yeah.
-    - yeah?
-    - yet cannot love nor write it.
-    - you'll need one more hit.
-    - '{suddenly unctious) Oh, but of course ...'
-    - '|      It''s not working. You''re not doing it properly.  Ni!'
-    - ~Don't shoot, G-Men..."
     labels:
     - Seems like she could get a date easy enough...
-    reward: 0
     text: The thing is, Cameron -- I'm at the mercy of a particularly hideous breed
       of loser.  My sister.  I can't date until she does.
 - - episode_done: true
     id: cornell_movie:double
-    label_candidates:
-    - '   We have twenty-one minutes. Captain,    we can still save V''ger... and
-      ourselves.'
-    - '  DELETE IN CUTTING'
-    - '  I''ve got the real thing!'
-    - '  Oh, we''re screwed, Night...'
-    - ' ...Mason.  I can barely hear you.'
-    - ' ...checked.'
-    - ' ...or it''ll ignite.'
-    - ' ...shit!'
-    - ' ...the night is a tunnel... a hole into tomorrow... if we''re to have a tomorrow...'
-    - ' All right.  Hey.'
-    - xxxxxx
-    - y-y-yes--
-    - yeah, yes, hi, hello.
-    - yeah.
-    - yeah?
-    - yet cannot love nor write it.
-    - you'll need one more hit.
-    - '{suddenly unctious) Oh, but of course ...'
-    - '|      It''s not working. You''re not doing it properly.  Ni!'
-    - ~Don't shoot, G-Men..."
     labels:
     - The thing is, Cameron -- I'm at the mercy of a particularly hideous breed of
       loser.  My sister.  I can't date until she does.
     text: Cameron.
 - - episode_done: true
     id: cornell_movie:double
-    label_candidates:
-    - '   We have twenty-one minutes. Captain,    we can still save V''ger... and
-      ourselves.'
-    - '  DELETE IN CUTTING'
-    - '  I''ve got the real thing!'
-    - '  Oh, we''re screwed, Night...'
-    - ' ...Mason.  I can barely hear you.'
-    - ' ...checked.'
-    - ' ...or it''ll ignite.'
-    - ' ...shit!'
-    - ' ...the night is a tunnel... a hole into tomorrow... if we''re to have a tomorrow...'
-    - ' All right.  Hey.'
-    - xxxxxx
-    - y-y-yes--
-    - yeah, yes, hi, hello.
-    - yeah.
-    - yeah?
-    - yet cannot love nor write it.
-    - you'll need one more hit.
-    - '{suddenly unctious) Oh, but of course ...'
-    - '|      It''s not working. You''re not doing it properly.  Ni!'
-    - ~Don't shoot, G-Men..."
     labels:
     - Unsolved mystery.  She used to be really popular when she started high school,
       then it was just like she got sick of it or something.
-    reward: 0
     text: Why?
-num_episodes: 102401
-num_examples: 176975
+num_episodes: 66478
+num_examples: 110496
diff --git a/parlai/tasks/cornell_movie/test/cornell_movie_double_valid.yml b/parlai/tasks/cornell_movie/test/cornell_movie_double_valid.yml
--- a/parlai/tasks/cornell_movie/test/cornell_movie_double_valid.yml
+++ b/parlai/tasks/cornell_movie/test/cornell_movie_double_valid.yml
@@ -3,167 +3,28 @@ acts:
     eval_labels:
     - Well, I thought we'd start with pronunciation, if that's okay with you.
     id: cornell_movie:double
-    label_candidates:
-    - '  Hey, actually...'
-    - ' ...-onds...with just a fraction of what''s in...'
-    - ' No...'
-    - ' That''s why I didn''t give it...'
-    - ' Yeah...'
-    - '"A faithful heart makes wishes come true."'
-    - '"Age of Enlightenment". Shit. Like some waitress in a Las Vegas coffee shop
-      is going to get an obscure French philosophical reference. How demeaning. I
-      may as well have just said "Let me jump your ignorant bones."...'
-    - '"Are not capable of love" is what you mean.'
-    - '"At first I thought it was infatuation ...But oh it''s lasted so long..."'
-    - '"At the center of the investigation are well-known Washington-area attorneys
-      Robert Dean and Rachel Banks."'
-    - think that's, uh-
-    - uh uh.
-    - wHAT'S My status?! I've lost three men and your worthless fuck! After I kill
-      this asshole I'm coming your Yllo!
-    - was fascinated with you....  a real live detective....  You used to tell me
-      the most wonderful stories.  Were they true?
-    - we've got kidnapping, grand theft auto, burglary, and two counts of murder on
-      you, and I'm gonna see to it personally that it sticks!
-    - xxxxxx
-    - yeah, good, ok.
-    - you woke me up to tell me that?
-    - you're gonna like night clubs, they're really a lotta fun.
-    - '{baffled) I don''t follow you.'
-    reward: 0
     text: Can we make this quick?  Roxanne Korrine and Andrew Barrett are having an
       incredibly horrendous public break- up on the quad.  Again.
 - - episode_done: true
     eval_labels:
     - Okay... then how 'bout we try out some French cuisine.  Saturday?  Night?
     id: cornell_movie:double
-    label_candidates:
-    - '  Hey, actually...'
-    - ' ...-onds...with just a fraction of what''s in...'
-    - ' No...'
-    - ' That''s why I didn''t give it...'
-    - ' Yeah...'
-    - '"A faithful heart makes wishes come true."'
-    - '"Age of Enlightenment". Shit. Like some waitress in a Las Vegas coffee shop
-      is going to get an obscure French philosophical reference. How demeaning. I
-      may as well have just said "Let me jump your ignorant bones."...'
-    - '"Are not capable of love" is what you mean.'
-    - '"At first I thought it was infatuation ...But oh it''s lasted so long..."'
-    - '"At the center of the investigation are well-known Washington-area attorneys
-      Robert Dean and Rachel Banks."'
-    - think that's, uh-
-    - uh uh.
-    - wHAT'S My status?! I've lost three men and your worthless fuck! After I kill
-      this asshole I'm coming your Yllo!
-    - was fascinated with you....  a real live detective....  You used to tell me
-      the most wonderful stories.  Were they true?
-    - we've got kidnapping, grand theft auto, burglary, and two counts of murder on
-      you, and I'm gonna see to it personally that it sticks!
-    - xxxxxx
-    - yeah, good, ok.
-    - you woke me up to tell me that?
-    - you're gonna like night clubs, they're really a lotta fun.
-    - '{baffled) I don''t follow you.'
-    reward: 0
     text: Not the hacking and gagging and spitting part.  Please.
 - - episode_done: true
     eval_labels:
     - Not the hacking and gagging and spitting part.  Please.
     id: cornell_movie:double
-    label_candidates:
-    - '  Hey, actually...'
-    - ' ...-onds...with just a fraction of what''s in...'
-    - ' No...'
-    - ' That''s why I didn''t give it...'
-    - ' Yeah...'
-    - '"A faithful heart makes wishes come true."'
-    - '"Age of Enlightenment". Shit. Like some waitress in a Las Vegas coffee shop
-      is going to get an obscure French philosophical reference. How demeaning. I
-      may as well have just said "Let me jump your ignorant bones."...'
-    - '"Are not capable of love" is what you mean.'
-    - '"At first I thought it was infatuation ...But oh it''s lasted so long..."'
-    - '"At the center of the investigation are well-known Washington-area attorneys
-      Robert Dean and Rachel Banks."'
-    - think that's, uh-
-    - uh uh.
-    - wHAT'S My status?! I've lost three men and your worthless fuck! After I kill
-      this asshole I'm coming your Yllo!
-    - was fascinated with you....  a real live detective....  You used to tell me
-      the most wonderful stories.  Were they true?
-    - we've got kidnapping, grand theft auto, burglary, and two counts of murder on
-      you, and I'm gonna see to it personally that it sticks!
-    - xxxxxx
-    - yeah, good, ok.
-    - you woke me up to tell me that?
-    - you're gonna like night clubs, they're really a lotta fun.
-    - '{baffled) I don''t follow you.'
     text: Well, I thought we'd start with pronunciation, if that's okay with you.
 - - episode_done: true
     eval_labels:
     - Eber's Deep Conditioner every two days. And I never, ever use a blowdryer without
       the diffuser attachment.
     id: cornell_movie:double
-    label_candidates:
-    - '  Hey, actually...'
-    - ' ...-onds...with just a fraction of what''s in...'
-    - ' No...'
-    - ' That''s why I didn''t give it...'
-    - ' Yeah...'
-    - '"A faithful heart makes wishes come true."'
-    - '"Age of Enlightenment". Shit. Like some waitress in a Las Vegas coffee shop
-      is going to get an obscure French philosophical reference. How demeaning. I
-      may as well have just said "Let me jump your ignorant bones."...'
-    - '"Are not capable of love" is what you mean.'
-    - '"At first I thought it was infatuation ...But oh it''s lasted so long..."'
-    - '"At the center of the investigation are well-known Washington-area attorneys
-      Robert Dean and Rachel Banks."'
-    - think that's, uh-
-    - uh uh.
-    - wHAT'S My status?! I've lost three men and your worthless fuck! After I kill
-      this asshole I'm coming your Yllo!
-    - was fascinated with you....  a real live detective....  You used to tell me
-      the most wonderful stories.  Were they true?
-    - we've got kidnapping, grand theft auto, burglary, and two counts of murder on
-      you, and I'm gonna see to it personally that it sticks!
-    - xxxxxx
-    - yeah, good, ok.
-    - you woke me up to tell me that?
-    - you're gonna like night clubs, they're really a lotta fun.
-    - '{baffled) I don''t follow you.'
-    reward: 0
     text: How do you get your hair to look like that?
 - - episode_done: false
     eval_labels:
     - What good stuff?
     id: cornell_movie:double
-    label_candidates:
-    - '  Hey, actually...'
-    - ' ...-onds...with just a fraction of what''s in...'
-    - ' No...'
-    - ' That''s why I didn''t give it...'
-    - ' Yeah...'
-    - '"A faithful heart makes wishes come true."'
-    - '"Age of Enlightenment". Shit. Like some waitress in a Las Vegas coffee shop
-      is going to get an obscure French philosophical reference. How demeaning. I
-      may as well have just said "Let me jump your ignorant bones."...'
-    - '"Are not capable of love" is what you mean.'
-    - '"At first I thought it was infatuation ...But oh it''s lasted so long..."'
-    - '"At the center of the investigation are well-known Washington-area attorneys
-      Robert Dean and Rachel Banks."'
-    - think that's, uh-
-    - uh uh.
-    - wHAT'S My status?! I've lost three men and your worthless fuck! After I kill
-      this asshole I'm coming your Yllo!
-    - was fascinated with you....  a real live detective....  You used to tell me
-      the most wonderful stories.  Were they true?
-    - we've got kidnapping, grand theft auto, burglary, and two counts of murder on
-      you, and I'm gonna see to it personally that it sticks!
-    - xxxxxx
-    - yeah, good, ok.
-    - you woke me up to tell me that?
-    - you're gonna like night clubs, they're really a lotta fun.
-    - '{baffled) I don''t follow you.'
-    reward: 0
     text: I figured you'd get to the good stuff eventually.
-num_episodes: 12806
-num_examples: 22349
+num_episodes: 8310
+num_examples: 13914
diff --git a/parlai/tasks/cornell_movie/test/cornell_movie_test.yml b/parlai/tasks/cornell_movie/test/cornell_movie_test.yml
--- a/parlai/tasks/cornell_movie/test/cornell_movie_test.yml
+++ b/parlai/tasks/cornell_movie/test/cornell_movie_test.yml
@@ -3,127 +3,29 @@ acts:
     eval_labels:
     - You're sweet.
     id: cornell_movie
-    label_candidates:
-    - ' And he agreed?'
-    - ' The only one who understands what this me-...'
-    - '"...unsure whether or not Enemy Action..."'
-    - '"A Reason to Love."'
-    - '"A people is a detour of nature to get 6 or 7 great men - Yes, and then to
-      get around them..." Nietzsche said that.'
-    - '"Cora" is my part. You''ve got to tell Lloyd it''s for me.'
-    - '"Debbie Does Dallas"...  Hell, it''s in Russian.  I can''t read it...'
-    - '"Deserve" don''t mean shit, Little Bill.'
-    - '"Deutchland, Deutchland ... "'
-    - '"Do chickens give milk?"'
-    - '[Your brother wants to talk to you.]'
-    - '`Cause you know I can do other stuff. I mean,  if you wanted me to talk or...'
-    - and HERBERT exchange a glance. HONORA smiles at Juliet.
-    - frowns at Henry.
-    - how much weight have you lost?
-    - kissed a lotta tadpoles.  Listen, I been thinking about your problem. I'm not
-      the guy to sponsor you. It would be unethical. But, there is something I could
-      do for you.  Putt-putt golf.
-    - ooohhhhhh no.
-    - xxxxxx
-    - yeah, I had a bad night.
-    - yeah.
-    reward: 0
     text: You have my word.  As a gentleman
 - - episode_done: false
     eval_labels:
     - What crap?
     id: cornell_movie
-    label_candidates:
-    - ' And he agreed?'
-    - ' The only one who understands what this me-...'
-    - '"...unsure whether or not Enemy Action..."'
-    - '"A Reason to Love."'
-    - '"A people is a detour of nature to get 6 or 7 great men - Yes, and then to
-      get around them..." Nietzsche said that.'
-    - '"Cora" is my part. You''ve got to tell Lloyd it''s for me.'
-    - '"Debbie Does Dallas"...  Hell, it''s in Russian.  I can''t read it...'
-    - '"Deserve" don''t mean shit, Little Bill.'
-    - '"Deutchland, Deutchland ... "'
-    - '"Do chickens give milk?"'
-    - '[Your brother wants to talk to you.]'
-    - '`Cause you know I can do other stuff. I mean,  if you wanted me to talk or...'
-    - and HERBERT exchange a glance. HONORA smiles at Juliet.
-    - frowns at Henry.
-    - how much weight have you lost?
-    - kissed a lotta tadpoles.  Listen, I been thinking about your problem. I'm not
-      the guy to sponsor you. It would be unethical. But, there is something I could
-      do for you.  Putt-putt golf.
-    - ooohhhhhh no.
-    - xxxxxx
-    - yeah, I had a bad night.
-    - yeah.
-    reward: 0
     text: do you listen to this crap?
 - - episode_done: true
     eval_labels:
     - Thank God!  If I had to hear one more story about your coiffure...
     id: cornell_movie
-    label_candidates:
-    - ' And he agreed?'
-    - ' The only one who understands what this me-...'
-    - '"...unsure whether or not Enemy Action..."'
-    - '"A Reason to Love."'
-    - '"A people is a detour of nature to get 6 or 7 great men - Yes, and then to
-      get around them..." Nietzsche said that.'
-    - '"Cora" is my part. You''ve got to tell Lloyd it''s for me.'
-    - '"Debbie Does Dallas"...  Hell, it''s in Russian.  I can''t read it...'
-    - '"Deserve" don''t mean shit, Little Bill.'
-    - '"Deutchland, Deutchland ... "'
-    - '"Do chickens give milk?"'
-    - '[Your brother wants to talk to you.]'
-    - '`Cause you know I can do other stuff. I mean,  if you wanted me to talk or...'
-    - and HERBERT exchange a glance. HONORA smiles at Juliet.
-    - frowns at Henry.
-    - how much weight have you lost?
-    - kissed a lotta tadpoles.  Listen, I been thinking about your problem. I'm not
-      the guy to sponsor you. It would be unethical. But, there is something I could
-      do for you.  Putt-putt golf.
-    - ooohhhhhh no.
-    - xxxxxx
-    - yeah, I had a bad night.
-    - yeah.
-    reward: 0
     text: Me.  This endless ...blonde babble. I'm like, boring myself.
-- - episode_done: false
+- - episode_done: true
     eval_labels:
     - Sometimes I wonder if the guys we're supposed to want to go out with are the
       ones we actually want to go out with, you know?
     id: cornell_movie
-    label_candidates:
-    - ' And he agreed?'
-    - ' The only one who understands what this me-...'
-    - '"...unsure whether or not Enemy Action..."'
-    - '"A Reason to Love."'
-    - '"A people is a detour of nature to get 6 or 7 great men - Yes, and then to
-      get around them..." Nietzsche said that.'
-    - '"Cora" is my part. You''ve got to tell Lloyd it''s for me.'
-    - '"Debbie Does Dallas"...  Hell, it''s in Russian.  I can''t read it...'
-    - '"Deserve" don''t mean shit, Little Bill.'
-    - '"Deutchland, Deutchland ... "'
-    - '"Do chickens give milk?"'
-    - '[Your brother wants to talk to you.]'
-    - '`Cause you know I can do other stuff. I mean,  if you wanted me to talk or...'
-    - and HERBERT exchange a glance. HONORA smiles at Juliet.
-    - frowns at Henry.
-    - how much weight have you lost?
-    - kissed a lotta tadpoles.  Listen, I been thinking about your problem. I'm not
-      the guy to sponsor you. It would be unethical. But, there is something I could
-      do for you.  Putt-putt golf.
-    - ooohhhhhh no.
-    - xxxxxx
-    - yeah, I had a bad night.
-    - yeah.
-    reward: 0
     text: Bianca, I don't think the highlights of dating Joey Dorsey are going to
       include door-opening and coat-holding.
 - - episode_done: true
+    eval_labels:
+    - Can you at least start wearing a bra?
     id: cornell_movie
-    reward: 0
-    text: All I know is -- I'd give up my private line to go out with a guy like Joey.
+    text: I have the potential to smack the crap out of you if you don't get out of
+      my way.
 num_episodes: 8309
-num_examples: 16611
+num_examples: 13725
diff --git a/parlai/tasks/cornell_movie/test/cornell_movie_train.yml b/parlai/tasks/cornell_movie/test/cornell_movie_train.yml
--- a/parlai/tasks/cornell_movie/test/cornell_movie_train.yml
+++ b/parlai/tasks/cornell_movie/test/cornell_movie_train.yml
@@ -1,121 +1,30 @@
 acts:
 - - episode_done: true
     id: cornell_movie
-    label_candidates:
-    - '   We have twenty-one minutes. Captain,    we can still save V''ger... and
-      ourselves.'
-    - '  DELETE IN CUTTING'
-    - '  I''ve got the real thing!'
-    - '  Oh, we''re screwed, Night...'
-    - ' ...Mason.  I can barely hear you.'
-    - ' ...checked.'
-    - ' ...or it''ll ignite.'
-    - ' ...shit!'
-    - ' ...the night is a tunnel... a hole into tomorrow... if we''re to have a tomorrow...'
-    - ' All right.  Hey.'
-    - xxxxxx
-    - y-y-yes--
-    - yeah, yes, hi, hello.
-    - yeah.
-    - yeah?
-    - yet cannot love nor write it.
-    - you'll need one more hit.
-    - '{suddenly unctious) Oh, but of course ...'
-    - '|      It''s not working. You''re not doing it properly.  Ni!'
-    - ~Don't shoot, G-Men..."
     labels:
     - Forget it.
-    reward: 0
     text: You're asking me out.  That's so cute. What's your name again?
 - - episode_done: false
     id: cornell_movie
-    label_candidates:
-    - '   We have twenty-one minutes. Captain,    we can still save V''ger... and
-      ourselves.'
-    - '  DELETE IN CUTTING'
-    - '  I''ve got the real thing!'
-    - '  Oh, we''re screwed, Night...'
-    - ' ...Mason.  I can barely hear you.'
-    - ' ...checked.'
-    - ' ...or it''ll ignite.'
-    - ' ...shit!'
-    - ' ...the night is a tunnel... a hole into tomorrow... if we''re to have a tomorrow...'
-    - ' All right.  Hey.'
-    - xxxxxx
-    - y-y-yes--
-    - yeah, yes, hi, hello.
-    - yeah.
-    - yeah?
-    - yet cannot love nor write it.
-    - you'll need one more hit.
-    - '{suddenly unctious) Oh, but of course ...'
-    - '|      It''s not working. You''re not doing it properly.  Ni!'
-    - ~Don't shoot, G-Men..."
     labels:
     - Cameron.
-    reward: 0
     text: No, no, it's my fault -- we didn't have a proper introduction ---
 - - episode_done: true
     id: cornell_movie
-    label_candidates:
-    - '   We have twenty-one minutes. Captain,    we can still save V''ger... and
-      ourselves.'
-    - '  DELETE IN CUTTING'
-    - '  I''ve got the real thing!'
-    - '  Oh, we''re screwed, Night...'
-    - ' ...Mason.  I can barely hear you.'
-    - ' ...checked.'
-    - ' ...or it''ll ignite.'
-    - ' ...shit!'
-    - ' ...the night is a tunnel... a hole into tomorrow... if we''re to have a tomorrow...'
-    - ' All right.  Hey.'
-    - xxxxxx
-    - y-y-yes--
-    - yeah, yes, hi, hello.
-    - yeah.
-    - yeah?
-    - yet cannot love nor write it.
-    - you'll need one more hit.
-    - '{suddenly unctious) Oh, but of course ...'
-    - '|      It''s not working. You''re not doing it properly.  Ni!'
-    - ~Don't shoot, G-Men..."
     labels:
     - Seems like she could get a date easy enough...
-    reward: 0
     text: The thing is, Cameron -- I'm at the mercy of a particularly hideous breed
       of loser.  My sister.  I can't date until she does.
-- - episode_done: false
+- - episode_done: true
     id: cornell_movie
-    label_candidates:
-    - '   We have twenty-one minutes. Captain,    we can still save V''ger... and
-      ourselves.'
-    - '  DELETE IN CUTTING'
-    - '  I''ve got the real thing!'
-    - '  Oh, we''re screwed, Night...'
-    - ' ...Mason.  I can barely hear you.'
-    - ' ...checked.'
-    - ' ...or it''ll ignite.'
-    - ' ...shit!'
-    - ' ...the night is a tunnel... a hole into tomorrow... if we''re to have a tomorrow...'
-    - ' All right.  Hey.'
-    - xxxxxx
-    - y-y-yes--
-    - yeah, yes, hi, hello.
-    - yeah.
-    - yeah?
-    - yet cannot love nor write it.
-    - you'll need one more hit.
-    - '{suddenly unctious) Oh, but of course ...'
-    - '|      It''s not working. You''re not doing it properly.  Ni!'
-    - ~Don't shoot, G-Men..."
     labels:
     - Unsolved mystery.  She used to be really popular when she started high school,
       then it was just like she got sick of it or something.
-    reward: 0
     text: Why?
 - - episode_done: true
     id: cornell_movie
-    reward: 0
-    text: That's a shame.
+    labels:
+    - Let me see what I can do.
+    text: Gosh, if only we could find Kat a boyfriend...
 num_episodes: 66478
-num_examples: 133125
+num_examples: 110496
diff --git a/parlai/tasks/cornell_movie/test/cornell_movie_valid.yml b/parlai/tasks/cornell_movie/test/cornell_movie_valid.yml
--- a/parlai/tasks/cornell_movie/test/cornell_movie_valid.yml
+++ b/parlai/tasks/cornell_movie/test/cornell_movie_valid.yml
@@ -3,168 +3,28 @@ acts:
     eval_labels:
     - Well, I thought we'd start with pronunciation, if that's okay with you.
     id: cornell_movie
-    label_candidates:
-    - '  Hey, actually...'
-    - ' ...-onds...with just a fraction of what''s in...'
-    - ' No...'
-    - ' That''s why I didn''t give it...'
-    - ' Yeah...'
-    - '"A faithful heart makes wishes come true."'
-    - '"Age of Enlightenment". Shit. Like some waitress in a Las Vegas coffee shop
-      is going to get an obscure French philosophical reference. How demeaning. I
-      may as well have just said "Let me jump your ignorant bones."...'
-    - '"Are not capable of love" is what you mean.'
-    - '"At first I thought it was infatuation ...But oh it''s lasted so long..."'
-    - '"At the center of the investigation are well-known Washington-area attorneys
-      Robert Dean and Rachel Banks."'
-    - think that's, uh-
-    - uh uh.
-    - wHAT'S My status?! I've lost three men and your worthless fuck! After I kill
-      this asshole I'm coming your Yllo!
-    - was fascinated with you....  a real live detective....  You used to tell me
-      the most wonderful stories.  Were they true?
-    - we've got kidnapping, grand theft auto, burglary, and two counts of murder on
-      you, and I'm gonna see to it personally that it sticks!
-    - xxxxxx
-    - yeah, good, ok.
-    - you woke me up to tell me that?
-    - you're gonna like night clubs, they're really a lotta fun.
-    - '{baffled) I don''t follow you.'
-    reward: 0
     text: Can we make this quick?  Roxanne Korrine and Andrew Barrett are having an
       incredibly horrendous public break- up on the quad.  Again.
 - - episode_done: true
     eval_labels:
     - Okay... then how 'bout we try out some French cuisine.  Saturday?  Night?
     id: cornell_movie
-    label_candidates:
-    - '  Hey, actually...'
-    - ' ...-onds...with just a fraction of what''s in...'
-    - ' No...'
-    - ' That''s why I didn''t give it...'
-    - ' Yeah...'
-    - '"A faithful heart makes wishes come true."'
-    - '"Age of Enlightenment". Shit. Like some waitress in a Las Vegas coffee shop
-      is going to get an obscure French philosophical reference. How demeaning. I
-      may as well have just said "Let me jump your ignorant bones."...'
-    - '"Are not capable of love" is what you mean.'
-    - '"At first I thought it was infatuation ...But oh it''s lasted so long..."'
-    - '"At the center of the investigation are well-known Washington-area attorneys
-      Robert Dean and Rachel Banks."'
-    - think that's, uh-
-    - uh uh.
-    - wHAT'S My status?! I've lost three men and your worthless fuck! After I kill
-      this asshole I'm coming your Yllo!
-    - was fascinated with you....  a real live detective....  You used to tell me
-      the most wonderful stories.  Were they true?
-    - we've got kidnapping, grand theft auto, burglary, and two counts of murder on
-      you, and I'm gonna see to it personally that it sticks!
-    - xxxxxx
-    - yeah, good, ok.
-    - you woke me up to tell me that?
-    - you're gonna like night clubs, they're really a lotta fun.
-    - '{baffled) I don''t follow you.'
-    reward: 0
     text: Not the hacking and gagging and spitting part.  Please.
 - - episode_done: true
     eval_labels:
     - Eber's Deep Conditioner every two days. And I never, ever use a blowdryer without
       the diffuser attachment.
     id: cornell_movie
-    label_candidates:
-    - '  Hey, actually...'
-    - ' ...-onds...with just a fraction of what''s in...'
-    - ' No...'
-    - ' That''s why I didn''t give it...'
-    - ' Yeah...'
-    - '"A faithful heart makes wishes come true."'
-    - '"Age of Enlightenment". Shit. Like some waitress in a Las Vegas coffee shop
-      is going to get an obscure French philosophical reference. How demeaning. I
-      may as well have just said "Let me jump your ignorant bones."...'
-    - '"Are not capable of love" is what you mean.'
-    - '"At first I thought it was infatuation ...But oh it''s lasted so long..."'
-    - '"At the center of the investigation are well-known Washington-area attorneys
-      Robert Dean and Rachel Banks."'
-    - think that's, uh-
-    - uh uh.
-    - wHAT'S My status?! I've lost three men and your worthless fuck! After I kill
-      this asshole I'm coming your Yllo!
-    - was fascinated with you....  a real live detective....  You used to tell me
-      the most wonderful stories.  Were they true?
-    - we've got kidnapping, grand theft auto, burglary, and two counts of murder on
-      you, and I'm gonna see to it personally that it sticks!
-    - xxxxxx
-    - yeah, good, ok.
-    - you woke me up to tell me that?
-    - you're gonna like night clubs, they're really a lotta fun.
-    - '{baffled) I don''t follow you.'
-    reward: 0
     text: How do you get your hair to look like that?
 - - episode_done: false
     eval_labels:
     - What good stuff?
     id: cornell_movie
-    label_candidates:
-    - '  Hey, actually...'
-    - ' ...-onds...with just a fraction of what''s in...'
-    - ' No...'
-    - ' That''s why I didn''t give it...'
-    - ' Yeah...'
-    - '"A faithful heart makes wishes come true."'
-    - '"Age of Enlightenment". Shit. Like some waitress in a Las Vegas coffee shop
-      is going to get an obscure French philosophical reference. How demeaning. I
-      may as well have just said "Let me jump your ignorant bones."...'
-    - '"Are not capable of love" is what you mean.'
-    - '"At first I thought it was infatuation ...But oh it''s lasted so long..."'
-    - '"At the center of the investigation are well-known Washington-area attorneys
-      Robert Dean and Rachel Banks."'
-    - think that's, uh-
-    - uh uh.
-    - wHAT'S My status?! I've lost three men and your worthless fuck! After I kill
-      this asshole I'm coming your Yllo!
-    - was fascinated with you....  a real live detective....  You used to tell me
-      the most wonderful stories.  Were they true?
-    - we've got kidnapping, grand theft auto, burglary, and two counts of murder on
-      you, and I'm gonna see to it personally that it sticks!
-    - xxxxxx
-    - yeah, good, ok.
-    - you woke me up to tell me that?
-    - you're gonna like night clubs, they're really a lotta fun.
-    - '{baffled) I don''t follow you.'
-    reward: 0
     text: I figured you'd get to the good stuff eventually.
 - - episode_done: true
     eval_labels:
     - Like my fear of wearing pastels?
     id: cornell_movie
-    label_candidates:
-    - '  Hey, actually...'
-    - ' ...-onds...with just a fraction of what''s in...'
-    - ' No...'
-    - ' That''s why I didn''t give it...'
-    - ' Yeah...'
-    - '"A faithful heart makes wishes come true."'
-    - '"Age of Enlightenment". Shit. Like some waitress in a Las Vegas coffee shop
-      is going to get an obscure French philosophical reference. How demeaning. I
-      may as well have just said "Let me jump your ignorant bones."...'
-    - '"Are not capable of love" is what you mean.'
-    - '"At first I thought it was infatuation ...But oh it''s lasted so long..."'
-    - '"At the center of the investigation are well-known Washington-area attorneys
-      Robert Dean and Rachel Banks."'
-    - think that's, uh-
-    - uh uh.
-    - wHAT'S My status?! I've lost three men and your worthless fuck! After I kill
-      this asshole I'm coming your Yllo!
-    - was fascinated with you....  a real live detective....  You used to tell me
-      the most wonderful stories.  Were they true?
-    - we've got kidnapping, grand theft auto, burglary, and two counts of murder on
-      you, and I'm gonna see to it personally that it sticks!
-    - xxxxxx
-    - yeah, good, ok.
-    - you woke me up to tell me that?
-    - you're gonna like night clubs, they're really a lotta fun.
-    - '{baffled) I don''t follow you.'
-    reward: 0
     text: The "real you".
 num_episodes: 8310
-num_examples: 16759
+num_examples: 13914