Repository: facebookresearch/ParlAI

Issue:
Display_data script does not work for dealnodeal
**Bug description**
When running python examples/display_data.py -t dealnodeal, an error occurs, but it works for other datasets like twitter and babi.

**Reproduction steps**
I installed ParlAI, ran the suggested "python examples/display_data.py -t twitter", which worked! I want to work with the Deal or No Deal dataset, which led to an error about the 'TeacherMetrics' object having no attribute 'update'. Logs included below.

**Expected behavior**
For the twitter and babi datasets, I saw a few examples of the data and expected a similar output for the Deal or No Deal dataset.

**Logs**
/persist/anaconda3/lib/python3.7/site-packages/sklearn/feature_extraction/text.py:17: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
  from collections import Mapping, defaultdict
[ optional arguments: ] 
[  display_ignore_fields: agent_reply ]
[  max_display_len: 1000 ]
[  num_examples: 10 ]
[ Main ParlAI Arguments: ] 
[  batchsize: 1 ]
[  datapath: /persist/ParlAI/data ]
[  datatype: train:stream ]
[  download_path: /persist/ParlAI/downloads ]
[  dynamic_batching: None ]
[  hide_labels: False ]
[  image_mode: raw ]
[  init_opt: None ]
[  multitask_weights: [1] ]
[  numthreads: 1 ]
[  show_advanced_args: False ]
[  task: dealnodeal ]
[ ParlAI Model Arguments: ] 
[  dict_class: None ]
[  init_model: None ]
[  model: None ]
[  model_file: None ]
[ ParlAI Image Preprocessing Arguments: ] 
[  image_cropsize: 224 ]
[  image_size: 256 ]
[ Current ParlAI commit: b1a1f16ba9b1197d3ddfdb3bc3cec7917042ce03 ]
[creating task(s): dealnodeal]
loading: /persist/ParlAI/data/negotiation/end-to-end-negotiator-master/src/data/negotiate/train.txt
Traceback (most recent call last):
  File "/persist/ParlAI/examples/display_data.py", line 23, in <module>
    display_data(opt)
  File "/persist/ParlAI/parlai/scripts/display_data.py", line 46, in display_data
    world.parley()
  File "/persist/ParlAI/parlai/core/worlds.py", line 343, in parley
    agents[0].observe(validate(acts[1]))
  File "/persist/ParlAI/parlai/tasks/dealnodeal/agents.py", line 111, in observe
    self.metrics.update(observation, self.expected_reponse)
AttributeError: 'TeacherMetrics' object has no attribute 'update'

**Additional context**
Add any other context about the problem here. (like proxy settings, network setup, overall goals, etc.)



Discussion:
Thanks for filing!

Resolving patch:
diff --git a/parlai/scripts/verify_data.py b/parlai/scripts/verify_data.py
--- a/parlai/scripts/verify_data.py
+++ b/parlai/scripts/verify_data.py
@@ -27,7 +27,7 @@ def setup_args(parser=None):
     # Get command line arguments
     parser.add_argument('-ltim', '--log-every-n-secs', type=float, default=2)
     parser.add_argument('-d', '--display-examples', type='bool', default=False)
-    parser.set_defaults(datatype='train:stream')
+    parser.set_defaults(datatype='train:ordered')
     return parser
 
 
diff --git a/parlai/tasks/dealnodeal/agents.py b/parlai/tasks/dealnodeal/agents.py
--- a/parlai/tasks/dealnodeal/agents.py
+++ b/parlai/tasks/dealnodeal/agents.py
@@ -46,7 +46,8 @@ class NegotiationTeacher(Teacher):
     def __init__(self, opt, shared=None):
         super().__init__(opt, shared)
         self.datatype = opt['datatype'].split(':')[0]
-        self.random = self.datatype == 'train'
+        self.datatype_ = opt['datatype']
+        self.random = self.datatype_ == 'train'
         build(opt)
 
         filename = 'val' if self.datatype == 'valid' else self.datatype
@@ -86,6 +87,9 @@ def num_examples(self):
         )
         return num_exs
 
+    def num_episodes(self):
+        return len(self.episodes)
+
     def reset(self):
         super().reset()
         self.episode_idx = self.data_offset - self.step_size
@@ -108,7 +112,7 @@ def observe(self, observation):
         Process observation for metrics.
         """
         if self.expected_reponse is not None:
-            self.metrics.update(observation, self.expected_reponse)
+            self.metrics.evaluate_response(observation, self.expected_reponse)
             self.expected_reponse = None
         return observation
 


Test patch: