Repository: facebookresearch/ParlAI

Issue:
Wizard Of Wikipedia mturk evaluation error
When I try to run /projects/wizard_of_wikipedia/mturk_evaluation_task/run.py I get the following error:

08/30/2019 09:40:51 PM: [ Manager received: Packet <{'id': '08c62114-d886-4405-8092-a0b0c490c529', 'type': 'message', 'sender_id': 'A2ACW7CBIDO9XE', 'receiver_id': '[World_mturk_evaluation_task_1567215583]', 'assignment_id': '3VHP9MDGROQH5K9SZ46ODHXJTRLCF2', 'conversation_id': 't_1', 'data': {'text': 'I like Siamese cats.', 'id': 'PERSON_1', 'message_id': '8e579d58-a466-47c0-a479-6a110a570a05', 'episode_done': False}}> ]
Exception in thread task-t_1:
Traceback (most recent call last):
  File "/home/namanahuja/anaconda3/envs/fall19/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/home/namanahuja/anaconda3/envs/fall19/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/namanahuja/latest/ParlAI/parlai/mturk/core/mturk_manager.py", line 1209, in _task_function
    save_data = task_function(mturk_manager=self, opt=opt, workers=agents)
  File "run.py", line 249, in run_conversation
    world.parley()
  File "/home/namanahuja/ParlAI/projects/wizard_of_wikipedia/mturk_evaluation_task/worlds.py", line 295, in parley
    act = self.model_agent.act()
  File "/home/namanahuja/latest/ParlAI/projects/wizard_of_wikipedia/interactive_retrieval/interactive_retrieval.py", line 278, in act
    responder_act = self.responder.act()
  File "/home/namanahuja/latest/ParlAI/parlai/core/torch_agent.py", line 1687, in act
    'act() will misbehave in batching mode. Set batchsize to 1, or '
RuntimeError: act() will misbehave in batching mode. Set batchsize to 1, or --interactive-mode true



Discussion:
Can you share the full args? But the answer is to add `—interactive-mode true` to your command line args.
@stephenroller  Thanks for your reply. I am running the experiment via:
python run.py --interactive-mode true

I can begin the Mturk experiment and select a topic. But after that, I get the error:

09/01/2019 01:59:56 PM: [ Starting task t_1... ]
09/01/2019 01:59:57 PM: [ All agents joined the conversation t_1! ]
conversation t_1 is at turn 1...
<b><span style="color:blue">Automobile repair shop
</span></b>
09/01/2019 01:59:57 PM: [ Manager sending: Packet <{'id': 'A2ACW7CBIDO9XE_070c94e8-ca0a-46ec-9718-79aa990e0233', 'type': 'message', 'sender_id': '[World_mturk_evaluation_task_1567360767]', 'receiver_id': 'A2ACW7CBIDO9XE', 'assignment_id': '3V5Q80FXIYX9RQ1XIGGG0VZ0EA623D', 'conversation_id': 't_1', 'data': {'episode_done': False, 'id': 'SYSTEM', 'chosen_topic': '<b><span style="color:blue">Automobile repair shop\n</span></b>', 'text': '\nSuccessfully matched.         Now let\'s talk about something through the chat!        <b><span style="color:red">You can find your chosen topic in the         bottom left corner.</span></b> \n         You need to finish at least <b>5 chat turns</b>,         after which you can click the "Done" button to end the chat. ', 'type': 'MESSAGE', 'message_id': '632814c9-3ea0-40b1-834d-ad9e626e587f'}}> ]
09/01/2019 01:59:57 PM: [ Expiring all unassigned HITs... ]
Expiring all unassigned HITs...
Exception in thread task-t_1:
Traceback (most recent call last):
  File "/home/namanahuja/anaconda3/envs/fall19/lib/python3.7/threading.py", line 926, in _bootstrap_inner
    self.run()
  File "/home/namanahuja/anaconda3/envs/fall19/lib/python3.7/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/home/namanahuja/latest/ParlAI/parlai/mturk/core/mturk_manager.py", line 1209, in _task_function
    save_data = task_function(mturk_manager=self, opt=opt, workers=agents)
  File "run.py", line 249, in run_conversation
    world.parley()
  File "/home/namanahuja/ParlAI/projects/wizard_of_wikipedia/mturk_evaluation_task/worlds.py", line 248, in parley
    model_act = self.model_agent.act()
  File "/home/namanahuja/latest/ParlAI/projects/wizard_of_wikipedia/interactive_retrieval/interactive_retrieval.py", line 278, in act
    responder_act = self.responder.act()
  File "/home/namanahuja/latest/ParlAI/parlai/core/torch_agent.py", line 1687, in act
    'act() will misbehave in batching mode. Set batchsize to 1, or '
RuntimeError: act() will misbehave in batching mode. Set batchsize to 1, or --interactive-mode true

Okay cool, thanks for reporting.

I think you can add `’interactive_mode’: True,` to this dictionary:


https://github.com/facebookresearch/ParlAI/blob/e122f33b85755f1b00b4117db09bb3dbdc5d0bfb/projects/wizard_of_wikipedia/interactive_retrieval/interactive_retrieval.py#L98
Thanks for the suggestion. I can now type a message. But, as soon as I press send, I get the same error!
Okay, one of us will take a look next week when we’re back from the holiday weekend.

Resolving patch:
diff --git a/parlai/tasks/wizard_of_wikipedia/worlds.py b/parlai/tasks/wizard_of_wikipedia/worlds.py
--- a/parlai/tasks/wizard_of_wikipedia/worlds.py
+++ b/parlai/tasks/wizard_of_wikipedia/worlds.py
@@ -12,6 +12,7 @@
 
 from parlai.core.params import ParlaiParser
 from parlai.core.agents import create_agent
+from parlai.core.message import Message
 from parlai.core.worlds import create_task
 from parlai.core.worlds import DialogPartnerWorld, validate
 from parlai.agents.repeat_label.repeat_label import RepeatLabelAgent
@@ -76,7 +77,7 @@ def parley(self):
 
         # possibly get human act first
         if self.cnt == 0 and not self.human_first:
-            self.acts[0] = act = {'text': '', 'episode_done': False}
+            self.acts[0] = act = Message({'text': '', 'episode_done': False})
             act = self.acts[0]
         else:
             self.acts[0] = self.human_agent.act()
@@ -86,7 +87,7 @@ def parley(self):
         if self.cnt == 0 and self.topic != NO_TOPIC:
             # add the chosen_topic to the message
             act['chosen_topic'] = self.topic
-            act['text'] = '\n'.join([self.topic, act.get('text', 'hi')])
+            act.force_set('text', '\n'.join([self.topic, act.get('text', 'hi')]))
         self.model_agent.observe(validate(act))
 
         # model agent act
diff --git a/projects/wizard_of_wikipedia/interactive_retrieval/interactive_retrieval.py b/projects/wizard_of_wikipedia/interactive_retrieval/interactive_retrieval.py
--- a/projects/wizard_of_wikipedia/interactive_retrieval/interactive_retrieval.py
+++ b/projects/wizard_of_wikipedia/interactive_retrieval/interactive_retrieval.py
@@ -109,6 +109,8 @@ def _set_up_responder(self, opt):
             'legacy': True,
             'no_cuda': True,
             'encode_candidate_vecs': True,
+            'batchsize': 1,
+            'interactive_mode': True,
         }
         for k, v in override_opts.items():
             responder_opts[k] = v
diff --git a/projects/wizard_of_wikipedia/mturk_evaluation_task/worlds.py b/projects/wizard_of_wikipedia/mturk_evaluation_task/worlds.py
--- a/projects/wizard_of_wikipedia/mturk_evaluation_task/worlds.py
+++ b/projects/wizard_of_wikipedia/mturk_evaluation_task/worlds.py
@@ -143,7 +143,7 @@ def __init__(
         opt,
         agents=None,
         shared=None,
-        range_turn=[3, 5],
+        range_turn=(3, 5),
         max_turn=5,
         max_resp_time=120,
         model_agent_opt=None,
@@ -246,7 +246,7 @@ def parley(self):
                 }
                 self.model_agent.observe(chosen_act)
                 model_act = self.model_agent.act()
-                model_act['id'] = 'PERSON_2'
+                model_act.force_set('id', 'PERSON_2')
                 self.dialog.append((1, model_act.get('text')))
                 self.eval_agent.observe(model_act)
             else:
@@ -300,8 +300,8 @@ def parley(self):
                     (' !', '!'),
                     ('i ', 'I '),
                 ]:
-                    act['text'] = act['text'].replace(sb_0, sb_1)
-                act['id'] = 'PERSON_2'
+                    act.force_set('text', act['text'].replace(sb_0, sb_1))
+                act.force_set('id', 'PERSON_2')
                 # NOTE: your model may or may not need to observe itself here
                 # If it does, call model_observes_itself or some other specialized
                 # function


Test patch: