Repository: facebookresearch/ParlAI

Issue:
Error in training: module 'parlai.tasks.fromfile.agents' has no attribute 'create_agents'
**Bug description**
Error while trying to train model.

**Reproduction steps**
Tried to train model with these args: `parlai train_model -t fromfile:parlaiformat --fromfile_datapath ./parlai_utilities/covid_qa+FAQ_parlaiformat.txt -m transformer/generator --multitask-weights 1 --num-epochs 10.0 --beam-block-ngram 3 --beam-context-block-ngram 3 --init-model zoo:blender/blender_90M/model --dict-file zoo:blender/blender_90M/model.dict --embedding-size 512 --n-layers 8 --ffn-size 2048 --dropout 0.1 --n-heads 16 --learn-positional-embeddings True  --n-positions 512 --variant xlm --activation gelu --skip-generation True --fp16 True --text-truncate 256 --label-truncate 256 --dict-tokenizer bpe --dict-lower True -lr 1e-06 --optimizer adamax --lr-scheduler reduceonplateau --gradient-clip 0.1 -veps 0.25 --betas 0.9,0.999 --update-freq 1 --attention-dropout 0.0 --relu-dropout 0.0 --skip-generation True -vp 15 -stim 60 -vme 20000 -bs 32 -vmt ppl -vmm min --save-after-valid True --model-file  covidmodel1`

**Logs**

```
  File "/ParlAI/parlai/core/worlds.py", line 1581, in _create_task_agents
    task_agents = my_module.create_agents(opt)  # type: ignore
AttributeError: module 'parlai.tasks.fromfile.agents' has no attribute 'create_agents'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/parlai", line 33, in <module>
    sys.exit(load_entry_point('parlai', 'console_scripts', 'parlai')())
  File "/ParlAI/parlai/core/script.py", line 272, in superscript_main
    SCRIPT_REGISTRY[cmd].klass._run_from_parser_and_opt(opt, parser)
  File "/ParlAI/parlai/core/script.py", line 88, in _run_from_parser_and_opt
    return script.run()
  File "/ParlAI/parlai/scripts/train_model.py", line 768, in run
    self.train_loop = TrainLoop(self.opt)
  File "/ParlAI/parlai/scripts/train_model.py", line 279, in __init__
    self.world = create_task(opt, self.agent)
  File "/ParlAI/parlai/core/worlds.py", line 1633, in create_task
    world = create_task_world(opt, user_agents, default_world=default_world)
  File "/ParlAI/parlai/core/worlds.py", line 1596, in create_task_world
    task_agents = _create_task_agents(opt)
  File "/ParlAI/parlai/core/worlds.py", line 1584, in _create_task_agents
    return create_task_agent_from_taskname(opt)
  File "/ParlAI/parlai/core/teachers.py", line 2402, in create_task_agent_from_taskname
    task_agents = teacher_class(opt)
  File "/ParlAI/parlai/tasks/fromfile/agents.py", line 87, in __init__
    self._setup_data(datafile)
  File "/ParlAI/parlai/core/teachers.py", line 1394, in _setup_data
    for line_no, line in enumerate(read, 1):
  File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 2204: ordinal not in range(128)
```



Discussion:
I think your environment is set to non-utf8 encodings.

Short term fix: add these variables (or similar ones) to your bash environment
```
LANG=en_US.UTF-8
PYTHONIOENCODING=utf-8
```

The real fix: this line needs to explicitly say opening with utf-8.
https://github.com/facebookresearch/ParlAI/blob/0fe83071f796c73930c79761d14d7b27070baced/parlai/core/teachers.py#L1393

Would you be willing to make a PR to add `encoding='utf-8'` to that call to `open`?
Thanks for the quick answer.
I tried to do a PR but I'm very new to all this and I got stuck because pre-commit 'failed to find interpreter for Builtin discover of python_spec='python3.6' while committing and now I'm not able to go deeper into the problem, I'm sorry.
Thanks for your attempt.

Resolving patch:
diff --git a/parlai/core/teachers.py b/parlai/core/teachers.py
--- a/parlai/core/teachers.py
+++ b/parlai/core/teachers.py
@@ -981,10 +981,10 @@ def load_length(self):
             for episode in self._read_episode(self.data_loader(self.datafile)):
                 num_eps += 1
                 num_exs += len(episode)
-            with open(length_file, 'w') as f:
+            with open(length_file, 'w', encoding="utf-8") as f:
                 f.write("{}\n{}".format(num_eps, num_exs))
         else:
-            with open(length_file, 'r') as f:
+            with open(length_file, 'r', encoding="utf-8") as f:
                 num_eps, num_exs = f.readlines()
         return int(num_eps), int(num_exs)
 
@@ -1148,7 +1148,7 @@ def load_cands(self, path):
         lines_have_ids = False
         cands_are_replies = False
         cnt = 0
-        with open(path) as read:
+        with open(path, encoding="utf-8") as read:
             for line in read:
                 line = line.strip().replace('\\n', '\n')
                 if len(line) > 0:
@@ -1203,7 +1203,7 @@ def setup_data(self, path):
             new_episode = False (this is the second example in the episode)
         """
         logging.info(f"loading fbdialog data: {path}")
-        with open(path) as read:
+        with open(path, encoding="utf-8") as read:
             start = True
             x = ''
             reward = 0
@@ -1390,7 +1390,7 @@ def _setup_data(self, path):
         self.episodes = []
         self.num_exs = 0
         eps = []
-        with open(path, newline='\n') as read:
+        with open(path, newline='\n', encoding="utf-8") as read:
             for line_no, line in enumerate(read, 1):
                 msg = str_to_msg(line.rstrip('\n'))
                 if msg and 'eval_labels' in msg:
@@ -1796,7 +1796,7 @@ def load_data(self, data_path, opt):
         data_file = os.path.join(self.data_path, '%s.json' % dt)
 
         # Load the text data and image number indexes
-        with open(data_file) as f:
+        with open(data_file, encoding="utf-8") as f:
             self.data = json.load(f)
 
         if len(self.data) > 0 and self.image_id_key not in self.data[0]:


Test patch: