Repository: facebookresearch/ParlAI

Issue:
Model goes out of sync when using pre-trained word embeddings in multiprocessing_train
**Bug description**
Using pretrained word embeddings does not work for `multiprocessing_train.py`

**Reproduction steps**
When training the model(seq2seq, transformer/generator) with `multiprocessing_train.py`,
select any pretrained word embeddings(`glove`, `fasttext` etc.) other than `random`.
Then the model's parameters go out of sync.

**Expected behavior**
Needs to be trained properly. For a single GPU, it works fine. However, in the case of multiGPUs, it does not work.

**Logs**
Please paste the command line output:

```
❯❯❯ python -m parlai.scripts.multiprocessing_train -t convai2 -m transformer/generator -emb fasttext_cc -mf ./tmp/tg_fasttext_2gpu
/data/sunlok/workspace/parlai_practice/ParlAI/parlai/agents/transformer/modules.py:32: UserWarning: Installing APEX can give a significant speed boost.
  warn_once("Installing APEX can give a significant speed boost.")
[ Main ParlAI Arguments: ]
[  batchsize: 1 ]
[  datapath: /data/sunlok/workspace/parlai_practice/ParlAI/data ]
[  datatype: train ]
[  download_path: /data/sunlok/workspace/parlai_practice/ParlAI/downloads ]
[  hide_labels: False ]
[  image_mode: raw ]
[  init_opt: None ]
[  multitask_weights: [1] ]
[  numthreads: 1 ]
[  show_advanced_args: False ]
[  task: convai2 ]
[ ParlAI Model Arguments: ]
[  dict_class: parlai.core.dict:DictionaryAgent ]
[  init_model: None ]
[  model: transformer/generator ]
[  model_file: ./tmp/tg_fasttext_2gpu ]
[ Training Loop Arguments: ]
[  aggregate_micro: False ]
[  dict_build_first: True ]
[  display_examples: False ]
[  eval_batchsize: None ]
[  evaltask: None ]
[  load_from_checkpoint: False ]
[  max_train_time: -1 ]
[  metrics: default ]
[  num_epochs: -1 ]
[  save_after_valid: False ]
[  save_every_n_secs: -1 ]
[  short_final_eval: False ]
[  validation_cutoff: 1.0 ]
[  validation_every_n_epochs: -1 ]
[  validation_every_n_secs: 210 ]
[  validation_max_exs: 1024 ]
[  validation_metric: ppl ]
[  validation_metric_mode: min ]
[  validation_patience: 12 ]
[  validation_share_agent: False ]
[ Tensorboard Arguments: ]
[  tensorboard_log: False ]
[ PytorchData Arguments: ]
[  batch_length_range: 5 ]
[  batch_sort_cache_type: pop ]
[  batch_sort_field: text ]
[  numworkers: 4 ]
[  pytorch_context_length: -1 ]
[  pytorch_datapath: None ]
[  pytorch_include_labels: True ]
[  pytorch_preprocess: False ]
[  pytorch_teacher_batch_sort: False ]
[  pytorch_teacher_dataset: None ]
[  pytorch_teacher_task: None ]
[  shuffle: False ]
[ Dictionary Loop Arguments: ]
[  dict_include_test: False ]
[  dict_include_valid: True ]
[  dict_maxexs: -1 ]
[  log_every_n_secs: 10 ]
[ Distributed Training: ]
[  distributed_world_size: 2 ]
[  verbose: False ]
[ ParlAI Image Preprocessing Arguments: ]
[  image_cropsize: 224 ]
[  image_size: 256 ]
[ Transformer Arguments: ]
[  activation: relu ]
[  attention_dropout: 0.0 ]
[  dropout: 0.0 ]
[  embedding_size: 300 ]
[  embeddings_scale: True ]
[  ffn_size: 300 ]
[  learn_positional_embeddings: False ]
[  n_heads: 2 ]
[  n_layers: 2 ]
[  n_positions: None ]
[  n_segments: 0 ]
[  output_scaling: 1.0 ]
[  relu_dropout: 0.0 ]
[  share_word_embeddings: True ]
[  variant: aiayn ]
[ Torch Generator Agent: ]
[  beam_min_length: 1 ]
[  beam_min_n_best: 3 ]
[  beam_size: 1 ]
[  inference: greedy ]
[  skip_generation: False ]
[  topk: 10 ]
[  topp: 0.9 ]
[ TorchAgent Arguments: ]
[  add_p1_after_newln: False ]
[  delimiter:
]
[  embedding_projection: random ]
[  embedding_type: fasttext_cc ]
[  fp16: False ]
[  gpu: -1 ]
[  history_size: -1 ]
[  interactive_mode: False ]
[  label_truncate: None ]
[  no_cuda: False ]
[  person_tokens: False ]
[  rank_candidates: False ]
[  split_lines: False ]
[  text_truncate: None ]
[  truncate: -1 ]
[  use_reply: label ]
[ Optimizer Arguments: ]
[  adam_eps: 1e-08 ]
[  betas: (0.9, 0.999) ]
[  gradient_clip: 0.1 ]
[  learningrate: 1 ]
[  momentum: 0 ]
[  nesterov: True ]
[  nus: (0.7,) ]
[  optimizer: sgd ]
[  weight_decay: None ]
[ Learning Rate Scheduler: ]
[  lr_scheduler: reduceonplateau ]
[  lr_scheduler_decay: 0.5 ]
[  lr_scheduler_patience: 3 ]
[  update_freq: 1 ]
[  warmup_rate: 0.0001 ]
[  warmup_updates: -1 ]
[ Dictionary Arguments: ]
[  bpe_debug: False ]
[  dict_endtoken: __end__ ]
[  dict_file: /data/sunlok/workspace/parlai_practice/ParlAI/data/models/controllable_dialogue/dict_twit30k_train_split ]
[  dict_initpath: None ]
[  dict_language: english ]
[  dict_lower: True ]
[  dict_max_ngram_size: -1 ]
[  dict_maxtokens: -1 ]
[  dict_minfreq: 0 ]
[  dict_nulltoken: __null__ ]
[  dict_starttoken: __start__ ]
[  dict_textfields: text,labels ]
[  dict_tokenizer: re ]
[  dict_unktoken: __unk__ ]
[ Current ParlAI commit: e8d0a75d291c7bb4b4e5565d60a899f794c10963 ]
Distributed group initialized
[ building dictionary first... ]
[ dictionary already built .]
[ no model with opt yet at: ./tmp/tg_fasttext_2gpu(.opt) ]

[ Using CUDA ]
Dictionary: loading dictionary from /data/sunlok/workspace/parlai_practice/ParlAI/data/models/controllable_dialogue/dict_twit30k_train_split
/data/sunlok/workspace/parlai_practice/ParlAI/parlai/agents/transformer/modules.py:32: UserWarning: Installing APEX can give a significant speed boost.
  warn_once("Installing APEX can give a significant speed boost.")
[ num words =  35213 ]
[TransformerGenerator: full interactive mode on.]
Initialized embeddings for 32438 tokens (92.1%) from fasttext_cc.
Traceback (most recent call last):
  File "/home/sunlok/.miniconda3/envs/parlai_practice/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/home/sunlok/.miniconda3/envs/parlai_practice/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/data/sunlok/workspace/parlai_practice/ParlAI/parlai/scripts/multiprocessing_train.py", line 150, in <module>
    main()
  File "/data/sunlok/workspace/parlai_practice/ParlAI/parlai/scripts/multiprocessing_train.py", line 146, in main
    return launch_and_train(opt, port)
  File "/data/sunlok/workspace/parlai_practice/ParlAI/parlai/scripts/multiprocessing_train.py", line 110, in launch_and_train
    retval = multiprocess_train(0, opt, port)
  File "/data/sunlok/workspace/parlai_practice/ParlAI/parlai/scripts/multiprocessing_train.py", line 94, in multiprocess_train
    return single_train.TrainLoop(opt).train()
  File "/data/sunlok/workspace/parlai_practice/ParlAI/parlai/scripts/train_model.py", line 376, in __init__
    self.agent = create_agent(opt)
  File "/data/sunlok/workspace/parlai_practice/ParlAI/parlai/core/agents.py", line 694, in create_agent
    model = model_class(opt)
  File "/data/sunlok/workspace/parlai_practice/ParlAI/parlai/core/torch_generator_agent.py", line 346, in __init__
    check_synced_parameters(self.model)
  File "/data/sunlok/workspace/parlai_practice/ParlAI/parlai/core/distributed_utils.py", line 250, in check_synced_parameters
    " ".join(str(x) for x in all_versions)
AssertionError: Some models parameters were out of sync. Got the following norms: 1096695.0 355013.28125
```

**Additional context**
Add any other context about the problem here. (like proxy settings, network setup, overall goals, etc.)



Discussion:
Boy do they! Thanks for filing.

Resolving patch:
diff --git a/parlai/core/torch_agent.py b/parlai/core/torch_agent.py
--- a/parlai/core/torch_agent.py
+++ b/parlai/core/torch_agent.py
@@ -1172,10 +1172,6 @@ def _copy_embeddings(self, weight, emb_type, log=True):
         :param emb_type:
             pretrained embedding type
         """
-        if not is_primary_worker():
-            # we're in distributed mode, copying embeddings in the workers
-            # slows things down considerably
-            return
         embs, name = self._get_embtype(emb_type)
         cnt = 0
         for w, i in self.dict.tok2ind.items():


Test patch: