Encoder-Decoder NMT Refactor
State: closed | Pull request: yes

This PR will refactor the `TransformerMTModel` so that pre-trained HF and Megatron encoders and decoders can easily be used along with the `TransformerEncoder` and `TransformerDecoder` for NMT.

# Plan for this PR
- rnd -> rnd

# Plan for future PRs
- pretrained encoder -> rnd
- pretrained encoder -> pretrained decoder

# TODO
- [ ] Organize machine translation naming and folder structure
- [x] Jenkins test
- [ ] Unit Tests
- [ ] Instantiate tokenizers with hydra

# Usage
```
python examples/nlp/machine_translation/enc_dec_nmt.py \
        --config-path=${CONFIG_PATH} \
        --config-name=${CONFIG_NAME} \
        model.max_generation_delta=10 \
        model.train_ds.src_file_name=/raid/data/68792/train.clean.en.shuffled.dev \
        model.train_ds.tgt_file_name=/raid/data/68792/train.clean.de.shuffled.dev \
        model.train_ds.tokens_in_batch=8192 \
        model.validation_ds.src_file_name=/raid/data/68792/wmt14-en-de.src \
        model.validation_ds.tgt_file_name=/raid/data/68792/wmt14-en-de.ref \
        model.validation_ds.tokens_in_batch=1024 \
        model.test_ds.src_file_name=/raid/data/68792/wmt14-en-de.src \
        model.test_ds.tgt_file_name=/raid/data/68792/wmt14-en-de.ref \
        model.encoder_tokenizer.tokenizer_model=/raid/data/68792/tokenizer.BPE.37K.model \
        model.decoder_tokenizer.tokenizer_model=/raid/data/68792/tokenizer.BPE.37K.model \
        trainer.gpus=4 \
        trainer.val_check_interval=1.0 \
```

This pull request **introduces 4 alerts** when merging 17d0dd296bac97e6824925ddabcacee075b7fe8b into cfc84cebd98d92fa06089cdd42da685e60521d26 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-d88619b7202d9dc56b09659798de28b51248b54a)

**new alerts:**

* 4 for Unused import

This pull request **introduces 8 alerts** when merging d858915a912e78f218bab6cc94c4ffd7f26e8942 into cfc84cebd98d92fa06089cdd42da685e60521d26 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-fac0620f18e4c7816b09ac9699617a22d3891dc8)

**new alerts:**

* 8 for Unused import

This pull request **introduces 15 alerts** when merging 25e710ba74a4cf3fe8de1ee91e4da5692f15322d into cfc84cebd98d92fa06089cdd42da685e60521d26 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-cb1027ef7da23281742250f5ec617c752b3e94b7)

**new alerts:**

* 14 for Unused import
* 1 for Unused local variable

This pull request **introduces 17 alerts** when merging 9b8c49661679715d867d3f9d3dbd1d66f8164019 into ddfae2e8988cd4d518e241daad1a75660735f2ba - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-e37798307f4f0826ba5748ae1440065385f20778)

**new alerts:**

* 16 for Unused import
* 1 for Unused local variable

This pull request **introduces 17 alerts** when merging b0a4ad0d051cfcece9f0093025a9d80c336b718f into ddfae2e8988cd4d518e241daad1a75660735f2ba - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-7cad3cd5bf38ad1c85e4352af7bf04e08d90dede)

**new alerts:**

* 16 for Unused import
* 1 for Unused local variable

This pull request **introduces 17 alerts** when merging 6f5888bd158b94ea2701f4f6292bdd81ed4479ca into ddfae2e8988cd4d518e241daad1a75660735f2ba - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-d5afc5df3bbd364e9ef00b343147958b04770cd2)

**new alerts:**

* 16 for Unused import
* 1 for Unused local variable

This pull request **introduces 20 alerts** when merging 8cfa7020797a2ccfde516e79baba2d98fa346419 into ddfae2e8988cd4d518e241daad1a75660735f2ba - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-b01d9f00d0e8fd6c8f44a2004610d745d27ea7c7)

**new alerts:**

* 17 for Unused import
* 3 for Unused local variable

This pull request **introduces 21 alerts** when merging 35bd74eefd05d4c3285447e766dc9a043de1d24d into 631eaf9845d77708d6922dae5b5ac17805d19026 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-8076f782ed762815182fd9292e00ebf374b410d8)

**new alerts:**

* 20 for Unused import
* 1 for Unused local variable

This pull request **introduces 21 alerts** when merging 24752737851d1897dde01e83367358aaad13f201 into 631eaf9845d77708d6922dae5b5ac17805d19026 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-979784d5ef428b35c4cb2cc901b29e2105339368)

**new alerts:**

* 20 for Unused import
* 1 for Unused local variable

This pull request **introduces 21 alerts** when merging bac2a86c8be4f0251c22957a8ef33ed4e3e628ab into 631eaf9845d77708d6922dae5b5ac17805d19026 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-3ac6b004473f9efcd8ddec8353d3b80b2a967b81)

**new alerts:**

* 20 for Unused import
* 1 for Unused local variable

This pull request **introduces 23 alerts** when merging 8ccd8187fbf801e26e8ade3c2a64676dc3e13197 into 631eaf9845d77708d6922dae5b5ac17805d19026 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-c1d536eda4b30d51018ecc4a725c606e46c3510d)

**new alerts:**

* 20 for Unused import
* 2 for Wrong name for an argument in a call
* 1 for Unused local variable

This pull request **introduces 20 alerts** when merging 6a0437917fbf0a08a763281d4875f4ef15975ca7 into 631eaf9845d77708d6922dae5b5ac17805d19026 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-3b152ab26ffa3cba325d89ef133b2b87dc604a83)

**new alerts:**

* 20 for Unused import

This pull request **introduces 20 alerts** when merging 98a5ae1e06d88733e57f9e5422091d7e5371c007 into 631eaf9845d77708d6922dae5b5ac17805d19026 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-e5913671bedb07fc4205107d78658d964ecd6243)

**new alerts:**

* 20 for Unused import

This pull request **introduces 14 alerts** when merging 9b4f476a33127d676daffd1d671ac27c5505b55b into 631eaf9845d77708d6922dae5b5ac17805d19026 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-6f66a05ee92cb7395f1a5624e5b6ef4de1c82936)

**new alerts:**

* 14 for Unused import

This pull request **introduces 14 alerts** when merging 5eb4d8817cbc13d694f4ecce94dd1a8f627b0b9a into 631eaf9845d77708d6922dae5b5ac17805d19026 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-66d5ffd00bdaeb24916cc8a46cabc8abedb36627)

**new alerts:**

* 14 for Unused import

This pull request **introduces 16 alerts** when merging 0625b5fc64bec78371876ca975fd1446c8fc1e1b into 96ccc9dd0effd784ba3d31298b14785f8e9a93db - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-9b31984501b38705e04a63a26d5dfe966747b95a)

**new alerts:**

* 16 for Unused import

This pull request **introduces 16 alerts** when merging cc591b64ef98835bdbba149d2a8a8ed5f7aadeba into 96ccc9dd0effd784ba3d31298b14785f8e9a93db - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-be5cbcb7239b94c1dcad7c31b4990bf7c799a773)

**new alerts:**

* 16 for Unused import

This pull request **introduces 19 alerts** when merging 4a4a662aaba4b5357c59e44d07b998a82b90fbc9 into cf6d49c7d84b2edb2c48e9ef09fa088f8379a0c3 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-3d5aad3b34e2c27949cc7281b3508552fd29b907)

**new alerts:**

* 19 for Unused import

This pull request **introduces 19 alerts** when merging 750d615c8c8c905864016c6acfd326d1b2da97b4 into cf6d49c7d84b2edb2c48e9ef09fa088f8379a0c3 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-f0d280028b2cf66308279a76e1afe843d7683a3d)

**new alerts:**

* 19 for Unused import

This pull request **introduces 19 alerts** when merging 37df969755128a7190dcac724e8150772176f658 into cf6d49c7d84b2edb2c48e9ef09fa088f8379a0c3 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-952910ee1718de6add68587c6793c576669e19b5)

**new alerts:**

* 19 for Unused import

This pull request **introduces 18 alerts** when merging e81229acad79f89d1918598a9a0c9263b5bc0851 into cf6d49c7d84b2edb2c48e9ef09fa088f8379a0c3 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-596f499013fbd64035308cccf7788d72977a3c25)

**new alerts:**

* 18 for Unused import

This pull request **introduces 1 alert** when merging d1b56f60e3f42fc0dedfa57cb0aa437be6d3a6f4 into cf6d49c7d84b2edb2c48e9ef09fa088f8379a0c3 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-89439b9c71975e32c138c2be0ae9ff20fbf1b95a)

**new alerts:**

* 1 for Unused import

This pull request **introduces 12 alerts** when merging 8111e38ea99ffcd056e5c2c8f1f377adf7c22353 into cf6d49c7d84b2edb2c48e9ef09fa088f8379a0c3 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-5dd19041e5d3723dec2ae64bbe16fd75f64bac1b)

**new alerts:**

* 12 for Unused import

This pull request **introduces 5 alerts** when merging 08966b703988212246ef3943f5acf09bedd4983e into c1330f384a9d4a986cc48ad94f079e39190fd49b - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-d6bb2c644884ed551798475f6a4e284c52402cb7)

**new alerts:**

* 5 for Unused import

This pull request **introduces 2 alerts** when merging 1711319c67ad3513f45df07b45cdaa6095d8a0df into c1330f384a9d4a986cc48ad94f079e39190fd49b - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-61d27e0f98acf155410b3827bf82564e7a683297)

**new alerts:**

* 2 for Unused import

This pull request **introduces 2 alerts** when merging 88021aea642061ed9c16d7d2771edd82b7ac6598 into c1330f384a9d4a986cc48ad94f079e39190fd49b - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-4dcdf0277a67d6b2f8f33d47f8432d003c03164c)

**new alerts:**

* 2 for Unused import