NMT export, _prepare_for_export refactor
State: closed | Pull request: yes

Here, added export tp NMT's encoder and decoder. 
The design decision is is to have those 2 parts explicitly exportable, instead of having whole model's export() produce 2 files. 
Test added. 

This pull request **introduces 4 alerts** when merging 03cb3ef0aad58dcaea2e849989897a94c849dcdc into 4d868b662af0fab87a4ad4078156486885bbb0a4 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-477438a4d8b220885f444f3455bf533d4efff55d)

**new alerts:**

* 2 for Modification of dictionary returned by locals\(\)
* 1 for Signature mismatch in overriding method
* 1 for Unused import

As per @MaximumEntropy suggestion, triangle mask can actually be dropped for inference (not just for export). I have verified that on a 100-sentence set (no diffs).

This pull request **introduces 4 alerts** when merging 87d2435ef7a81a51cc3771ddda6231fad2d933b9 into 4d868b662af0fab87a4ad4078156486885bbb0a4 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-604b80f9af4f4b75ec5f1e8e4fddfba76c75e73a)

**new alerts:**

* 2 for Modification of dictionary returned by locals\(\)
* 1 for Signature mismatch in overriding method
* 1 for Unused import

This pull request **introduces 4 alerts** when merging d14de501f99f55b642cc61aa80c8b39ee1f99af2 into e816905e286df545ce344e865ea6cf8e4b0c85ca - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-a0fadd69ffd0b4f84de7709fc606afd5480bf2c7)

**new alerts:**

* 2 for Modification of dictionary returned by locals\(\)
* 1 for Signature mismatch in overriding method
* 1 for Unused import

This pull request **introduces 4 alerts** when merging d934479f956f5db25dea8a1c5a38b4684deaa87f into e816905e286df545ce344e865ea6cf8e4b0c85ca - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-41c0615a46a7ec0c6eaf82a52f20acec4313b320)

**new alerts:**

* 2 for Modification of dictionary returned by locals\(\)
* 1 for Signature mismatch in overriding method
* 1 for Unused import

This pull request **introduces 4 alerts** when merging 9868414685036be59fa91740da4615a6ba24e3bb into 5d80627ca696384dc16e462f1e17efd33917a9dd - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-f2678947f893bee51552520cb19713b8d0e6632f)

**new alerts:**

* 2 for Modification of dictionary returned by locals\(\)
* 1 for Signature mismatch in overriding method
* 1 for Unused import

@borisfom unittests are failing

> @borisfom unittests are failing

Was there a change in CI machines setup? The tests still work on my machine, and I do not see changes in PT version in Dockerfile.
================================================================================ slowest durations =================================================================================
16.87s call     tests/collections/nlp/test_nmt_model.py::TestMTEncDecModel::test_creation_saving_restoring
10.79s call     tests/collections/nlp/test_nmt_model.py::TestMTEncDecModel::test_cpu_export
10.51s call     tests/collections/nlp/test_nmt_model.py::TestMTEncDecModel::test_gpu_export

(6 durations < 0.005s hidden.  Use -vv to show these durations.)
========================================================================= 3 passed, 20 warnings in 42.35s ==========================================================================
boris@snikolaev-DGXStation:~/git/NeMo$ sudo pip3 list | grep torch
pytorch-lightning             1.2.2
torch                         1.7.1+cu110
torch-stft                    0.1.4
torchaudio                    0.7.2
torchtext                     0.8.0
torchvision                   0.8.2+cu110
trtorch                       0.2.0a0


@borisfom  can you try inside the container? main branch uses 21.02-py3 for CI

This pull request **introduces 4 alerts** when merging 162a4b41dde512b057aad5f990c2344bce2908c1 into 05eb2587a95e512b5e95705d9f77582634abbd95 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-ac780a854af2a49debda5da204bee3d09754fd53)

**new alerts:**

* 2 for Modification of dictionary returned by locals\(\)
* 1 for Signature mismatch in overriding method
* 1 for Unused import

This pull request **introduces 4 alerts** when merging feb2aa98a412a0522af824985c077d0ea6380b94 into 05eb2587a95e512b5e95705d9f77582634abbd95 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-91c724a41e11640e779d8bba07b580a716b769b0)

**new alerts:**

* 2 for Modification of dictionary returned by locals\(\)
* 1 for Signature mismatch in overriding method
* 1 for Unused import

> @borisfom can you try inside the container? main branch uses 21.02-py3 for CI

Right, so I did - also bumped the base in Dockerfile and enabled Megatron export test that works with 21.02

This pull request **introduces 4 alerts** when merging c9c2978935a5618d4452bd69b00c9db35658ada2 into 646aff11aa788b7d874dae930ce6ffcd658c1f16 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-18195fa22e39d5a4c0088a69831cfed2ce88b4a8)

**new alerts:**

* 2 for Modification of dictionary returned by locals\(\)
* 1 for Signature mismatch in overriding method
* 1 for Unused import

> > @borisfom can you try inside the container? main branch uses 21.02-py3 for CI
> 
> Right, so I did - also bumped the base in Dockerfile and enabled Megatron export test that works with 21.02

After fixing Torch compatibility issues with upgraded PT, the CI fails on unrelated test:
cd tools/ctc_segmentation && pip install -r requirements.txt && DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends ffmpeg
test -f /etc/shinit_v2
+ source /etc/shinit_v2
+++ sed -n 's/^NVRM.*Kernel Module *\([^() ]*\).*$/\1/p' /proc/driver/nvidia/version
+++ sed 's/^$/unknown/'
++ NV_DRIVER_VERS=460.39
++ export _CUDA_COMPAT_PATH=/usr/local/cuda/compat
++ _CUDA_COMPAT_PATH=/usr/local/cuda/compat
+++ hostname
++ _CUDA_COMPAT_CHECKFILE=/usr/local/cuda/compat/.460.39.7fb21172b5cf.checked
++ _CUDA_COMPAT_REALLIB=/usr/local/cuda/compat/lib.real
++ _CUDA_COMPAT_SYMLINK=/usr/local/cuda/compat/lib
++ '[' -n 460.39 -a -e /dev/nvidiactl -a '!' -e /usr/local/cuda/compat/.460.39.7fb21172b5cf.checked ']'
+++ cat /sys/module/mlx5_core/version
+++ true
++ DETECTED_MOFED=
++ dpkg --compare-versions 0 lt 4.9
++ export 'OMPI_MCA_pml=^ucx'
++ OMPI_MCA_pml='^ucx'
++ unset _CUDA_COMPAT_CHECKFILE
++ unset _CUDA_COMPAT_REALLIB
++ unset _CUDA_COMPAT_SYMLINK
+ '[' -z '' ']'
+ return
+ cd tools/ctc_segmentation
+ pip install -r requirements.txt
Collecting ctc_segmentation==1.1.0
  Downloading ctc_segmentation-1.1.0.tar.gz (57 kB)
Collecting num2words
  Downloading num2words-0.5.10-py3-none-any.whl (101 kB)
Requirement already satisfied: setuptools in /opt/conda/lib/python3.8/site-packages (from ctc_segmentation==1.1.0->-r requirements.txt (line 1)) (50.3.1.post20201107)
Requirement already satisfied: numpy in /opt/conda/lib/python3.8/site-packages (from ctc_segmentation==1.1.0->-r requirements.txt (line 1)) (1.19.2)
Requirement already satisfied: Cython in /opt/conda/lib/python3.8/site-packages (from ctc_segmentation==1.1.0->-r requirements.txt (line 1)) (0.28.4)
Collecting docopt>=0.6.2
  Downloading docopt-0.6.2.tar.gz (25 kB)
Building wheels for collected packages: ctc-segmentation, docopt
  Building wheel for ctc-segmentation (setup.py): started
  Building wheel for ctc-segmentation (setup.py): finished with status 'done'
  Created wheel for ctc-segmentation: filename=ctc_segmentation-1.1.0-cp38-cp38-linux_x86_64.whl size=120481 sha256=6583c5221f311b58278d169b5ad53ba162917094963459610a404440fb6d6759
  Stored in directory: /root/.cache/pip/wheels/4e/6e/9e/2904d4b4f46b220690572ef09688e6cf248a6340cd2758742c
  Building wheel for docopt (setup.py): started
  Building wheel for docopt (setup.py): finished with status 'done'
  Created wheel for docopt: filename=docopt-0.6.2-py2.py3-none-any.whl size=13704 sha256=09cec6b7a65ec64f9ee6bb37422283e3ed48fa51d054016adb6da1201a39e327
  Stored in directory: /root/.cache/pip/wheels/56/ea/58/ead137b087d9e326852a851351d1debf4ada529b6ac0ec4e8c
Successfully built ctc-segmentation docopt
Installing collected packages: ctc-segmentation, docopt, num2words
Successfully installed ctc-segmentation-1.1.0 docopt-0.6.2 num2words-0.5.10
+ DEBIAN_FRONTEND=noninteractive
+ apt-get install -y --no-install-recommends ffmpeg
Reading package lists...
Building dependency tree...
Reading state information...
E: Unable to locate package ffmpeg
script returned exit code 100
Clearly CI issue - ffmpeg os one of the pre-reqs to install NeMo, mentoned both in README and Dockerfile:
Use this installation mode if you want the latest released version.
.. code-block:: bash
  apt-get update && apt-get install -y libsndfile1 ffmpeg
  pip install Cython
  pip install nemo_toolkit[all]==1.0.0b3


This pull request **introduces 4 alerts** when merging 874f1f65c64abf867643f8f7094cfd1ad32f99c9 into b0f9f94469b1b7bacb10fb1851f8fabfd8b4323d - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-fbdba08fb50cdc68c493f8dca8e7c382bbc5a95a)

**new alerts:**

* 2 for Modification of dictionary returned by locals\(\)
* 1 for Signature mismatch in overriding method
* 1 for Unused import