NLP Update to Model Parallel
State: closed | Pull request: yes

PTL significantly changed their API which broke our existing model parallel support.

This PR updates `NLPModel` so that model parallel training and auto-resume works again.

# TODO:

- Add model parallel support to .nemo files.

This pull request **introduces 1 alert** when merging d5f866692e69b78a5565dd635e396521ebecbd67 into 91146cf360415e9656e62c635b48ad639b1b45af - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-a6442172376caa13bb8863b03862186b4e24cc86)

**new alerts:**

* 1 for Unused import

This pull request **introduces 1 alert** when merging 00fed41bdbdd9eb4fc955af647ec4a1dd4f9e731 into 91146cf360415e9656e62c635b48ad639b1b45af - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-b5ddf66a90906faf3a185779453eb1f91747d55b)

**new alerts:**

* 1 for Unused import

This pull request **introduces 1 alert** when merging 472a28dfaab024e4482513001c214de86f57769e into 91146cf360415e9656e62c635b48ad639b1b45af - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-6f080e2f767e616f56d728929f095d8b0bf0e951)

**new alerts:**

* 1 for Unused import

This pull request **introduces 1 alert** when merging d76389a45d60d479dcd2762af651bda1c07f87e4 into 91146cf360415e9656e62c635b48ad639b1b45af - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-9761851acbb4a9f8f36290a3dcaf07290b0c6e0f)

**new alerts:**

* 1 for Unused import

This pull request **introduces 1 alert** when merging f3f232e5bc91449ef5adc2f740c952352f87e9ff into 19af99bc487aeafa091bc5e8901919f0904f8cdb - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-6818890be86776575324f89deefdc5fb0eccba71)

**new alerts:**

* 1 for Unused import

This pull request **introduces 1 alert** when merging 63d783df6b838b80bd1f9b34f15fff7b6d8027bf into 19af99bc487aeafa091bc5e8901919f0904f8cdb - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-b57b931e9ffed948bf2f98ee6c773c48d1034a94)

**new alerts:**

* 1 for Unused import

This pull request **introduces 1 alert** when merging 51d302ca066cf57217f351213c5adef9a813616f into 19af99bc487aeafa091bc5e8901919f0904f8cdb - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-936321f8ef7862cbc8b13c48442ba22ab52c82e0)

**new alerts:**

* 1 for Unused import

This pull request **introduces 1 alert** when merging 5e2ee7b2268bc2367c46f156fe77e187d2ffb264 into 19af99bc487aeafa091bc5e8901919f0904f8cdb - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-7ffe27a5669017e30f173fdba287ba4cc49b7138)

**new alerts:**

* 1 for Unused import

This pull request **introduces 1 alert** when merging 62b6dd4201308a642bc4fa7da39c724f82ab596f into 19af99bc487aeafa091bc5e8901919f0904f8cdb - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-cb184c85ad5630f8c510750591039cc0cd1ced5b)

**new alerts:**

* 1 for Unused import

This pull request **introduces 1 alert** when merging 78245dd858b28c45189fcd9e6b325e07f4c96476 into 4bcb2321cd16835f63afe3dfe993e6d56bcf2c0c - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-886eb38056d6fb503030edeb79864b6fc8b558de)

**new alerts:**

* 1 for Unused import

This pull request **introduces 1 alert** when merging 70852f1a84d70a4f28257f85fcb364470febc5e6 into 4bcb2321cd16835f63afe3dfe993e6d56bcf2c0c - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-348c3ebf285a19fc689c5bf856c93f9a34e3e0fa)

**new alerts:**

* 1 for Unused import

This pull request **introduces 2 alerts** when merging b9724dc5b6b41c94d909be88b74f80c437b396ae into df0cbdb278e7e916c37d43126a6746870868f17e - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-59a3b39ed5ef209db4754d99b8a1d8f64e0771cd)

**new alerts:**

* 2 for Unused import

This pull request **introduces 5 alerts** when merging e4e8f0ca4830608f346875d493a4fb45a8a8c26c into 360eb0422f53f1d3bb41fa941a44591bba4d7de9 - [view on LGTM.com](https://lgtm.com/projects/g/NVIDIA/NeMo/rev/pr-4508cbe5ae5363157b52fdd19b59385c58ba7573)

**new alerts:**

* 3 for Unused import
* 1 for Except block handles &#39;BaseException&#39;
* 1 for Module is imported with &#39;import&#39; and &#39;import from&#39;

Close in favor of #2015