Enable ucc backend for pp
State: closed | Pull request: yes
Labels: name: stale

name: NLP

# What does this PR do ?

This MR provides an interface to enable the UCC backend for PP communication.
To enable the UCC backend, set the following argument:

`training.model.pipeline_model_parallel_comm_backend=ucc`

Requires a related MCore MR ([link](https://gitlab-master.nvidia.com/ADLR/megatron-lm/-/merge_requests/2116)).


<!-- pylint-output-strict-mode-false -->

beep boop 🤖: 🙏 The following files have warnings. In case you are familiar with these, please try helping us to improve the code base.

---

Your code was analyzed with PyLint. The following annotations have been identified:

```
************* Module nemo.collections.nlp.models.language_modeling.megatron_base_model
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:103:0: C0301: Line too long (149/119) (line-too-long)
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:111:0: C0301: Line too long (134/119) (line-too-long)
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:253:0: C0301: Line too long (125/119) (line-too-long)
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:271:0: C0301: Line too long (120/119) (line-too-long)
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:346:0: C0301: Line too long (133/119) (line-too-long)
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:606:0: C0301: Line too long (123/119) (line-too-long)
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:661:0: C0301: Line too long (153/119) (line-too-long)
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:833:0: C0301: Line too long (122/119) (line-too-long)
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:961:0: C0301: Line too long (121/119) (line-too-long)
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:980:0: C0301: Line too long (136/119) (line-too-long)
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:1097:0: C0301: Line too long (195/119) (line-too-long)
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:1154:0: C0301: Line too long (135/119) (line-too-long)
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:1202:0: C0301: Line too long (120/119) (line-too-long)
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:1285:0: C0301: Line too long (128/119) (line-too-long)
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:330:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:461:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:466:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:699:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:751:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:922:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:1026:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:1260:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:1291:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:1295:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/nlp/models/language_modeling/megatron_base_model.py:1299:4: C0116: Missing function or method docstring (missing-function-docstring)
************* Module nemo.collections.nlp.modules.common.megatron.megatron_init
nemo/collections/nlp/modules/common/megatron/megatron_init.py:265:0: C0301: Line too long (122/119) (line-too-long)
nemo/collections/nlp/modules/common/megatron/megatron_init.py:266:0: C0301: Line too long (133/119) (line-too-long)
nemo/collections/nlp/modules/common/megatron/megatron_init.py:267:0: C0301: Line too long (121/119) (line-too-long)
nemo/collections/nlp/modules/common/megatron/megatron_init.py:88:0: C0116: Missing function or method docstring (missing-function-docstring)
************* Module nemo.collections.nlp.parts.nlp_overrides
nemo/collections/nlp/parts/nlp_overrides.py:207:0: C0301: Line too long (140/119) (line-too-long)
nemo/collections/nlp/parts/nlp_overrides.py:212:0: C0301: Line too long (149/119) (line-too-long)
nemo/collections/nlp/parts/nlp_overrides.py:228:0: C0301: Line too long (123/119) (line-too-long)
nemo/collections/nlp/parts/nlp_overrides.py:420:0: C0301: Line too long (136/119) (line-too-long)
nemo/collections/nlp/parts/nlp_overrides.py:506:0: C0301: Line too long (152/119) (line-too-long)
nemo/collections/nlp/parts/nlp_overrides.py:726:0: C0301: Line too long (140/119) (line-too-long)
nemo/collections/nlp/parts/nlp_overrides.py:731:0: C0301: Line too long (149/119) (line-too-long)
nemo/collections/nlp/parts/nlp_overrides.py:997:0: C0301: Line too long (128/119) (line-too-long)
nemo/collections/nlp/parts/nlp_overrides.py:1001:0: C0301: Line too long (141/119) (line-too-long)
nemo/collections/nlp/parts/nlp_overrides.py:1005:0: C0301: Line too long (149/119) (line-too-long)
nemo/collections/nlp/parts/nlp_overrides.py:1062:0: C0301: Line too long (135/119) (line-too-long)
nemo/collections/nlp/parts/nlp_overrides.py:1189:0: C0301: Line too long (121/119) (line-too-long)
nemo/collections/nlp/parts/nlp_overrides.py:1767:0: C0301: Line too long (152/119) (line-too-long)
nemo/collections/nlp/parts/nlp_overrides.py:243:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/nlp/parts/nlp_overrides.py:376:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/nlp/parts/nlp_overrides.py:421:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/nlp/parts/nlp_overrides.py:598:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/nlp/parts/nlp_overrides.py:614:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/nlp/parts/nlp_overrides.py:633:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/nlp/parts/nlp_overrides.py:874:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/nlp/parts/nlp_overrides.py:993:0: C0115: Missing class docstring (missing-class-docstring)
nemo/collections/nlp/parts/nlp_overrides.py:1678:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/nlp/parts/nlp_overrides.py:1766:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/collections/nlp/parts/nlp_overrides.py:18:0: W0611: Unused import re (unused-import)
nemo/collections/nlp/parts/nlp_overrides.py:106:4: W0611: Unused tensorstore imported from megatron.core.dist_checkpointing.strategies (unused-import)
************* Module nemo.utils.app_state
nemo/utils/app_state.py:424:0: C0301: Line too long (138/119) (line-too-long)
nemo/utils/app_state.py:432:0: C0301: Line too long (135/119) (line-too-long)
nemo/utils/app_state.py:23:0: C0115: Missing class docstring (missing-class-docstring)
nemo/utils/app_state.py:29:0: C0115: Missing class docstring (missing-class-docstring)
nemo/utils/app_state.py:255:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/utils/app_state.py:692:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/utils/app_state.py:702:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/utils/app_state.py:711:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/utils/app_state.py:716:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/utils/app_state.py:722:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/utils/app_state.py:730:4: C0116: Missing function or method docstring (missing-function-docstring)
nemo/utils/app_state.py:738:4: C0116: Missing function or method docstring (missing-function-docstring)

-----------------------------------
Your code has been rated at 9.65/10
```

---

Thank you for improving NeMo's documentation!

This PR is stale because it has been open for 14 days with no activity. Remove stale label or comment or update or this will be closed in 7 days.

This PR was closed because it has been inactive for 7 days since being marked as stale.