Add a fix for single-GPU nsys.
State: closed | Pull request: yes
Labels: name: Run CICD

The SingleDeviceStrategy never sets the requisite attribute; workaround it in the profiling code.

# What does this PR do ?

Fixes using the nsys callback for single-GPU benchmarking.

**Collection**: core

# Changelog 
- Fix for cuda profiling of single-GPU runs

# Usage

```python
   nsyscb = nl.pytorch.callbacks.nsys.NsysCallback(
     start_step=0,
     end_step=40,
     ranks=[0],
     gen_shape=False,
   )
  llm.api.finetune(..., callbacks=[nsyscb])
```
is how I am using it.

# GitHub Actions CI

The Jenkins CI system has been replaced by GitHub Actions self-hosted runners.

The GitHub Actions CI will run automatically when the "Run CICD" label is added to the PR.
To re-run CI remove and add the label again.
To run CI on an untrusted fork, a NeMo user with write access must first click "Approve and run".

# Before your PR is "Ready for review"
**Pre checks**:
- [x] Make sure you read and followed [Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md)
- [ ] Did you write any new necessary tests? *Not sure how to test this. Let's first get some confirmation that this is even the right approach*
- [ ] Did you add or update any necessary documentation? *Let's first get some confirmation that this is even the right approach*
- [ ] Does the PR affect components that are optional to install? (Ex: Numba, Pynini, Apex etc) N/A
  - [ ] Reviewer: Does the PR have correct import guards for all optional libraries?
  
**PR Type**:
- [ ] New Feature
- [x] Bugfix
- [ ] Documentation

If you haven't finished some of the above items you can still open "Draft" PR.


## Who can review?

Anyone in the community is free to review the PR once the checks have passed. 
[Contributor guidelines](https://github.com/NVIDIA/NeMo/blob/main/CONTRIBUTING.md) contains specific people who can review PRs to various areas.

# Additional Information
* Related to # (issue)

@tfogal NeMo commits need to have --signoff can you amend and push again? In the meantime I've started CI since it takes a few hours to complete. If CI has passed before you push we can ask someone to force merge if needed.

The `Isort and ...` issues are apparently coming up because the source branch is in another repo:

```
  Warning: Unable to find merge base between 34f740822a11802c2f0cea1c8039163b2b8bff92 and ecc8a1f3a2d532bf771746ed5a81367acf866da9
  Error: Unable to locate the commit sha: 34f740822a11802c2f0cea1c8039163b2b8bff92
  Error: Please verify that the commit sha is correct, and increase the 'fetch_depth' input if needed
  Warning: fatal: bad object 34f740822a11802c2f0cea1c8039163b2b8bff92
  
  Warning: If this pull request is from a forked repository, please set the checkout action `repository` input to the same repository as the pull request.
  Warning: This can be done by setting actions/checkout `repository` to ${{ github.event.pull_request.head.repo.full_name }}
  Error: Unable to determine a difference between 34f740822a11802c2f0cea1c8039163b2b8bff92..ecc8a1f3a2d532bf771746ed5a81367acf866da9
```

This is not something I can fix, though, since I can't make branches in NeMo.

Unrelatedly, marking ready for review. I still question whether this is the *correct* fix but it's definitely an improvement and we could always refine.

<!-- pylint-output-strict-mode-false -->

beep boop 🤖: 🙏 The following files have warnings. In case you are familiar with these, please try helping us to improve the code base.

---

Your code was analyzed with PyLint. The following annotations have been identified:

```

------------------------------------
Your code has been rated at 10.00/10
```

---

Thank you for improving NeMo's documentation!

@pablo-garay https://github.com/NVIDIA/NeMo/pull/11354#issuecomment-2498721924 any solution?