How do I output in a nice table in the terminal the mapping of the gpu id, the pid and the **username**?

Brando_Miranda: <p>I saw: <a href="https://stackoverflow.com/questions/50264491/how-do-i-customize-nvidia-smi-s-output-to-show-pid-username" class="inline-onebox" rel="noopener nofollow ugc">cuda - How do I customize nvidia-smi 's output to show PID username? - Stack Overflow</a> but doesn’t do what I want. I want the output to look:</p>
<pre><code class="lang-auto">USER        GPU PID    hostname %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
brando9       0 1234   ampere3  ... etc... whatever don't really care
</code></pre>
<p>but instead I see:</p>
<pre><code class="lang-auto">(metalearning_gpu) brando9~ $ nvidia-smi; ps -up `nvidia-smi -q -x | grep pid | sed -e 's/&lt;pid&gt;//g' -e 's/&lt;\/pid&gt;//g' -e 's/^[[:space:]]*//'`; hostname
Mon Feb  6 19:19:59 2023
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 515.43.04    Driver Version: 515.43.04    CUDA Version: 11.7     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|

|   0  NVIDIA A100-SXM...  On   | 00000000:07:00.0 Off |                    0 |
| N/A   31C    P0    67W / 400W |      2MiB / 81920MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
|   1  NVIDIA A100-SXM...  On   | 00000000:0A:00.0 Off |                    0 |
| N/A   28C    P0    61W / 400W |      2MiB / 81920MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
|   2  NVIDIA A100-SXM...  On   | 00000000:44:00.0 Off |                    0 |
| N/A   29C    P0    63W / 400W |      2MiB / 81920MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
|   3  NVIDIA A100-SXM...  On   | 00000000:4A:00.0 Off |                    0 |
| N/A   32C    P0    65W / 400W |      2MiB / 81920MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
|   4  NVIDIA A100-SXM...  On   | 00000000:84:00.0 Off |                    0 |
| N/A   33C    P0    65W / 400W |      2MiB / 81920MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
|   5  NVIDIA A100-SXM...  On   | 00000000:8A:00.0 Off |                    0 |
| N/A   30C    P0    71W / 400W |  66729MiB / 81920MiB |     14%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
|   6  NVIDIA A100-SXM...  On   | 00000000:C0:00.0 Off |                    0 |
| N/A   30C    P0    62W / 400W |      2MiB / 81920MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+
|   7  NVIDIA A100-SXM...  On   | 00000000:C3:00.0 Off |                    0 |
| N/A   32C    P0    64W / 400W |      2MiB / 81920MiB |      0%      Default |
|                               |                      |             Disabled |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    5   N/A  N/A     49854      C   .../envs/a100_env/bin/python    66727MiB |
+-----------------------------------------------------------------------------+
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
kexinh     49854  359  0.3 130510112 6749364 ?   Rsl  18:16 226:30 /dfs/user/kexinh/miniconda3/envs/a100_env/bin/python -m ipykernel_launcher -f /afs/cs.stanford.edu/u/kexinh/.local/share/jupyter/runtime/kernel-bbc9f45e-4513-4643-82c3-0f67dde751
ampere3
</code></pre>
<p>How do I add a column such that I can see the gpu id, pid and user name easily in bash/the terminal?</p>
<p>Even a command using python is fine e.g.</p>
<pre><code class="lang-auto">python -c 'some one liner python script that works'
</code></pre>
<hr>
<p>related:</p>
<ul>
<li>related: <a href="https://stackoverflow.com/questions/50264491/how-do-i-customize-nvidia-smi-s-output-to-show-pid-username?noredirect=1&amp;lq=1" class="inline-onebox" rel="noopener nofollow ugc">cuda - How do I customize nvidia-smi 's output to show PID username? - Stack Overflow</a>
</li>
<li>cross: <a href="https://www.reddit.com/r/nvidia/comments/10vr808/how_do_i_output_in_a_nice_table_in_the_terminal/" rel="noopener nofollow ugc">https://www.reddit.com/r/nvidia/comments/10vr808/how_do_i_output_in_a_nice_table_in_the_terminal/</a>
</li>
</ul>

Brando_Miranda: <p>Given my requirements that I want to display Pid, username, gpu id and process/app name, here is the solution:</p>
<p>Answer:</p>
<pre><code class="lang-auto">(echo "GPU_ID PID UID APP" ; for GPU in 0 1 2 3 ; do for PID in $( nvidia-smi -q --id=${GPU} --display=PIDS | awk '/Process ID/{print $NF}') ; do echo -n "${GPU} ${PID} " ; ps -up ${PID} | awk 'NR-1 {print $1,$NF}' ; done ; done) | column -t
</code></pre>
<p>credit:</p><aside class="onebox allowlistedgeneric" data-onebox-src="https://www.reddit.com/r/HPC/comments/10x9w6x/comment/j7sg7w2/?utm_source=share&amp;utm_medium=web2x&amp;context=3">
  <header class="source">
      <img src="https://www.redditstatic.com/desktop2x/img/favicon/android-icon-192x192.png" class="site-icon" width="192" height="192">

      <a href="https://www.reddit.com/r/HPC/comments/10x9w6x/comment/j7sg7w2/?utm_source=share&amp;utm_medium=web2x&amp;context=3" target="_blank" rel="noopener nofollow ugc">reddit</a>
  </header>

  <article class="onebox-body">
    <img src="https://external-preview.redd.it/QgPvRTknlY3rMNDqH1k4I37XGiq9tZF_FsygC_Xht4o.jpg?auto=webp&amp;v=enabled&amp;s=19b4a59f036ea2f314ff2033c11e54cdc240f8d8" class="thumbnail onebox-avatar" width="316" height="316">

<h3><a href="https://www.reddit.com/r/HPC/comments/10x9w6x/comment/j7sg7w2/?utm_source=share&amp;utm_medium=web2x&amp;context=3" target="_blank" rel="noopener nofollow ugc">r/HPC - How do I output in a nice table in the terminal the mapping of the...</a></h3>

  <p>3 votes and 2 comments so far on Reddit</p>


  </article>

  <div class="onebox-metadata">
    
    
  </div>

  <div style="clear: both"></div>
</aside>