报错：[2024-06-13 23:19:24,422] torch.distributed.run: [WARNING] 
[2024-06-13 23:19:24,422] torch.distributed.run: [WARNING] *****************************************
[2024-06-13 23:19:24,422] torch.distributed.run: [WARNING] Setting OMP_NUM_THREADS environment variable for each process to be 1 in default, to avoid your system being overloaded, please further tune the variable for optimal performance in your application as needed. 
[2024-06-13 23:19:24,422] torch.distributed.run: [WARNING] *****************************************
usage: train.py [-h] [--script SCRIPT] [--config CONFIG] [--save_dir SAVE_DIR] [--mode {single,multiple,multi_node}] --nproc_per_node NPROC_PER_NODE [--use_lmdb {0,1}] [--script_prv SCRIPT_PRV]
                [--config_prv CONFIG_PRV] [--use_wandb {0,1}] [--distill {0,1}] [--script_teacher SCRIPT_TEACHER] [--config_teacher CONFIG_TEACHER] [--rank RANK] [--world-size WORLD_SIZE] [--ip IP] [--port PORT]
train.py: error: the following arguments are required: --nproc_per_node
usage: train.py [-h] [--script SCRIPT] [--config CONFIG] [--save_dir SAVE_DIR] [--mode {single,multiple,multi_node}] --nproc_per_node NPROC_PER_NODE [--use_lmdb {0,1}] [--script_prv SCRIPT_PRV]
                [--config_prv CONFIG_PRV] [--use_wandb {0,1}] [--distill {0,1}] [--script_teacher SCRIPT_TEACHER] [--config_teacher CONFIG_TEACHER] [--rank RANK] [--world-size WORLD_SIZE] [--ip IP] [--port PORT]
train.py: error: the following arguments are required: --nproc_per_node
usage: train.py [-h] [--script SCRIPT] [--config CONFIG] [--save_dir SAVE_DIR] [--mode {single,multiple,multi_node}] --nproc_per_node NPROC_PER_NODE [--use_lmdb {0,1}] [--script_prv SCRIPT_PRV]
                [--config_prv CONFIG_PRV] [--use_wandb {0,1}] [--distill {0,1}] [--script_teacher SCRIPT_TEACHER] [--config_teacher CONFIG_TEACHER] [--rank RANK] [--world-size WORLD_SIZE] [--ip IP] [--port PORT]
train.py: error: the following arguments are required: --nproc_per_node
[2024-06-13 23:19:29,433] torch.distributed.elastic.multiprocessing.api: [ERROR] failed (exitcode: 2) local_rank: 0 (pid: 136781) of binary: /home/heyuting/anaconda3/envs/mamba/bin/python
Traceback (most recent call last):
  File "/home/heyuting/anaconda3/envs/mamba/bin/torchrun", line 8, in <module>
    sys.exit(main())
  File "/home/heyuting/anaconda3/envs/mamba/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/__init__.py", line 346, in wrapper
    return f(*args, **kwargs)
  File "/home/heyuting/anaconda3/envs/mamba/lib/python3.10/site-packages/torch/distributed/run.py", line 806, in main
    run(args)
  File "/home/heyuting/anaconda3/envs/mamba/lib/python3.10/site-packages/torch/distributed/run.py", line 797, in run
    elastic_launch(
  File "/home/heyuting/anaconda3/envs/mamba/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 134, in __call__
    return launch_agent(self._config, self._entrypoint, list(args))
  File "/home/heyuting/anaconda3/envs/mamba/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 264, in launch_agent
    raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError: 
============================================================
tracking/train.py FAILED
------------------------------------------------------------
Failures:
[1]:
  time      : 2024-06-13_23:19:29
  host      : server12
  rank      : 1 (local_rank: 1)
  exitcode  : 2 (pid: 136782)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
[2]:
  time      : 2024-06-13_23:19:29
  host      : server12
  rank      : 2 (local_rank: 2)
  exitcode  : 2 (pid: 136783)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
------------------------------------------------------------
Root Cause (first observed failure):
[0]:
  time      : 2024-06-13_23:19:29
  host      : server12
  rank      : 0 (local_rank: 0)
  exitcode  : 2 (pid: 136781)
  error_file: <N/A>
  traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html
============================================================