Spaces:
Running
on
CPU Upgrade
Running
on
CPU Upgrade
add multi_xpu configuration
#5
by
faaany
- opened
code_samples/training_configuration/multi_xpu
ADDED
@@ -0,0 +1,42 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
##
|
2 |
+
<pre>
|
3 |
+
compute_environment: LOCAL_MACHINE
|
4 |
+
debug: false
|
5 |
+
distributed_type: MULTI_XPU
|
6 |
+
downcast_bf16: 'no'
|
7 |
+
enable_cpu_affinity: false
|
8 |
+
gpu_ids: 0,1,2,3
|
9 |
+
ipex_config:
|
10 |
+
ipex: true
|
11 |
+
machine_rank: 0
|
12 |
+
main_training_function: main
|
13 |
+
mixed_precision: 'no'
|
14 |
+
num_machines: 1
|
15 |
+
num_processes: 4
|
16 |
+
rdzv_backend: static
|
17 |
+
same_network: true
|
18 |
+
tpu_env: []
|
19 |
+
tpu_use_cluster: false
|
20 |
+
tpu_use_sudo: false
|
21 |
+
use_cpu: false</pre>
|
22 |
+
##
|
23 |
+
None
|
24 |
+
##
|
25 |
+
If the YAML was generated through the `accelerate config` command:
|
26 |
+
```
|
27 |
+
accelerate launch {script_name.py} {--arg1} {--arg2} ...
|
28 |
+
```
|
29 |
+
|
30 |
+
If the YAML is saved to a `~/config.yaml` file:
|
31 |
+
```
|
32 |
+
accelerate launch --config_file ~/config.yaml {script_name.py} {--arg1} {--arg2} ...
|
33 |
+
```
|
34 |
+
|
35 |
+
##
|
36 |
+
Launching on multi-XPU instances requires a different launch command than just `python myscript.py`. Accelerate will wrap around the proper launching script to delegate and call, reading in how to set their configuration based on the parameters passed in. It is a passthrough to the `torchrun` command.
|
37 |
+
|
38 |
+
**Remember that you can always use the `accelerate launch` functionality, even if the code in your script does not use the `Accelerator`**
|
39 |
+
##
|
40 |
+
To learn more checkout the related documentation:
|
41 |
+
- <a href="https://huggingface.co/docs/accelerate/main/en/basic_tutorials/launch" target="_blank">Launching distributed code</a>
|
42 |
+
- <a href="https://huggingface.co/docs/accelerate/main/en/package_reference/cli" target="_blank">The Command Line</a>
|