add multi_xpu configuration

#5
by faaany - opened
code_samples/training_configuration/multi_xpu ADDED
@@ -0,0 +1,42 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ##
2
+ <pre>
3
+ compute_environment: LOCAL_MACHINE
4
+ debug: false
5
+ distributed_type: MULTI_XPU
6
+ downcast_bf16: 'no'
7
+ enable_cpu_affinity: false
8
+ gpu_ids: 0,1,2,3
9
+ ipex_config:
10
+ ipex: true
11
+ machine_rank: 0
12
+ main_training_function: main
13
+ mixed_precision: 'no'
14
+ num_machines: 1
15
+ num_processes: 4
16
+ rdzv_backend: static
17
+ same_network: true
18
+ tpu_env: []
19
+ tpu_use_cluster: false
20
+ tpu_use_sudo: false
21
+ use_cpu: false</pre>
22
+ ##
23
+ None
24
+ ##
25
+ If the YAML was generated through the `accelerate config` command:
26
+ ```
27
+ accelerate launch {script_name.py} {--arg1} {--arg2} ...
28
+ ```
29
+
30
+ If the YAML is saved to a `~/config.yaml` file:
31
+ ```
32
+ accelerate launch --config_file ~/config.yaml {script_name.py} {--arg1} {--arg2} ...
33
+ ```
34
+
35
+ ##
36
+ Launching on multi-XPU instances requires a different launch command than just `python myscript.py`. Accelerate will wrap around the proper launching script to delegate and call, reading in how to set their configuration based on the parameters passed in. It is a passthrough to the `torchrun` command.
37
+
38
+ **Remember that you can always use the `accelerate launch` functionality, even if the code in your script does not use the `Accelerator`**
39
+ ##
40
+ To learn more checkout the related documentation:
41
+ - <a href="https://huggingface.co/docs/accelerate/main/en/basic_tutorials/launch" target="_blank">Launching distributed code</a>
42
+ - <a href="https://huggingface.co/docs/accelerate/main/en/package_reference/cli" target="_blank">The Command Line</a>