regisss HF staff commited on
Commit
a05195e
1 Parent(s): d2bd057

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +40 -0
README.md CHANGED
@@ -1,3 +1,43 @@
1
  ---
2
  license: apache-2.0
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
  ---
4
+
5
+ [Optimum Habana](https://github.com/huggingface/optimum-habana) is the interface between the Transformers library and Habana's Gaudi processor (HPU). It provides a set of tools enabling easy and fast model loading and fine-tuning on single- and multi-HPU settings for different downstream tasks.
6
+ Learn more about how to take advantage of the power of Habana HPUs to train Transformers models at [hf.co/hardware/habana](https://huggingface.co/hardware/habana).
7
+
8
+ ## Swin Transformer model HPU configuration
9
+
10
+ This model contains just the `GaudiConfig` file for running the [Swin Transformer](https://huggingface.co/microsoft/swin-base-patch4-window7-224-in22k) model on Habana's Gaudi processors (HPU).
11
+
12
+ **This model contains no model weights, only a GaudiConfig.**
13
+
14
+ This enables to specify:
15
+ - `use_habana_mixed_precision`: whether to use Habana Mixed Precision (HMP)
16
+ - `hmp_opt_level`: optimization level for HMP, see [here](https://docs.habana.ai/en/latest/PyTorch/PyTorch_User_Guide/PT_Mixed_Precision.html#configuration-options) for a detailed explanation
17
+ - `hmp_bf16_ops`: list of operators that should run in bf16
18
+ - `hmp_fp32_ops`: list of operators that should run in fp32
19
+ - `hmp_is_verbose`: verbosity
20
+ - `use_fused_adam`: whether to use Habana's custom AdamW implementation
21
+ - `use_fused_clip_norm`: whether to use Habana's fused gradient norm clipping operator
22
+
23
+ ## Usage
24
+
25
+ The model is instantiated the same way as in the Transformers library.
26
+ The only difference is that there are a few new training arguments specific to HPUs:
27
+
28
+ ```
29
+ from optimum.habana import GaudiTrainer, GaudiTrainingArguments
30
+ from transformers import SwinForImageClassification
31
+ model = SwinForImageClassification.from_pretrained('microsoft/swin-base-patch4-window7-224-in22k')
32
+ args = GaudiTrainingArguments(
33
+ output_dir="/tmp/output_dir",
34
+ use_habana=True,
35
+ use_lazy_mode=True,
36
+ gaudi_config_name="Habana/swin",
37
+ )
38
+ trainer = GaudiTrainer(
39
+ model=model,
40
+ args=args,
41
+ )
42
+ trainer.train()
43
+ ```