regisss HF staff jwieczorekhabana commited on
Commit
b593bc0
1 Parent(s): ac388e7

Remove hmp from gaudi_config.json and README (#1)

Browse files

- Update gaudi_config.json (625264ef30d7eb5dd14a34e8e9c88738be07c72e)
- Update README.md (3efb9a31752c18411d9edc99f0e69366a65ea92b)
- Update gaudi_config.json (5184fd0ce2e5ebdc14e081cf9c0c50d79ec12849)
- Update README.md (6e1e0c091f00fd9e3d46f91c87a9e20b3bf30ea9)
- Update README.md (bf55e34199d11075e338178b5c323d100844979d)


Co-authored-by: Jan Wieczorek <jwieczorekhabana@users.noreply.huggingface.co>

Files changed (2) hide show
  1. README.md +6 -8
  2. gaudi_config.json +1 -25
README.md CHANGED
@@ -13,18 +13,15 @@ This model only contains the `GaudiConfig` file for running the [Swin Transforme
13
  **This model contains no model weights, only a GaudiConfig.**
14
 
15
  This enables to specify:
16
- - `use_habana_mixed_precision`: whether to use Habana Mixed Precision (HMP)
17
- - `hmp_opt_level`: optimization level for HMP, see [here](https://docs.habana.ai/en/latest/PyTorch/PyTorch_Mixed_Precision/PT_Mixed_Precision.html#configuration-options) for a detailed explanation
18
- - `hmp_bf16_ops`: list of operators that should run in bf16
19
- - `hmp_fp32_ops`: list of operators that should run in fp32
20
- - `hmp_is_verbose`: verbosity
21
  - `use_fused_adam`: whether to use Habana's custom AdamW implementation
22
  - `use_fused_clip_norm`: whether to use Habana's fused gradient norm clipping operator
23
-
 
24
  ## Usage
25
 
26
  The model is instantiated the same way as in the Transformers library.
27
- The only difference is that there are a few new training arguments specific to HPUs.
 
28
 
29
  [Here](https://github.com/huggingface/optimum-habana/blob/main/examples/image-classification/run_image_classification.py) is an image classification example script to fine-tune a model. You can run it with Swin with the following command:
30
  ```bash
@@ -48,7 +45,8 @@ python run_image_classification.py \
48
  --use_lazy_mode \
49
  --gaudi_config_name Habana/swin \
50
  --throughput_warmup_steps 2 \
51
- --ignore_mismatched_sizes
 
52
  ```
53
 
54
  Check the [documentation](https://huggingface.co/docs/optimum/habana/index) out for more advanced usage and examples.
13
  **This model contains no model weights, only a GaudiConfig.**
14
 
15
  This enables to specify:
 
 
 
 
 
16
  - `use_fused_adam`: whether to use Habana's custom AdamW implementation
17
  - `use_fused_clip_norm`: whether to use Habana's fused gradient norm clipping operator
18
+ - `use_torch_autocast`: whether to use Torch Autocast for managing mixed precision
19
+
20
  ## Usage
21
 
22
  The model is instantiated the same way as in the Transformers library.
23
+ The only difference is that there are a few new training arguments specific to HPUs.\
24
+ It is strongly recommended to train this model doing bf16 mixed-precision training for optimal performance and accuracy.
25
 
26
  [Here](https://github.com/huggingface/optimum-habana/blob/main/examples/image-classification/run_image_classification.py) is an image classification example script to fine-tune a model. You can run it with Swin with the following command:
27
  ```bash
45
  --use_lazy_mode \
46
  --gaudi_config_name Habana/swin \
47
  --throughput_warmup_steps 2 \
48
+ --ignore_mismatched_sizes \
49
+ --bf16
50
  ```
51
 
52
  Check the [documentation](https://huggingface.co/docs/optimum/habana/index) out for more advanced usage and examples.
gaudi_config.json CHANGED
@@ -1,29 +1,5 @@
1
  {
2
- "use_habana_mixed_precision": true,
3
- "hmp_is_verbose": false,
4
  "use_fused_adam": true,
5
  "use_fused_clip_norm": true,
6
- "hmp_bf16_ops": [
7
- "add",
8
- "addmm",
9
- "bmm",
10
- "dropout",
11
- "gelu",
12
- "iadd",
13
- "linear",
14
- "layer_norm",
15
- "matmul",
16
- "mm",
17
- "rsub",
18
- "softmax",
19
- "truediv",
20
- "avg_pool2d",
21
- "conv2d"
22
- ],
23
- "hmp_fp32_ops": [
24
- "nll_loss",
25
- "log_softmax",
26
- "embedding",
27
- "cross_entropy"
28
- ]
29
  }
1
  {
 
 
2
  "use_fused_adam": true,
3
  "use_fused_clip_norm": true,
4
+ "use_torch_autocast": true
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
5
  }