Kaihui commited on
Commit
2549f5d
1 Parent(s): 39ceb0b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +18 -1
README.md CHANGED
@@ -16,6 +16,23 @@ tags:
16
  - neural-compressor
17
  ---
18
  # INT8 GPT-J 6B
 
 
19
  GPT-J 6B is a transformer model trained using Ben Wang's [Mesh Transformer JAX](https://github.com/kingoflolz/mesh-transformer-jax/). "GPT-J" refers to the class of model, while "6B" represents the number of trainable parameters.
20
 
21
- This int8 PyTorch model is generated by [neural-compressor](https://github.com/intel/neural-compressor).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
16
  - neural-compressor
17
  ---
18
  # INT8 GPT-J 6B
19
+
20
+ ## Model Description
21
  GPT-J 6B is a transformer model trained using Ben Wang's [Mesh Transformer JAX](https://github.com/kingoflolz/mesh-transformer-jax/). "GPT-J" refers to the class of model, while "6B" represents the number of trainable parameters.
22
 
23
+ This int8 PyTorch model is generated by [intel-extension-for-transformers](https://github.com/intel/intel-extension-for-transformers).
24
+ | Package | Version |
25
+ |----------------------|------------|
26
+ | intel-extension-for-transformers| a4aba8ddb07c9b744b6ac106502ec059e0c47960 |
27
+ | neural-compressor | 2.4.1 |
28
+ | torch | 2.1.0+cpu |
29
+ | intel-extension-for-pytorch | 2.1.0 |
30
+ | transformers | 4.32.0 |
31
+
32
+ ## Evaluation results
33
+ Evaluating the accuracy of the optimized model of gpt-j-6b using the lambada_openai dataset in lm_eval.
34
+
35
+ | Dtype | Dataset | Precision |
36
+ |------ |--------|--------|
37
+ | FP32 |Lambada_openai  | 0.6831 |
38
+ | INT8 |Lambada_openai  | 0.6835 |