Hugging Face's logo

Mrw33554432
/
bitLinear-phi-1.5

Text Generation
Transformers
Safetensors
phi
custom_code
Inference Endpoints
text-generation-inference
Model card Files Community
1
Mrw33554432 commited on
Commit
d013df6
•
1 Parent(s): a18a3b8

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -1
README.md CHANGED
@@ -11,7 +11,9 @@ Our BitLinear layer will only apply 1-bit quantization to the weight, all other
11
 
12
  The model structure is from [phi-1.5](https://huggingface.co/microsoft/phi-1_5), with all linear layers except lm_head replaced with our custom BitLinear layer.
13
 
14
- It was trained on a small subset of the [wikipedia dataset](https://huggingface.co/datasets/wikipedia) dataset, for research validation purpose only.
 
 
15
 
16
  ```python
17
  dataset = load_dataset("wikipedia", "20220301.en")
 
11
 
12
  The model structure is from [phi-1.5](https://huggingface.co/microsoft/phi-1_5), with all linear layers except lm_head replaced with our custom BitLinear layer.
13
 
14
+ It was trained on a small subset of the [wikipedia dataset](https://huggingface.co/datasets/wikipedia) dataset, for research validation purpose only.
15
+
16
+ Please notice the kernel is not optimzed for 1-bit matrix yet.
17
 
18
  ```python
19
  dataset = load_dataset("wikipedia", "20220301.en")