|
--- |
|
pipeline_tag: text-generation |
|
inference: true |
|
widget: |
|
- text: 'Hello!' |
|
example_title: Hello world |
|
group: Python |
|
library_name: transformers |
|
--- |
|
|
|
This model is randomly initialized, using the config from [tiiuae/falcon-40b](https://huggingface.co/tiiuae/falcon-40b) but with smaller size. |
|
Note: |
|
- The model uses "new architecture" in Falcon-40b. |
|
- The model is in float16. |
|
|
|
Codes: |
|
```python |
|
import transformers |
|
from optimum.intel.openvino import OVModelForCausalLM |
|
import torch |
|
import os |
|
from huggingface_hub import create_repo, upload_folder |
|
|
|
source_model_id = 'tiiuae/falcon-40b' |
|
save_path = '/tmp/yujiepan/falcon-new-tiny64-random' |
|
repo_id = 'yujiepan/falcon-new-tiny64-random' |
|
|
|
config = transformers.AutoConfig.from_pretrained( |
|
source_model_id, trust_remote_code=True) |
|
config.hidden_size = 64 |
|
config.num_attention_heads = 2 |
|
config.num_hidden_layers = 1 |
|
config.num_kv_heads = 2 |
|
config.torch_dtype = torch.float16 |
|
|
|
model = transformers.AutoModelForCausalLM.from_config( |
|
config, trust_remote_code=True) |
|
model = model.half() |
|
model.save_pretrained(save_path) |
|
|
|
tokenizer = transformers.AutoTokenizer.from_pretrained( |
|
source_model_id, trust_remote_code=True) |
|
tokenizer.save_pretrained(save_path) |
|
|
|
# current not supported, might add this later |
|
# ovmodel = OVModelForCausalLM.from_pretrained( |
|
# save_path, export=True, trust_remote_code=True) |
|
# ovmodel.save_pretrained(save_path) |
|
|
|
os.system(f'ls -alh {save_path}') |
|
create_repo(repo_id, exist_ok=True) |
|
upload_folder(repo_id=repo_id, folder_path=save_path) |
|
``` |