andrewrreed
/

falcon-7b-guanaco-qlora-arr

generated_from_trainer

Model card Files Files and versions Metrics Training metrics Community

falcon-7b-guanaco-qlora-arr / adapter_config.json

andrewrreed's picture

andrewrreed HF staff

mid run manual checkpoint

b49b212 11 months ago

raw history blame contribute delete

No virus

464 Bytes

	{
	"base_model_name_or_path": "tiiuae/falcon-7b",
	"bias": "none",
	"fan_in_fan_out": false,
	"inference_mode": true,
	"init_lora_weights": true,
	"layers_pattern": null,
	"layers_to_transform": null,
	"lora_alpha": 16,
	"lora_dropout": 0.1,
	"modules_to_save": null,
	"peft_type": "LORA",
	"r": 64,
	"revision": null,
	"target_modules": [
	"query_key_value",
	"dense",
	"dense_h_to_4h",
	"dense_4h_to_h"
	],
	"task_type": "CAUSAL_LM"
	}