Попытался перевести модель в формат awq, но встретил ошибку KeyError: 'self_attn.q_proj' в процессе работы.
Однако вижу, что в конфиге для peft данный слой присутствует. базовый yarn переводится в этот формат без проблем. Подскажите, в чем может быть проблема?
Далее прилагаю код перевода и полный текст ошибки

from awq import AutoAWQForCausalLM
from transformers import AutoTokenizer

model_path = './PavelGPT-7B-128K-v0.1-LoRA/'
quant_path = 'PavelGPT-7B-128K-v0.1-awq'
quant_config = { "zero_point": True, "q_group_size": 128, "w_bit": 4, "version": "GEMM" }

Load model

model = AutoAWQForCausalLM.from_pretrained(model_path)
tokenizer = AutoTokenizer.from_pretrained(model_path, trust_remote_code=True)

Quantize

model.quantize(tokenizer, quant_config=quant_config)

Save quantized model

model.save_quantized(quant_path)
tokenizer.save_pretrained(quant_path)

Token indices sequence length is longer than the specified maximum sequence length for this model (77696 > 32768). Running this sequence through the model will result in indexing errors
AWQ: 0%| | 0/32 [00:02<?, ?it/s]
Traceback (most recent call last):
File "/home/epimachok/awq/qawq.py", line 13, in
model.quantize(tokenizer, quant_config=quant_config)
File "/home/epimachok/anaconda3/envs/autoawq/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "/home/epimachok/anaconda3/envs/autoawq/lib/python3.10/site-packages/awq/models/base.py", line 93, in quantize
quantizer.quantize()
File "/home/epimachok/anaconda3/envs/autoawq/lib/python3.10/site-packages/awq/quantize/quantizer.py", line 99, in quantize
module_config: List[Dict] = self.awq_model.get_layers_for_scaling(
File "/home/epimachok/anaconda3/envs/autoawq/lib/python3.10/site-packages/awq/models/mistral.py", line 46, in get_layers_for_scaling
inp=input_feat['self_attn.q_proj'],
KeyError: 'self_attn.q_proj'