tiiuae

/

falcon-7b-instruct

like 876

Hi there! Using the 7b instruct model, i used the code provided in model card, saved as main.py and when i run it, i keep receiving this error after 3-5 mins processing, any ideas of what may be wrong?

C:\Users\veilsidebr/.cache\huggingface\modules\transformers_modules\tiiuae\falcon-7b-instruct\b6 │
│ efaea5d78e4313145bda4d688675414a76fc22\modelling_RW.py:478 in _convert_to_rw_cache │
│ │
│ 475 │ def _convert_to_rw_cache( │
│ 476 │ │ past_key_value: Tuple[Tuple[torch.Tensor, torch.Tensor]] │
│ 477 │ ) -> Tuple[Tuple[torch.Tensor, torch.Tensor]]: │
│ ❱ 478 │ │ batch_size, num_heads, head_dim, seq_length = past_key_value[0][0].shape │
│ 479 │ │ batch_size_times_num_heads = batch_size * num_heads │
│ 480 │ │ # key: [batch_size, num_heads, head_dim, seq_length] -> [batch_size * num_heads │
│ 481 │ │ # value: [batch_size, num_heads, seq_length, head_dim] -> [batch_size * num_head │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
ValueError: not enough values to unpack (expected 4, got 3)
PS C:\Models\Falcon-7B>