Embedding index getting out of range while running onlplab/alephbert-base model
Author: MatanBenChorin
Pull request: no

## Environment info
- Platform: Google colab 

## Information

Model I am using: 'onlplab/alephbert-base'

The problem arises when using:
 [* ] the official example scripts: 
I tried to implement the code written in your official course (https://huggingface.co/course/chapter7/7?fw=pt)
When the only change I want to make is to replace the model with an  'onlplab/alephbert-base'.
I mean I want to do a finetuning question answering to the  'onlplab/alephbert-base'.

The tasks I am working on is:
 [ *] an official task: SQUaD


## To reproduce

Steps to reproduce the behavior:

1.Run the script:

      
      !pip install datasets transformers[sentencepiece]
      !pip install accelerate
      !apt install git-lfs
      
      from huggingface_hub import notebook_login
      from datasets import load_dataset
      from transformers import AutoTokenizer,BertTokenizerFast
      from transformers import AutoModelForQuestionAnswering, BertForQuestionAnswering
      import collections
      import numpy as np
      from datasets import load_metric
      from tqdm.auto import tqdm
      from transformers import TrainingArguments
      from transformers import Trainer
      from transformers import pipeline
      import torch
      
      raw_datasets = load_dataset("squad")
      raw_datasets["train"].filter(lambda x: len(x["answers"]["text"]) != 1)
      
      # model_checkpoint = "bert-base-cased"
      model_checkpoint = "onlplab/alephbert-base"
      
      # tokenizer = AutoTokenizer.from_pretrained(model_checkpoint)
      tokenizer = BertTokenizerFast.from_pretrained(model_checkpoint)
      
      context = raw_datasets["train"][0]["context"]
      question = raw_datasets["train"][0]["question"]
      
      inputs = tokenizer(question, context)
      tokenizer.decode(inputs["input_ids"])
      
      max_length = 384
      stride = 128
      
      
      def preprocess_training_examples(examples):
          questions = [q.strip() for q in examples["question"]]
          inputs = tokenizer(
              questions,
              examples["context"],
              max_length=max_length,
              truncation="only_second",
              stride=stride,
              return_overflowing_tokens=True,
              return_offsets_mapping=True,
              padding="max_length",
          )
      
          offset_mapping = inputs.pop("offset_mapping")
          sample_map = inputs.pop("overflow_to_sample_mapping")
          answers = examples["answers"]
          start_positions = []
          end_positions = []
      
          for i, offset in enumerate(offset_mapping):
              sample_idx = sample_map[i]
              answer = answers[sample_idx]
              start_char = answer["answer_start"][0]
              end_char = answer["answer_start"][0] + len(answer["text"][0])
              sequence_ids = inputs.sequence_ids(i)
      
              # Find the start and end of the context
              idx = 0
              while sequence_ids[idx] != 1:
                  idx += 1
              context_start = idx
              while sequence_ids[idx] == 1:
                  idx += 1
              context_end = idx - 1
      
              # If the answer is not fully inside the context, label is (0, 0)
              if offset[context_start][0] > end_char or offset[context_end][1] < start_char:
                  start_positions.append(0)
                  end_positions.append(0)
              else:
                  # Otherwise it's the start and end token positions
                  idx = context_start
                  while idx <= context_end and offset[idx][0] <= start_char:
                      idx += 1
                  start_positions.append(idx - 1)
      
                  idx = context_end
                  while idx >= context_start and offset[idx][1] >= end_char:
                      idx -= 1
                  end_positions.append(idx + 1)
      
          inputs["start_positions"] = start_positions
          inputs["end_positions"] = end_positions
          return inputs
      
      train_dataset = raw_datasets["train"].map(
          preprocess_training_examples,
          batched=True,
          remove_columns=raw_datasets["train"].column_names,
      )
      len(raw_datasets["train"]), len(train_dataset)
      
      def preprocess_validation_examples(examples):
          questions = [q.strip() for q in examples["question"]]
          inputs = tokenizer(
              questions,
              examples["context"],
              max_length=max_length,
              truncation="only_second",
              stride=stride,
              return_overflowing_tokens=True,
              return_offsets_mapping=True,
              padding="max_length",
          )
      
          sample_map = inputs.pop("overflow_to_sample_mapping")
          example_ids = []
      
          for i in range(len(inputs["input_ids"])):
              sample_idx = sample_map[i]
              example_ids.append(examples["id"][sample_idx])
      
              sequence_ids = inputs.sequence_ids(i)
              offset = inputs["offset_mapping"][i]
              inputs["offset_mapping"][i] = [
                  o if sequence_ids[k] == 1 else None for k, o in enumerate(offset)
              ]
      
          inputs["example_id"] = example_ids
          return inputs
      
      validation_dataset = raw_datasets["validation"].map(
          preprocess_validation_examples,
          batched=True,
          remove_columns=raw_datasets["validation"].column_names,
      )
      len(raw_datasets["validation"]), len(validation_dataset)
      
      
      metric = load_metric("squad")
      
      n_best = 20
      max_answer_length = 30
      
      def compute_metrics(start_logits, end_logits, features, examples):
          example_to_features = collections.defaultdict(list)
          for idx, feature in enumerate(features):
              example_to_features[feature["example_id"]].append(idx)
      
          predicted_answers = []
          for example in tqdm(examples):
              example_id = example["id"]
              context = example["context"]
              answers = []
      
              # Loop through all features associated with that example
              for feature_index in example_to_features[example_id]:
                  start_logit = start_logits[feature_index]
                  end_logit = end_logits[feature_index]
                  offsets = features[feature_index]["offset_mapping"]
      
                  start_indexes = np.argsort(start_logit)[-1 : -n_best - 1 : -1].tolist()
                  end_indexes = np.argsort(end_logit)[-1 : -n_best - 1 : -1].tolist()
                  for start_index in start_indexes:
                      for end_index in end_indexes:
                          # Skip answers that are not fully in the context
                          if offsets[start_index] is None or offsets[end_index] is None:
                              continue
                          # Skip answers with a length that is either < 0 or > max_answer_length
                          if (
                              end_index < start_index
                              or end_index - start_index + 1 > max_answer_length
                          ):
                              continue
      
                          answer = {
                              "text": context[offsets[start_index][0] : offsets[end_index][1]],
                              "logit_score": start_logit[start_index] + end_logit[end_index],
                          }
                          answers.append(answer)
      
              # Select the answer with the best score
              if len(answers) > 0:
                  best_answer = max(answers, key=lambda x: x["logit_score"])
                  predicted_answers.append(
                      {"id": example_id, "prediction_text": best_answer["text"]}
                  )
              else:
                  predicted_answers.append({"id": example_id, "prediction_text": ""})
      
          theoretical_answers = [{"id": ex["id"], "answers": ex["answers"]} for ex in examples]
          return metric.compute(predictions=predicted_answers, references=theoretical_answers)
      
      # model = AutoModelForQuestionAnswering.from_pretrained(model_checkpoint)
      model = BertForQuestionAnswering.from_pretrained(model_checkpoint)
      
      args = TrainingArguments(
          "alephbert-finetuned-squad",
          evaluation_strategy="no",
          save_strategy="epoch",
          learning_rate=2e-5,
          num_train_epochs=3,
          weight_decay=0.01,
          fp16=False,
          push_to_hub=False, #TODO enable it if we eant push to hub
      ) 
      
      trainer = Trainer(
          model=model,
          args=args,
          train_dataset=train_dataset,
          eval_dataset=validation_dataset,
          tokenizer=tokenizer,
      )
      trainer.train() #TODO enable it if we want to train
      
      predictions, _ = trainer.predict(validation_dataset)
      start_logits, end_logits = predictions
      compute_metrics(start_logits, end_logits, validation_dataset, raw_datasets["validation"])
      
      # Save the model
      output_dir = "aleph_bert-finetuned-squad"
      trainer.save_model(output_dir)
      
      
      # Replace this with your own checkpoint
      #model_checkpoint = "huggingface-course/bert-finetuned-squad"
      model_checkpoint=output_dir
      question_answerer = pipeline("question-answering", model=model_checkpoint)
      
      context = """
      🤗 Transformers is backed by the three most popular deep learning libraries — Jax, PyTorch and TensorFlow — with a seamless integration
      between them. It's straightforward to train your models with one before loading them for inference with the other.
      """
      question = "Which deep learning libraries back 🤗 Transformers?"
      question_answerer(question=question, context=context)
      
 
2. The problem arises on trainer.train() 
3. If i change the model back to Bert the code runs well.
4. I dont know what is the problem

Thanks

## Expected behavior
from:  torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
IndexError: index out of range in self


        
        --> 211 trainer.train() #TODO enable it if we want to train
            212 
            213 predictions, _ = trainer.predict(validation_dataset)
        
        11 frames
        [/usr/local/lib/python3.7/dist-packages/torch/nn/functional.py](https://localhost:8080/#) in embedding(input, weight, padding_idx, max_norm, norm_type, scale_grad_by_freq, sparse)
           2042         # remove once script supports set_grad_enabled
           2043         _no_grad_embedding_renorm_(weight, input, max_norm, norm_type)
        -> 2044     return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
           2045 
           2046 
        
        IndexError: index out of range in self
        


[Question_answering_(PyTorch).zip](https://github.com/huggingface/transformers/files/8142807/Question_answering_.PyTorch.zip)



Comments:
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.