text: 
  
  TITLE
  Getting `raise NotImplementedError` for base_model.get_input_embeddings() when upgrading from pytorch-transformers
  
  COMMENTS
  6
  
  REACTIONS
  +1: 0
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  # 📚 Migration
  
  ## Information
  
  <!-- Important information -->
  Getting `raise NotImplementedError` at 
   the lines - https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_utils.py#L474-L477 when I am trying to upgrade my code from pytorch-transformers to transformers
  
  Model I am using (Bert, XLNet ...): Bert
  
  Language I am using the model on (English, Chinese ...): English
  
  The problem arises when using:
  * [ ] the official example scripts: (give details below): Not sure
  * [ ] my own modified scripts: (give details below): Yes
  
  The tasks I am working on is:
  * [ ] an official GLUE/SQUaD task: (give the name) No 
  * [ ] my own task or dataset: (give details below): No
  
  ## Details
  
  <!-- A clear and concise description of the migration issue.
      If you have code snippets, please provide it here as well.
      Important! Use code tags to correctly format your code. See https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks#syntax-highlighting
      Do not use screenshots, as they are hard to read and (more importantly) don't allow others to copy-and-paste your code.
      -->
  I am using Oscar repo (https://github.com/microsoft/Oscar), which uses an older version of Huggingface pytorch-transformers (https://github.com/huggingface/transformers/tree/067923d3267325f525f4e46f357360c191ba562e). I am trying to upgrade the repo to use latest version of transformers (https://github.com/huggingface/transformers). However, I am getting below error : 
  ```
  Traceback (most recent call last):
    File "oscar/run_captioning_airsplay.py", line 1019, in <module>
      main()
    File "oscar/run_captioning_airsplay.py", line 966, in main
      from_tf=bool('.ckpt' in args.model_name_or_path), config=config)
    File "/home/default/ephemeral_drive/work/image_captioning/Oscar_edited/transformers/src/transformers/modeling_utils.py", line 1188, in from_pretrained
      model.tie_weights()
    File "/home/default/ephemeral_drive/work/image_captioning/Oscar_edited/transformers/src/transformers/modeling_utils.py", line 504, in tie_weights
      self._tie_or_clone_weights(output_embeddings, self.get_input_embeddings())
    File "/home/default/ephemeral_drive/work/image_captioning/Oscar_edited/transformers/src/transformers/modeling_utils.py", line 469, in get_input_embeddings
      return base_model.get_input_embeddings()
    File "/home/default/ephemeral_drive/work/image_captioning/Oscar_edited/transformers/src/transformers/modeling_utils.py", line 471, in get_input_embeddings
      raise NotImplementedError
  NotImplementedError
  ```
  The error occurs at this block in the transformers code - https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_utils.py#L474-L477. My code runs fine when I use an older version of hugging face transformers - https://github.com/huggingface/transformers/tree/067923d3267325f525f4e46f357360c191ba562e, possibly, because pytorch-transformers did not have a requirement that `set_input_embeddings()` should be defined for base_model. The base model that I am using is a custom defined model `BertForImageCaptioning` (https://github.com/microsoft/Oscar/blob/df79152b708c3c46f2dc93324776a27406ccc634/oscar/modeling/modeling_bert.py#L604), which has a custom defined parent class ` CaptionPreTrainedModel` (https://github.com/microsoft/Oscar/blob/df79152b708c3c46f2dc93324776a27406ccc634/oscar/modeling/modeling_utils.py#L21), which has a parent class `BertPreTrainedModel`.
  
  I have not seen any mention of how to deal with this issue in the migration documents from Pytorch-transformers or from transformers 3.x. (https://huggingface.co/transformers/migration.html#migrating-from-transformers-v3-x-to-v4-x). 
  
  I have looked into examples to check how to define the function, but this did not give enough details to define the function at my side - https://github.com/huggingface/transformers/blob/master/examples/research_projects/movement-pruning/emmental/modeling_bert_masked.py#L487-L488. 
  
  How should I define the function `get_input_embeddings()` for my use case and what are the guidelines for doing the same. Are there any examples explaining the process of defining the function.  
  
  ## Environment info
  <!-- You can run the command `python transformers-cli env` and copy-and-paste its output below.
       Don't forget to fill out the missing fields in that output! -->
   
  - `transformers` version: https://github.com/huggingface/transformers
  - Platform: x86_64 GNU/Linux
  - Python version: 3.6.8
  - PyTorch version (GPU?):  1.7.0+cu101 (GPU)
  - Tensorflow version (GPU?): 2.3.0 (GPU)
  - Using GPU in script?: yes
  - Using distributed or parallel set-up in script?: No
  
  <!-- IMPORTANT: which version of the former library do you use? -->
  * `pytorch-transformers` or `pytorch-pretrained-bert` version (or branch): https://github.com/huggingface/transformers/tree/067923d3267325f525f4e46f357360c191ba562e
  
  
  ## Checklist
  
  - [ Yes] I have read the migration guide in the readme.
   ([pytorch-transformers](https://github.com/huggingface/transformers#migrating-from-pytorch-transformers-to-transformers);
    [pytorch-pretrained-bert](https://github.com/huggingface/transformers#migrating-from-pytorch-pretrained-bert-to-transformers))
  - [ yes] I checked if a related official extension example runs on my machine.
  