Adaptiv fixed positional embeddings
State: closed | Pull request: yes

This PR enhances fixed positional embeddings to be adaptive to input sequences which are longer than the pre-defined max-sequence-length.

The PR also fixes two issues with checkpoint averaging `NeMo/scripts/checkpoint_averaging/checkpoint_averaging.py`:

1. Checkpoint averaging now ignores `-last.ckpt` file (which is not part of top k saved models).
2. The script allows now to add list of file to import before restoring checkpoint. Useful when some classes were defined in the training script, and as such the class cannot be imported.
   
    Example for error:
    ```
    Traceback (most recent call last):                                                                             
      File "~/code/NeMo/scripts/checkpoint_averaging/checkpoint_averaging.py", line 136, in <module>                                                                                                               
        main()                                                                                                                                                                                                                    
      File "~/code/NeMo/scripts/checkpoint_averaging/checkpoint_averaging.py", line 103, in main                                                                                                                   
        checkpoint = torch.load(path, map_location=device)                                                                                                                                                                        
      File "~/anaconda3/lib/python3.8/site-packages/torch/serialization.py", line 607, in load                                                                                                                         
        return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)                                                                                                                                               File "~/anaconda3/lib/python3.8/site-packages/torch/serialization.py", line 882, in _load                                                                                                                        
        result = unpickler.load()                                                                                                                                                                                                 
      File "~/anaconda3/lib/python3.8/site-packages/torch/serialization.py", line 875, in find_class                                                                                                                       return super().find_class(mod_name, name)                                                                                                                                                                                 
    AttributeError: Can't get attribute 'MTBottleneckConfig' on <module '__main__' from '~/code/NeMo/scripts/checkpoint_averaging/checkpoint_averaging.py'>                                                        (    
    ```
    Usage example:
    ```
    ~/code/NeMo/scripts/checkpoint_averaging/checkpoint_averaging.py \
        --import_fname_list ~/code/NeMo/examples/nlp/machine_translation/enc_dec_nmt-bottleneck.py \
        -- ./nll-perceiver_6x6_steps32_byte-level_sp-bpe_ls0.1_lp0.0-en_de/checkpoints
    ```