text: 
  
  TITLE
  FastFormers into transformers
  
  COMMENTS
  5
  
  REACTIONS
  +1: 5
  -1: 0
  laugh: 0
  hooray: 0
  heart: 12
  rocket: 4
  eyes: 3
  
  BODY
  # 🌟 New model addition
  
  ## Model description
  
  We just open-sourced [FastFormers](https://arxiv.org/abs/2010.13382) which are our SustaiNLP 2020 systems (FastFormers: Highly Efficient Transformer Models for Natural Language Understanding [paper](https://arxiv.org/abs/2010.13382)).
  
  Currently, we are hosting this on our repository, but would like to merge it back to the transformers repository as an example.
  our repo - https://github.com/microsoft/fastformers
  
  For the purpose of the shared task, this is purely implemented with SuperGLUE data set.
  So, it's dependent on Alex Wang(@W4ngatang)'s SuperGLUE data processing pipeline.
  Also, many parts of the implementations are based on Alex'.
  (https://github.com/W4ngatang/transformers/tree/superglue)
  
  What would be the best way to merge this back?
  
  ## Open source status
  
  * [x] the model implementation is available: https://github.com/microsoft/fastformers/blob/main/examples/fastformers/run_superglue.py
  * [x] the model weights are available: demo systems are uploaded. https://github.com/microsoft/fastformers/releases/tag/v0.1-model
  * [x] who are the authors: @ykim362 
  