text: 
  
  TITLE
  AST: Audio Spectrogram Transformer
  
  COMMENTS
  7
  
  REACTIONS
  +1: 0
  -1: 0
  laugh: 0
  hooray: 0
  heart: 0
  rocket: 0
  eyes: 0
  
  BODY
  # 🌟 New model addition
  
  ## Model description
  
  In the past decade, convolutional neural networks (CNNs)
  have been widely adopted as the main building block for endto-end audio classification models, which aim to learn a direct
  mapping from audio spectrograms to corresponding labels. To
  better capture long-range global context, a recent trend is to
  add a self-attention mechanism on top of the CNN, forming a
  CNN-attention hybrid model. However, it is unclear whether
  the reliance on a CNN is necessary, and if neural networks
  purely based on attention are sufficient to obtain good performance in audio classification. In this paper, we answer the question by introducing the Audio Spectrogram Transformer (AST),
  the first convolution-free, purely attention-based model for audio classification. We evaluate AST on various audio classification benchmarks, where it achieves new state-of-the-art results of 0.485 mAP on AudioSet, 95.6% accuracy on ESC-50,
  and 98.1% accuracy on Speech Commands V2.
  Index Terms: audio classification, self-attention
  
  ## Open source status
  
  * [x] the model implementation is available: https://github.com/YuanGongND/ast
  * [x] the model weights are available: https://github.com/YuanGongND/ast#Pretrained-Models
  * [x] who are the authors: @YuanGongND
  
  
  Happy to supervise anyone interested in porting the model :-)