GitHub repository: JohnGiorgi/compact-multi-head-self-attention-pytorch
Primary language: Python
Created: 2019-12-08T23:52:41Z
Description: A PyTorch implementation of the Compact Multi-Head Self-Attention Mechanism from the paper: "Low Rank Factorization for Compact Multi-Head Self-Attention"