GitHub repository: kyegomez/SimplifiedTransformers
Primary language: Python
Created: 2023-12-02T18:55:00Z
Description: SimplifiedTransformer simplifies transformer block without affecting training. Skip connections, projection parameters, sequential sub-blocks, and normalization layers are removed. Experimental results confirm similar training speed and performance.