mpt-30b-chat / fc.py
daking's picture
LLM-foundry update October 17, 2023 23:04:30 (#21)
1e3a152
raw
history blame
167 Bytes
from torch import nn
FC_CLASS_REGISTRY = {'torch': nn.Linear}
try:
import transformer_engine.pytorch as te
FC_CLASS_REGISTRY['te'] = te.Linear
except:
pass