internlm

/

internlm-7b

like 93

I find it weird that in the official internlm repo there is actually dropout in the MHA module, but there is not a single dropout module in the modeling_internlm.py file. Neither in Attention nor in MLP.