https://gist.github.com/xzuyn/106324ba725a8bfc20d6041be904088b
----Zeroed Layers----
model.layers.3.self_attn.o_proj.weight
model.layers.3.mlp.down_proj.weight
model.layers.7.self_attn.o_proj.weight
model.layers.7.mlp.down_proj.weight
model.layers.11.self_attn.o_proj.weight
model.layers.11.mlp.down_proj.weight
model.layers.15.self_attn.o_proj.weight
model.layers.15.mlp.down_proj.weight
model.layers.19.self_attn.o_proj.weight
model.layers.19.mlp.down_proj.weight
model.layers.23.self_attn.o_proj.weight
model.layers.23.mlp.down_proj.weight
model.layers.27.self_attn.o_proj.weight
model.layers.27.mlp.down_proj.weight
model.layers.31.self_attn.o_proj.weight
model.layers.31.mlp.down_proj.weight
----Zeroed Layers----
- Downloads last month
- 1