Spaces:

howard-hou
/

VisualRWKV-Gradio-1

Sleeping

howard-hou commited on Dec 30, 2023

Commit

898a24b

•

1 Parent(s): c25fbe0

Update modeling_rwkv.py

Files changed (1) hide show

modeling_rwkv.py CHANGED Viewed

@@ -320,8 +320,8 @@ class RWKV(MyModule):
                     w['emb.weight'] = F.layer_norm(w['emb.weight'], (args.n_embd,), weight=w['blocks.0.ln0.weight'], bias=w['blocks.0.ln0.bias'])
                 except:
                     w['emb.weight'] = F.layer_norm(w['emb.weight'].float(), (args.n_embd,), weight=w['blocks.0.ln0.weight'].float(), bias=w['blocks.0.ln0.bias'].float())
-                del w['blocks.0.ln0.weight']
-                del w['blocks.0.ln0.bias']
             print_need_newline = False

                     w['emb.weight'] = F.layer_norm(w['emb.weight'], (args.n_embd,), weight=w['blocks.0.ln0.weight'], bias=w['blocks.0.ln0.bias'])
                 except:
                     w['emb.weight'] = F.layer_norm(w['emb.weight'].float(), (args.n_embd,), weight=w['blocks.0.ln0.weight'].float(), bias=w['blocks.0.ln0.bias'].float())
+                # del w['blocks.0.ln0.weight']
+                # del w['blocks.0.ln0.bias']
             print_need_newline = False