potetoadmin's picture
Upload 5933 files
320718e

Change Log

2023/03/29 Update to 0.1.4

  • cp decomposition is default to disable now
  • add 4 more layer to train (conv_in/out, time_embedding)

2023/03/12 Update to 0.1.0

  • Add cp-decomposition implementation for convolution layer
    • Both LoRA(LoCon) and LoHa can use this more parameter-efficient decomposition
  • Add sparse bias for extracted LoRA
    • Will add to training in the future (Maybe)
  • Change weight initialization method in LoHa
    • Use lower std to avoid loss to go high or NaN when using normal lr (like 0.5 in Dadap)