FPHam commited on
Commit
034eb5c
1 Parent(s): 39bc73e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -33,8 +33,8 @@ Edit: It could prevent overfitting though and hence help with generalization. It
33
 
34
  - my favorite scheduler is warmup, hold for 1 epoch then cosine down for the next 1- x epochs.
35
 
36
- - rank is literally how many trainable parameters you get - you don't have to try to find some other meaning (style vs knowledge). It's like an image taken with 1Mpixel vs 16Mpixel. You always get the whole image, but on 1Mpixel the details are very mushy.
37
- the problem of course is - do you have enough diverse training data to fill those parameters with? If not, you'd be creating very specific model that would have hard time to generalize. Lowring rank will help with generalizations, but also the mundane details will be lost.
38
 
39
  **Anything else?**
40
 
 
33
 
34
  - my favorite scheduler is warmup, hold for 1 epoch then cosine down for the next 1- x epochs.
35
 
36
+ - rank is literally how many trainable parameters you get - you don't have to try to find some other meaning (style vs knowledge). It's like an image taken with 1Mpixel vs 16Mpixel. You always get the whole image, but on 1Mpixel the details are very mushy - while you can still see the big subject, you better not expect the details will be fine.
37
+ The problem of course is - do you have enough diverse training data to fill those parameters with? If not, you'd be creating very specific model that would have hard time to generalize. Lowring rank will help with generalizations, but also the mundane details will be lost.
38
 
39
  **Anything else?**
40