Hugging Face's logo

billpsomas
/
resnet50_dino_official_ep100

Image Classification
English
resnet
convolutional neural network
simpool
dino
computer vision
deep learning
Model card Files Community
Bill Psomas commited on
Commit
5eaa633
•
1 Parent(s): 744957a

model first commit

Browse files
Files changed (4) hide show
  1. README.md +33 -0
  2. checkpoint.pth +3 -0
  3. log.txt +94 -0
  4. resnet50_dino_official.yaml +44 -0
README.md CHANGED
@@ -1,3 +1,36 @@
1
  ---
2
  license: cc-by-4.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: cc-by-4.0
3
+ datasets:
4
+ - imagenet-1k
5
+ metrics:
6
+ - accuracy
7
+ pipeline_tag: image-classification
8
+ language:
9
+ - en
10
+ tags:
11
+ - resnet
12
+ - convolutional neural network
13
+ - simpool
14
+ - dino
15
+ - computer vision
16
+ - deep learning
17
  ---
18
+
19
+ # Self-supervised ResNet-50 model.
20
+
21
+ ResNet-50 official model trained on ImageNet-1k for 100 epochs. Self-supervision with [DINO](https://arxiv.org/abs/2104.14294). Reproduced for ICCV 2023 [SimPool](https://arxiv.org/abs/2309.06891) paper.
22
+ SimPool is a simple attention-based pooling method at the end of network, released in this [repository](https://github.com/billpsomas/simpool/).
23
+ Disclaimer: This model card is written by the author of SimPool, i.e. [Bill Psomas](http://users.ntua.gr/psomasbill/).
24
+
25
+ ## BibTeX entry and citation info
26
+
27
+ ```
28
+ @misc{psomas2023simpool,
29
+ title={Keep It SimPool: Who Said Supervised Transformers Suffer from Attention Deficit?},
30
+ author={Bill Psomas and Ioannis Kakogeorgiou and Konstantinos Karantzalos and Yannis Avrithis},
31
+ year={2023},
32
+ eprint={2309.06891},
33
+ archivePrefix={arXiv},
34
+ primaryClass={cs.CV}
35
+ }
36
+ ```
checkpoint.pth ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:ea508c24ee1c394f56b7b474c07878edba3dfedc7d603469d24ffcf1a5aae71d
3
+ size 575033254
log.txt ADDED
@@ -0,0 +1,94 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {"train_loss": 10.822090088129043, "train_entropy": 9.827102318257094, "train_KL_div": 0.9949876400455833, "train_lr": 0.09364451806475811, "train_wd": 1.0000000000000158e-06, "epoch": 0, "k-NN": {"10": {"top1": 2.442, "top5": 5.96}, "20": {"top1": 2.74, "top5": 6.58}, "100": {"top1": 3.18, "top5": 8.55}, "200": {"top1": 3.29, "top5": 9.032}}}
2
+ {"train_loss": 10.05876412793994, "train_entropy": 9.373386079855264, "train_KL_div": 0.6853780091530643, "train_lr": 0.2811679584948117, "train_wd": 1.0000000000000158e-06, "epoch": 1}
3
+ {"train_loss": 9.60993558689952, "train_entropy": 8.670826627910138, "train_KL_div": 0.939108896390535, "train_lr": 0.46869139892486567, "train_wd": 1.0000000000000158e-06, "epoch": 2}
4
+ {"train_loss": 8.790879895091058, "train_entropy": 7.5420682119950655, "train_KL_div": 1.2488117322931067, "train_lr": 0.6562148393549191, "train_wd": 1.0000000000000158e-06, "epoch": 3}
5
+ {"train_loss": 7.993955350928008, "train_entropy": 6.436534951590001, "train_KL_div": 1.5574203992821276, "train_lr": 0.8437382797849705, "train_wd": 1.0000000000000158e-06, "epoch": 4}
6
+ {"train_loss": 6.594060190549985, "train_entropy": 4.534123720582344, "train_KL_div": 2.0599364807621834, "train_lr": 1.0969034762065477, "train_wd": 1.0000000000000167e-06, "epoch": 6}
7
+ {"train_loss": 6.145517218501817, "train_entropy": 3.9467236174540794, "train_KL_div": 2.1987936053554975, "train_lr": 1.2656724603442453, "train_wd": 1.0000000000000167e-06, "epoch": 7}
8
+ {"train_loss": 5.859462107428013, "train_entropy": 3.597953138824791, "train_KL_div": 2.261508968335035, "train_lr": 1.4344414444819436, "train_wd": 1.0000000000000167e-06, "epoch": 8}
9
+ {"train_loss": 5.664201904771298, "train_entropy": 3.3804880579592003, "train_KL_div": 2.2837138420769145, "train_lr": 1.603210428619642, "train_wd": 1.0000000000000167e-06, "epoch": 9}
10
+ {"train_loss": 5.508502964663693, "train_entropy": 3.23153561740879, "train_KL_div": 2.27696732400641, "train_lr": 1.6873294386510405, "train_wd": 1.0000000000000167e-06, "epoch": 10, "k-NN": {"10": {"top1": 32.778, "top5": 52.704}, "20": {"top1": 33.346, "top5": 55.678}, "100": {"top1": 31.58, "top5": 58.022}, "200": {"top1": 29.84, "top5": 57.024}}}
11
+ {"train_loss": 5.377465474759768, "train_entropy": 3.125733437467077, "train_KL_div": 2.2517320454305016, "train_lr": 1.6863051676003853, "train_wd": 1.0000000000000167e-06, "epoch": 11}
12
+ {"train_loss": 5.275276106274168, "train_entropy": 3.051392647191564, "train_KL_div": 2.2238834706733277, "train_lr": 1.6842572968939906, "train_wd": 1.0000000000000167e-06, "epoch": 12}
13
+ {"train_loss": 5.192265950218765, "train_entropy": 2.9948497413750843, "train_KL_div": 2.1974162314719905, "train_lr": 1.681188321546868, "train_wd": 1.0000000000000167e-06, "epoch": 13}
14
+ {"train_loss": 5.119674654859794, "train_entropy": 2.952211366779893, "train_KL_div": 2.1674633239164396, "train_lr": 1.6771019806327307, "train_wd": 1.0000000000000167e-06, "epoch": 14}
15
+ {"train_loss": 5.003808929979801, "train_entropy": 2.8945611042678356, "train_KL_div": 2.1092478329315782, "train_lr": 1.480805013415583, "train_wd": 1.000000000000015e-06, "epoch": 16}
16
+ {"train_loss": 4.9625270928442475, "train_entropy": 2.8773487904965878, "train_KL_div": 2.0851783088222144, "train_lr": 1.4744928631053371, "train_wd": 1.000000000000015e-06, "epoch": 17}
17
+ {"train_loss": 4.927669084042311, "train_entropy": 2.866431937843561, "train_KL_div": 2.0612371537834404, "train_lr": 1.4673009538712791, "train_wd": 1.000000000000015e-06, "epoch": 18}
18
+ {"train_loss": 4.898154962658882, "train_entropy": 2.8602684118151664, "train_KL_div": 2.037886555157602, "train_lr": 1.4592380479469769, "train_wd": 1.000000000000015e-06, "epoch": 19}
19
+ {"train_loss": 4.874116347461939, "train_entropy": 2.859779030621052, "train_KL_div": 2.0143373258411885, "train_lr": 1.4503139687413085, "train_wd": 1.000000000000015e-06, "epoch": 20, "k-NN": {"10": {"top1": 48.166, "top5": 69.3}, "20": {"top1": 48.45, "top5": 72.09}, "100": {"top1": 46.046, "top5": 74.084}, "200": {"top1": 44.446, "top5": 73.19}}}
20
+ {"train_loss": 4.8534459759891035, "train_entropy": 2.8620242937207223, "train_KL_div": 1.991421680390835, "train_lr": 1.4405395888701316, "train_wd": 1.000000000000015e-06, "epoch": 21}
21
+ {"train_loss": 4.835293414950371, "train_entropy": 2.8677912113666535, "train_KL_div": 1.9675021969228983, "train_lr": 1.4299268169096953, "train_wd": 1.000000000000015e-06, "epoch": 22}
22
+ {"train_loss": 4.822021477669478, "train_entropy": 2.8775864255130292, "train_KL_div": 1.9444350484833122, "train_lr": 1.4184885828878602, "train_wd": 1.000000000000015e-06, "epoch": 23}
23
+ {"train_loss": 4.811153393208981, "train_entropy": 2.8894384362250567, "train_KL_div": 1.9217149555161595, "train_lr": 1.4062388225308557, "train_wd": 1.000000000000015e-06, "epoch": 24}
24
+ {"train_loss": 4.7969759936630725, "train_entropy": 2.9197964306920765, "train_KL_div": 1.877179551832378, "train_lr": 1.3793653911322934, "train_wd": 1.000000000000015e-06, "epoch": 26}
25
+ {"train_loss": 4.794415944129229, "train_entropy": 2.938823475107551, "train_KL_div": 1.8555924566686153, "train_lr": 1.3647744612273613, "train_wd": 1.000000000000015e-06, "epoch": 27}
26
+ {"train_loss": 4.791651093184948, "train_entropy": 2.958687356635928, "train_KL_div": 1.8329637124985456, "train_lr": 1.3494374473704787, "train_wd": 1.000000000000015e-06, "epoch": 28}
27
+ {"train_loss": 4.792230256944895, "train_entropy": 2.9786513569206, "train_KL_div": 1.8135788771882653, "train_lr": 1.3333730353505442, "train_wd": 1.000000000000015e-06, "epoch": 29}
28
+ {"train_loss": 4.792146694540977, "train_entropy": 3.0004933481663465, "train_KL_div": 1.7916533227860927, "train_lr": 1.3166007971790663, "train_wd": 1.000000000000015e-06, "epoch": 30, "k-NN": {"10": {"top1": 54.76, "top5": 74.924}, "20": {"top1": 54.762, "top5": 77.532}, "100": {"top1": 52.676, "top5": 79.334}, "200": {"top1": 51.034, "top5": 78.712}}}
29
+ {"train_loss": 4.7967462888062, "train_entropy": 3.0265396140813827, "train_KL_div": 1.7702066550701856, "train_lr": 1.2991411672447015, "train_wd": 1.000000000000015e-06, "epoch": 31}
30
+ {"train_loss": 4.798162660479545, "train_entropy": 3.049789741307497, "train_KL_div": 1.7483728961199523, "train_lr": 1.2810154174170678, "train_wd": 1.000000000000015e-06, "epoch": 32}
31
+ {"train_loss": 4.8031692865788935, "train_entropy": 3.07608109036088, "train_KL_div": 1.7270881644412874, "train_lr": 1.2622456311302717, "train_wd": 1.000000000000015e-06, "epoch": 33}
32
+ {"train_loss": 4.8089889549613, "train_entropy": 3.104274053379893, "train_KL_div": 1.7047148696556687, "train_lr": 1.242854676477644, "train_wd": 1.000000000000015e-06, "epoch": 34}
33
+ {"train_loss": 4.829103384852409, "train_entropy": 3.1655568590760232, "train_KL_div": 1.6635464930683375, "train_lr": 1.2023044896547548, "train_wd": 1.000000000000015e-06, "epoch": 36}
34
+ {"train_loss": 4.837519742786884, "train_entropy": 3.196329554066062, "train_KL_div": 1.6411901547834278, "train_lr": 1.181194661640857, "train_wd": 1.000000000000015e-06, "epoch": 37}
35
+ {"train_loss": 4.848754864513874, "train_entropy": 3.229165882885456, "train_KL_div": 1.6195889372006058, "train_lr": 1.1595624133825075, "train_wd": 1.000000000000015e-06, "epoch": 38}
36
+ {"train_loss": 4.860455645412206, "train_entropy": 3.263007918730378, "train_KL_div": 1.5974476896449923, "train_lr": 1.1374341004420114, "train_wd": 1.000000000000015e-06, "epoch": 39}
37
+ {"train_loss": 4.874580739080906, "train_entropy": 3.2985379841774702, "train_KL_div": 1.5760427118837834, "train_lr": 1.1148366827600853, "train_wd": 1.000000000000015e-06, "epoch": 40, "k-NN": {"10": {"top1": 58.09, "top5": 77.52}, "20": {"top1": 58.258, "top5": 79.912}, "100": {"top1": 56.074, "top5": 81.754}, "200": {"top1": 54.522, "top5": 81.224}}}
38
+ {"train_loss": 4.890512124627828, "train_entropy": 3.336365274503827, "train_KL_div": 1.5541468119695783, "train_lr": 1.0917976918093049, "train_wd": 1.000000000000015e-06, "epoch": 41}
39
+ {"train_loss": 4.908562789708376, "train_entropy": 3.3768500670939683, "train_KL_div": 1.531712687984109, "train_lr": 1.0683451970512654, "train_wd": 1.000000000000015e-06, "epoch": 42}
40
+ {"train_loss": 4.928154284030199, "train_entropy": 3.417848773986101, "train_KL_div": 1.510305480621755, "train_lr": 1.0445077717382412, "train_wd": 1.000000000000015e-06, "epoch": 43}
41
+ {"train_loss": 4.9484223972857, "train_entropy": 3.4602653556019067, "train_KL_div": 1.4881570169180631, "train_lr": 1.020314458101109, "train_wd": 1.000000000000015e-06, "epoch": 44}
42
+ {"train_loss": 4.99522451877594, "train_entropy": 3.552194246917963, "train_KL_div": 1.4430302546471356, "train_lr": 0.9709784668417525, "train_wd": 1.000000000000015e-06, "epoch": 46}
43
+ {"train_loss": 5.0213593964278695, "train_entropy": 3.6024799616336822, "train_KL_div": 1.418879418335855, "train_lr": 0.9458958975252503, "train_wd": 1.000000000000015e-06, "epoch": 47}
44
+ {"train_loss": 5.051733556836844, "train_entropy": 3.6536365574598313, "train_KL_div": 1.3980969729349018, "train_lr": 0.9205775832633721, "train_wd": 1.000000000000015e-06, "epoch": 48}
45
+ {"train_loss": 5.083297366380692, "train_entropy": 3.7092409436404705, "train_KL_div": 1.3740564118251204, "train_lr": 0.8950543705220572, "train_wd": 1.000000000000015e-06, "epoch": 49}
46
+ {"train_loss": 5.079889129787683, "train_entropy": 3.7109459673166274, "train_KL_div": 1.3689431369677185, "train_lr": 0.8693573554044856, "train_wd": 1.000000000000015e-06, "epoch": 50, "k-NN": {"10": {"top1": 59.956, "top5": 79.048}, "20": {"top1": 60.128, "top5": 81.234}, "100": {"top1": 58.068, "top5": 83.028}, "200": {"top1": 56.37, "top5": 82.502}}}
47
+ {"train_loss": 5.074975584924221, "train_entropy": 3.7125961907058955, "train_KL_div": 1.3623793781027198, "train_lr": 0.8435178457652496, "train_wd": 1.000000000000015e-06, "epoch": 51}
48
+ {"train_loss": 5.067102880030871, "train_entropy": 3.7102181621938946, "train_KL_div": 1.3568847026452422, "train_lr": 0.8175673230665799, "train_wd": 1.000000000000015e-06, "epoch": 52}
49
+ {"train_loss": 5.059842837810517, "train_entropy": 3.7084912073761225, "train_KL_div": 1.3513516155928373, "train_lr": 0.7915374040230088, "train_wd": 1.000000000000015e-06, "epoch": 53}
50
+ {"train_loss": 5.049668218880892, "train_entropy": 3.70326385524869, "train_KL_div": 1.3464043413177134, "train_lr": 0.7654598020812906, "train_wd": 1.000000000000015e-06, "epoch": 54}
51
+ {"train_loss": 5.029793396353722, "train_entropy": 3.693082193136215, "train_KL_div": 1.3367111805230378, "train_lr": 0.7132886550530276, "train_wd": 1.000000000000015e-06, "epoch": 56}
52
+ {"train_loss": 5.01936347541213, "train_entropy": 3.6865864253491165, "train_KL_div": 1.3327770313173533, "train_lr": 0.6872586724727882, "train_wd": 1.000000000000015e-06, "epoch": 57}
53
+ {"train_loss": 5.009163070976734, "train_entropy": 3.681334529519081, "train_KL_div": 1.327828518934548, "train_lr": 0.6613080545658879, "train_wd": 1.000000000000015e-06, "epoch": 58}
54
+ {"train_loss": 4.999786652803421, "train_entropy": 3.6771736212968826, "train_KL_div": 1.3226130114644765, "train_lr": 0.6354684181628611, "train_wd": 1.000000000000015e-06, "epoch": 59}
55
+ {"train_loss": 4.990401573687792, "train_entropy": 3.671037443250418, "train_KL_div": 1.3193641085885466, "train_lr": 0.6097712448803728, "train_wd": 1.000000000000015e-06, "epoch": 60}
56
+ {"train_loss": 4.977557312488556, "train_entropy": 3.664313763603568, "train_KL_div": 1.3132435307390988, "train_lr": 0.5842478427657233, "train_wd": 1.000000000000015e-06, "epoch": 61}
57
+ {"train_loss": 4.967382822066545, "train_entropy": 3.65863646055758, "train_KL_div": 1.3087463394477963, "train_lr": 0.5589293081528083, "train_wd": 1.000000000000015e-06, "epoch": 62}
58
+ {"train_loss": 4.957185649484396, "train_entropy": 3.652737945601344, "train_KL_div": 1.3044476847499609, "train_lr": 0.5338464877760345, "train_wd": 1.000000000000015e-06, "epoch": 63}
59
+ {"train_loss": 4.94571715375781, "train_entropy": 3.6468128121197223, "train_KL_div": 1.2989043239951135, "train_lr": 0.5090299411883172, "train_wd": 1.000000000000015e-06, "epoch": 64}
60
+ {"train_loss": 4.9350747437775135, "train_entropy": 3.640355280071497, "train_KL_div": 1.2947194381020963, "train_lr": 0.48450990352897855, "train_wd": 1.000000000000015e-06, "epoch": 65}
61
+ {"train_loss": 4.9227733147442345, "train_entropy": 3.6332145245969296, "train_KL_div": 1.2895587622299791, "train_lr": 0.46031624868688437, "train_wd": 1.000000000000015e-06, "epoch": 66}
62
+ {"train_loss": 4.913880233585835, "train_entropy": 3.62899993519485, "train_KL_div": 1.2848802708685398, "train_lr": 0.43647845290371157, "train_wd": 1.000000000000015e-06, "epoch": 67}
63
+ {"train_loss": 4.902535814225674, "train_entropy": 3.6222265422046185, "train_KL_div": 1.2803092468045651, "train_lr": 0.41302555886169284, "train_wd": 1.000000000000015e-06, "epoch": 68}
64
+ {"train_loss": 4.892583984464407, "train_entropy": 3.6170960906147958, "train_KL_div": 1.275487869143486, "train_lr": 0.3899861402995754, "train_wd": 1.000000000000015e-06, "epoch": 69}
65
+ {"train_loss": 4.882353959828615, "train_entropy": 3.611496497899294, "train_KL_div": 1.2708574409261346, "train_lr": 0.36738826719992784, "train_wd": 1.000000000000015e-06, "epoch": 70}
66
+ {"train_loss": 4.872393114387989, "train_entropy": 3.6066741693764923, "train_KL_div": 1.2657189122363925, "train_lr": 0.34525947159018566, "train_wd": 1.000000000000015e-06, "epoch": 71}
67
+ {"train_loss": 4.862843583613634, "train_entropy": 3.6021431097984316, "train_KL_div": 1.2607004480622708, "train_lr": 0.32362671399911996, "train_wd": 1.000000000000015e-06, "epoch": 72}
68
+ {"train_loss": 4.85355372440815, "train_entropy": 3.5975622773915528, "train_KL_div": 1.2559914231039584, "train_lr": 0.3025163506095845, "train_wd": 1.000000000000015e-06, "epoch": 73}
69
+ {"train_loss": 4.844503538608551, "train_entropy": 3.5934438712596894, "train_KL_div": 1.2510596433915198, "train_lr": 0.2819541011475686, "train_wd": 1.000000000000015e-06, "epoch": 74}
70
+ {"train_loss": 4.83454293718934, "train_entropy": 3.5882278201431035, "train_KL_div": 1.2463150889016688, "train_lr": 0.26196501754666823, "train_wd": 1.000000000000015e-06, "epoch": 75}
71
+ {"train_loss": 4.828565942674875, "train_entropy": 3.585548872411251, "train_KL_div": 1.243017045415938, "train_lr": 0.2425734534261701, "train_wd": 1.000000000000015e-06, "epoch": 76}
72
+ {"train_loss": 4.819540807932615, "train_entropy": 3.582259978726506, "train_KL_div": 1.2372808025628328, "train_lr": 0.2238030344199124, "train_wd": 1.000000000000015e-06, "epoch": 77}
73
+ {"train_loss": 4.813406818687916, "train_entropy": 3.5795128671079874, "train_KL_div": 1.2338939197398722, "train_lr": 0.2056766293920937, "train_wd": 1.000000000000015e-06, "epoch": 78}
74
+ {"train_loss": 4.805546234995127, "train_entropy": 3.576568000867963, "train_KL_div": 1.2289781992137432, "train_lr": 0.18821632257508172, "train_wd": 1.000000000000015e-06, "epoch": 79}
75
+ {"train_loss": 4.798792516380549, "train_entropy": 3.5742871756106616, "train_KL_div": 1.2245053135678172, "train_lr": 0.17144338666317846, "train_wd": 1.000000000000015e-06, "epoch": 80}
76
+ {"train_loss": 4.792729702472687, "train_entropy": 3.571892802014947, "train_KL_div": 1.2208368703462182, "train_lr": 0.1553782568951198, "train_wd": 1.000000000000015e-06, "epoch": 81}
77
+ {"train_loss": 4.785656943127513, "train_entropy": 3.5692378340661524, "train_KL_div": 1.2164190668463708, "train_lr": 0.1400405061568854, "train_wd": 1.000000000000015e-06, "epoch": 82}
78
+ {"train_loss": 4.781258459031582, "train_entropy": 3.5680705535560846, "train_KL_div": 1.2131878745630384, "train_lr": 0.1254488211351497, "train_wd": 1.000000000000015e-06, "epoch": 83}
79
+ {"train_loss": 4.774896655663848, "train_entropy": 3.5651439200788735, "train_KL_div": 1.2097526989988983, "train_lr": 0.11162097955043504, "train_wd": 1.000000000000015e-06, "epoch": 84}
80
+ {"train_loss": 4.770819726124406, "train_entropy": 3.564972313299775, "train_KL_div": 1.2058473794460296, "train_lr": 0.09857382849769694, "train_wd": 1.000000000000015e-06, "epoch": 85}
81
+ {"train_loss": 4.767567071944475, "train_entropy": 3.564078944429755, "train_KL_div": 1.2034880973026156, "train_lr": 0.08632326392073303, "train_wd": 1.000000000000015e-06, "epoch": 86}
82
+ {"train_loss": 4.76243833181262, "train_entropy": 3.561860580652952, "train_KL_div": 1.200577720247209, "train_lr": 0.07488421124542637, "train_wd": 1.000000000000015e-06, "epoch": 87}
83
+ {"train_loss": 4.758417877838015, "train_entropy": 3.561635865196586, "train_KL_div": 1.196781981755048, "train_lr": 0.06427060719540985, "train_wd": 1.000000000000015e-06, "epoch": 88}
84
+ {"train_loss": 4.755804740428925, "train_entropy": 3.561266101881862, "train_KL_div": 1.1945386027172207, "train_lr": 0.05449538281231875, "train_wd": 1.000000000000015e-06, "epoch": 89}
85
+ {"train_loss": 4.752958665117621, "train_entropy": 3.5605754030942918, "train_KL_div": 1.1923832238391041, "train_lr": 0.04557044770130585, "train_wd": 1.000000000000015e-06, "epoch": 90}
86
+ {"train_loss": 4.751380822688341, "train_entropy": 3.5606866656392815, "train_KL_div": 1.1906941169276835, "train_lr": 0.037506675521022874, "train_wd": 1.000000000000015e-06, "epoch": 91}
87
+ {"train_loss": 4.749019042447209, "train_entropy": 3.5608795890659093, "train_KL_div": 1.1881394159644842, "train_lr": 0.03031389073574286, "train_wd": 1.000000000000015e-06, "epoch": 92}
88
+ {"train_loss": 4.74601147761941, "train_entropy": 3.5594451666623352, "train_KL_div": 1.186566278476268, "train_lr": 0.024000856645763048, "train_wd": 1.000000000000015e-06, "epoch": 93}
89
+ {"train_loss": 4.745018689453602, "train_entropy": 3.5594830311089756, "train_KL_div": 1.1855356308333576, "train_lr": 0.018575264710673705, "train_wd": 1.000000000000015e-06, "epoch": 94}
90
+ {"train_loss": 4.743301714554429, "train_entropy": 3.560469861537218, "train_KL_div": 1.1828318185918034, "train_lr": 0.014043725178499328, "train_wd": 1.000000000000015e-06, "epoch": 95}
91
+ {"train_loss": 4.742933311417699, "train_entropy": 3.5589320993423463, "train_KL_div": 1.1840011742711067, "train_lr": 0.01041175903212971, "train_wd": 1.000000000000015e-06, "epoch": 96}
92
+ {"train_loss": 4.741570005580783, "train_entropy": 3.5594317411631344, "train_KL_div": 1.1821382378973067, "train_lr": 0.0076837912628526265, "train_wd": 1.000000000000015e-06, "epoch": 97}
93
+ {"train_loss": 4.742103420898318, "train_entropy": 3.559510856553912, "train_KL_div": 1.1825925338231027, "train_lr": 0.005863145479183704, "train_wd": 1.000000000000015e-06, "epoch": 98}
94
+ {"train_loss": 4.740088093295693, "train_entropy": 3.5589667666852476, "train_KL_div": 1.181121308118105, "train_lr": 0.004952039857561546, "train_wd": 1.000000000000015e-06, "epoch": 99, "k-NN": {"10": {"top1": 61.84, "top5": 80.35}, "20": {"top1": 62.174, "top5": 82.75}, "100": {"top1": 60.088, "top5": 84.216}, "200": {"top1": 58.544, "top5": 83.834}}}
resnet50_dino_official.yaml ADDED
@@ -0,0 +1,44 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ arch: resnet50
2
+ backend: nccl
3
+ batch_size_per_gpu: 90
4
+ clip_grad: 0.0
5
+ data_path: /path/to/imagenet/
6
+ dist_url: env://
7
+ drop_path_rate: 0.1
8
+ epochs: 100
9
+ eval_every: 5
10
+ freeze_last_layer: 1
11
+ global_crops_scale:
12
+ - 0.14
13
+ - 1.0
14
+ local_crops_number: 6
15
+ local_crops_scale:
16
+ - 0.05
17
+ - 0.14
18
+ local_rank: 0
19
+ lr: 0.3
20
+ min_lr: 0.0048
21
+ momentum_teacher: 0.996
22
+ nb_knn:
23
+ - 10
24
+ - 20
25
+ - 100
26
+ - 200
27
+ norm_last_layer: true
28
+ num_workers: 10
29
+ optimizer: lars
30
+ out_dim: 60000
31
+ output_dir: /path/to/output/
32
+ patch_size: 16
33
+ saveckp_freq: 20
34
+ seed: 0
35
+ subset: -1
36
+ teacher_temp: 0.07
37
+ temperature: 0.07
38
+ use_bn_in_head: true
39
+ use_fp16: false
40
+ warmup_epochs: 10
41
+ warmup_teacher_temp: 0.04
42
+ warmup_teacher_temp_epochs: 50
43
+ weight_decay: 1.0e-06
44
+ weight_decay_end: 1.0e-06