1 |
1 |
Baseline SGD w/ LR 0.01 |
0.105 |
0.09 @0.114 |
Best
optimiser |
2 |
2 |
Baseline Adam w/ LR 0.001 |
0.043 |
0.03 @0.071 |
|
3 |
3 |
SGD Momentum @0.9 |
0.089 |
0.07 @0.133 |
|
4 |
4 |
SGD Momentum @0.905 |
0.088 |
0.07 @0.153 |
Best value |
5 |
5 |
SGD Momentum @0.91 |
0.094 |
0.07 @0.148 |
|
6 |
6 |
SGD Momentum @0.92 |
0.104 |
0.08 @0.106 |
|
7 |
7 |
SGD Warmup Epochs @2.5 |
0.109 |
0.09 @0.095 |
Best value |
8 |
8 |
SGD Warmup Epochs @2.4 |
0.105 |
0.08 @0.125 |
|
9 |
9 |
SGD Warmup Momentum @0.801 |
0.108 |
0.09 @0.096 |
Best value |
10 |
10 |
SGD Warmup Momentum @0.799 |
0.107 |
0.09 @0.1 |
|
11 |
11 |
SGD Weight Decay @0.000499 |
0.111 |
0.09 @0.113 |
Best value |
12 |
12 |
SGD Weight Decay @0.000498 |
0.101 |
0.08 @0.09 |
|
13 |
13 |
SGD Warmup Bias LR @0.099 |
0.109 |
0.08 @0.099 |
|
14 |
14 |
SGD Warmup Bias LR @0.101 |
0.128 |
0.09 @0.116 |
Best
vlaue |
15 |
15 |
SGD Warmup Bias LR @0.102 |
0.107 |
0.08 @0.094 |
|