lr_scaler
zeus.util.lr_scaler
Classes that enclose learning rate scaling rules.
SquareRootScaler
dataclass
Square root scaling.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
bs |
int
|
The initial batch size |
required |
lr |
float
|
The initial learning rate |
required |
Source code in zeus/util/lr_scaler.py
21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
|
compute_lr
compute_lr(new_bs)
Compute the scaled learning rate given the new batch size.
Source code in zeus/util/lr_scaler.py
33 34 35 |
|
LinearScaler
dataclass
Linear scaling.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
bs |
int
|
The initial batch size |
required |
lr |
float
|
The initial learning rate |
required |
Source code in zeus/util/lr_scaler.py
38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 |
|
compute_lr
compute_lr(new_bs)
Compute the scaled learning rate given the new batch size.
Source code in zeus/util/lr_scaler.py
50 51 52 |
|