Exponential backoff stepsize reduction policy for parallel SGD. More...
Public Member Functions | |
| ExponentialBackoff (const size_t firstBackoffEpoch, const double step, const double beta) | |
| Member initializer constructor to construct the exponential backoff policy with the required parameters. More... | |
| double | StepSize (const size_t numEpoch) |
| Get the step size for the current gradient update. More... | |
Detailed Description
Exponential backoff stepsize reduction policy for parallel SGD.
For more information, see the following.
{1106.5730, Author = {Feng Niu and Benjamin Recht and Christopher Re and Stephen J. Wright}, Title = {HOGWILD!: A Lock-Free Approach to Parallelizing Stochastic Gradient Descent}, Year = {2011}, Eprint = {arXiv:1106.5730}, }
This stepsize update scheme gives robust 1/k convergence rates to the implementation of parallel SGD.
Definition at line 38 of file exponential_backoff.hpp.
Constructor & Destructor Documentation
◆ ExponentialBackoff()
|
inline |
Member initializer constructor to construct the exponential backoff policy with the required parameters.
- Parameters
-
firstBackoffEpoch The number of updates to run before the first stepsize backoff. step The initial stepsize(gamma). beta The reduction factor. This should be a value in range (0, 1).
Definition at line 50 of file exponential_backoff.hpp.
Member Function Documentation
◆ StepSize()
|
inline |
Get the step size for the current gradient update.
- Parameters
-
numEpoch The iteration number of the current update.
- Returns
- The stepsize for the current iteration.
Definition at line 65 of file exponential_backoff.hpp.
The documentation for this class was generated from the following file:
- src/mlpack/core/optimizers/parallel_sgd/decay_policies/exponential_backoff.hpp