CyclicalDecay Class Reference

Simulate a new warm-started run/restart once a number of epochs are performed. More...

Public Member Functions
	CyclicalDecay (const size_t epochRestart, const double multFactor, const double stepSize)
	Construct the CyclicalDecay technique a restart method, where the step size decays after each batch and peridically resets to its initial value. More...
double	EpochBatches () const
	Get the restart fraction. More...
double &	EpochBatches ()
	Modify the restart fraction. More...
double	StepSize () const
	Get the step size. More...
double &	StepSize ()
	Modify the step size. More...
void	Update (arma::mat &, double &stepSize, const arma::mat &)
	This function is called in each iteration after the policy update. More...

Detailed Description

Simulate a new warm-started run/restart once a number of epochs are performed.

Importantly, the restarts are not performed from scratch but emulated by increasing the step size while the old step size value of as an initial parameter.

For more information, please refer to:

@article{Loshchilov2016,
  title   = {Learning representations by back-propagating errors},
  author  = {Ilya Loshchilov and Frank Hutter},
  title   = {{SGDR:} Stochastic Gradient Descent with Restarts},
  journal = {CoRR},
  year    = {2016},
  url     = {https://arxiv.org/abs/1608.03983}
}

Definition at line 40 of file cyclical_decay.hpp.

Constructor & Destructor Documentation

CyclicalDecay()

CyclicalDecay ( const size_t  epochRestart, const double  multFactor, const double  stepSize  )

inline

Construct the CyclicalDecay technique a restart method, where the step size decays after each batch and peridically resets to its initial value.

Parameters

epochRestart	Initial epoch where decay is applied.
multFactor	Factor to increase the number of epochs before a restart.
stepSize	Initial step size for each restart.
batchSize	Size of each mini-batch.
numFunctions	The number of separable functions (the number of predictor points).

Definition at line 55 of file cyclical_decay.hpp.

Member Function Documentation

EpochBatches() [1/2]

double EpochBatches ( ) const

inline

Get the restart fraction.

Definition at line 110 of file cyclical_decay.hpp.

EpochBatches() [2/2]

double& EpochBatches ( )

inline

Modify the restart fraction.

Definition at line 112 of file cyclical_decay.hpp.

StepSize() [1/2]

double StepSize ( ) const

inline

Get the step size.

Definition at line 105 of file cyclical_decay.hpp.

StepSize() [2/2]

double& StepSize ( )

inline

Modify the step size.

Definition at line 107 of file cyclical_decay.hpp.

Update()

void Update ( arma::mat &  , double &  stepSize, const arma::mat &    )

inline

This function is called in each iteration after the policy update.

Parameters

iterate	Parameters that minimize the function.
stepSize	Step size to be used for the given iteration.
gradient	The gradient matrix.

Definition at line 74 of file cyclical_decay.hpp.

References M_PI.

---

The documentation for this class was generated from the following file:

• src/mlpack/core/optimizers/sgdr/cyclical_decay.hpp