BarzilaiBorweinDecay Class Reference

Barzilai-Borwein decay policy for Stochastic variance reduced gradient (SVRG). More...

Public Member Functions
	BarzilaiBorweinDecay (const double maxStepSize=DBL_MAX, const double eps=1e-7)
void	Initialize (const size_t, const size_t)
	The Initialize method is called by SVRG Optimizer method before the start of the iteration update process. More...
void	Update (const arma::mat &iterate, const arma::mat &iterate0, const arma::mat &, const arma::mat &fullGradient, const size_t numBatches, double &stepSize)
	Barzilai-Borwein update step for SVRG. More...

Detailed Description

Barzilai-Borwein decay policy for Stochastic variance reduced gradient (SVRG).

For more information, please refer to:

@incollection{Tan2016,
  title     = {Barzilai-Borwein Step Size for Stochastic Gradient Descent},
  author    = {Tan, Conghui and Ma, Shiqian and Dai, Yu-Hong
               and Qian, Yuqiu},
  booktitle = {Advances in Neural Information Processing Systems 29},
  editor    = {D. D. Lee and M. Sugiyama and U. V. Luxburg and I. Guyon
               and R. Garnett},
  pages     = {685--693},
  year      = {2016},
  publisher = {Curran Associates, Inc.}
}

Definition at line 40 of file barzilai_borwein_decay.hpp.

Constructor & Destructor Documentation

BarzilaiBorweinDecay()

BarzilaiBorweinDecay ( const double  maxStepSize = DBL_MAX, const double  eps = 1e-7  )

inline

Definition at line 50 of file barzilai_borwein_decay.hpp.

Member Function Documentation

Initialize()

void Initialize ( const size_t  , const size_t    )

inline

The Initialize method is called by SVRG Optimizer method before the start of the iteration update process.

Parameters

rows	Number of rows in the gradient matrix.
cols	Number of columns in the gradient matrix.

Definition at line 63 of file barzilai_borwein_decay.hpp.

Update()

void Update ( const arma::mat &  iterate, const arma::mat &  iterate0, const arma::mat &  , const arma::mat &  fullGradient, const size_t  numBatches, double &  stepSize  )

inline

Barzilai-Borwein update step for SVRG.

Parameters

iterate	The current function parameter at time t.
iterate0	The last function parameters at time t - 1.
gradient	The current gradient matrix at time t.
fullGradient	The computed full gradient.
numBatches	The number of batches.
stepSize	Step size to be used for the given iteration.

Definition at line 76 of file barzilai_borwein_decay.hpp.

---

The documentation for this class was generated from the following file:

• src/mlpack/core/optimizers/svrg/barzilai_borwein_decay.hpp