mlpack
Public Member Functions | List of all members
SVRGUpdate Class Reference

Vanilla update policy for Stochastic variance reduced gradient (SVRG). More...

Public Member Functions

void Initialize (const size_t, const size_t)
 The Initialize method is called by SVRG Optimizer method before the start of the iteration update process. More...

 
void Update (arma::mat &iterate, const arma::mat &fullGradient, const arma::mat &gradient, const arma::mat &gradient0, const size_t batchSize, const double stepSize)
 Update step for SVRG. More...

 

Detailed Description

Vanilla update policy for Stochastic variance reduced gradient (SVRG).

The following update scheme is used to update SGD in every iteration:

Definition at line 24 of file svrg_update.hpp.

Member Function Documentation

◆ Initialize()

void Initialize ( const size_t  ,
const size_t   
)
inline

The Initialize method is called by SVRG Optimizer method before the start of the iteration update process.

The vanilla update doesn't initialize anything.

Parameters
rowsNumber of rows in the gradient matrix.
colsNumber of columns in the gradient matrix.

Definition at line 35 of file svrg_update.hpp.

◆ Update()

void Update ( arma::mat &  iterate,
const arma::mat &  fullGradient,
const arma::mat &  gradient,
const arma::mat &  gradient0,
const size_t  batchSize,
const double  stepSize 
)
inline

Update step for SVRG.

The function parameters are updated in the negative direction of the gradient.

Parameters
iterateParameters that minimize the function.
fullGradientThe computed full gradient.
gradientThe current gradient matrix at time t.
gradient0The old gradient matrix at time t - 1.
batchSizeBatch size to be used for the given iteration.
stepSizeStep size to be used for the given iteration.

Definition at line 49 of file svrg_update.hpp.

The documentation for this class was generated from the following file: