mlpack
Public Member Functions | List of all members
VanillaUpdate Class Reference

Vanilla update policy for Stochastic Gradient Descent (SGD). More...

Public Member Functions

void Initialize (const size_t, const size_t)
 The Initialize method is called by SGD Optimizer method before the start of the iteration update process. More...

 
void Update (arma::mat &iterate, const double stepSize, const arma::mat &gradient)
 Update step for SGD. More...

 

Detailed Description

Vanilla update policy for Stochastic Gradient Descent (SGD).

The following update scheme is used to update SGD in every iteration:

\[ A_{j + 1} = A_j + \alpha \nabla f_i(A) \]

where $ \alpha $ is a parameter which specifies the step size. $ i $ is chosen according to $ j $ (the iteration number).

Definition at line 31 of file vanilla_update.hpp.

Member Function Documentation

◆ Initialize()

void Initialize ( const size_t  ,
const size_t   
)
inline

The Initialize method is called by SGD Optimizer method before the start of the iteration update process.

The vanilla update doesn't initialize anything.

Parameters
rowsNumber of rows in the gradient matrix.
colsNumber of columns in the gradient matrix.

Definition at line 42 of file vanilla_update.hpp.

◆ Update()

void Update ( arma::mat &  iterate,
const double  stepSize,
const arma::mat &  gradient 
)
inline

Update step for SGD.

The function parameters are updated in the negative direction of the gradient.

Parameters
iterateParameters that minimize the function.
stepSizeStep size to be used for the given iteration.
gradientThe gradient matrix.

Definition at line 53 of file vanilla_update.hpp.

The documentation for this class was generated from the following file: