AdaMaxUpdate Class Reference

AdaMax is a variant of Adam, an optimizer that computes individual adaptive learning rates for different parameters from estimates of first and second moments of the gradients.based on the infinity norm as given in the section 7 of the following paper. More...

Public Member Functions
	AdaMaxUpdate (const double epsilon=1e-8, const double beta1=0.9, const double beta2=0.999)
	Construct the AdaMax update policy with the given parameters. More...
double	Beta1 () const
	Get the smoothing parameter. More...
double &	Beta1 ()
	Modify the smoothing parameter. More...
double	Beta2 () const
	Get the second moment coefficient. More...
double &	Beta2 ()
	Modify the second moment coefficient. More...
double	Epsilon () const
	Get the value used to initialise the squared gradient parameter. More...
double &	Epsilon ()
	Modify the value used to initialise the squared gradient parameter. More...
void	Initialize (const size_t rows, const size_t cols)
	The Initialize method is called by SGD Optimizer method before the start of the iteration update process. More...
void	Update (arma::mat &iterate, const double stepSize, const arma::mat &gradient)
	Update step for Adam. More...

Detailed Description

AdaMax is a variant of Adam, an optimizer that computes individual adaptive learning rates for different parameters from estimates of first and second moments of the gradients.based on the infinity norm as given in the section 7 of the following paper.

For more information, see the following.

@article{Kingma2014,
  author    = {Diederik P. Kingma and Jimmy Ba},
  title     = {Adam: {A} Method for Stochastic Optimization},
  journal   = {CoRR},
  year      = {2014},
  url       = {http://arxiv.org/abs/1412.6980}
}

Definition at line 44 of file adamax_update.hpp.

Constructor & Destructor Documentation

AdaMaxUpdate()

AdaMaxUpdate ( const double  epsilon = 1e-8, const double  beta1 = 0.9, const double  beta2 = 0.999  )

inline

Construct the AdaMax update policy with the given parameters.

Parameters

epsilon	The epsilon value used to initialise the squared gradient parameter.
beta1	The smoothing parameter.
beta2	The second moment coefficient.

Definition at line 55 of file adamax_update.hpp.

Member Function Documentation

Beta1() [1/2]

double Beta1 ( ) const

inline

Get the smoothing parameter.

Definition at line 113 of file adamax_update.hpp.

Beta1() [2/2]

double& Beta1 ( )

inline

Modify the smoothing parameter.

Definition at line 115 of file adamax_update.hpp.

Beta2() [1/2]

double Beta2 ( ) const

inline

Get the second moment coefficient.

Definition at line 118 of file adamax_update.hpp.

Beta2() [2/2]

double& Beta2 ( )

inline

Modify the second moment coefficient.

Definition at line 120 of file adamax_update.hpp.

Epsilon() [1/2]

double Epsilon ( ) const

inline

Get the value used to initialise the squared gradient parameter.

Definition at line 108 of file adamax_update.hpp.

Epsilon() [2/2]

double& Epsilon ( )

inline

Modify the value used to initialise the squared gradient parameter.

Definition at line 110 of file adamax_update.hpp.

Initialize()

void Initialize ( const size_t  rows, const size_t  cols  )

inline

The Initialize method is called by SGD Optimizer method before the start of the iteration update process.

Parameters

rows	Number of rows in the gradient matrix.
cols	Number of columns in the gradient matrix.

Definition at line 73 of file adamax_update.hpp.

Update()

void Update ( arma::mat &  iterate, const double  stepSize, const arma::mat &  gradient  )

inline

Update step for Adam.

Parameters

iterate	Parameters that minimize the function.
stepSize	Step size to be used for the given iteration.
gradient	The gradient matrix.

Definition at line 86 of file adamax_update.hpp.

---

The documentation for this class was generated from the following file:

• src/mlpack/core/optimizers/adam/adamax_update.hpp