NadaMaxUpdate Class Reference

NadaMax is an optimizer that combines the AdaMax and NAG. More...

Public Member Functions
	NadaMaxUpdate (const double epsilon=1e-8, const double beta1=0.9, const double beta2=0.99, const double scheduleDecay=4e-3)
	Construct the NadaMax update policy with the given parameters. More...
double	Beta1 () const
	Get the smoothing parameter. More...
double &	Beta1 ()
	Modify the smoothing parameter. More...
double	Beta2 () const
	Get the second moment coefficient. More...
double &	Beta2 ()
	Modify the second moment coefficient. More...
double	CumBeta1 () const
	Get the value of the cumulative product of decay coefficients. More...
double &	CumBeta1 ()
	Modify the value of the cumulative product of decay coefficients. More...
double	Epsilon () const
	Get the value used to initialise the squared gradient parameter. More...
double &	Epsilon ()
	Modify the value used to initialise the squared gradient parameter. More...
void	Initialize (const size_t rows, const size_t cols)
	The Initialize() method is called by the optimizer before the start of the iteration update process. More...
double	ScheduleDecay () const
	Get the decay parameter for decay coefficients. More...
double &	ScheduleDecay ()
	Modify the decay parameter for decay coefficients. More...
void	Update (arma::mat &iterate, const double stepSize, const arma::mat &gradient)
	Update step for NadaMax. More...

Detailed Description

NadaMax is an optimizer that combines the AdaMax and NAG.

For more information, see the following.

@techreport{Dozat2015,
  title       = {Incorporating Nesterov momentum into Adam},
  author      = {Timothy Dozat},
  institution = {Stanford University},
  address     = {Stanford},
  year        = {2015},
  url         = {https://openreview.net/pdf?id=OM0jvwB8jIp57ZJjtNEZ}
}

Definition at line 37 of file nadamax_update.hpp.

Constructor & Destructor Documentation

NadaMaxUpdate()

NadaMaxUpdate ( const double  epsilon = 1e-8, const double  beta1 = 0.9, const double  beta2 = 0.99, const double  scheduleDecay = 4e-3  )

inline

Construct the NadaMax update policy with the given parameters.

Parameters

epsilon	The epsilon value used to initialise the squared gradient parameter.
beta1	The smoothing parameter.
beta2	The second moment coefficient
scheduleDecay	The decay parameter for decay coefficients

Definition at line 49 of file nadamax_update.hpp.

Member Function Documentation

Beta1() [1/2]

double Beta1 ( ) const

inline

Get the smoothing parameter.

Definition at line 126 of file nadamax_update.hpp.

Beta1() [2/2]

double& Beta1 ( )

inline

Modify the smoothing parameter.

Definition at line 128 of file nadamax_update.hpp.

Beta2() [1/2]

double Beta2 ( ) const

inline

Get the second moment coefficient.

Definition at line 131 of file nadamax_update.hpp.

Beta2() [2/2]

double& Beta2 ( )

inline

Modify the second moment coefficient.

Definition at line 133 of file nadamax_update.hpp.

CumBeta1() [1/2]

double CumBeta1 ( ) const

inline

Get the value of the cumulative product of decay coefficients.

Definition at line 121 of file nadamax_update.hpp.

CumBeta1() [2/2]

double& CumBeta1 ( )

inline

Modify the value of the cumulative product of decay coefficients.

Definition at line 123 of file nadamax_update.hpp.

Epsilon() [1/2]

double Epsilon ( ) const

inline

Get the value used to initialise the squared gradient parameter.

Definition at line 116 of file nadamax_update.hpp.

Epsilon() [2/2]

double& Epsilon ( )

inline

Modify the value used to initialise the squared gradient parameter.

Definition at line 118 of file nadamax_update.hpp.

Initialize()

void Initialize ( const size_t  rows, const size_t  cols  )

inline

The Initialize() method is called by the optimizer before the start of the iteration update process.

Parameters

rows	Number of rows in the gradient matrix.
cols	Number of columns in the gradient matrix.

Definition at line 70 of file nadamax_update.hpp.

ScheduleDecay() [1/2]

double ScheduleDecay ( ) const

inline

Get the decay parameter for decay coefficients.

Definition at line 136 of file nadamax_update.hpp.

ScheduleDecay() [2/2]

double& ScheduleDecay ( )

inline

Modify the decay parameter for decay coefficients.

Definition at line 138 of file nadamax_update.hpp.

Update()

void Update ( arma::mat &  iterate, const double  stepSize, const arma::mat &  gradient  )

inline

Update step for NadaMax.

Parameters

iterate	Parameters that minimize the function.
stepSize	Step size to be used for the given iteration.
gradient	The gradient matrix.

Definition at line 83 of file nadamax_update.hpp.

---

The documentation for this class was generated from the following file:

• src/mlpack/core/optimizers/adam/nadamax_update.hpp