AdaptiveStepsize Class Reference

Definition of the adaptive stepize technique, a non-monotonic stepsize scheme that uses curvature estimates to propose new stepsize choices. More...

Public Member Functions
	AdaptiveStepsize (const double backtrackStepSize=0.1, const double searchParameter=0.1)
	Construct the AdaptiveStepsize object with the given function and parameters. More...
double	BacktrackStepSize () const
	Get the backtracking step size. More...
double &	BacktrackStepSize ()
	Modify the backtracking step size. More...
double	SearchParameter () const
	Get the search parameter. More...
double &	SearchParameter ()
	Modify the search parameter. More...
template < typename DecomposableFunctionType >
void	Update (DecomposableFunctionType &function, double &stepSize, arma::mat &iterate, const arma::mat &gradient, const double gradientNorm, const double sampleVariance, const size_t offset, const size_t batchSize, const size_t backtrackingBatchSize, const bool)
	This function is called in each iteration. More...

Detailed Description

Definition of the adaptive stepize technique, a non-monotonic stepsize scheme that uses curvature estimates to propose new stepsize choices.

direction.

For more information, please refer to:

@article{De2017,
  title   = {Big Batch {SGD:} Automated Inference using Adaptive Batch
             Sizes},
  author  = {Soham De and Abhay Kumar Yadav and David W. Jacobs and
             Tom Goldstein},
  journal = {CoRR},
  year    = {2017},
  url     = {http://arxiv.org/abs/1610.05792},
}

Definition at line 40 of file adaptive_stepsize.hpp.

Constructor & Destructor Documentation

AdaptiveStepsize()

AdaptiveStepsize ( const double  backtrackStepSize = 0.1, const double  searchParameter = 0.1  )

inline

Construct the AdaptiveStepsize object with the given function and parameters.

The defaults here are not necessarily good for the given problem, so it is suggested that the values used be tailored to the task at hand.

Parameters

backtrackStepSize	The backtracking step size for each iteration.
searchParameter	The backtracking search parameter for each iteration.

Definition at line 53 of file adaptive_stepsize.hpp.

Member Function Documentation

BacktrackStepSize() [1/2]

double BacktrackStepSize ( ) const

inline

Get the backtracking step size.

Definition at line 112 of file adaptive_stepsize.hpp.

BacktrackStepSize() [2/2]

double& BacktrackStepSize ( )

inline

Modify the backtracking step size.

Definition at line 114 of file adaptive_stepsize.hpp.

SearchParameter() [1/2]

double SearchParameter ( ) const

inline

Get the search parameter.

Definition at line 117 of file adaptive_stepsize.hpp.

SearchParameter() [2/2]

double& SearchParameter ( )

inline

Modify the search parameter.

Definition at line 119 of file adaptive_stepsize.hpp.

Update()

void Update ( DecomposableFunctionType &  function, double &  stepSize, arma::mat &  iterate, const arma::mat &  gradient, const double  gradientNorm, const double  sampleVariance, const size_t  offset, const size_t  batchSize, const size_t  backtrackingBatchSize, const bool    )

inline

This function is called in each iteration.

Template Parameters

DecomposableFunctionType

Type of the function to be optimized.

Parameters

function	Function to be optimized (minimized).
stepSize	Step size to be used for the given iteration.
iterate	Parameters that minimize the function.
gradient	The gradient matrix.
gradientNorm	The gradient norm to be used for the given iteration.
offset	The batch offset to be used for the given iteration.
batchSize	Batch size to be used for the given iteration.
backtrackingBatchSize	Backtracking batch size to be used for the given iteration.
reset	Reset the step size decay parameter.

Definition at line 75 of file adaptive_stepsize.hpp.

---

The documentation for this class was generated from the following file:

• src/mlpack/core/optimizers/bigbatch_sgd/adaptive_stepsize.hpp