SnapshotEnsembles Class Reference

Simulate a new warm-started run/restart once a number of epochs are performed. More...

Public Member Functions
	SnapshotEnsembles (const size_t epochRestart, const double multFactor, const double stepSize, const size_t maxIterations, const size_t snapshots)
	Construct the CyclicalDecay technique a restart method, where the step size decays after each batch and peridically resets to its initial value. More...
double	EpochBatches () const
	Get the restart fraction. More...
double &	EpochBatches ()
	Modify the restart fraction. More...
std::vector< arma::mat >	Snapshots () const
	Get the snapshots. More...
std::vector< arma::mat > &	Snapshots ()
	Modify the snapshots. More...
double	StepSize () const
	Get the step size. More...
double &	StepSize ()
	Modify the step size. More...
void	Update (arma::mat &iterate, double &stepSize, const arma::mat &)
	This function is called in each iteration after the policy update. More...

Detailed Description

Simulate a new warm-started run/restart once a number of epochs are performed.

Importantly, the restarts are not performed from scratch but emulated by increasing the step size while the old step size value of as an initial parameter.

For more information, please refer to:

@inproceedings{Huang2017,
  title     = {Snapshot ensembles: Train 1, get m for free},
  author    = {Gao Huang, Yixuan Li, Geoff Pleiss, Zhuang Liu,
               John E. Hopcroft, and Kilian Q. Weinberger},
  booktitle = {Proceedings of the International Conference on Learning
               Representations (ICLR)},
  year      = {2017},
  url       = {https://arxiv.org/abs/1704.00109}
}

Definition at line 40 of file snapshot_ensembles.hpp.

Constructor & Destructor Documentation

SnapshotEnsembles()

SnapshotEnsembles ( const size_t  epochRestart, const double  multFactor, const double  stepSize, const size_t  maxIterations, const size_t  snapshots  )

inline

Construct the CyclicalDecay technique a restart method, where the step size decays after each batch and peridically resets to its initial value.

Parameters

epochRestart	Initial epoch where decay is applied.
multFactor	Factor to increase the number of epochs before a restart.
stepSize	Initial step size for each restart.
batchSize	Size of each mini-batch.
numFunctions	The number of separable functions (the number of predictor points).
maxIterations	Maximum number of iterations allowed (0 means no limit).
snapshots	Maximum number of snapshots.

Definition at line 58 of file snapshot_ensembles.hpp.

Member Function Documentation

EpochBatches() [1/2]

double EpochBatches ( ) const

inline

Get the restart fraction.

Definition at line 135 of file snapshot_ensembles.hpp.

EpochBatches() [2/2]

double& EpochBatches ( )

inline

Modify the restart fraction.

Definition at line 137 of file snapshot_ensembles.hpp.

Snapshots() [1/2]

std::vector<arma::mat> Snapshots ( ) const

inline

Get the snapshots.

Definition at line 140 of file snapshot_ensembles.hpp.

Referenced by SnapshotSGDR< UpdatePolicyType >::Snapshots().

Snapshots() [2/2]

std::vector<arma::mat>& Snapshots ( )

inline

Modify the snapshots.

Definition at line 142 of file snapshot_ensembles.hpp.

StepSize() [1/2]

double StepSize ( ) const

inline

Get the step size.

Definition at line 130 of file snapshot_ensembles.hpp.

StepSize() [2/2]

double& StepSize ( )

inline

Modify the step size.

Definition at line 132 of file snapshot_ensembles.hpp.

Update()

void Update ( arma::mat &  iterate, double &  stepSize, const arma::mat &    )

inline

This function is called in each iteration after the policy update.

Parameters

iterate	Parameters that minimize the function.
stepSize	Step size to be used for the given iteration.
gradient	The gradient matrix.

Definition at line 93 of file snapshot_ensembles.hpp.

References M_PI.

---

The documentation for this class was generated from the following file:

• src/mlpack/core/optimizers/sgdr/snapshot_ensembles.hpp