mlpack
Public Member Functions | List of all members
GradientClipping< UpdatePolicyType > Class Template Reference

Interface for wrapping around update policies (e.g., VanillaUpdate) and feeding a clipped gradient to them instead of the normal one. More...

Public Member Functions

 GradientClipping (const double minGradient, const double maxGradient, UpdatePolicyType &updatePolicy)
 Constructor for creating a GradientClipping instance. More...

 
void Initialize (const size_t rows, const size_t cols)
 The Initialize method is called by SGD Optimizer method before the start of the iteration update process. More...

 
double MaxGradient () const
 Get the maximum gradient value. More...

 
double & MaxGradient ()
 Modify the maximum gradient value. More...

 
double MinGradient () const
 Get the minimum gradient value. More...

 
double & MinGradient ()
 Modify the minimum gradient value. More...

 
void Update (arma::mat &iterate, const double stepSize, const arma::mat &gradient)
 Update step. More...

 
UpdatePolicyType & UpdatePolicy () const
 Get the update policy. More...

 
UpdatePolicyType & UpdatePolicy ()
 Modify the update policy. More...

 

Detailed Description


template
<
typename
UpdatePolicyType
>

class mlpack::optimization::GradientClipping< UpdatePolicyType >

Interface for wrapping around update policies (e.g., VanillaUpdate) and feeding a clipped gradient to them instead of the normal one.

(Clipping here is implemented as $ g_{\text{clipped}} = \max(g_{\text{min}}, \min(g_{\text{min}}, g))) $.)

Template Parameters
UpdatePolicyA type of UpdatePolicy that sould be wrapped around.

Definition at line 29 of file gradient_clipping.hpp.

Constructor & Destructor Documentation

◆ GradientClipping()

GradientClipping ( const double  minGradient,
const double  maxGradient,
UpdatePolicyType &  updatePolicy 
)
inline

Constructor for creating a GradientClipping instance.

Parameters
minGradientMinimum possible value of gradient element.
maxGradientMaximum possible value of gradient element.
updatePolicyAn instance of the UpdatePolicyType used for actual optimization.

Definition at line 40 of file gradient_clipping.hpp.

Member Function Documentation

◆ Initialize()

void Initialize ( const size_t  rows,
const size_t  cols 
)
inline

The Initialize method is called by SGD Optimizer method before the start of the iteration update process.

Here we just do whatever initialization is needed for the actual update policy.

Parameters
rowsNumber of rows in the gradient matrix.
colsNumber of columns in the gradient matrix.

Definition at line 58 of file gradient_clipping.hpp.

◆ MaxGradient() [1/2]

double MaxGradient ( ) const
inline

Get the maximum gradient value.

Definition at line 92 of file gradient_clipping.hpp.

◆ MaxGradient() [2/2]

double& MaxGradient ( )
inline

Modify the maximum gradient value.

Definition at line 94 of file gradient_clipping.hpp.

◆ MinGradient() [1/2]

double MinGradient ( ) const
inline

Get the minimum gradient value.

Definition at line 87 of file gradient_clipping.hpp.

◆ MinGradient() [2/2]

double& MinGradient ( )
inline

Modify the minimum gradient value.

Definition at line 89 of file gradient_clipping.hpp.

◆ Update()

void Update ( arma::mat &  iterate,
const double  stepSize,
const arma::mat &  gradient 
)
inline

Update step.

First, the gradient is clipped, and then the actual update policy does whatever update it needs to do.

Parameters
iterateParameters that minimize the function.
stepSizeStep size to be used for the given iteration.
gradientThe gradient matrix.

Definition at line 71 of file gradient_clipping.hpp.

◆ UpdatePolicy() [1/2]

UpdatePolicyType& UpdatePolicy ( ) const
inline

Get the update policy.

Definition at line 82 of file gradient_clipping.hpp.

◆ UpdatePolicy() [2/2]

UpdatePolicyType& UpdatePolicy ( )
inline

Modify the update policy.

Definition at line 84 of file gradient_clipping.hpp.

The documentation for this class was generated from the following file: