Gradient Descent is a technique to minimize a function. More...

Public Member Functions
	GradientDescent (const double stepSize=0.01, const size_t maxIterations=100000, const double tolerance=1e-5)
	Construct the Gradient Descent optimizer with the given function and parameters. More...

size_t	MaxIterations () const
	Get the maximum number of iterations (0 indicates no limit). More...

size_t &	MaxIterations ()
	Modify the maximum number of iterations (0 indicates no limit). More...

template < typename FunctionType >
double	Optimize (FunctionType &function, arma::mat &iterate)
	Optimize the given function using gradient descent. More...

template < typename FunctionType >
double	Optimize (FunctionType &function, arma::mat &iterate, data::DatasetMapper< data::IncrementPolicy, double > &datasetInfo)
	Assert all dimensions are numeric and optimize the given function using gradient descent. More...

double	StepSize () const
	Get the step size. More...

double &	StepSize ()
	Modify the step size. More...

double	Tolerance () const
	Get the tolerance for termination. More...

double &	Tolerance ()
	Modify the tolerance for termination. More...

Detailed Description

Gradient Descent is a technique to minimize a function.

To find a local minimum of a function using gradient descent, one takes steps proportional to the negative of the gradient of the function at the current point, producing the following update scheme:

$A_{j + 1} = A_j + \alpha \nabla F(A)$

where $\alpha$ is a parameter which specifies the step size. $ F $ is the function being optimized. The algorithm continues until $ j $ reaches the maximum number of iterations—or when an update produces an improvement within a certain tolerance $\epsilon$ . That is,

$| F(A_{j + 1}) - F(A_j) | < \epsilon.$

The parameter $\epsilon$ is specified by the tolerance parameter to the constructor.

For Gradient Descent to work, a FunctionType template parameter is required. This class must implement the following function:

double Evaluate(const arma::mat& coordinates); void Gradient(const arma::mat& coordinates, arma::mat& gradient);

Definition at line 49 of file gradient_descent.hpp.

Constructor & Destructor Documentation

◆ GradientDescent()

GradientDescent	(	const double	stepSize = `0.01`,
		const size_t	maxIterations = `100000`,
		const double	tolerance = `1e-5`
	)

Construct the Gradient Descent optimizer with the given function and parameters.

The defaults here are not necessarily good for the given problem, so it is suggested that the values used be tailored to the task at hand.

Parameters

function	Function to be optimized (minimized).
stepSize	Step size for each iteration.
maxIterations	Maximum number of iterations allowed (0 means no limit).
tolerance	Maximum absolute tolerance to terminate algorithm.

Member Function Documentation

◆ MaxIterations() [1/2]

size_t MaxIterations ( ) const

inline

Get the maximum number of iterations (0 indicates no limit).

Definition at line 108 of file gradient_descent.hpp.

◆ MaxIterations() [2/2]

size_t& MaxIterations ( )

inline

Modify the maximum number of iterations (0 indicates no limit).

Definition at line 110 of file gradient_descent.hpp.

◆ Optimize() [1/2]

double Optimize	(	FunctionType &	function,
		arma::mat &	iterate
	)

Optimize the given function using gradient descent.

The given starting point will be modified to store the finishing point of the algorithm, and the final objective value is returned.

Template Parameters

FunctionType Type of the function to optimize.

Parameters

function	Function to optimize.
iterate	Starting point (will be modified).

Returns: Objective value of the final point.

◆ Optimize() [2/2]

double Optimize	(	FunctionType &	function,
		arma::mat &	iterate,
		data::DatasetMapper< data::IncrementPolicy, double > &	datasetInfo
	)

Assert all dimensions are numeric and optimize the given function using gradient descent.

The given starting point will be modified to store the finishing point of the algorithm, and the final objective value is returned.

This overload is intended to be used primarily by the hyper-parameter tuning module.

Template Parameters

FunctionType Type of the function to optimize.

Parameters

function	Function to optimize.
iterate	Starting point (will be modified).
datasetInfo	Type information for each dimension of the dataset.

Returns: Objective value of the final point.

◆ StepSize() [1/2]

double StepSize ( ) const

inline

Get the step size.

Definition at line 103 of file gradient_descent.hpp.

◆ StepSize() [2/2]

double& StepSize ( )

inline

Modify the step size.

Definition at line 105 of file gradient_descent.hpp.

◆ Tolerance() [1/2]

double Tolerance ( ) const

inline

Get the tolerance for termination.

Definition at line 113 of file gradient_descent.hpp.

◆ Tolerance() [2/2]

double& Tolerance ( )

inline

Modify the tolerance for termination.

Definition at line 115 of file gradient_descent.hpp.

The documentation for this class was generated from the following file:

src/mlpack/core/optimizers/gradient_descent/gradient_descent.hpp

Public Member Functions

Detailed Description

Constructor & Destructor Documentation

◆ GradientDescent()

Member Function Documentation

◆ MaxIterations() [1/2]

◆ MaxIterations() [2/2]

◆ Optimize() [1/2]

◆ Optimize() [2/2]

◆ StepSize() [1/2]

◆ StepSize() [2/2]

◆ Tolerance() [1/2]

◆ Tolerance() [2/2]