L_BFGS Class Reference

The generic L-BFGS optimizer, which uses a back-tracking line search algorithm to minimize a function. More...

Public Member Functions
	L_BFGS (const size_t numBasis=10, const size_t maxIterations=10000, const double armijoConstant=1e-4, const double wolfe=0.9, const double minGradientNorm=1e-6, const double factr=1e-15, const size_t maxLineSearchTrials=50, const double minStep=1e-20, const double maxStep=1e20)
	Initialize the L-BFGS object. More...
double	ArmijoConstant () const
	Get the Armijo condition constant. More...
double &	ArmijoConstant ()
	Modify the Armijo condition constant. More...
double	Factr () const
	Get the factr value. More...
double &	Factr ()
	Modify the factr value. More...
size_t	MaxIterations () const
	Get the maximum number of iterations. More...
size_t &	MaxIterations ()
	Modify the maximum number of iterations. More...
size_t	MaxLineSearchTrials () const
	Get the maximum number of line search trials. More...
size_t &	MaxLineSearchTrials ()
	Modify the maximum number of line search trials. More...
double	MaxStep () const
	Return the maximum line search step size. More...
double &	MaxStep ()
	Modify the maximum line search step size. More...
double	MinGradientNorm () const
	Get the minimum gradient norm. More...
double &	MinGradientNorm ()
	Modify the minimum gradient norm. More...
const std::pair< arma::mat, double > &	MinPointIterate () const
	Return the point where the lowest function value has been found. More...
double	MinStep () const
	Return the minimum line search step size. More...
double &	MinStep ()
	Modify the minimum line search step size. More...
size_t	NumBasis () const
	Get the memory size. More...
size_t &	NumBasis ()
	Modify the memory size. More...
template < typename FunctionType >
double	Optimize (FunctionType &function, arma::mat &iterate)
	Use L-BFGS to optimize the given function, starting at the given iterate point and finding the minimum. More...
double	Wolfe () const
	Get the Wolfe parameter. More...
double &	Wolfe ()
	Modify the Wolfe parameter. More...

Detailed Description

The generic L-BFGS optimizer, which uses a back-tracking line search algorithm to minimize a function.

The parameters for the algorithm (number of memory points, maximum step size, and so forth) are all configurable via either the constructor or standalone modifier functions. A function which can be optimized by this class's Optimize() method must implement the following methods:

• a default constructor
• double Evaluate(const arma::mat& coordinates);
• void Gradient(const arma::mat& coordinates, arma::mat& gradient);
• arma::mat& GetInitialPoint();

Definition at line 35 of file lbfgs.hpp.

Constructor & Destructor Documentation

L_BFGS()

L_BFGS	(	const size_t	numBasis = 10,
		const size_t	maxIterations = 10000,
		const double	armijoConstant = 1e-4,
		const double	wolfe = 0.9,
		const double	minGradientNorm = 1e-6,
		const double	factr = 1e-15,
		const size_t	maxLineSearchTrials = 50,
		const double	minStep = 1e-20,
		const double	maxStep = 1e20
	)

Initialize the L-BFGS object.

There are many parameters that can be set for the optimization, but default values are given for each of them.

Parameters

numBasis	Number of memory points to be stored (default 5).
maxIterations	Maximum number of iterations for the optimization (0 means no limit and may run indefinitely).
armijoConstant	Controls the accuracy of the line search routine for determining the Armijo condition.
wolfe	Parameter for detecting the Wolfe condition.
minGradientNorm	Minimum gradient norm required to continue the optimization.
maxLineSearchTrials	The maximum number of trials for the line search (before giving up).
minStep	The minimum step of the line search.
maxStep	The maximum step of the line search.

Member Function Documentation

ArmijoConstant() [1/2]

double ArmijoConstant ( ) const

inline

Get the Armijo condition constant.

Definition at line 100 of file lbfgs.hpp.

ArmijoConstant() [2/2]

double& ArmijoConstant ( )

inline

Modify the Armijo condition constant.

Definition at line 102 of file lbfgs.hpp.

Factr() [1/2]

double Factr ( ) const

inline

Get the factr value.

Definition at line 115 of file lbfgs.hpp.

Factr() [2/2]

double& Factr ( )

inline

Modify the factr value.

Definition at line 117 of file lbfgs.hpp.

MaxIterations() [1/2]

size_t MaxIterations ( ) const

inline

Get the maximum number of iterations.

Definition at line 95 of file lbfgs.hpp.

MaxIterations() [2/2]

size_t& MaxIterations ( )

inline

Modify the maximum number of iterations.

Definition at line 97 of file lbfgs.hpp.

MaxLineSearchTrials() [1/2]

size_t MaxLineSearchTrials ( ) const

inline

Get the maximum number of line search trials.

Definition at line 120 of file lbfgs.hpp.

MaxLineSearchTrials() [2/2]

size_t& MaxLineSearchTrials ( )

inline

Modify the maximum number of line search trials.

Definition at line 122 of file lbfgs.hpp.

MaxStep() [1/2]

double MaxStep ( ) const

inline

Return the maximum line search step size.

Definition at line 130 of file lbfgs.hpp.

MaxStep() [2/2]

double& MaxStep ( )

inline

Modify the maximum line search step size.

Definition at line 132 of file lbfgs.hpp.

MinGradientNorm() [1/2]

double MinGradientNorm ( ) const

inline

Get the minimum gradient norm.

Definition at line 110 of file lbfgs.hpp.

MinGradientNorm() [2/2]

double& MinGradientNorm ( )

inline

Modify the minimum gradient norm.

Definition at line 112 of file lbfgs.hpp.

MinPointIterate()

const std::pair<arma::mat, double>& MinPointIterate

(

)

const

Return the point where the lowest function value has been found.

Returns

arma::vec representing the point and a double with the function value at that point.

MinStep() [1/2]

double MinStep ( ) const

inline

Return the minimum line search step size.

Definition at line 125 of file lbfgs.hpp.

MinStep() [2/2]

double& MinStep ( )

inline

Modify the minimum line search step size.

Definition at line 127 of file lbfgs.hpp.

NumBasis() [1/2]

size_t NumBasis ( ) const

inline

Get the memory size.

Definition at line 90 of file lbfgs.hpp.

NumBasis() [2/2]

size_t& NumBasis ( )

inline

Modify the memory size.

Definition at line 92 of file lbfgs.hpp.

Optimize()

double Optimize	(	FunctionType &	function,
		arma::mat &	iterate
	)

Use L-BFGS to optimize the given function, starting at the given iterate point and finding the minimum.

The maximum number of iterations is set in the constructor (or with MaxIterations()). Alternately, another overload is provided which takes a maximum number of iterations as a parameter. The given starting point will be modified to store the finishing point of the algorithm, and the final objective value is returned.

Template Parameters

FunctionType

Type of the function to be optimized.

Parameters

function	Function to optimize; must have Evaluate() and Gradient().
iterate	Starting point (will be modified).

Returns

Objective value of the final point.

Wolfe() [1/2]

double Wolfe ( ) const

inline

Get the Wolfe parameter.

Definition at line 105 of file lbfgs.hpp.

Wolfe() [2/2]

double& Wolfe ( )

inline

Modify the Wolfe parameter.

Definition at line 107 of file lbfgs.hpp.

---

The documentation for this class was generated from the following file:

• src/mlpack/core/optimizers/lbfgs/lbfgs.hpp