Implementation of a standard feed forward network. More...

Public Types
using	NetworkType = FFN< OutputLayerType, InitializationRuleType >
	Convenience typedef for the internal model construction. More...

Public Member Functions
	FFN (OutputLayerType outputLayer=OutputLayerType(), InitializationRuleType initializeRule=InitializationRuleType())
	Create the FFN object. More...

	FFN (const FFN &)
	Copy constructor. More...

	FFN (FFN &&)
	Move constructor. More...

	~FFN ()
	Destructor to release allocated memory. More...

template<class LayerType , class... Args>
void	Add (Args... args)

void	Add (LayerTypes< CustomLayers... > layer)

double	Backward (arma::mat targets, arma::mat &gradients)
	Perform the backward pass of the data in real batch mode. More...

double	Evaluate (const arma::mat &parameters)
	Evaluate the feedforward network with the given parameters. More...

double	Evaluate (const arma::mat &parameters, const size_t begin, const size_t batchSize, const bool deterministic)
	Evaluate the feedforward network with the given parameters, but using only a number of data points. More...

double	Evaluate (const arma::mat &parameters, const size_t begin, const size_t batchSize)
	Evaluate the feedforward network with the given parameters, but using only a number of data points. More...

template < typename GradType >
double	EvaluateWithGradient (const arma::mat &parameters, GradType &gradient)
	Evaluate the feedforward network with the given parameters. More...

template < typename GradType >
double	EvaluateWithGradient (const arma::mat &parameters, const size_t begin, GradType &gradient, const size_t batchSize, const bool deterministic)
	Evaluate the feedforward network with the given parameters, but using only a number of data points. More...

template < typename GradType >
double	EvaluateWithGradient (const arma::mat &parameters, const size_t begin, GradType &gradient, const size_t batchSize)
	Evaluate the feedforward network with the given parameters, but using only a number of data points. More...

void	Forward (arma::mat inputs, arma::mat &results)
	Perform the forward pass of the data in real batch mode. More...

void	Forward (arma::mat inputs, arma::mat &results, const size_t begin, const size_t end)
	Perform a partial forward pass of the data. More...

void	Gradient (const arma::mat &parameters, const size_t begin, arma::mat &gradient, const size_t batchSize)
	Evaluate the gradient of the feedforward network with the given parameters, and with respect to only a number of points in the dataset. More...

size_t	NumFunctions () const
	Return the number of separable functions (the number of predictor points). More...

FFN &	operator= (FFN)
	Copy/move assignment operator. More...

const arma::mat &	Parameters () const
	Return the initial point for the optimization. More...

arma::mat &	Parameters ()
	Modify the initial point for the optimization. More...

void	Predict (arma::mat predictors, arma::mat &results)
	Predict the responses to a given set of predictors. More...

const arma::mat &	Predictors () const
	Get the matrix of data points (predictors). More...

arma::mat &	Predictors ()
	Modify the matrix of data points (predictors). More...

void	ResetParameters ()
	Reset the module infomration (weights/parameters). More...

const arma::mat &	Responses () const
	Get the matrix of responses to the input data points. More...

arma::mat &	Responses ()
	Modify the matrix of responses to the input data points. More...

template < typename Archive >
void	serialize (Archive &ar, const unsigned int)
	Serialize the model. More...

void	Shuffle ()
	Shuffle the order of function visitation. More...

template < typename OptimizerType >
void	Train (arma::mat predictors, arma::mat responses, OptimizerType &optimizer)
	Train the feedforward network on the given input data using the given optimizer. More...

template < typename OptimizerType = mlpack::optimization::RMSProp >
void	Train (arma::mat predictors, arma::mat responses)
	Train the feedforward network on the given input data. More...

Detailed Description

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization, typename... CustomLayers>
class mlpack::ann::FFN< OutputLayerType, InitializationRuleType, CustomLayers >

Implementation of a standard feed forward network.

Template Parameters

OutputLayerType	The output layer type used to evaluate the network.
InitializationRuleType	Rule used to initialize the weight matrix.
CustomLayers	Any set of custom layers that could be a part of the feed forward network.

Definition at line 50 of file ffn.hpp.

Member Typedef Documentation

◆ NetworkType

using NetworkType = FFN<OutputLayerType, InitializationRuleType>

Convenience typedef for the internal model construction.

Definition at line 54 of file ffn.hpp.

Constructor & Destructor Documentation

◆ FFN() [1/3]

FFN	(	OutputLayerType	outputLayer = `OutputLayerType()`,
		InitializationRuleType	initializeRule = `InitializationRuleType()`
	)

Create the FFN object.

Optionally, specify which initialize rule and performance function should be used.

If you want to pass in a parameter and discard the original parameter object, be sure to use std::move to avoid unnecessary copy.

Parameters

outputLayer	Output layer used to evaluate the network.
initializeRule	Optional instantiated InitializationRule object for initializing the network parameter.

◆ FFN() [2/3]

FFN ( const FFN< OutputLayerType, InitializationRuleType, CustomLayers > & )

Copy constructor.

◆ FFN() [3/3]

FFN ( FFN< OutputLayerType, InitializationRuleType, CustomLayers > && )

Move constructor.

◆ ~FFN()

~FFN ( )

Destructor to release allocated memory.

Member Function Documentation

◆ Add() [1/2]

void Add ( Args... args )

inline

Definition at line 260 of file ffn.hpp.

◆ Add() [2/2]

void Add ( LayerTypes< CustomLayers... > layer )

inline

Definition at line 267 of file ffn.hpp.

◆ Backward()

double Backward	(	arma::mat	targets,
		arma::mat &	gradients
	)

Perform the backward pass of the data in real batch mode.

Forward and Backward should be used as a pair, and they are designed mainly for advanced users. User should try to use Predict and Train unless those two functions can't satisfy some special requirements.

Parameters

targets	The training target.
gradients	Computed gradients.

Returns: Training error of the current pass.

Referenced by FFN< OutputLayerType, InitializationRuleType, CustomLayers >::Predictors().

◆ Evaluate() [1/3]

double Evaluate ( const arma::mat & parameters )

Evaluate the feedforward network with the given parameters.

This function is usually called by the optimizer to train the model.

Parameters

parameters	Matrix model parameters.
deterministic	Whether or not to train or test the model. Note some layer act differently in training or testing mode.

◆ Evaluate() [2/3]

double Evaluate	(	const arma::mat &	parameters,
		const size_t	begin,
		const size_t	batchSize,
		const bool	deterministic
	)

Evaluate the feedforward network with the given parameters, but using only a number of data points.

This is useful for optimizers such as SGD, which require a separable objective function.

Parameters

parameters	Matrix model parameters.
begin	Index of the starting point to use for objective function evaluation.
batchSize	Number of points to be passed at a time to use for objective function evaluation.
deterministic	Whether or not to train or test the model. Note some layer act differently in training or testing mode.

◆ Evaluate() [3/3]

double Evaluate	(	const arma::mat &	parameters,
		const size_t	begin,
		const size_t	batchSize
	)

Evaluate the feedforward network with the given parameters, but using only a number of data points.

This is useful for optimizers such as SGD, which require a separable objective function. This just calls the overload of Evaluate() with deterministic = true.

Parameters

parameters	Matrix model parameters.
begin	Index of the starting point to use for objective function evaluation.
batchSize	Number of points to be passed at a time to use for objective function evaluation.

◆ EvaluateWithGradient() [1/3]

double EvaluateWithGradient	(	const arma::mat &	parameters,
		GradType &	gradient
	)

Evaluate the feedforward network with the given parameters.

This function is usually called by the optimizer to train the model.

Parameters

parameters	Matrix model parameters.
gradient	Matrix to output gradient into.

◆ EvaluateWithGradient() [2/3]

double EvaluateWithGradient	(	const arma::mat &	parameters,
		const size_t	begin,
		GradType &	gradient,
		const size_t	batchSize,
		const bool	deterministic
	)

Evaluate the feedforward network with the given parameters, but using only a number of data points.

This is useful for optimizers such as SGD, which require a separable objective function.

Parameters

parameters	Matrix model parameters.
begin	Index of the starting point to use for objective function evaluation.
gradient	Matrix to output gradient into.
batchSize	Number of points to be passed at a time to use for objective function evaluation.
deterministic	Whether or not to train or test the model. Note some layer act differently in training or testing mode.

◆ EvaluateWithGradient() [3/3]

double EvaluateWithGradient	(	const arma::mat &	parameters,
		const size_t	begin,
		GradType &	gradient,
		const size_t	batchSize
	)

Evaluate the feedforward network with the given parameters, but using only a number of data points.

This is useful for optimizers such as SGD, which require a separable objective function. This just calls the overload of Evaluate() with deterministic = true.

Parameters

parameters	Matrix model parameters.
begin	Index of the starting point to use for objective function evaluation.
gradient	Matrix to output gradient into.
batchSize	Number of points to be passed at a time to use for objective function evaluation.

◆ Forward() [1/2]

void Forward	(	arma::mat	inputs,
		arma::mat &	results
	)

Perform the forward pass of the data in real batch mode.

Forward and Backward should be used as a pair, and they are designed mainly for advanced users. User should try to use Predict and Train unless those two functions can't satisfy some special requirements.

Parameters

inputs	The input data.
results	The predicted results.

Referenced by FFN< OutputLayerType, InitializationRuleType, CustomLayers >::Predictors().

◆ Forward() [2/2]

void Forward	(	arma::mat	inputs,
		arma::mat &	results,
		const size_t	begin,
		const size_t	end
	)

Perform a partial forward pass of the data.

This function is meant for the cases when users require a forward pass only through certain layers and not the entire network.

Parameters

inputs	The input data for the specified first layer.
results	The predicted results from the specified last layer.
begin	The index of the first layer.
end	The index of the last layer.

◆ Gradient()

void Gradient	(	const arma::mat &	parameters,
		const size_t	begin,
		arma::mat &	gradient,
		const size_t	batchSize
	)

Evaluate the gradient of the feedforward network with the given parameters, and with respect to only a number of points in the dataset.

This is useful for optimizers such as SGD, which require a separable objective function.

Parameters

parameters	Matrix of the model parameters to be optimized.
begin	Index of the starting point to use for objective function gradient evaluation.
gradient	Matrix to output gradient into.
batchSize	Number of points to be processed as a batch for objective function gradient evaluation.

Referenced by FFN< OutputLayerType, InitializationRuleType, CustomLayers >::Predictors().

◆ NumFunctions()

size_t NumFunctions ( ) const

inline

Return the number of separable functions (the number of predictor points).

Definition at line 270 of file ffn.hpp.

◆ operator=()

FFN& operator= ( FFN< OutputLayerType, InitializationRuleType, CustomLayers > )

Copy/move assignment operator.

◆ Parameters() [1/2]

const arma::mat& Parameters ( ) const

inline

Return the initial point for the optimization.

Definition at line 273 of file ffn.hpp.

◆ Parameters() [2/2]

arma::mat& Parameters ( )

inline

Modify the initial point for the optimization.

Definition at line 275 of file ffn.hpp.

◆ Predict()

void Predict	(	arma::mat	predictors,
		arma::mat &	results
	)

Predict the responses to a given set of predictors.

The responses will reflect the output of the given output layer as returned by the output layer function.

If you want to pass in a parameter and discard the original parameter object, be sure to use std::move to avoid unnecessary copy.

Parameters

predictors	Input predictors.
results	Matrix to put output predictions of responses into.

◆ Predictors() [1/2]

const arma::mat& Predictors ( ) const

inline

Get the matrix of data points (predictors).

Definition at line 283 of file ffn.hpp.

◆ Predictors() [2/2]

arma::mat& Predictors ( )

inline

Modify the matrix of data points (predictors).

Definition at line 285 of file ffn.hpp.

References FFN< OutputLayerType, InitializationRuleType, CustomLayers >::Backward(), FFN< OutputLayerType, InitializationRuleType, CustomLayers >::Forward(), FFN< OutputLayerType, InitializationRuleType, CustomLayers >::Gradient(), FFN< OutputLayerType, InitializationRuleType, CustomLayers >::ResetParameters(), and FFN< OutputLayerType, InitializationRuleType, CustomLayers >::serialize().

◆ ResetParameters()

void ResetParameters ( )

Reset the module infomration (weights/parameters).

Referenced by FFN< OutputLayerType, InitializationRuleType, CustomLayers >::Predictors().

◆ Responses() [1/2]

const arma::mat& Responses ( ) const

inline

Get the matrix of responses to the input data points.

Definition at line 278 of file ffn.hpp.

◆ Responses() [2/2]

arma::mat& Responses ( )

inline

Modify the matrix of responses to the input data points.

Definition at line 280 of file ffn.hpp.

◆ serialize()

void serialize	(	Archive &	ar,
		const unsigned	int
	)

Serialize the model.

Referenced by FFN< OutputLayerType, InitializationRuleType, CustomLayers >::Predictors().

◆ Shuffle()

void Shuffle ( )

Shuffle the order of function visitation.

This may be called by the optimizer.

◆ Train() [1/2]

void Train	(	arma::mat	predictors,
		arma::mat	responses,
		OptimizerType &	optimizer
	)

Train the feedforward network on the given input data using the given optimizer.

This will use the existing model parameters as a starting point for the optimization. If this is not what you want, then you should access the parameters vector directly with Parameters() and modify it as desired.

If you want to pass in a parameter and discard the original parameter object, be sure to use std::move to avoid unnecessary copy.

Template Parameters

OptimizerType Type of optimizer to use to train the model.

Parameters

predictors	Input training variables.
responses	Outputs results from input training variables.
optimizer	Instantiated optimizer used to train the model.

◆ Train() [2/2]

void Train	(	arma::mat	predictors,
		arma::mat	responses
	)

Train the feedforward network on the given input data.

By default, the RMSProp optimization algorithm is used, but others can be specified (such as mlpack::optimization::SGD).

This will use the existing model parameters as a starting point for the optimization. If this is not what you want, then you should access the parameters vector directly with Parameters() and modify it as desired.

If you want to pass in a parameter and discard the original parameter object, be sure to use std::move to avoid unnecessary copy.

Template Parameters

OptimizerType Type of optimizer to use to train the model.

Parameters

predictors	Input training variables.
responses	Outputs results from input training variables.

The documentation for this class was generated from the following file:

/home/ryan/src/mlpack.org/_src/mlpack-3.0.4/src/mlpack/methods/ann/ffn.hpp

Public Types

Public Member Functions

Detailed Description

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization, typename... CustomLayers> class mlpack::ann::FFN< OutputLayerType, InitializationRuleType, CustomLayers >

Member Typedef Documentation

◆ NetworkType

Constructor & Destructor Documentation

◆ FFN() [1/3]

◆ FFN() [2/3]

◆ FFN() [3/3]

◆ ~FFN()

Member Function Documentation

◆ Add() [1/2]

◆ Add() [2/2]

◆ Backward()

◆ Evaluate() [1/3]

◆ Evaluate() [2/3]

◆ Evaluate() [3/3]

◆ EvaluateWithGradient() [1/3]

◆ EvaluateWithGradient() [2/3]

◆ EvaluateWithGradient() [3/3]

◆ Forward() [1/2]

◆ Forward() [2/2]

◆ Gradient()

◆ NumFunctions()

◆ operator=()

◆ Parameters() [1/2]

◆ Parameters() [2/2]

◆ Predict()

◆ Predictors() [1/2]

◆ Predictors() [2/2]

◆ ResetParameters()

◆ Responses() [1/2]

◆ Responses() [2/2]

◆ serialize()

◆ Shuffle()

◆ Train() [1/2]

◆ Train() [2/2]

template<typename OutputLayerType = NegativeLogLikelihood<>, typename InitializationRuleType = RandomInitialization, typename... CustomLayers>
class mlpack::ann::FFN< OutputLayerType, InitializationRuleType, CustomLayers >