An implementation of a lstm network layer. More...

Public Member Functions
	LSTM ()
	Create the LSTM object. More...

	LSTM (const size_t inSize, const size_t outSize, const size_t rho=std::numeric_limits< size_t >::max())
	Create the LSTM layer object using the specified parameters. More...

template < typename InputType , typename ErrorType , typename GradientType >
void	Backward (const InputType &&input, ErrorType &&gy, GradientType &&g)
	Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f. More...

OutputDataType const &	Delta () const
	Get the delta. More...

OutputDataType &	Delta ()
	Modify the delta. More...

template < typename InputType , typename OutputType >
void	Forward (InputType &&input, OutputType &&output)
	Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. More...

template < typename InputType , typename ErrorType , typename GradientType >
void	Gradient (InputType &&input, ErrorType &&error, GradientType &&gradient)

OutputDataType const &	Gradient () const
	Get the gradient. More...

OutputDataType &	Gradient ()
	Modify the gradient. More...

InputDataType const &	InputParameter () const
	Get the input parameter. More...

InputDataType &	InputParameter ()
	Modify the input parameter. More...

OutputDataType const &	OutputParameter () const
	Get the output parameter. More...

OutputDataType &	OutputParameter ()
	Modify the output parameter. More...

OutputDataType const &	Parameters () const
	Get the parameters. More...

OutputDataType &	Parameters ()
	Modify the parameters. More...

void	Reset ()

void	ResetCell (const size_t size)

size_t	Rho () const
	Get the maximum number of steps to backpropagate through time (BPTT). More...

size_t &	Rho ()
	Modify the maximum number of steps to backpropagate through time (BPTT). More...

template < typename Archive >
void	serialize (Archive &ar, const unsigned int)
	Serialize the layer. More...

Detailed Description

template
<
typename
InputDataType
=
arma::mat
,
typename
OutputDataType
=
arma::mat
>

class mlpack::ann::LSTM< InputDataType, OutputDataType >

An implementation of a lstm network layer.

This class allows specification of the type of the activation functions used for the gates and cells and also of the type of the function used to initialize and update the peephole weights.

The implementation corresponds to the following algorithm:

$\begin{eqnarray} i &=& sigmoid(W \cdot x + W \cdot h + W \cdot c + b) \\ f &=& sigmoid(W \cdot x + W \cdot h + W \cdot c + b) \\ z &=& tanh(W \cdot x + W \cdot h + b) \\ c &=& f \cdot c + i \cdot z \\ o &=& sigmoid(W \cdot x + W \cdot h + W \cdot c + b) \\ h &=& o \cdot tanh(c) \end{eqnarray}$

For more information, see the following.

@article{Graves2013,
  author  = {Alex Graves and Abdel{-}rahman Mohamed and Geoffrey E. Hinton},
  title   = {Speech Recognition with Deep Recurrent Neural Networks},
  journal = CoRR},
  year    = {2013},
  url     = {http://arxiv.org/abs/1303.5778},
}

See also: FastLSTM for a faster LSTM version which combines the calculation of the input, forget, output gates and hidden state in a single step.

Template Parameters

InputDataType	Type of the input data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).
OutputDataType	Type of the output data (arma::colvec, arma::mat, arma::sp_mat or arma::cube).

Definition at line 55 of file layer_types.hpp.

Constructor & Destructor Documentation

◆ LSTM() [1/2]

LSTM ( )

Create the LSTM object.

◆ LSTM() [2/2]

LSTM	(	const size_t	inSize,
		const size_t	outSize,
		const size_t	rho = `std::numeric_limits< size_t >::max()`
	)

Create the LSTM layer object using the specified parameters.

Parameters

inSize	The number of input units.
outSize	The number of output units.
rho	Maximum number of steps to backpropagate through time (BPTT).

Member Function Documentation

◆ Backward()

void Backward	(	const InputType &&	input,
		ErrorType &&	gy,
		GradientType &&	g
	)

Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards trough f.

Using the results from the feed forward pass.

Parameters

input	The propagated input activation.
gy	The backpropagated error.
g	The calculated gradient.

◆ Delta() [1/2]

OutputDataType const& Delta ( ) const

inline

Get the delta.

Definition at line 150 of file lstm.hpp.

◆ Delta() [2/2]

OutputDataType& Delta ( )

inline

Modify the delta.

Definition at line 152 of file lstm.hpp.

◆ Forward()

void Forward	(	InputType &&	input,
		OutputType &&	output
	)

Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f.

Parameters

input	Input data used for evaluating the specified function.
output	Resulting output activation.

◆ Gradient() [1/3]

void Gradient	(	InputType &&	input,
		ErrorType &&	error,
		GradientType &&	gradient
	)

◆ Gradient() [2/3]

OutputDataType const& Gradient ( ) const

inline

Get the gradient.

Definition at line 155 of file lstm.hpp.

◆ Gradient() [3/3]

OutputDataType& Gradient ( )

inline

Modify the gradient.

Definition at line 157 of file lstm.hpp.

References LSTM< InputDataType, OutputDataType >::serialize().

◆ InputParameter() [1/2]

InputDataType const& InputParameter ( ) const

inline

Get the input parameter.

Definition at line 140 of file lstm.hpp.

◆ InputParameter() [2/2]

InputDataType& InputParameter ( )

inline

Modify the input parameter.

Definition at line 142 of file lstm.hpp.

◆ OutputParameter() [1/2]

OutputDataType const& OutputParameter ( ) const

inline

Get the output parameter.

Definition at line 145 of file lstm.hpp.

◆ OutputParameter() [2/2]

OutputDataType& OutputParameter ( )

inline

Modify the output parameter.

Definition at line 147 of file lstm.hpp.

◆ Parameters() [1/2]

OutputDataType const& Parameters ( ) const

inline

Get the parameters.

Definition at line 135 of file lstm.hpp.

◆ Parameters() [2/2]

OutputDataType& Parameters ( )

inline

Modify the parameters.

Definition at line 137 of file lstm.hpp.

◆ Reset()

void Reset ( )

◆ ResetCell()

void ResetCell ( const size_t size )

◆ Rho() [1/2]

size_t Rho ( ) const

inline

Get the maximum number of steps to backpropagate through time (BPTT).

Definition at line 130 of file lstm.hpp.

◆ Rho() [2/2]

size_t& Rho ( )

inline

Modify the maximum number of steps to backpropagate through time (BPTT).

Definition at line 132 of file lstm.hpp.

◆ serialize()

void serialize	(	Archive &	ar,
		const unsigned	int
	)

Serialize the layer.

Referenced by LSTM< InputDataType, OutputDataType >::Gradient().

The documentation for this class was generated from the following files:

src/mlpack/methods/ann/layer/layer_types.hpp
src/mlpack/methods/ann/layer/lstm.hpp

Public Member Functions

Detailed Description

template<typenameInputDataType=arma::mat,typenameOutputDataType=arma::mat> class mlpack::ann::LSTM< InputDataType, OutputDataType >

Constructor & Destructor Documentation

◆ LSTM() [1/2]

◆ LSTM() [2/2]

Member Function Documentation

◆ Backward()

◆ Delta() [1/2]

◆ Delta() [2/2]

◆ Forward()

◆ Gradient() [1/3]

◆ Gradient() [2/3]

◆ Gradient() [3/3]

◆ InputParameter() [1/2]

◆ InputParameter() [2/2]

◆ OutputParameter() [1/2]

◆ OutputParameter() [2/2]

◆ Parameters() [1/2]

◆ Parameters() [2/2]

◆ Reset()

◆ ResetCell()

◆ Rho() [1/2]

◆ Rho() [2/2]

◆ serialize()

template
<
typename
InputDataType
=
arma::mat
,
typename
OutputDataType
=
arma::mat
>

class mlpack::ann::LSTM< InputDataType, OutputDataType >