DuelingDQN< CompleteNetworkType, FeatureNetworkType, AdvantageNetworkType, ValueNetworkType > Class Template Reference

Implementation of the Dueling Deep Q-Learning network. More...

Public Member Functions
	DuelingDQN ()
	Default constructor. More...
	DuelingDQN (const int inputDim, const int h1, const int h2, const int outputDim, const bool isNoisy=false)
	Construct an instance of DuelingDQN class. More...
	DuelingDQN (FeatureNetworkType featureNetwork, AdvantageNetworkType advantageNetwork, ValueNetworkType valueNetwork, const bool isNoisy=false)
	DuelingDQN (const DuelingDQN &model)
	Copy constructor. More...
void	Backward (const arma::mat state, arma::mat &target, arma::mat &gradient)
	Perform the backward pass of the state in real batch mode. More...
void	Forward (const arma::mat state, arma::mat &actionValue)
	Perform the forward pass of the states in real batch mode. More...
void	operator= (const DuelingDQN &model)
	Copy assignment operator. More...
const arma::mat &	Parameters () const
	Return the Parameters. More...
arma::mat &	Parameters ()
	Modify the Parameters. More...
void	Predict (const arma::mat state, arma::mat &actionValue)
	Predict the responses to a given set of predictors. More...
void	ResetNoise ()
	Resets noise of the network, if the network is of type noisy. More...
void	ResetParameters ()
	Resets the parameters of the network. More...

Detailed Description

template<typename CompleteNetworkType = FFN<EmptyLoss<>, GaussianInitialization>, typename FeatureNetworkType = Sequential<>, typename AdvantageNetworkType = Sequential<>, typename ValueNetworkType = Sequential<>>
class mlpack::rl::DuelingDQN< CompleteNetworkType, FeatureNetworkType, AdvantageNetworkType, ValueNetworkType >

Implementation of the Dueling Deep Q-Learning network.

For more information, see the following.

@misc{wang2015dueling,
  author  = {Ziyu Wang, Tom Schaul, Matteo Hessel,Hado van Hasselt,
             Marc Lanctot, Nando de Freitas},
  title   = {Dueling Network Architectures for Deep Reinforcement Learning},
  year    = {2015},
  url     = {https://arxiv.org/abs/1511.06581}
}

Template Parameters

CompleteNetworkType	The type of network used for full dueling dqn.
FeatureNetworkType	The type of network used for feature network.
AdvantageNetworkType	The type of network used for advantage network.
ValueNetworkType	The type of network used for value network.

Definition at line 52 of file dueling_dqn.hpp.

Constructor & Destructor Documentation

DuelingDQN() [1/4]

DuelingDQN ( )

inline

Default constructor.

Definition at line 56 of file dueling_dqn.hpp.

References Sequential< InputDataType, OutputDataType, Residual, CustomLayers >::Add().

DuelingDQN() [2/4]

DuelingDQN ( const int  inputDim, const int  h1, const int  h2, const int  outputDim, const bool  isNoisy = false  )

inline

Construct an instance of DuelingDQN class.

Parameters

inputDim	Number of inputs.
h1	Number of neurons in hiddenlayer-1.
h2	Number of neurons in hiddenlayer-2.
outputDim	Number of neurons in output layer.
isNoisy	Specifies whether the network needs to be of type noisy.

Definition at line 79 of file dueling_dqn.hpp.

References Sequential< InputDataType, OutputDataType, Residual, CustomLayers >::Add(), and Concat< InputDataType, OutputDataType, CustomLayers >::Add().

DuelingDQN() [3/4]

DuelingDQN ( FeatureNetworkType  featureNetwork, AdvantageNetworkType  advantageNetwork, ValueNetworkType  valueNetwork, const bool  isNoisy = false  )

inline

Definition at line 128 of file dueling_dqn.hpp.

References Concat< InputDataType, OutputDataType, CustomLayers >::Add().

DuelingDQN() [4/4]

DuelingDQN ( const DuelingDQN< CompleteNetworkType, FeatureNetworkType, AdvantageNetworkType, ValueNetworkType > &  model )

inline

Copy constructor.

Definition at line 147 of file dueling_dqn.hpp.

Member Function Documentation

Backward()

void Backward ( const arma::mat  state, arma::mat &  target, arma::mat &  gradient  )

inline

Perform the backward pass of the state in real batch mode.

Parameters

state	The input state.
target	The training target.
gradient	The gradient.

Definition at line 205 of file dueling_dqn.hpp.

Forward()

void Forward ( const arma::mat  state, arma::mat &  actionValue  )

inline

Perform the forward pass of the states in real batch mode.

Parameters

state	The input state.
actionValue	Matrix to put output action values of states input.

Definition at line 187 of file dueling_dqn.hpp.

operator=()

void operator= ( const DuelingDQN< CompleteNetworkType, FeatureNetworkType, AdvantageNetworkType, ValueNetworkType > &  model )

inline

Copy assignment operator.

Definition at line 151 of file dueling_dqn.hpp.

Parameters() [1/2]

const arma::mat& Parameters ( ) const

inline

Return the Parameters.

Definition at line 240 of file dueling_dqn.hpp.

Parameters() [2/2]

arma::mat& Parameters ( )

inline

Modify the Parameters.

Definition at line 242 of file dueling_dqn.hpp.

Predict()

void Predict ( const arma::mat  state, arma::mat &  actionValue  )

inline

Predict the responses to a given set of predictors.

The responses will reflect the output of the given output layer as returned by the output layer function.

If you want to pass in a parameter and discard the original parameter object, be sure to use std::move to avoid unnecessary copy.

Parameters

state	Input state.
actionValue	Matrix to put output action values of states input.

Definition at line 171 of file dueling_dqn.hpp.

ResetNoise()

void ResetNoise ( )

inline

Resets noise of the network, if the network is of type noisy.

Definition at line 228 of file dueling_dqn.hpp.

ResetParameters()

void ResetParameters ( )

inline

Resets the parameters of the network.

Definition at line 220 of file dueling_dqn.hpp.

---

The documentation for this class was generated from the following file:

• /var/www/mlpack.ratml.org/mlpack.org/_src/mlpack-3.3.2/src/mlpack/methods/reinforcement_learning/q_networks/dueling_dqn.hpp