\section{Categorical\+D\+QN$<$ Output\+Layer\+Type, Init\+Type, Network\+Type $>$ Class Template Reference} \label{classmlpack_1_1rl_1_1CategoricalDQN}\index{Categorical\+D\+Q\+N$<$ Output\+Layer\+Type, Init\+Type, Network\+Type $>$@{Categorical\+D\+Q\+N$<$ Output\+Layer\+Type, Init\+Type, Network\+Type $>$}} Implementation of the Categorical Deep Q-\/\+Learning network. \subsection*{Public Member Functions} \begin{DoxyCompactItemize} \item \textbf{ Categorical\+D\+QN} () \begin{DoxyCompactList}\small\item\em Default constructor. \end{DoxyCompactList}\item \textbf{ Categorical\+D\+QN} (const int input\+Dim, const int h1, const int h2, const int output\+Dim, \textbf{ Training\+Config} config, const bool is\+Noisy=false, Init\+Type init=Init\+Type(), Output\+Layer\+Type output\+Layer=Output\+Layer\+Type()) \begin{DoxyCompactList}\small\item\em Construct an instance of \doxyref{Categorical\+D\+QN}{p.}{classmlpack_1_1rl_1_1CategoricalDQN} class. \end{DoxyCompactList}\item \textbf{ Categorical\+D\+QN} (Network\+Type \&network, \textbf{ Training\+Config} config, const bool is\+Noisy=false) \begin{DoxyCompactList}\small\item\em Construct an instance of \doxyref{Categorical\+D\+QN}{p.}{classmlpack_1_1rl_1_1CategoricalDQN} class from a pre-\/constructed network. \end{DoxyCompactList}\item void \textbf{ Backward} (const arma\+::mat state, arma\+::mat \&loss\+Gradients, arma\+::mat \&gradient) \begin{DoxyCompactList}\small\item\em Perform the backward pass of the state in real batch mode. \end{DoxyCompactList}\item void \textbf{ Forward} (const arma\+::mat state, arma\+::mat \&dist) \begin{DoxyCompactList}\small\item\em Perform the forward pass of the states in real batch mode. \end{DoxyCompactList}\item const arma\+::mat \& \textbf{ Parameters} () const \begin{DoxyCompactList}\small\item\em Return the Parameters. \end{DoxyCompactList}\item arma\+::mat \& \textbf{ Parameters} () \begin{DoxyCompactList}\small\item\em Modify the Parameters. \end{DoxyCompactList}\item void \textbf{ Predict} (const arma\+::mat state, arma\+::mat \&action\+Value) \begin{DoxyCompactList}\small\item\em Predict the responses to a given set of predictors. \end{DoxyCompactList}\item void \textbf{ Reset\+Noise} () \begin{DoxyCompactList}\small\item\em Resets noise of the network, if the network is of type noisy. \end{DoxyCompactList}\item void \textbf{ Reset\+Parameters} () \begin{DoxyCompactList}\small\item\em Resets the parameters of the network. \end{DoxyCompactList}\end{DoxyCompactItemize} \subsection{Detailed Description} \subsubsection*{template$<$typename Output\+Layer\+Type = Empty\+Loss$<$$>$, typename Init\+Type = Gaussian\+Initialization, typename Network\+Type = F\+F\+N$<$\+Output\+Layer\+Type, Init\+Type$>$$>$\newline class mlpack\+::rl\+::\+Categorical\+D\+Q\+N$<$ Output\+Layer\+Type, Init\+Type, Network\+Type $>$} Implementation of the Categorical Deep Q-\/\+Learning network. For more information, see the following. \begin{DoxyCode} @misc\{bellemare2017distributional, author = \{Marc G. Bellemare, Will Dabney, Rémi Munos\}, title = \{A Distributional Perspective on Reinforcement Learning\}, year = \{2017\}, url = \{http:\textcolor{comment}{//arxiv.org/abs/1707.06887\}} \} \end{DoxyCode} \begin{DoxyTemplParams}{Template Parameters} {\em Output\+Layer\+Type} & The output layer type of the network. \\ \hline {\em Init\+Type} & The initialization type used for the network. \\ \hline {\em Network\+Type} & The type of network used for simple dqn. \\ \hline \end{DoxyTemplParams} Definition at line 50 of file categorical\+\_\+dqn.\+hpp. \subsection{Constructor \& Destructor Documentation} \mbox{\label{classmlpack_1_1rl_1_1CategoricalDQN_a45e43f83c396a89b9306568d431c8c62}} \index{mlpack\+::rl\+::\+Categorical\+D\+QN@{mlpack\+::rl\+::\+Categorical\+D\+QN}!Categorical\+D\+QN@{Categorical\+D\+QN}} \index{Categorical\+D\+QN@{Categorical\+D\+QN}!mlpack\+::rl\+::\+Categorical\+D\+QN@{mlpack\+::rl\+::\+Categorical\+D\+QN}} \subsubsection{Categorical\+D\+Q\+N()\hspace{0.1cm}{\footnotesize\ttfamily [1/3]}} {\footnotesize\ttfamily \textbf{ Categorical\+D\+QN} (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Default constructor. Definition at line 56 of file categorical\+\_\+dqn.\+hpp. \mbox{\label{classmlpack_1_1rl_1_1CategoricalDQN_a66f0f1c43d7a21c0b020126f2eeaddef}} \index{mlpack\+::rl\+::\+Categorical\+D\+QN@{mlpack\+::rl\+::\+Categorical\+D\+QN}!Categorical\+D\+QN@{Categorical\+D\+QN}} \index{Categorical\+D\+QN@{Categorical\+D\+QN}!mlpack\+::rl\+::\+Categorical\+D\+QN@{mlpack\+::rl\+::\+Categorical\+D\+QN}} \subsubsection{Categorical\+D\+Q\+N()\hspace{0.1cm}{\footnotesize\ttfamily [2/3]}} {\footnotesize\ttfamily \textbf{ Categorical\+D\+QN} (\begin{DoxyParamCaption}\item[{const int}]{input\+Dim, }\item[{const int}]{h1, }\item[{const int}]{h2, }\item[{const int}]{output\+Dim, }\item[{\textbf{ Training\+Config}}]{config, }\item[{const bool}]{is\+Noisy = {\ttfamily false}, }\item[{Init\+Type}]{init = {\ttfamily InitType()}, }\item[{Output\+Layer\+Type}]{output\+Layer = {\ttfamily OutputLayerType()} }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Construct an instance of \doxyref{Categorical\+D\+QN}{p.}{classmlpack_1_1rl_1_1CategoricalDQN} class. \begin{DoxyParams}{Parameters} {\em input\+Dim} & Number of inputs. \\ \hline {\em h1} & Number of neurons in hiddenlayer-\/1. \\ \hline {\em h2} & Number of neurons in hiddenlayer-\/2. \\ \hline {\em output\+Dim} & Number of neurons in output layer. \\ \hline {\em config} & Hyper-\/parameters for categorical dqn. \\ \hline {\em is\+Noisy} & Specifies whether the network needs to be of type noisy. \\ \hline {\em init} & Specifies the initialization rule for the network. \\ \hline {\em output\+Layer} & Specifies the output layer type for network. \\ \hline \end{DoxyParams} Definition at line 71 of file categorical\+\_\+dqn.\+hpp. \mbox{\label{classmlpack_1_1rl_1_1CategoricalDQN_a69a18509f64711b6f24e53733f85b013}} \index{mlpack\+::rl\+::\+Categorical\+D\+QN@{mlpack\+::rl\+::\+Categorical\+D\+QN}!Categorical\+D\+QN@{Categorical\+D\+QN}} \index{Categorical\+D\+QN@{Categorical\+D\+QN}!mlpack\+::rl\+::\+Categorical\+D\+QN@{mlpack\+::rl\+::\+Categorical\+D\+QN}} \subsubsection{Categorical\+D\+Q\+N()\hspace{0.1cm}{\footnotesize\ttfamily [3/3]}} {\footnotesize\ttfamily \textbf{ Categorical\+D\+QN} (\begin{DoxyParamCaption}\item[{Network\+Type \&}]{network, }\item[{\textbf{ Training\+Config}}]{config, }\item[{const bool}]{is\+Noisy = {\ttfamily false} }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Construct an instance of \doxyref{Categorical\+D\+QN}{p.}{classmlpack_1_1rl_1_1CategoricalDQN} class from a pre-\/constructed network. \begin{DoxyParams}{Parameters} {\em network} & The network to be used by \doxyref{Categorical\+D\+QN}{p.}{classmlpack_1_1rl_1_1CategoricalDQN} class. \\ \hline {\em config} & Hyper-\/parameters for categorical dqn. \\ \hline {\em is\+Noisy} & Specifies whether the network needs to be of type noisy. \\ \hline \end{DoxyParams} Definition at line 110 of file categorical\+\_\+dqn.\+hpp. \subsection{Member Function Documentation} \mbox{\label{classmlpack_1_1rl_1_1CategoricalDQN_afac2a0f2e66fb7e384d881b2c12c9187}} \index{mlpack\+::rl\+::\+Categorical\+D\+QN@{mlpack\+::rl\+::\+Categorical\+D\+QN}!Backward@{Backward}} \index{Backward@{Backward}!mlpack\+::rl\+::\+Categorical\+D\+QN@{mlpack\+::rl\+::\+Categorical\+D\+QN}} \subsubsection{Backward()} {\footnotesize\ttfamily void Backward (\begin{DoxyParamCaption}\item[{const arma\+::mat}]{state, }\item[{arma\+::mat \&}]{loss\+Gradients, }\item[{arma\+::mat \&}]{gradient }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Perform the backward pass of the state in real batch mode. \begin{DoxyParams}{Parameters} {\em state} & The input state. \\ \hline {\em loss\+Gradients} & The loss gradients. \\ \hline {\em gradient} & The gradient. \\ \hline \end{DoxyParams} Definition at line 201 of file categorical\+\_\+dqn.\+hpp. \mbox{\label{classmlpack_1_1rl_1_1CategoricalDQN_a52f0f2f01f7936617bdff04df3565253}} \index{mlpack\+::rl\+::\+Categorical\+D\+QN@{mlpack\+::rl\+::\+Categorical\+D\+QN}!Forward@{Forward}} \index{Forward@{Forward}!mlpack\+::rl\+::\+Categorical\+D\+QN@{mlpack\+::rl\+::\+Categorical\+D\+QN}} \subsubsection{Forward()} {\footnotesize\ttfamily void Forward (\begin{DoxyParamCaption}\item[{const arma\+::mat}]{state, }\item[{arma\+::mat \&}]{dist }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Perform the forward pass of the states in real batch mode. \begin{DoxyParams}{Parameters} {\em state} & The input state. \\ \hline {\em dist} & The predicted distributions. \\ \hline \end{DoxyParams} Definition at line 154 of file categorical\+\_\+dqn.\+hpp. \mbox{\label{classmlpack_1_1rl_1_1CategoricalDQN_aa68d74dc1e86e4352e00a3cab83a0e4a}} \index{mlpack\+::rl\+::\+Categorical\+D\+QN@{mlpack\+::rl\+::\+Categorical\+D\+QN}!Parameters@{Parameters}} \index{Parameters@{Parameters}!mlpack\+::rl\+::\+Categorical\+D\+QN@{mlpack\+::rl\+::\+Categorical\+D\+QN}} \subsubsection{Parameters()\hspace{0.1cm}{\footnotesize\ttfamily [1/2]}} {\footnotesize\ttfamily const arma\+::mat\& Parameters (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}} Return the Parameters. Definition at line 190 of file categorical\+\_\+dqn.\+hpp. \mbox{\label{classmlpack_1_1rl_1_1CategoricalDQN_a043f0ccd62e6711a18e0d81047be9a0a}} \index{mlpack\+::rl\+::\+Categorical\+D\+QN@{mlpack\+::rl\+::\+Categorical\+D\+QN}!Parameters@{Parameters}} \index{Parameters@{Parameters}!mlpack\+::rl\+::\+Categorical\+D\+QN@{mlpack\+::rl\+::\+Categorical\+D\+QN}} \subsubsection{Parameters()\hspace{0.1cm}{\footnotesize\ttfamily [2/2]}} {\footnotesize\ttfamily arma\+::mat\& Parameters (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Modify the Parameters. Definition at line 192 of file categorical\+\_\+dqn.\+hpp. \mbox{\label{classmlpack_1_1rl_1_1CategoricalDQN_aefcbb9daa8a256f06abf569abc4ebd1e}} \index{mlpack\+::rl\+::\+Categorical\+D\+QN@{mlpack\+::rl\+::\+Categorical\+D\+QN}!Predict@{Predict}} \index{Predict@{Predict}!mlpack\+::rl\+::\+Categorical\+D\+QN@{mlpack\+::rl\+::\+Categorical\+D\+QN}} \subsubsection{Predict()} {\footnotesize\ttfamily void Predict (\begin{DoxyParamCaption}\item[{const arma\+::mat}]{state, }\item[{arma\+::mat \&}]{action\+Value }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Predict the responses to a given set of predictors. The responses will reflect the output of the given output layer as returned by the output layer function. If you want to pass in a parameter and discard the original parameter object, be sure to use std\+::move to avoid unnecessary copy. \begin{DoxyParams}{Parameters} {\em state} & Input state. \\ \hline {\em action\+Value} & Matrix to put output action values of states input. \\ \hline \end{DoxyParams} Definition at line 131 of file categorical\+\_\+dqn.\+hpp. \mbox{\label{classmlpack_1_1rl_1_1CategoricalDQN_a13df76c46bb1caaacde3c67531e52382}} \index{mlpack\+::rl\+::\+Categorical\+D\+QN@{mlpack\+::rl\+::\+Categorical\+D\+QN}!Reset\+Noise@{Reset\+Noise}} \index{Reset\+Noise@{Reset\+Noise}!mlpack\+::rl\+::\+Categorical\+D\+QN@{mlpack\+::rl\+::\+Categorical\+D\+QN}} \subsubsection{Reset\+Noise()} {\footnotesize\ttfamily void Reset\+Noise (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Resets noise of the network, if the network is of type noisy. Definition at line 180 of file categorical\+\_\+dqn.\+hpp. \mbox{\label{classmlpack_1_1rl_1_1CategoricalDQN_a7178038c3cb8d247eadb94cd2058c432}} \index{mlpack\+::rl\+::\+Categorical\+D\+QN@{mlpack\+::rl\+::\+Categorical\+D\+QN}!Reset\+Parameters@{Reset\+Parameters}} \index{Reset\+Parameters@{Reset\+Parameters}!mlpack\+::rl\+::\+Categorical\+D\+QN@{mlpack\+::rl\+::\+Categorical\+D\+QN}} \subsubsection{Reset\+Parameters()} {\footnotesize\ttfamily void Reset\+Parameters (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Resets the parameters of the network. Definition at line 172 of file categorical\+\_\+dqn.\+hpp. The documentation for this class was generated from the following file\+:\begin{DoxyCompactItemize} \item /var/www/mlpack.\+ratml.\+org/mlpack.\+org/\+\_\+src/mlpack-\/git/src/mlpack/methods/reinforcement\+\_\+learning/q\+\_\+networks/\textbf{ categorical\+\_\+dqn.\+hpp}\end{DoxyCompactItemize}