\section{N\+Step\+Q\+Learning\+Worker$<$ Environment\+Type, Network\+Type, Updater\+Type, Policy\+Type $>$ Class Template Reference} \label{classmlpack_1_1rl_1_1NStepQLearningWorker}\index{N\+Step\+Q\+Learning\+Worker$<$ Environment\+Type, Network\+Type, Updater\+Type, Policy\+Type $>$@{N\+Step\+Q\+Learning\+Worker$<$ Environment\+Type, Network\+Type, Updater\+Type, Policy\+Type $>$}} Forward declaration of \doxyref{N\+Step\+Q\+Learning\+Worker}{p.}{classmlpack_1_1rl_1_1NStepQLearningWorker}. \subsection*{Public Types} \begin{DoxyCompactItemize} \item using \textbf{ Action\+Type} = typename Environment\+Type\+::\+Action \item using \textbf{ State\+Type} = typename Environment\+Type\+::\+State \item using \textbf{ Transition\+Type} = std\+::tuple$<$ \textbf{ State\+Type}, \textbf{ Action\+Type}, double, \textbf{ State\+Type} $>$ \end{DoxyCompactItemize} \subsection*{Public Member Functions} \begin{DoxyCompactItemize} \item \textbf{ N\+Step\+Q\+Learning\+Worker} (const Updater\+Type \&updater, const Environment\+Type \&environment, const \textbf{ Training\+Config} \&config, bool deterministic) \begin{DoxyCompactList}\small\item\em Construct N-\/step Q-\/\+Learning worker with the given parameters and environment. \end{DoxyCompactList}\item \textbf{ N\+Step\+Q\+Learning\+Worker} (const \textbf{ N\+Step\+Q\+Learning\+Worker} \&other) \begin{DoxyCompactList}\small\item\em Copy another \doxyref{N\+Step\+Q\+Learning\+Worker}{p.}{classmlpack_1_1rl_1_1NStepQLearningWorker}. \end{DoxyCompactList}\item \textbf{ N\+Step\+Q\+Learning\+Worker} (\textbf{ N\+Step\+Q\+Learning\+Worker} \&\&other) \begin{DoxyCompactList}\small\item\em Take ownership of another \doxyref{N\+Step\+Q\+Learning\+Worker}{p.}{classmlpack_1_1rl_1_1NStepQLearningWorker}. \end{DoxyCompactList}\item \textbf{ $\sim$\+N\+Step\+Q\+Learning\+Worker} () \begin{DoxyCompactList}\small\item\em Clean memory. \end{DoxyCompactList}\item void \textbf{ Initialize} (Network\+Type \&learning\+Network) \begin{DoxyCompactList}\small\item\em Initialize the worker. \end{DoxyCompactList}\item \textbf{ N\+Step\+Q\+Learning\+Worker} \& \textbf{ operator=} (const \textbf{ N\+Step\+Q\+Learning\+Worker} \&other) \begin{DoxyCompactList}\small\item\em Copy another \doxyref{N\+Step\+Q\+Learning\+Worker}{p.}{classmlpack_1_1rl_1_1NStepQLearningWorker}. \end{DoxyCompactList}\item \textbf{ N\+Step\+Q\+Learning\+Worker} \& \textbf{ operator=} (\textbf{ N\+Step\+Q\+Learning\+Worker} \&\&other) \begin{DoxyCompactList}\small\item\em Take ownership of another \doxyref{N\+Step\+Q\+Learning\+Worker}{p.}{classmlpack_1_1rl_1_1NStepQLearningWorker}. \end{DoxyCompactList}\item bool \textbf{ Step} (Network\+Type \&learning\+Network, Network\+Type \&target\+Network, size\+\_\+t \&total\+Steps, Policy\+Type \&policy, double \&total\+Reward) \begin{DoxyCompactList}\small\item\em The agent will execute one step. \end{DoxyCompactList}\end{DoxyCompactItemize} \subsection{Detailed Description} \subsubsection*{template$<$typename Environment\+Type, typename Network\+Type, typename Updater\+Type, typename Policy\+Type$>$\newline class mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker$<$ Environment\+Type, Network\+Type, Updater\+Type, Policy\+Type $>$} Forward declaration of \doxyref{N\+Step\+Q\+Learning\+Worker}{p.}{classmlpack_1_1rl_1_1NStepQLearningWorker}. N-\/step Q-\/\+Learning worker. \begin{DoxyTemplParams}{Template Parameters} {\em Environment\+Type} & The type of the reinforcement learning task. \\ \hline {\em Network\+Type} & The type of the network model. \\ \hline {\em Updater\+Type} & The type of the optimizer. \\ \hline {\em Policy\+Type} & The type of the behavior policy. \\ \hline \end{DoxyTemplParams} Definition at line 179 of file async\+\_\+learning.\+hpp. \subsection{Member Typedef Documentation} \mbox{\label{classmlpack_1_1rl_1_1NStepQLearningWorker_aaf7b2dc5d49d01961601c7c16be76777}} \index{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker@{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker}!Action\+Type@{Action\+Type}} \index{Action\+Type@{Action\+Type}!mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker@{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker}} \subsubsection{Action\+Type} {\footnotesize\ttfamily using \textbf{ Action\+Type} = typename Environment\+Type\+::\+Action} Definition at line 39 of file n\+\_\+step\+\_\+q\+\_\+learning\+\_\+worker.\+hpp. \mbox{\label{classmlpack_1_1rl_1_1NStepQLearningWorker_ada68ef405b7c331a2bee337614f00088}} \index{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker@{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker}!State\+Type@{State\+Type}} \index{State\+Type@{State\+Type}!mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker@{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker}} \subsubsection{State\+Type} {\footnotesize\ttfamily using \textbf{ State\+Type} = typename Environment\+Type\+::\+State} Definition at line 38 of file n\+\_\+step\+\_\+q\+\_\+learning\+\_\+worker.\+hpp. \mbox{\label{classmlpack_1_1rl_1_1NStepQLearningWorker_a810951afc4d825b2409bf5378e871dc5}} \index{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker@{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker}!Transition\+Type@{Transition\+Type}} \index{Transition\+Type@{Transition\+Type}!mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker@{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker}} \subsubsection{Transition\+Type} {\footnotesize\ttfamily using \textbf{ Transition\+Type} = std\+::tuple$<$\textbf{ State\+Type}, \textbf{ Action\+Type}, double, \textbf{ State\+Type}$>$} Definition at line 40 of file n\+\_\+step\+\_\+q\+\_\+learning\+\_\+worker.\+hpp. \subsection{Constructor \& Destructor Documentation} \mbox{\label{classmlpack_1_1rl_1_1NStepQLearningWorker_ad7c7dd8a63c39d20e9cda5d2ab348b91}} \index{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker@{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker}!N\+Step\+Q\+Learning\+Worker@{N\+Step\+Q\+Learning\+Worker}} \index{N\+Step\+Q\+Learning\+Worker@{N\+Step\+Q\+Learning\+Worker}!mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker@{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker}} \subsubsection{N\+Step\+Q\+Learning\+Worker()\hspace{0.1cm}{\footnotesize\ttfamily [1/3]}} {\footnotesize\ttfamily \textbf{ N\+Step\+Q\+Learning\+Worker} (\begin{DoxyParamCaption}\item[{const Updater\+Type \&}]{updater, }\item[{const Environment\+Type \&}]{environment, }\item[{const \textbf{ Training\+Config} \&}]{config, }\item[{bool}]{deterministic }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Construct N-\/step Q-\/\+Learning worker with the given parameters and environment. \begin{DoxyParams}{Parameters} {\em updater} & The optimizer. \\ \hline {\em environment} & The reinforcement learning task. \\ \hline {\em config} & Hyper-\/parameters. \\ \hline {\em deterministic} & Whether it should be deterministic. \\ \hline \end{DoxyParams} Definition at line 51 of file n\+\_\+step\+\_\+q\+\_\+learning\+\_\+worker.\+hpp. \mbox{\label{classmlpack_1_1rl_1_1NStepQLearningWorker_afe4d1c5b3fee4c0291044180f6ced66f}} \index{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker@{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker}!N\+Step\+Q\+Learning\+Worker@{N\+Step\+Q\+Learning\+Worker}} \index{N\+Step\+Q\+Learning\+Worker@{N\+Step\+Q\+Learning\+Worker}!mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker@{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker}} \subsubsection{N\+Step\+Q\+Learning\+Worker()\hspace{0.1cm}{\footnotesize\ttfamily [2/3]}} {\footnotesize\ttfamily \textbf{ N\+Step\+Q\+Learning\+Worker} (\begin{DoxyParamCaption}\item[{const \textbf{ N\+Step\+Q\+Learning\+Worker}$<$ Environment\+Type, Network\+Type, Updater\+Type, Policy\+Type $>$ \&}]{other }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Copy another \doxyref{N\+Step\+Q\+Learning\+Worker}{p.}{classmlpack_1_1rl_1_1NStepQLearningWorker}. \begin{DoxyParams}{Parameters} {\em other} & \doxyref{N\+Step\+Q\+Learning\+Worker}{p.}{classmlpack_1_1rl_1_1NStepQLearningWorker} to copy. \\ \hline \end{DoxyParams} Definition at line 71 of file n\+\_\+step\+\_\+q\+\_\+learning\+\_\+worker.\+hpp. \mbox{\label{classmlpack_1_1rl_1_1NStepQLearningWorker_a24d36c163a368e04fd6df657e0bec023}} \index{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker@{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker}!N\+Step\+Q\+Learning\+Worker@{N\+Step\+Q\+Learning\+Worker}} \index{N\+Step\+Q\+Learning\+Worker@{N\+Step\+Q\+Learning\+Worker}!mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker@{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker}} \subsubsection{N\+Step\+Q\+Learning\+Worker()\hspace{0.1cm}{\footnotesize\ttfamily [3/3]}} {\footnotesize\ttfamily \textbf{ N\+Step\+Q\+Learning\+Worker} (\begin{DoxyParamCaption}\item[{\textbf{ N\+Step\+Q\+Learning\+Worker}$<$ Environment\+Type, Network\+Type, Updater\+Type, Policy\+Type $>$ \&\&}]{other }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Take ownership of another \doxyref{N\+Step\+Q\+Learning\+Worker}{p.}{classmlpack_1_1rl_1_1NStepQLearningWorker}. \begin{DoxyParams}{Parameters} {\em other} & \doxyref{N\+Step\+Q\+Learning\+Worker}{p.}{classmlpack_1_1rl_1_1NStepQLearningWorker} to take ownership of. \\ \hline \end{DoxyParams} Definition at line 101 of file n\+\_\+step\+\_\+q\+\_\+learning\+\_\+worker.\+hpp. \mbox{\label{classmlpack_1_1rl_1_1NStepQLearningWorker_acdb590ece8e681e2a51b765c3c379cdb}} \index{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker@{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker}!````~N\+Step\+Q\+Learning\+Worker@{$\sim$\+N\+Step\+Q\+Learning\+Worker}} \index{````~N\+Step\+Q\+Learning\+Worker@{$\sim$\+N\+Step\+Q\+Learning\+Worker}!mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker@{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker}} \subsubsection{$\sim$\+N\+Step\+Q\+Learning\+Worker()} {\footnotesize\ttfamily $\sim$\textbf{ N\+Step\+Q\+Learning\+Worker} (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Clean memory. Definition at line 203 of file n\+\_\+step\+\_\+q\+\_\+learning\+\_\+worker.\+hpp. \subsection{Member Function Documentation} \mbox{\label{classmlpack_1_1rl_1_1NStepQLearningWorker_acf86e5b1010bd1f15be7b3fab93dfd75}} \index{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker@{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker}!Initialize@{Initialize}} \index{Initialize@{Initialize}!mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker@{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker}} \subsubsection{Initialize()} {\footnotesize\ttfamily void Initialize (\begin{DoxyParamCaption}\item[{Network\+Type \&}]{learning\+Network }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Initialize the worker. \begin{DoxyParams}{Parameters} {\em learning\+Network} & The shared network. \\ \hline \end{DoxyParams} Definition at line 214 of file n\+\_\+step\+\_\+q\+\_\+learning\+\_\+worker.\+hpp. \mbox{\label{classmlpack_1_1rl_1_1NStepQLearningWorker_a55a2bb2bc1015fd5bc079b0c1e3e65a2}} \index{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker@{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker}!operator=@{operator=}} \index{operator=@{operator=}!mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker@{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker}} \subsubsection{operator=()\hspace{0.1cm}{\footnotesize\ttfamily [1/2]}} {\footnotesize\ttfamily \textbf{ N\+Step\+Q\+Learning\+Worker}\& operator= (\begin{DoxyParamCaption}\item[{const \textbf{ N\+Step\+Q\+Learning\+Worker}$<$ Environment\+Type, Network\+Type, Updater\+Type, Policy\+Type $>$ \&}]{other }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Copy another \doxyref{N\+Step\+Q\+Learning\+Worker}{p.}{classmlpack_1_1rl_1_1NStepQLearningWorker}. \begin{DoxyParams}{Parameters} {\em other} & \doxyref{N\+Step\+Q\+Learning\+Worker}{p.}{classmlpack_1_1rl_1_1NStepQLearningWorker} to copy. \\ \hline \end{DoxyParams} Definition at line 131 of file n\+\_\+step\+\_\+q\+\_\+learning\+\_\+worker.\+hpp. \mbox{\label{classmlpack_1_1rl_1_1NStepQLearningWorker_a42109fefb4743f4a4346c2955f4b016e}} \index{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker@{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker}!operator=@{operator=}} \index{operator=@{operator=}!mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker@{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker}} \subsubsection{operator=()\hspace{0.1cm}{\footnotesize\ttfamily [2/2]}} {\footnotesize\ttfamily \textbf{ N\+Step\+Q\+Learning\+Worker}\& operator= (\begin{DoxyParamCaption}\item[{\textbf{ N\+Step\+Q\+Learning\+Worker}$<$ Environment\+Type, Network\+Type, Updater\+Type, Policy\+Type $>$ \&\&}]{other }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Take ownership of another \doxyref{N\+Step\+Q\+Learning\+Worker}{p.}{classmlpack_1_1rl_1_1NStepQLearningWorker}. \begin{DoxyParams}{Parameters} {\em other} & \doxyref{N\+Step\+Q\+Learning\+Worker}{p.}{classmlpack_1_1rl_1_1NStepQLearningWorker} to take ownership of. \\ \hline \end{DoxyParams} Definition at line 168 of file n\+\_\+step\+\_\+q\+\_\+learning\+\_\+worker.\+hpp. \mbox{\label{classmlpack_1_1rl_1_1NStepQLearningWorker_ad3920ee0951fcd4ce236584e19db8256}} \index{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker@{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker}!Step@{Step}} \index{Step@{Step}!mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker@{mlpack\+::rl\+::\+N\+Step\+Q\+Learning\+Worker}} \subsubsection{Step()} {\footnotesize\ttfamily bool Step (\begin{DoxyParamCaption}\item[{Network\+Type \&}]{learning\+Network, }\item[{Network\+Type \&}]{target\+Network, }\item[{size\+\_\+t \&}]{total\+Steps, }\item[{Policy\+Type \&}]{policy, }\item[{double \&}]{total\+Reward }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} The agent will execute one step. \begin{DoxyParams}{Parameters} {\em learning\+Network} & The shared learning network. \\ \hline {\em target\+Network} & The shared target network. \\ \hline {\em total\+Steps} & The shared counter for total steps. \\ \hline {\em policy} & The shared behavior policy. \\ \hline {\em total\+Reward} & This will be the episode return if the episode ends after this step. Otherwise this is invalid. \\ \hline \end{DoxyParams} \begin{DoxyReturn}{Returns} Indicate whether current episode ends after this step. \end{DoxyReturn} Definition at line 243 of file n\+\_\+step\+\_\+q\+\_\+learning\+\_\+worker.\+hpp. References Training\+Config\+::\+Discount(), Training\+Config\+::\+Gradient\+Limit(), Training\+Config\+::\+Step\+Limit(), Training\+Config\+::\+Step\+Size(), Training\+Config\+::\+Target\+Network\+Sync\+Interval(), and Training\+Config\+::\+Update\+Interval(). The documentation for this class was generated from the following files\+:\begin{DoxyCompactItemize} \item /var/www/mlpack.\+ratml.\+org/mlpack.\+org/\+\_\+src/mlpack-\/3.\+3.\+1/src/mlpack/methods/reinforcement\+\_\+learning/\textbf{ async\+\_\+learning.\+hpp}\item /var/www/mlpack.\+ratml.\+org/mlpack.\+org/\+\_\+src/mlpack-\/3.\+3.\+1/src/mlpack/methods/reinforcement\+\_\+learning/worker/\textbf{ n\+\_\+step\+\_\+q\+\_\+learning\+\_\+worker.\+hpp}\end{DoxyCompactItemize}