\section{One\+Step\+Sarsa\+Worker$<$ Environment\+Type, Network\+Type, Updater\+Type, Policy\+Type $>$ Class Template Reference}
\label{classmlpack_1_1rl_1_1OneStepSarsaWorker}\index{One\+Step\+Sarsa\+Worker$<$ Environment\+Type, Network\+Type, Updater\+Type, Policy\+Type $>$@{One\+Step\+Sarsa\+Worker$<$ Environment\+Type, Network\+Type, Updater\+Type, Policy\+Type $>$}}


Forward declaration of \doxyref{One\+Step\+Sarsa\+Worker}{p.}{classmlpack_1_1rl_1_1OneStepSarsaWorker}.  


\subsection*{Public Types}
\begin{DoxyCompactItemize}
\item 
using \textbf{ Action\+Type} = typename Environment\+Type\+::\+Action
\item 
using \textbf{ State\+Type} = typename Environment\+Type\+::\+State
\item 
using \textbf{ Transition\+Type} = std\+::tuple$<$ \textbf{ State\+Type}, \textbf{ Action\+Type}, double, \textbf{ State\+Type}, \textbf{ Action\+Type} $>$
\end{DoxyCompactItemize}
\subsection*{Public Member Functions}
\begin{DoxyCompactItemize}
\item 
\textbf{ One\+Step\+Sarsa\+Worker} (const Updater\+Type \&updater, const Environment\+Type \&environment, const \textbf{ Training\+Config} \&config, bool deterministic)
\begin{DoxyCompactList}\small\item\em Construct one step sarsa worker with the given parameters and environment. \end{DoxyCompactList}\item 
\textbf{ One\+Step\+Sarsa\+Worker} (const \textbf{ One\+Step\+Sarsa\+Worker} \&other)
\begin{DoxyCompactList}\small\item\em Copy another \doxyref{One\+Step\+Sarsa\+Worker}{p.}{classmlpack_1_1rl_1_1OneStepSarsaWorker}. \end{DoxyCompactList}\item 
\textbf{ One\+Step\+Sarsa\+Worker} (\textbf{ One\+Step\+Sarsa\+Worker} \&\&other)
\begin{DoxyCompactList}\small\item\em Take ownership of another \doxyref{One\+Step\+Sarsa\+Worker}{p.}{classmlpack_1_1rl_1_1OneStepSarsaWorker}. \end{DoxyCompactList}\item 
\textbf{ $\sim$\+One\+Step\+Sarsa\+Worker} ()
\begin{DoxyCompactList}\small\item\em Clean memory. \end{DoxyCompactList}\item 
void \textbf{ Initialize} (Network\+Type \&learning\+Network)
\begin{DoxyCompactList}\small\item\em Initialize the worker. \end{DoxyCompactList}\item 
\textbf{ One\+Step\+Sarsa\+Worker} \& \textbf{ operator=} (const \textbf{ One\+Step\+Sarsa\+Worker} \&other)
\begin{DoxyCompactList}\small\item\em Copy another \doxyref{One\+Step\+Sarsa\+Worker}{p.}{classmlpack_1_1rl_1_1OneStepSarsaWorker}. \end{DoxyCompactList}\item 
\textbf{ One\+Step\+Sarsa\+Worker} \& \textbf{ operator=} (\textbf{ One\+Step\+Sarsa\+Worker} \&\&other)
\begin{DoxyCompactList}\small\item\em Take ownership of another \doxyref{One\+Step\+Sarsa\+Worker}{p.}{classmlpack_1_1rl_1_1OneStepSarsaWorker}. \end{DoxyCompactList}\item 
bool \textbf{ Step} (Network\+Type \&learning\+Network, Network\+Type \&target\+Network, size\+\_\+t \&total\+Steps, Policy\+Type \&policy, double \&total\+Reward)
\begin{DoxyCompactList}\small\item\em The agent will execute one step. \end{DoxyCompactList}\end{DoxyCompactItemize}


\subsection{Detailed Description}
\subsubsection*{template$<$typename Environment\+Type, typename Network\+Type, typename Updater\+Type, typename Policy\+Type$>$\newline
class mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker$<$ Environment\+Type, Network\+Type, Updater\+Type, Policy\+Type $>$}

Forward declaration of \doxyref{One\+Step\+Sarsa\+Worker}{p.}{classmlpack_1_1rl_1_1OneStepSarsaWorker}. 

One step Sarsa worker.


\begin{DoxyTemplParams}{Template Parameters}
{\em Environment\+Type} & The type of the reinforcement learning task. \\
\hline
{\em Network\+Type} & The type of the network model. \\
\hline
{\em Updater\+Type} & The type of the optimizer. \\
\hline
{\em Policy\+Type} & The type of the behavior policy. \\
\hline
\end{DoxyTemplParams}


Definition at line 163 of file async\+\_\+learning.\+hpp.



\subsection{Member Typedef Documentation}
\mbox{\label{classmlpack_1_1rl_1_1OneStepSarsaWorker_aaf7b2dc5d49d01961601c7c16be76777}} 
\index{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker@{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker}!Action\+Type@{Action\+Type}}
\index{Action\+Type@{Action\+Type}!mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker@{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker}}
\subsubsection{Action\+Type}
{\footnotesize\ttfamily using \textbf{ Action\+Type} =  typename Environment\+Type\+::\+Action}



Definition at line 39 of file one\+\_\+step\+\_\+sarsa\+\_\+worker.\+hpp.

\mbox{\label{classmlpack_1_1rl_1_1OneStepSarsaWorker_ada68ef405b7c331a2bee337614f00088}} 
\index{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker@{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker}!State\+Type@{State\+Type}}
\index{State\+Type@{State\+Type}!mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker@{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker}}
\subsubsection{State\+Type}
{\footnotesize\ttfamily using \textbf{ State\+Type} =  typename Environment\+Type\+::\+State}



Definition at line 38 of file one\+\_\+step\+\_\+sarsa\+\_\+worker.\+hpp.

\mbox{\label{classmlpack_1_1rl_1_1OneStepSarsaWorker_a72a12c59d4e69c0e93c00d255e29b3dd}} 
\index{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker@{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker}!Transition\+Type@{Transition\+Type}}
\index{Transition\+Type@{Transition\+Type}!mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker@{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker}}
\subsubsection{Transition\+Type}
{\footnotesize\ttfamily using \textbf{ Transition\+Type} =  std\+::tuple$<$\textbf{ State\+Type}, \textbf{ Action\+Type}, double, \textbf{ State\+Type}, \textbf{ Action\+Type}$>$}



Definition at line 41 of file one\+\_\+step\+\_\+sarsa\+\_\+worker.\+hpp.



\subsection{Constructor \& Destructor Documentation}
\mbox{\label{classmlpack_1_1rl_1_1OneStepSarsaWorker_a0a8e6f78863ec7193971f78c3b46c8f8}} 
\index{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker@{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker}!One\+Step\+Sarsa\+Worker@{One\+Step\+Sarsa\+Worker}}
\index{One\+Step\+Sarsa\+Worker@{One\+Step\+Sarsa\+Worker}!mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker@{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker}}
\subsubsection{One\+Step\+Sarsa\+Worker()\hspace{0.1cm}{\footnotesize\ttfamily [1/3]}}
{\footnotesize\ttfamily \textbf{ One\+Step\+Sarsa\+Worker} (\begin{DoxyParamCaption}\item[{const Updater\+Type \&}]{updater,  }\item[{const Environment\+Type \&}]{environment,  }\item[{const \textbf{ Training\+Config} \&}]{config,  }\item[{bool}]{deterministic }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}}



Construct one step sarsa worker with the given parameters and environment. 


\begin{DoxyParams}{Parameters}
{\em updater} & The optimizer. \\
\hline
{\em environment} & The reinforcement learning task. \\
\hline
{\em config} & Hyper-\/parameters. \\
\hline
{\em deterministic} & Whether it should be deterministic. \\
\hline
\end{DoxyParams}


Definition at line 52 of file one\+\_\+step\+\_\+sarsa\+\_\+worker.\+hpp.

\mbox{\label{classmlpack_1_1rl_1_1OneStepSarsaWorker_a2aa68a49d17730854b98756e2fb11269}} 
\index{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker@{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker}!One\+Step\+Sarsa\+Worker@{One\+Step\+Sarsa\+Worker}}
\index{One\+Step\+Sarsa\+Worker@{One\+Step\+Sarsa\+Worker}!mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker@{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker}}
\subsubsection{One\+Step\+Sarsa\+Worker()\hspace{0.1cm}{\footnotesize\ttfamily [2/3]}}
{\footnotesize\ttfamily \textbf{ One\+Step\+Sarsa\+Worker} (\begin{DoxyParamCaption}\item[{const \textbf{ One\+Step\+Sarsa\+Worker}$<$ Environment\+Type, Network\+Type, Updater\+Type, Policy\+Type $>$ \&}]{other }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}}



Copy another \doxyref{One\+Step\+Sarsa\+Worker}{p.}{classmlpack_1_1rl_1_1OneStepSarsaWorker}. 


\begin{DoxyParams}{Parameters}
{\em other} & \doxyref{One\+Step\+Sarsa\+Worker}{p.}{classmlpack_1_1rl_1_1OneStepSarsaWorker} to copy. \\
\hline
\end{DoxyParams}


Definition at line 72 of file one\+\_\+step\+\_\+sarsa\+\_\+worker.\+hpp.

\mbox{\label{classmlpack_1_1rl_1_1OneStepSarsaWorker_aa6de7d2c1bfef9909f3a4d00a0c8a7da}} 
\index{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker@{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker}!One\+Step\+Sarsa\+Worker@{One\+Step\+Sarsa\+Worker}}
\index{One\+Step\+Sarsa\+Worker@{One\+Step\+Sarsa\+Worker}!mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker@{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker}}
\subsubsection{One\+Step\+Sarsa\+Worker()\hspace{0.1cm}{\footnotesize\ttfamily [3/3]}}
{\footnotesize\ttfamily \textbf{ One\+Step\+Sarsa\+Worker} (\begin{DoxyParamCaption}\item[{\textbf{ One\+Step\+Sarsa\+Worker}$<$ Environment\+Type, Network\+Type, Updater\+Type, Policy\+Type $>$ \&\&}]{other }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}}



Take ownership of another \doxyref{One\+Step\+Sarsa\+Worker}{p.}{classmlpack_1_1rl_1_1OneStepSarsaWorker}. 


\begin{DoxyParams}{Parameters}
{\em other} & \doxyref{One\+Step\+Sarsa\+Worker}{p.}{classmlpack_1_1rl_1_1OneStepSarsaWorker} to take ownership of. \\
\hline
\end{DoxyParams}


Definition at line 103 of file one\+\_\+step\+\_\+sarsa\+\_\+worker.\+hpp.

\mbox{\label{classmlpack_1_1rl_1_1OneStepSarsaWorker_aae1e6636d737cde6b5426f51c93b286b}} 
\index{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker@{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker}!````~One\+Step\+Sarsa\+Worker@{$\sim$\+One\+Step\+Sarsa\+Worker}}
\index{````~One\+Step\+Sarsa\+Worker@{$\sim$\+One\+Step\+Sarsa\+Worker}!mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker@{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker}}
\subsubsection{$\sim$\+One\+Step\+Sarsa\+Worker()}
{\footnotesize\ttfamily $\sim$\textbf{ One\+Step\+Sarsa\+Worker} (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}}



Clean memory. 



Definition at line 208 of file one\+\_\+step\+\_\+sarsa\+\_\+worker.\+hpp.



\subsection{Member Function Documentation}
\mbox{\label{classmlpack_1_1rl_1_1OneStepSarsaWorker_acf86e5b1010bd1f15be7b3fab93dfd75}} 
\index{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker@{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker}!Initialize@{Initialize}}
\index{Initialize@{Initialize}!mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker@{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker}}
\subsubsection{Initialize()}
{\footnotesize\ttfamily void Initialize (\begin{DoxyParamCaption}\item[{Network\+Type \&}]{learning\+Network }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}}



Initialize the worker. 


\begin{DoxyParams}{Parameters}
{\em learning\+Network} & The shared network. \\
\hline
\end{DoxyParams}


Definition at line 219 of file one\+\_\+step\+\_\+sarsa\+\_\+worker.\+hpp.

\mbox{\label{classmlpack_1_1rl_1_1OneStepSarsaWorker_a1730ded96dff7297d23125adba23b369}} 
\index{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker@{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker}!operator=@{operator=}}
\index{operator=@{operator=}!mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker@{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker}}
\subsubsection{operator=()\hspace{0.1cm}{\footnotesize\ttfamily [1/2]}}
{\footnotesize\ttfamily \textbf{ One\+Step\+Sarsa\+Worker}\& operator= (\begin{DoxyParamCaption}\item[{const \textbf{ One\+Step\+Sarsa\+Worker}$<$ Environment\+Type, Network\+Type, Updater\+Type, Policy\+Type $>$ \&}]{other }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}}



Copy another \doxyref{One\+Step\+Sarsa\+Worker}{p.}{classmlpack_1_1rl_1_1OneStepSarsaWorker}. 


\begin{DoxyParams}{Parameters}
{\em other} & \doxyref{One\+Step\+Sarsa\+Worker}{p.}{classmlpack_1_1rl_1_1OneStepSarsaWorker} to copy. \\
\hline
\end{DoxyParams}


Definition at line 134 of file one\+\_\+step\+\_\+sarsa\+\_\+worker.\+hpp.

\mbox{\label{classmlpack_1_1rl_1_1OneStepSarsaWorker_acfb52a3ee867d85209f2c1cc43dece24}} 
\index{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker@{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker}!operator=@{operator=}}
\index{operator=@{operator=}!mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker@{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker}}
\subsubsection{operator=()\hspace{0.1cm}{\footnotesize\ttfamily [2/2]}}
{\footnotesize\ttfamily \textbf{ One\+Step\+Sarsa\+Worker}\& operator= (\begin{DoxyParamCaption}\item[{\textbf{ One\+Step\+Sarsa\+Worker}$<$ Environment\+Type, Network\+Type, Updater\+Type, Policy\+Type $>$ \&\&}]{other }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}}



Take ownership of another \doxyref{One\+Step\+Sarsa\+Worker}{p.}{classmlpack_1_1rl_1_1OneStepSarsaWorker}. 


\begin{DoxyParams}{Parameters}
{\em other} & \doxyref{One\+Step\+Sarsa\+Worker}{p.}{classmlpack_1_1rl_1_1OneStepSarsaWorker} to take ownership of. \\
\hline
\end{DoxyParams}


Definition at line 172 of file one\+\_\+step\+\_\+sarsa\+\_\+worker.\+hpp.

\mbox{\label{classmlpack_1_1rl_1_1OneStepSarsaWorker_ad3920ee0951fcd4ce236584e19db8256}} 
\index{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker@{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker}!Step@{Step}}
\index{Step@{Step}!mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker@{mlpack\+::rl\+::\+One\+Step\+Sarsa\+Worker}}
\subsubsection{Step()}
{\footnotesize\ttfamily bool Step (\begin{DoxyParamCaption}\item[{Network\+Type \&}]{learning\+Network,  }\item[{Network\+Type \&}]{target\+Network,  }\item[{size\+\_\+t \&}]{total\+Steps,  }\item[{Policy\+Type \&}]{policy,  }\item[{double \&}]{total\+Reward }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}}



The agent will execute one step. 


\begin{DoxyParams}{Parameters}
{\em learning\+Network} & The shared learning network. \\
\hline
{\em target\+Network} & The shared target network. \\
\hline
{\em total\+Steps} & The shared counter for total steps. \\
\hline
{\em policy} & The shared behavior policy. \\
\hline
{\em total\+Reward} & This will be the episode return if the episode ends after this step. Otherwise this is invalid. \\
\hline
\end{DoxyParams}
\begin{DoxyReturn}{Returns}
Indicate whether current episode ends after this step. 
\end{DoxyReturn}


Definition at line 248 of file one\+\_\+step\+\_\+sarsa\+\_\+worker.\+hpp.



References Training\+Config\+::\+Discount(), Training\+Config\+::\+Gradient\+Limit(), Training\+Config\+::\+Step\+Limit(), Training\+Config\+::\+Step\+Size(), Training\+Config\+::\+Target\+Network\+Sync\+Interval(), and Training\+Config\+::\+Update\+Interval().



The documentation for this class was generated from the following files\+:\begin{DoxyCompactItemize}
\item 
/var/www/mlpack.\+ratml.\+org/mlpack.\+org/\+\_\+src/mlpack-\/3.\+3.\+0/src/mlpack/methods/reinforcement\+\_\+learning/\textbf{ async\+\_\+learning.\+hpp}\item 
/var/www/mlpack.\+ratml.\+org/mlpack.\+org/\+\_\+src/mlpack-\/3.\+3.\+0/src/mlpack/methods/reinforcement\+\_\+learning/worker/\textbf{ one\+\_\+step\+\_\+sarsa\+\_\+worker.\+hpp}\end{DoxyCompactItemize}
