\section{Random\+Replay$<$ Environment\+Type $>$ Class Template Reference} \label{classmlpack_1_1rl_1_1RandomReplay}\index{Random\+Replay$<$ Environment\+Type $>$@{Random\+Replay$<$ Environment\+Type $>$}} Implementation of random experience replay. \subsection*{Public Types} \begin{DoxyCompactItemize} \item using \textbf{ Action\+Type} = typename Environment\+Type\+::\+Action \begin{DoxyCompactList}\small\item\em Convenient typedef for action. \end{DoxyCompactList}\item using \textbf{ State\+Type} = typename Environment\+Type\+::\+State \begin{DoxyCompactList}\small\item\em Convenient typedef for state. \end{DoxyCompactList}\end{DoxyCompactItemize} \subsection*{Public Member Functions} \begin{DoxyCompactItemize} \item \textbf{ Random\+Replay} () \item \textbf{ Random\+Replay} (const size\+\_\+t batch\+Size, const size\+\_\+t capacity, const size\+\_\+t dimension=State\+Type\+::dimension) \begin{DoxyCompactList}\small\item\em Construct an instance of random experience replay class. \end{DoxyCompactList}\item void \textbf{ Sample} (arma\+::mat \&sampled\+States, arma\+::icolvec \&sampled\+Actions, arma\+::colvec \&sampled\+Rewards, arma\+::mat \&sampled\+Next\+States, arma\+::icolvec \&is\+Terminal) \begin{DoxyCompactList}\small\item\em Sample some experiences. \end{DoxyCompactList}\item const size\+\_\+t \& \textbf{ Size} () \begin{DoxyCompactList}\small\item\em Get the number of transitions in the memory. \end{DoxyCompactList}\item void \textbf{ Store} (const \textbf{ State\+Type} \&state, \textbf{ Action\+Type} action, double reward, const \textbf{ State\+Type} \&next\+State, bool is\+End) \begin{DoxyCompactList}\small\item\em Store the given experience. \end{DoxyCompactList}\item void \textbf{ Update} (arma\+::mat, arma\+::icolvec, arma\+::mat, arma\+::mat \&) \begin{DoxyCompactList}\small\item\em Update the priorities of transitions and Update the gradients. \end{DoxyCompactList}\end{DoxyCompactItemize} \subsection{Detailed Description} \subsubsection*{template$<$typename Environment\+Type$>$\newline class mlpack\+::rl\+::\+Random\+Replay$<$ Environment\+Type $>$} Implementation of random experience replay. At each time step, interactions between the agent and the environment will be saved to a memory buffer. When necessary, we can simply sample previous experiences from the buffer to train the agent. Typically this would be a random sample and the memory will be a First-\/\+In-\/\+First-\/\+Out buffer. For more information, see the following. \begin{DoxyCode} @phdthesis\{lin1993reinforcement, title = \{Reinforcement learning \textcolor{keywordflow}{for} robots \textcolor{keyword}{using} neural networks\}, author = \{Lin, Long-Ji\}, year = \{1993\}, school = \{Fujitsu Laboratories Ltd\} \} \end{DoxyCode} \begin{DoxyTemplParams}{Template Parameters} {\em Environment\+Type} & Desired task. \\ \hline \end{DoxyTemplParams} Definition at line 43 of file random\+\_\+replay.\+hpp. \subsection{Member Typedef Documentation} \mbox{\label{classmlpack_1_1rl_1_1RandomReplay_aaf7b2dc5d49d01961601c7c16be76777}} \index{mlpack\+::rl\+::\+Random\+Replay@{mlpack\+::rl\+::\+Random\+Replay}!Action\+Type@{Action\+Type}} \index{Action\+Type@{Action\+Type}!mlpack\+::rl\+::\+Random\+Replay@{mlpack\+::rl\+::\+Random\+Replay}} \subsubsection{Action\+Type} {\footnotesize\ttfamily using \textbf{ Action\+Type} = typename Environment\+Type\+::\+Action} Convenient typedef for action. Definition at line 47 of file random\+\_\+replay.\+hpp. \mbox{\label{classmlpack_1_1rl_1_1RandomReplay_ada68ef405b7c331a2bee337614f00088}} \index{mlpack\+::rl\+::\+Random\+Replay@{mlpack\+::rl\+::\+Random\+Replay}!State\+Type@{State\+Type}} \index{State\+Type@{State\+Type}!mlpack\+::rl\+::\+Random\+Replay@{mlpack\+::rl\+::\+Random\+Replay}} \subsubsection{State\+Type} {\footnotesize\ttfamily using \textbf{ State\+Type} = typename Environment\+Type\+::\+State} Convenient typedef for state. Definition at line 50 of file random\+\_\+replay.\+hpp. \subsection{Constructor \& Destructor Documentation} \mbox{\label{classmlpack_1_1rl_1_1RandomReplay_a63672681bb61c707da6ca94ada59164a}} \index{mlpack\+::rl\+::\+Random\+Replay@{mlpack\+::rl\+::\+Random\+Replay}!Random\+Replay@{Random\+Replay}} \index{Random\+Replay@{Random\+Replay}!mlpack\+::rl\+::\+Random\+Replay@{mlpack\+::rl\+::\+Random\+Replay}} \subsubsection{Random\+Replay()\hspace{0.1cm}{\footnotesize\ttfamily [1/2]}} {\footnotesize\ttfamily \textbf{ Random\+Replay} (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Definition at line 52 of file random\+\_\+replay.\+hpp. \mbox{\label{classmlpack_1_1rl_1_1RandomReplay_a7e67cad3191a0d22fbd442a8e1fd2652}} \index{mlpack\+::rl\+::\+Random\+Replay@{mlpack\+::rl\+::\+Random\+Replay}!Random\+Replay@{Random\+Replay}} \index{Random\+Replay@{Random\+Replay}!mlpack\+::rl\+::\+Random\+Replay@{mlpack\+::rl\+::\+Random\+Replay}} \subsubsection{Random\+Replay()\hspace{0.1cm}{\footnotesize\ttfamily [2/2]}} {\footnotesize\ttfamily \textbf{ Random\+Replay} (\begin{DoxyParamCaption}\item[{const size\+\_\+t}]{batch\+Size, }\item[{const size\+\_\+t}]{capacity, }\item[{const size\+\_\+t}]{dimension = {\ttfamily StateType\+:\+:dimension} }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Construct an instance of random experience replay class. \begin{DoxyParams}{Parameters} {\em batch\+Size} & Number of examples returned at each sample. \\ \hline {\em capacity} & Total memory size in terms of number of examples. \\ \hline {\em dimension} & The dimension of an encoded state. \\ \hline \end{DoxyParams} Definition at line 66 of file random\+\_\+replay.\+hpp. \subsection{Member Function Documentation} \mbox{\label{classmlpack_1_1rl_1_1RandomReplay_aacbf6723cb49d015e918c82011889dcc}} \index{mlpack\+::rl\+::\+Random\+Replay@{mlpack\+::rl\+::\+Random\+Replay}!Sample@{Sample}} \index{Sample@{Sample}!mlpack\+::rl\+::\+Random\+Replay@{mlpack\+::rl\+::\+Random\+Replay}} \subsubsection{Sample()} {\footnotesize\ttfamily void Sample (\begin{DoxyParamCaption}\item[{arma\+::mat \&}]{sampled\+States, }\item[{arma\+::icolvec \&}]{sampled\+Actions, }\item[{arma\+::colvec \&}]{sampled\+Rewards, }\item[{arma\+::mat \&}]{sampled\+Next\+States, }\item[{arma\+::icolvec \&}]{is\+Terminal }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Sample some experiences. \begin{DoxyParams}{Parameters} {\em sampled\+States} & Sampled encoded states. \\ \hline {\em sampled\+Actions} & Sampled actions. \\ \hline {\em sampled\+Rewards} & Sampled rewards. \\ \hline {\em sampled\+Next\+States} & Sampled encoded next states. \\ \hline {\em is\+Terminal} & Indicate whether corresponding next state is terminal state. \\ \hline \end{DoxyParams} Definition at line 118 of file random\+\_\+replay.\+hpp. \mbox{\label{classmlpack_1_1rl_1_1RandomReplay_ab8983dc8f7847b4c77148b86d0e7fc8d}} \index{mlpack\+::rl\+::\+Random\+Replay@{mlpack\+::rl\+::\+Random\+Replay}!Size@{Size}} \index{Size@{Size}!mlpack\+::rl\+::\+Random\+Replay@{mlpack\+::rl\+::\+Random\+Replay}} \subsubsection{Size()} {\footnotesize\ttfamily const size\+\_\+t\& Size (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Get the number of transitions in the memory. \begin{DoxyReturn}{Returns} Actual used memory size \end{DoxyReturn} Definition at line 140 of file random\+\_\+replay.\+hpp. \mbox{\label{classmlpack_1_1rl_1_1RandomReplay_a2a1280d285e5c5dc89392ae9a9bb2c97}} \index{mlpack\+::rl\+::\+Random\+Replay@{mlpack\+::rl\+::\+Random\+Replay}!Store@{Store}} \index{Store@{Store}!mlpack\+::rl\+::\+Random\+Replay@{mlpack\+::rl\+::\+Random\+Replay}} \subsubsection{Store()} {\footnotesize\ttfamily void Store (\begin{DoxyParamCaption}\item[{const \textbf{ State\+Type} \&}]{state, }\item[{\textbf{ Action\+Type}}]{action, }\item[{double}]{reward, }\item[{const \textbf{ State\+Type} \&}]{next\+State, }\item[{bool}]{is\+End }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Store the given experience. \begin{DoxyParams}{Parameters} {\em state} & Given state. \\ \hline {\em action} & Given action. \\ \hline {\em reward} & Given reward. \\ \hline {\em next\+State} & Given next state. \\ \hline {\em is\+End} & Whether next state is terminal state. \\ \hline \end{DoxyParams} Definition at line 89 of file random\+\_\+replay.\+hpp. \mbox{\label{classmlpack_1_1rl_1_1RandomReplay_ad1c8f06aa1b5de703eed4ef3182784bf}} \index{mlpack\+::rl\+::\+Random\+Replay@{mlpack\+::rl\+::\+Random\+Replay}!Update@{Update}} \index{Update@{Update}!mlpack\+::rl\+::\+Random\+Replay@{mlpack\+::rl\+::\+Random\+Replay}} \subsubsection{Update()} {\footnotesize\ttfamily void Update (\begin{DoxyParamCaption}\item[{arma\+::mat}]{, }\item[{arma\+::icolvec}]{, }\item[{arma\+::mat}]{, }\item[{arma\+::mat \&}]{ }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Update the priorities of transitions and Update the gradients. \begin{DoxyParams}{Parameters} {\em $\ast$} & (target) The learned value \\ \hline {\em $\ast$} & (sampled\+Actions) Agent\textquotesingle{}s sampled action \\ \hline {\em $\ast$} & (next\+Action\+Values) Agent\textquotesingle{}s next action \\ \hline {\em $\ast$} & (gradients) The model\textquotesingle{}s gradients \\ \hline \end{DoxyParams} Definition at line 153 of file random\+\_\+replay.\+hpp. The documentation for this class was generated from the following file\+:\begin{DoxyCompactItemize} \item /var/www/mlpack.\+ratml.\+org/mlpack.\+org/\+\_\+src/mlpack-\/3.\+3.\+2/src/mlpack/methods/reinforcement\+\_\+learning/replay/\textbf{ random\+\_\+replay.\+hpp}\end{DoxyCompactItemize}