\section{Acrobot Class Reference}
\label{classmlpack_1_1rl_1_1Acrobot}\index{Acrobot@{Acrobot}}


Implementation of \doxyref{Acrobot}{p.}{classmlpack_1_1rl_1_1Acrobot} game.  


\subsection*{Classes}
\begin{DoxyCompactItemize}
\item 
class \textbf{ State}
\end{DoxyCompactItemize}
\subsection*{Public Types}
\begin{DoxyCompactItemize}
\item 
enum \textbf{ Action} \{ \newline
\textbf{ negative\+Torque}, 
\newline
\textbf{ zero\+Torque}, 
\newline
\textbf{ positive\+Torque}, 
\newline
\textbf{ size}
 \}
\end{DoxyCompactItemize}
\subsection*{Public Member Functions}
\begin{DoxyCompactItemize}
\item 
\textbf{ Acrobot} (const double gravity=9.\+81, const double link\+Length1=1.\+0, const double link\+Length2=1.\+0, const double link\+Mass1=1.\+0, const double link\+Mass2=1.\+0, const double link\+Com1=0.\+5, const double link\+Com2=0.\+5, const double link\+Moi=1.\+0, const double max\+Vel1=4 $\ast$\textbf{ M\+\_\+\+PI}, const double max\+Vel2=9 $\ast$\textbf{ M\+\_\+\+PI}, const double dt=0.\+2, const double done\+Reward=0, const size\+\_\+t max\+Steps=0)
\begin{DoxyCompactList}\small\item\em Construct a \doxyref{Acrobot}{p.}{classmlpack_1_1rl_1_1Acrobot} instance using the given constants. \end{DoxyCompactList}\item 
arma\+::colvec \textbf{ Dsdt} (arma\+::colvec state, const double torque) const
\begin{DoxyCompactList}\small\item\em This is the ordinary differential equations required for estimation of next\+State through R\+K4 method. \end{DoxyCompactList}\item 
\textbf{ State} \textbf{ Initial\+Sample} ()
\begin{DoxyCompactList}\small\item\em This function does random initialization of state space. \end{DoxyCompactList}\item 
bool \textbf{ Is\+Terminal} (const \textbf{ State} \&state) const
\begin{DoxyCompactList}\small\item\em This function checks if the acrobot has reached the terminal state. \end{DoxyCompactList}\item 
size\+\_\+t \textbf{ Max\+Steps} () const
\begin{DoxyCompactList}\small\item\em Get the maximum number of steps allowed. \end{DoxyCompactList}\item 
size\+\_\+t \& \textbf{ Max\+Steps} ()
\begin{DoxyCompactList}\small\item\em Set the maximum number of steps allowed. \end{DoxyCompactList}\item 
arma\+::colvec \textbf{ Rk4} (const arma\+::colvec state, const double torque) const
\begin{DoxyCompactList}\small\item\em This function calls the R\+K4 iterative method to estimate the next state based on given ordinary differential equation. \end{DoxyCompactList}\item 
double \textbf{ Sample} (const \textbf{ State} \&state, const \textbf{ Action} \&action, \textbf{ State} \&next\+State)
\begin{DoxyCompactList}\small\item\em Dynamics of the \doxyref{Acrobot}{p.}{classmlpack_1_1rl_1_1Acrobot} System. \end{DoxyCompactList}\item 
double \textbf{ Sample} (const \textbf{ State} \&state, const \textbf{ Action} \&action)
\begin{DoxyCompactList}\small\item\em Dynamics of the \doxyref{Acrobot}{p.}{classmlpack_1_1rl_1_1Acrobot} System. \end{DoxyCompactList}\item 
size\+\_\+t \textbf{ Steps\+Performed} () const
\begin{DoxyCompactList}\small\item\em Get the number of steps performed. \end{DoxyCompactList}\item 
double \textbf{ Torque} (const \textbf{ Action} \&action) const
\begin{DoxyCompactList}\small\item\em This function calculates the torque for a particular action. \end{DoxyCompactList}\item 
double \textbf{ Wrap} (double value, const double minimum, const double maximum) const
\begin{DoxyCompactList}\small\item\em Wrap funtion is required to truncate the angle value from -\/180 to 180. \end{DoxyCompactList}\end{DoxyCompactItemize}


\subsection{Detailed Description}
Implementation of \doxyref{Acrobot}{p.}{classmlpack_1_1rl_1_1Acrobot} game. 

\doxyref{Acrobot}{p.}{classmlpack_1_1rl_1_1Acrobot} is a 2-\/link pendulum with only the second joint actuated. Intitially, both links point downwards. The goal is to swing the end-\/effector at a height at least the length of one link above the base. Both links can swing freely and can pass by each other, i.\+e., they don\textquotesingle{}t collide when they have the same angle. 

Definition at line 28 of file acrobot.\+hpp.



\subsection{Member Enumeration Documentation}
\mbox{\label{classmlpack_1_1rl_1_1Acrobot_a8bb1ef53467e4f61410d12822d922498}} 
\index{mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}!Action@{Action}}
\index{Action@{Action}!mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}}
\subsubsection{Action}
{\footnotesize\ttfamily enum \textbf{ Action}}

\begin{DoxyEnumFields}{Enumerator}
\raisebox{\heightof{T}}[0pt][0pt]{\index{negative\+Torque@{negative\+Torque}!mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}}\index{mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}!negative\+Torque@{negative\+Torque}}}\mbox{\label{classmlpack_1_1rl_1_1Acrobot_a8bb1ef53467e4f61410d12822d922498af6e2c4a274a7061460fc52bb0e8d92dd}} 
negative\+Torque&\\
\hline

\raisebox{\heightof{T}}[0pt][0pt]{\index{zero\+Torque@{zero\+Torque}!mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}}\index{mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}!zero\+Torque@{zero\+Torque}}}\mbox{\label{classmlpack_1_1rl_1_1Acrobot_a8bb1ef53467e4f61410d12822d922498a88b15afa5c5e7a209c85650230e11752}} 
zero\+Torque&\\
\hline

\raisebox{\heightof{T}}[0pt][0pt]{\index{positive\+Torque@{positive\+Torque}!mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}}\index{mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}!positive\+Torque@{positive\+Torque}}}\mbox{\label{classmlpack_1_1rl_1_1Acrobot_a8bb1ef53467e4f61410d12822d922498a4e74e4f919288406e2c194abe8873de8}} 
positive\+Torque&\\
\hline

\raisebox{\heightof{T}}[0pt][0pt]{\index{size@{size}!mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}}\index{mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}!size@{size}}}\mbox{\label{classmlpack_1_1rl_1_1Acrobot_a8bb1ef53467e4f61410d12822d922498a052a2b4924452fb4a73f2cb52b6e9d41}} 
size&\\
\hline

\end{DoxyEnumFields}


Definition at line 88 of file acrobot.\+hpp.



\subsection{Constructor \& Destructor Documentation}
\mbox{\label{classmlpack_1_1rl_1_1Acrobot_a3bfb421378fe3713efed9d290b365821}} 
\index{mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}!Acrobot@{Acrobot}}
\index{Acrobot@{Acrobot}!mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}}
\subsubsection{Acrobot()}
{\footnotesize\ttfamily \textbf{ Acrobot} (\begin{DoxyParamCaption}\item[{const double}]{gravity = {\ttfamily 9.81},  }\item[{const double}]{link\+Length1 = {\ttfamily 1.0},  }\item[{const double}]{link\+Length2 = {\ttfamily 1.0},  }\item[{const double}]{link\+Mass1 = {\ttfamily 1.0},  }\item[{const double}]{link\+Mass2 = {\ttfamily 1.0},  }\item[{const double}]{link\+Com1 = {\ttfamily 0.5},  }\item[{const double}]{link\+Com2 = {\ttfamily 0.5},  }\item[{const double}]{link\+Moi = {\ttfamily 1.0},  }\item[{const double}]{max\+Vel1 = {\ttfamily 4~$\ast$~\textbf{ M\+\_\+\+PI}},  }\item[{const double}]{max\+Vel2 = {\ttfamily 9~$\ast$~\textbf{ M\+\_\+\+PI}},  }\item[{const double}]{dt = {\ttfamily 0.2},  }\item[{const double}]{done\+Reward = {\ttfamily 0},  }\item[{const size\+\_\+t}]{max\+Steps = {\ttfamily 0} }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}}



Construct a \doxyref{Acrobot}{p.}{classmlpack_1_1rl_1_1Acrobot} instance using the given constants. 


\begin{DoxyParams}{Parameters}
{\em gravity} & The gravity parameter. \\
\hline
{\em link\+Length1} & The length of link 1. \\
\hline
{\em link\+Length2} & The length of link 2. \\
\hline
{\em link\+Mass1} & The mass of link 1. \\
\hline
{\em link\+Mass2} & The mass of link 2. \\
\hline
{\em link\+Com1} & The position of the center of mass of link 1. \\
\hline
{\em link\+Com2} & The position of the center of mass of link 2. \\
\hline
{\em link\+Moi} & The moments of inertia for both links. \\
\hline
{\em max\+Vel1} & The max angular velocity of link1. \\
\hline
{\em max\+Vel2} & The max angular velocity of link2. \\
\hline
{\em dt} & The differential value. \\
\hline
{\em done\+Reward} & The reward recieved by the agent on success. \\
\hline
{\em max\+Steps} & The number of steps after which the episode terminates. If the value is 0, there is no limit. \\
\hline
\end{DoxyParams}


Definition at line 116 of file acrobot.\+hpp.



\subsection{Member Function Documentation}
\mbox{\label{classmlpack_1_1rl_1_1Acrobot_a781c862b1afe7d673da7025f0b2a244c}} 
\index{mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}!Dsdt@{Dsdt}}
\index{Dsdt@{Dsdt}!mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}}
\subsubsection{Dsdt()}
{\footnotesize\ttfamily arma\+::colvec Dsdt (\begin{DoxyParamCaption}\item[{arma\+::colvec}]{state,  }\item[{const double}]{torque }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}}



This is the ordinary differential equations required for estimation of next\+State through R\+K4 method. 


\begin{DoxyParams}{Parameters}
{\em state} & Current \doxyref{State}{p.}{classmlpack_1_1rl_1_1Acrobot_1_1State}. \\
\hline
{\em torque} & The torque Applied. \\
\hline
\end{DoxyParams}


Definition at line 243 of file acrobot.\+hpp.



References M\+\_\+\+PI.



Referenced by Acrobot\+::\+Rk4().

\mbox{\label{classmlpack_1_1rl_1_1Acrobot_aa9f537249fa0c1e62b38197996ab4c6a}} 
\index{mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}!Initial\+Sample@{Initial\+Sample}}
\index{Initial\+Sample@{Initial\+Sample}!mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}}
\subsubsection{Initial\+Sample()}
{\footnotesize\ttfamily \textbf{ State} Initial\+Sample (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}}



This function does random initialization of state space. 



Definition at line 207 of file acrobot.\+hpp.



References Acrobot\+::\+State\+::\+State().

\mbox{\label{classmlpack_1_1rl_1_1Acrobot_a7fd056133dfd315e4bf45c408f99326f}} 
\index{mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}!Is\+Terminal@{Is\+Terminal}}
\index{Is\+Terminal@{Is\+Terminal}!mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}}
\subsubsection{Is\+Terminal()}
{\footnotesize\ttfamily bool Is\+Terminal (\begin{DoxyParamCaption}\item[{const \textbf{ State} \&}]{state }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}}



This function checks if the acrobot has reached the terminal state. 


\begin{DoxyParams}{Parameters}
{\em state} & The current \doxyref{State}{p.}{classmlpack_1_1rl_1_1Acrobot_1_1State}. \\
\hline
\end{DoxyParams}
\begin{DoxyReturn}{Returns}
true if state is a terminal state, otherwise false. 
\end{DoxyReturn}


Definition at line 219 of file acrobot.\+hpp.



References Log\+::\+Info, Acrobot\+::\+State\+::\+Theta1(), and Acrobot\+::\+State\+::\+Theta2().



Referenced by Acrobot\+::\+Sample().

\mbox{\label{classmlpack_1_1rl_1_1Acrobot_ad8fd6d8f7581c82e73556491b79a8907}} 
\index{mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}!Max\+Steps@{Max\+Steps}}
\index{Max\+Steps@{Max\+Steps}!mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}}
\subsubsection{Max\+Steps()\hspace{0.1cm}{\footnotesize\ttfamily [1/2]}}
{\footnotesize\ttfamily size\+\_\+t Max\+Steps (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}}



Get the maximum number of steps allowed. 



Definition at line 344 of file acrobot.\+hpp.

\mbox{\label{classmlpack_1_1rl_1_1Acrobot_a64c84cebc489c6fdfd7f057e127b0aef}} 
\index{mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}!Max\+Steps@{Max\+Steps}}
\index{Max\+Steps@{Max\+Steps}!mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}}
\subsubsection{Max\+Steps()\hspace{0.1cm}{\footnotesize\ttfamily [2/2]}}
{\footnotesize\ttfamily size\+\_\+t\& Max\+Steps (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}}



Set the maximum number of steps allowed. 



Definition at line 346 of file acrobot.\+hpp.

\mbox{\label{classmlpack_1_1rl_1_1Acrobot_ae09ec5b585b264b14d2831e689021a5e}} 
\index{mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}!Rk4@{Rk4}}
\index{Rk4@{Rk4}!mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}}
\subsubsection{Rk4()}
{\footnotesize\ttfamily arma\+::colvec Rk4 (\begin{DoxyParamCaption}\item[{const arma\+::colvec}]{state,  }\item[{const double}]{torque }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}}



This function calls the R\+K4 iterative method to estimate the next state based on given ordinary differential equation. 


\begin{DoxyParams}{Parameters}
{\em state} & The current \doxyref{State}{p.}{classmlpack_1_1rl_1_1Acrobot_1_1State}. \\
\hline
{\em torque} & The torque applied. \\
\hline
\end{DoxyParams}


Definition at line 329 of file acrobot.\+hpp.



References Acrobot\+::\+Dsdt().



Referenced by Acrobot\+::\+Sample().

\mbox{\label{classmlpack_1_1rl_1_1Acrobot_a311ac19edc537dee94f37b7cce93d908}} 
\index{mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}!Sample@{Sample}}
\index{Sample@{Sample}!mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}}
\subsubsection{Sample()\hspace{0.1cm}{\footnotesize\ttfamily [1/2]}}
{\footnotesize\ttfamily double Sample (\begin{DoxyParamCaption}\item[{const \textbf{ State} \&}]{state,  }\item[{const \textbf{ Action} \&}]{action,  }\item[{\textbf{ State} \&}]{next\+State }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}}



Dynamics of the \doxyref{Acrobot}{p.}{classmlpack_1_1rl_1_1Acrobot} System. 

To get reward and next state based on current state and current action. Always return -\/1 reward.


\begin{DoxyParams}{Parameters}
{\em state} & The current \doxyref{State}{p.}{classmlpack_1_1rl_1_1Acrobot_1_1State}. \\
\hline
{\em action} & The action taken. \\
\hline
{\em next\+State} & The next state. \\
\hline
\end{DoxyParams}
\begin{DoxyReturn}{Returns}
reward, it\textquotesingle{}s always -\/1.\+0. 
\end{DoxyReturn}
The value of angular velocity is bounded in min and max value. 

Definition at line 154 of file acrobot.\+hpp.



References Acrobot\+::\+State\+::\+Angular\+Velocity1(), Acrobot\+::\+State\+::\+Angular\+Velocity2(), mlpack\+::math\+::\+Clamp\+Range(), Acrobot\+::\+Is\+Terminal(), M\+\_\+\+PI, Acrobot\+::\+Rk4(), Acrobot\+::\+State\+::\+Theta1(), Acrobot\+::\+State\+::\+Theta2(), Acrobot\+::\+Torque(), and Acrobot\+::\+Wrap().



Referenced by Acrobot\+::\+Sample().

\mbox{\label{classmlpack_1_1rl_1_1Acrobot_af2bb860eaefeaa62a40f5cf940793704}} 
\index{mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}!Sample@{Sample}}
\index{Sample@{Sample}!mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}}
\subsubsection{Sample()\hspace{0.1cm}{\footnotesize\ttfamily [2/2]}}
{\footnotesize\ttfamily double Sample (\begin{DoxyParamCaption}\item[{const \textbf{ State} \&}]{state,  }\item[{const \textbf{ Action} \&}]{action }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}}



Dynamics of the \doxyref{Acrobot}{p.}{classmlpack_1_1rl_1_1Acrobot} System. 

To get reward and next state based on current state and current action. This function calls the Sample function to estimate the next state return reward for taking a particular action.


\begin{DoxyParams}{Parameters}
{\em state} & The current \doxyref{State}{p.}{classmlpack_1_1rl_1_1Acrobot_1_1State}. \\
\hline
{\em action} & The action taken. \\
\hline
\end{DoxyParams}
\begin{DoxyReturn}{Returns}
next\+State The next state. 
\end{DoxyReturn}


Definition at line 198 of file acrobot.\+hpp.



References Acrobot\+::\+Sample().

\mbox{\label{classmlpack_1_1rl_1_1Acrobot_a5fe06563064ebcee88c593e1673f03d4}} 
\index{mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}!Steps\+Performed@{Steps\+Performed}}
\index{Steps\+Performed@{Steps\+Performed}!mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}}
\subsubsection{Steps\+Performed()}
{\footnotesize\ttfamily size\+\_\+t Steps\+Performed (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}}



Get the number of steps performed. 



Definition at line 341 of file acrobot.\+hpp.

\mbox{\label{classmlpack_1_1rl_1_1Acrobot_ae12a0a80a61e21f6e1b722f4ad7d0b81}} 
\index{mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}!Torque@{Torque}}
\index{Torque@{Torque}!mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}}
\subsubsection{Torque()}
{\footnotesize\ttfamily double Torque (\begin{DoxyParamCaption}\item[{const \textbf{ Action} \&}]{action }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}}



This function calculates the torque for a particular action. 

0 \+: negative torque, 1 \+: zero torque, 2 \+: positive torque.


\begin{DoxyParams}{Parameters}
{\em action} & Action taken. \\
\hline
\end{DoxyParams}


Definition at line 316 of file acrobot.\+hpp.



References mlpack\+::math\+::\+Random().



Referenced by Acrobot\+::\+Sample().

\mbox{\label{classmlpack_1_1rl_1_1Acrobot_a9f977aba34c92e5f5f9249d8738ec85a}} 
\index{mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}!Wrap@{Wrap}}
\index{Wrap@{Wrap}!mlpack\+::rl\+::\+Acrobot@{mlpack\+::rl\+::\+Acrobot}}
\subsubsection{Wrap()}
{\footnotesize\ttfamily double Wrap (\begin{DoxyParamCaption}\item[{double}]{value,  }\item[{const double}]{minimum,  }\item[{const double}]{maximum }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}}



Wrap funtion is required to truncate the angle value from -\/180 to 180. 

This function will make sure that value will always be between minimum to maximum.


\begin{DoxyParams}{Parameters}
{\em value} & Scalar value to wrap. \\
\hline
{\em minimum} & Minimum range of wrap. \\
\hline
{\em maximum} & Maximum range of wrap. \\
\hline
\end{DoxyParams}


Definition at line 292 of file acrobot.\+hpp.



Referenced by Acrobot\+::\+Sample().



The documentation for this class was generated from the following file\+:\begin{DoxyCompactItemize}
\item 
/var/www/mlpack.\+ratml.\+org/mlpack.\+org/\+\_\+src/mlpack-\/3.\+3.\+2/src/mlpack/methods/reinforcement\+\_\+learning/environment/\textbf{ acrobot.\+hpp}\end{DoxyCompactItemize}
