\section{Continuous\+Double\+Pole\+Cart Class Reference} \label{classmlpack_1_1rl_1_1ContinuousDoublePoleCart}\index{Continuous\+Double\+Pole\+Cart@{Continuous\+Double\+Pole\+Cart}} Implementation of Continuous Double Pole Cart Balancing task. \subsection*{Classes} \begin{DoxyCompactItemize} \item struct \textbf{ Action} \begin{DoxyCompactList}\small\item\em Implementation of action of Continuous Double Pole Cart. \end{DoxyCompactList}\item class \textbf{ State} \begin{DoxyCompactList}\small\item\em Implementation of the state of Continuous Double Pole Cart. \end{DoxyCompactList}\end{DoxyCompactItemize} \subsection*{Public Member Functions} \begin{DoxyCompactItemize} \item \textbf{ Continuous\+Double\+Pole\+Cart} (const double m1=0.\+1, const double m2=0.\+01, const double l1=0.\+5, const double l2=0.\+05, const double gravity=9.\+8, const double mass\+Cart=1.\+0, const double force\+Mag=10.\+0, const double tau=0.\+02, const double theta\+Threshold\+Radians=36 $\ast$2 $\ast$3.\+1416/360, const double x\+Threshold=2.\+4, const double done\+Reward=0.\+0, const size\+\_\+t max\+Steps=0) \begin{DoxyCompactList}\small\item\em Construct a Double Pole Cart instance using the given constants. \end{DoxyCompactList}\item void \textbf{ Dsdt} (const \textbf{ State} \&state, const \textbf{ Action} \&action, arma\+::vec \&dydx) \begin{DoxyCompactList}\small\item\em This is the ordinary differential equations required for estimation of next state through R\+K4 method. \end{DoxyCompactList}\item \textbf{ State} \textbf{ Initial\+Sample} () \begin{DoxyCompactList}\small\item\em Initial state representation is randomly generated within [-\/0.\+05, 0.\+05]. \end{DoxyCompactList}\item bool \textbf{ Is\+Terminal} (const \textbf{ State} \&state) const \begin{DoxyCompactList}\small\item\em This function checks if the car has reached the terminal state. \end{DoxyCompactList}\item size\+\_\+t \textbf{ Max\+Steps} () const \begin{DoxyCompactList}\small\item\em Get the maximum number of steps allowed. \end{DoxyCompactList}\item size\+\_\+t \& \textbf{ Max\+Steps} () \begin{DoxyCompactList}\small\item\em Set the maximum number of steps allowed. \end{DoxyCompactList}\item void \textbf{ R\+K4} (const \textbf{ State} \&state, const \textbf{ Action} \&action, arma\+::vec \&dydx, \textbf{ State} \&next\+State) \begin{DoxyCompactList}\small\item\em This function calls the R\+K4 iterative method to estimate the next state based on given ordinary differential equation. \end{DoxyCompactList}\item double \textbf{ Sample} (const \textbf{ State} \&state, const \textbf{ Action} \&action, \textbf{ State} \&next\+State) \begin{DoxyCompactList}\small\item\em Dynamics of Continuous Double Pole Cart instance. \end{DoxyCompactList}\item double \textbf{ Sample} (const \textbf{ State} \&state, const \textbf{ Action} \&action) \begin{DoxyCompactList}\small\item\em Dynamics of Continuous Double Pole Cart. \end{DoxyCompactList}\item size\+\_\+t \textbf{ Steps\+Performed} () const \begin{DoxyCompactList}\small\item\em Get the number of steps performed. \end{DoxyCompactList}\end{DoxyCompactItemize} \subsection{Detailed Description} Implementation of Continuous Double Pole Cart Balancing task. This is an extension of the existing \doxyref{Cart\+Pole}{p.}{classmlpack_1_1rl_1_1CartPole} environment. The environment comprises of a cart with two upright poles of different lengths and masses. The agent is meant to balance the poles by applying force on the cart. Definition at line 28 of file continuous\+\_\+double\+\_\+pole\+\_\+cart.\+hpp. \subsection{Constructor \& Destructor Documentation} \mbox{\label{classmlpack_1_1rl_1_1ContinuousDoublePoleCart_a949192c3e2a13c5af625e50f6bfe00d4}} \index{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart@{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart}!Continuous\+Double\+Pole\+Cart@{Continuous\+Double\+Pole\+Cart}} \index{Continuous\+Double\+Pole\+Cart@{Continuous\+Double\+Pole\+Cart}!mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart@{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart}} \subsubsection{Continuous\+Double\+Pole\+Cart()} {\footnotesize\ttfamily \textbf{ Continuous\+Double\+Pole\+Cart} (\begin{DoxyParamCaption}\item[{const double}]{m1 = {\ttfamily 0.1}, }\item[{const double}]{m2 = {\ttfamily 0.01}, }\item[{const double}]{l1 = {\ttfamily 0.5}, }\item[{const double}]{l2 = {\ttfamily 0.05}, }\item[{const double}]{gravity = {\ttfamily 9.8}, }\item[{const double}]{mass\+Cart = {\ttfamily 1.0}, }\item[{const double}]{force\+Mag = {\ttfamily 10.0}, }\item[{const double}]{tau = {\ttfamily 0.02}, }\item[{const double}]{theta\+Threshold\+Radians = {\ttfamily 36~$\ast$~2~$\ast$~3.1416~/~360}, }\item[{const double}]{x\+Threshold = {\ttfamily 2.4}, }\item[{const double}]{done\+Reward = {\ttfamily 0.0}, }\item[{const size\+\_\+t}]{max\+Steps = {\ttfamily 0} }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Construct a Double Pole Cart instance using the given constants. \begin{DoxyParams}{Parameters} {\em m1} & The mass of the first pole. \\ \hline {\em m2} & The mass of the second pole. \\ \hline {\em l1} & The length of the first pole. \\ \hline {\em l2} & The length of the second pole. \\ \hline {\em gravity} & The gravity constant. \\ \hline {\em mass\+Cart} & The mass of the cart. \\ \hline {\em force\+Mag} & The magnitude of the applied force. \\ \hline {\em tau} & The time interval. \\ \hline {\em theta\+Threshold\+Radians} & The maximum angle. \\ \hline {\em x\+Threshold} & The maximum position. \\ \hline {\em done\+Reward} & Reward recieved by agent on success. \\ \hline {\em max\+Steps} & The number of steps after which the episode terminates. If the value is 0, there is no limit. \\ \hline \end{DoxyParams} Definition at line 116 of file continuous\+\_\+double\+\_\+pole\+\_\+cart.\+hpp. \subsection{Member Function Documentation} \mbox{\label{classmlpack_1_1rl_1_1ContinuousDoublePoleCart_aec6c83f7450927b7520a0c057babc128}} \index{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart@{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart}!Dsdt@{Dsdt}} \index{Dsdt@{Dsdt}!mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart@{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart}} \subsubsection{Dsdt()} {\footnotesize\ttfamily void Dsdt (\begin{DoxyParamCaption}\item[{const \textbf{ State} \&}]{state, }\item[{const \textbf{ Action} \&}]{action, }\item[{arma\+::vec \&}]{dydx }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} This is the ordinary differential equations required for estimation of next state through R\+K4 method. \begin{DoxyParams}{Parameters} {\em state} & The current state. \\ \hline {\em action} & The action taken. \\ \hline {\em dydx} & The differential. \\ \hline \end{DoxyParams} Definition at line 191 of file continuous\+\_\+double\+\_\+pole\+\_\+cart.\+hpp. References Continuous\+Double\+Pole\+Cart\+::\+Action\+::action, Continuous\+Double\+Pole\+Cart\+::\+State\+::\+Angle(), and Continuous\+Double\+Pole\+Cart\+::\+State\+::\+Angular\+Velocity(). Referenced by Continuous\+Double\+Pole\+Cart\+::\+R\+K4(), and Continuous\+Double\+Pole\+Cart\+::\+Sample(). \mbox{\label{classmlpack_1_1rl_1_1ContinuousDoublePoleCart_aa9f537249fa0c1e62b38197996ab4c6a}} \index{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart@{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart}!Initial\+Sample@{Initial\+Sample}} \index{Initial\+Sample@{Initial\+Sample}!mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart@{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart}} \subsubsection{Initial\+Sample()} {\footnotesize\ttfamily \textbf{ State} Initial\+Sample (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Initial state representation is randomly generated within [-\/0.\+05, 0.\+05]. \begin{DoxyReturn}{Returns} Initial state for each episode. \end{DoxyReturn} Definition at line 283 of file continuous\+\_\+double\+\_\+pole\+\_\+cart.\+hpp. References Continuous\+Double\+Pole\+Cart\+::\+State\+::\+State(). \mbox{\label{classmlpack_1_1rl_1_1ContinuousDoublePoleCart_a7fd056133dfd315e4bf45c408f99326f}} \index{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart@{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart}!Is\+Terminal@{Is\+Terminal}} \index{Is\+Terminal@{Is\+Terminal}!mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart@{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart}} \subsubsection{Is\+Terminal()} {\footnotesize\ttfamily bool Is\+Terminal (\begin{DoxyParamCaption}\item[{const \textbf{ State} \&}]{state }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}} This function checks if the car has reached the terminal state. \begin{DoxyParams}{Parameters} {\em state} & The desired state. \\ \hline \end{DoxyParams} \begin{DoxyReturn}{Returns} true if state is a terminal state, otherwise false. \end{DoxyReturn} Definition at line 295 of file continuous\+\_\+double\+\_\+pole\+\_\+cart.\+hpp. References Continuous\+Double\+Pole\+Cart\+::\+State\+::\+Angle(), Log\+::\+Info, and Continuous\+Double\+Pole\+Cart\+::\+State\+::\+Position(). Referenced by Continuous\+Double\+Pole\+Cart\+::\+Sample(). \mbox{\label{classmlpack_1_1rl_1_1ContinuousDoublePoleCart_ad8fd6d8f7581c82e73556491b79a8907}} \index{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart@{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart}!Max\+Steps@{Max\+Steps}} \index{Max\+Steps@{Max\+Steps}!mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart@{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart}} \subsubsection{Max\+Steps()\hspace{0.1cm}{\footnotesize\ttfamily [1/2]}} {\footnotesize\ttfamily size\+\_\+t Max\+Steps (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}} Get the maximum number of steps allowed. Definition at line 321 of file continuous\+\_\+double\+\_\+pole\+\_\+cart.\+hpp. \mbox{\label{classmlpack_1_1rl_1_1ContinuousDoublePoleCart_a64c84cebc489c6fdfd7f057e127b0aef}} \index{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart@{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart}!Max\+Steps@{Max\+Steps}} \index{Max\+Steps@{Max\+Steps}!mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart@{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart}} \subsubsection{Max\+Steps()\hspace{0.1cm}{\footnotesize\ttfamily [2/2]}} {\footnotesize\ttfamily size\+\_\+t\& Max\+Steps (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Set the maximum number of steps allowed. Definition at line 323 of file continuous\+\_\+double\+\_\+pole\+\_\+cart.\+hpp. \mbox{\label{classmlpack_1_1rl_1_1ContinuousDoublePoleCart_afb8fc975886aa60312d63dff046af34a}} \index{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart@{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart}!R\+K4@{R\+K4}} \index{R\+K4@{R\+K4}!mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart@{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart}} \subsubsection{R\+K4()} {\footnotesize\ttfamily void R\+K4 (\begin{DoxyParamCaption}\item[{const \textbf{ State} \&}]{state, }\item[{const \textbf{ Action} \&}]{action, }\item[{arma\+::vec \&}]{dydx, }\item[{\textbf{ State} \&}]{next\+State }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} This function calls the R\+K4 iterative method to estimate the next state based on given ordinary differential equation. \begin{DoxyParams}{Parameters} {\em state} & The current state. \\ \hline {\em action} & The action to be applied. \\ \hline {\em dydx} & The differential. \\ \hline {\em next\+State} & The next state. \\ \hline \end{DoxyParams} Definition at line 232 of file continuous\+\_\+double\+\_\+pole\+\_\+cart.\+hpp. References Continuous\+Double\+Pole\+Cart\+::\+State\+::\+Data(), Continuous\+Double\+Pole\+Cart\+::\+Dsdt(), and Continuous\+Double\+Pole\+Cart\+::\+State\+::\+State(). Referenced by Continuous\+Double\+Pole\+Cart\+::\+Sample(). \mbox{\label{classmlpack_1_1rl_1_1ContinuousDoublePoleCart_a311ac19edc537dee94f37b7cce93d908}} \index{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart@{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart}!Sample@{Sample}} \index{Sample@{Sample}!mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart@{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart}} \subsubsection{Sample()\hspace{0.1cm}{\footnotesize\ttfamily [1/2]}} {\footnotesize\ttfamily double Sample (\begin{DoxyParamCaption}\item[{const \textbf{ State} \&}]{state, }\item[{const \textbf{ Action} \&}]{action, }\item[{\textbf{ State} \&}]{next\+State }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Dynamics of Continuous Double Pole Cart instance. Get reward and next state based on current state and current action. \begin{DoxyParams}{Parameters} {\em state} & The current state. \\ \hline {\em action} & The current action. \\ \hline {\em next\+State} & The next state. \\ \hline \end{DoxyParams} \begin{DoxyReturn}{Returns} reward, it\textquotesingle{}s always 1.\+0. \end{DoxyReturn} When done is false, it means that the cartpole has fallen down. For this case the reward is 1.\+0. Definition at line 153 of file continuous\+\_\+double\+\_\+pole\+\_\+cart.\+hpp. References Continuous\+Double\+Pole\+Cart\+::\+State\+::\+Angular\+Velocity(), Continuous\+Double\+Pole\+Cart\+::\+Dsdt(), Continuous\+Double\+Pole\+Cart\+::\+Is\+Terminal(), Continuous\+Double\+Pole\+Cart\+::\+R\+K4(), and Continuous\+Double\+Pole\+Cart\+::\+State\+::\+Velocity(). Referenced by Continuous\+Double\+Pole\+Cart\+::\+Sample(). \mbox{\label{classmlpack_1_1rl_1_1ContinuousDoublePoleCart_af2bb860eaefeaa62a40f5cf940793704}} \index{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart@{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart}!Sample@{Sample}} \index{Sample@{Sample}!mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart@{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart}} \subsubsection{Sample()\hspace{0.1cm}{\footnotesize\ttfamily [2/2]}} {\footnotesize\ttfamily double Sample (\begin{DoxyParamCaption}\item[{const \textbf{ State} \&}]{state, }\item[{const \textbf{ Action} \&}]{action }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Dynamics of Continuous Double Pole Cart. Get reward based on current state and current action. \begin{DoxyParams}{Parameters} {\em state} & The current state. \\ \hline {\em action} & The current action. \\ \hline \end{DoxyParams} \begin{DoxyReturn}{Returns} reward, it\textquotesingle{}s always 1.\+0. \end{DoxyReturn} Definition at line 272 of file continuous\+\_\+double\+\_\+pole\+\_\+cart.\+hpp. References Continuous\+Double\+Pole\+Cart\+::\+Sample(). \mbox{\label{classmlpack_1_1rl_1_1ContinuousDoublePoleCart_a5fe06563064ebcee88c593e1673f03d4}} \index{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart@{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart}!Steps\+Performed@{Steps\+Performed}} \index{Steps\+Performed@{Steps\+Performed}!mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart@{mlpack\+::rl\+::\+Continuous\+Double\+Pole\+Cart}} \subsubsection{Steps\+Performed()} {\footnotesize\ttfamily size\+\_\+t Steps\+Performed (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}} Get the number of steps performed. Definition at line 318 of file continuous\+\_\+double\+\_\+pole\+\_\+cart.\+hpp. The documentation for this class was generated from the following file\+:\begin{DoxyCompactItemize} \item /var/www/mlpack.\+ratml.\+org/mlpack.\+org/\+\_\+src/mlpack-\/git/src/mlpack/methods/reinforcement\+\_\+learning/environment/\textbf{ continuous\+\_\+double\+\_\+pole\+\_\+cart.\+hpp}\end{DoxyCompactItemize}