\section{Flexible\+Re\+LU$<$ Input\+Data\+Type, Output\+Data\+Type $>$ Class Template Reference}
\label{classmlpack_1_1ann_1_1FlexibleReLU}\index{Flexible\+Re\+L\+U$<$ Input\+Data\+Type, Output\+Data\+Type $>$@{Flexible\+Re\+L\+U$<$ Input\+Data\+Type, Output\+Data\+Type $>$}}


The \doxyref{Flexible\+Re\+LU}{p.}{classmlpack_1_1ann_1_1FlexibleReLU} activation function, defined by.  


\subsection*{Public Member Functions}
\begin{DoxyCompactItemize}
\item 
\textbf{ Flexible\+Re\+LU} (const double alpha=0)
\begin{DoxyCompactList}\small\item\em Create the \doxyref{Flexible\+Re\+LU}{p.}{classmlpack_1_1ann_1_1FlexibleReLU} object using the specified parameters. \end{DoxyCompactList}\item 
double const  \& \textbf{ Alpha} () const
\begin{DoxyCompactList}\small\item\em Get the parameter controlling the range of the relu function. \end{DoxyCompactList}\item 
double \& \textbf{ Alpha} ()
\begin{DoxyCompactList}\small\item\em Modify the parameter controlling the range of the relu function. \end{DoxyCompactList}\item 
{\footnotesize template$<$typename Data\+Type $>$ }\\void \textbf{ Backward} (const Data\+Type \&input, const Data\+Type \&gy, Data\+Type \&g)
\begin{DoxyCompactList}\small\item\em Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards through f. \end{DoxyCompactList}\item 
Output\+Data\+Type const  \& \textbf{ Delta} () const
\begin{DoxyCompactList}\small\item\em Get the delta. \end{DoxyCompactList}\item 
Output\+Data\+Type \& \textbf{ Delta} ()
\begin{DoxyCompactList}\small\item\em Modify the delta. \end{DoxyCompactList}\item 
{\footnotesize template$<$typename Input\+Type , typename Output\+Type $>$ }\\void \textbf{ Forward} (const Input\+Type \&input, Output\+Type \&output)
\begin{DoxyCompactList}\small\item\em Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. \end{DoxyCompactList}\item 
{\footnotesize template$<$typename eT $>$ }\\void \textbf{ Gradient} (const arma\+::\+Mat$<$ eT $>$ \&input, const arma\+::\+Mat$<$ eT $>$ \&error, arma\+::\+Mat$<$ eT $>$ \&gradient)
\begin{DoxyCompactList}\small\item\em Calculate the gradient using the output delta and the input activation. \end{DoxyCompactList}\item 
Output\+Data\+Type const  \& \textbf{ Gradient} () const
\begin{DoxyCompactList}\small\item\em Get the gradient. \end{DoxyCompactList}\item 
Output\+Data\+Type \& \textbf{ Gradient} ()
\begin{DoxyCompactList}\small\item\em Modify the gradient. \end{DoxyCompactList}\item 
Output\+Data\+Type const  \& \textbf{ Output\+Parameter} () const
\begin{DoxyCompactList}\small\item\em Get the output parameter. \end{DoxyCompactList}\item 
Output\+Data\+Type \& \textbf{ Output\+Parameter} ()
\begin{DoxyCompactList}\small\item\em Modify the output parameter. \end{DoxyCompactList}\item 
Output\+Data\+Type const  \& \textbf{ Parameters} () const
\begin{DoxyCompactList}\small\item\em Get the parameters. \end{DoxyCompactList}\item 
Output\+Data\+Type \& \textbf{ Parameters} ()
\begin{DoxyCompactList}\small\item\em Modify the parameters. \end{DoxyCompactList}\item 
void \textbf{ Reset} ()
\begin{DoxyCompactList}\small\item\em Reset the layer parameter. \end{DoxyCompactList}\item 
{\footnotesize template$<$typename Archive $>$ }\\void \textbf{ serialize} (Archive \&ar, const unsigned int)
\begin{DoxyCompactList}\small\item\em Serialize the layer. \end{DoxyCompactList}\end{DoxyCompactItemize}


\subsection{Detailed Description}
\subsubsection*{template$<$typename Input\+Data\+Type = arma\+::mat, typename Output\+Data\+Type = arma\+::mat$>$\newline
class mlpack\+::ann\+::\+Flexible\+Re\+L\+U$<$ Input\+Data\+Type, Output\+Data\+Type $>$}

The \doxyref{Flexible\+Re\+LU}{p.}{classmlpack_1_1ann_1_1FlexibleReLU} activation function, defined by. 

\begin{eqnarray*} f(x) &=& \max(0,x)+alpha \\ f'(x) &=& \left\{ \begin{array}{lr} 1 & : x > 0 \\ 0 & : x \le 0 \end{array} \right. \end{eqnarray*}

For more information, read the following paper\+:


\begin{DoxyCode}
@article\{Qiu2018,
 author  = \{Suo Qiu, Xiangmin Xu and Bolun Cai\},
 title   = \{FReLU: Flexible Rectified Linear Units \textcolor{keywordflow}{for} Improving
            Convolutional Neural Networks\}
 journal = \{arxiv preprint\},
 URL     = \{https:\textcolor{comment}{//arxiv.org/abs/1706.08098\},}
 year    = \{2018\}
\}
\end{DoxyCode}



\begin{DoxyTemplParams}{Template Parameters}
{\em Input\+Data\+Type} & Type of the input data (arma\+::colvec, arma\+::mar, arma\+::sp\+\_\+mat or arma\+::cube) \\
\hline
{\em Output\+Data\+Type} & Type of the output data (arma\+::colvec, arma\+::mat, arma\+::sp\+\_\+mat or arma\+::cube) \\
\hline
\end{DoxyTemplParams}


Definition at line 59 of file flexible\+\_\+relu.\+hpp.



\subsection{Constructor \& Destructor Documentation}
\mbox{\label{classmlpack_1_1ann_1_1FlexibleReLU_a76fe1c4417c6044ce1275ba7fd9afc76}} 
\index{mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}!Flexible\+Re\+LU@{Flexible\+Re\+LU}}
\index{Flexible\+Re\+LU@{Flexible\+Re\+LU}!mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}}
\subsubsection{Flexible\+Re\+L\+U()}
{\footnotesize\ttfamily \textbf{ Flexible\+Re\+LU} (\begin{DoxyParamCaption}\item[{const double}]{alpha = {\ttfamily 0} }\end{DoxyParamCaption})}



Create the \doxyref{Flexible\+Re\+LU}{p.}{classmlpack_1_1ann_1_1FlexibleReLU} object using the specified parameters. 

The non zero parameter can be adjusted by specifying the parameter alpha which controls the range of the relu function. (Default alpha = 0) This parameter is trainable.


\begin{DoxyParams}{Parameters}
{\em alpha} & Parameter for adjusting the range of the relu function. \\
\hline
\end{DoxyParams}


\subsection{Member Function Documentation}
\mbox{\label{classmlpack_1_1ann_1_1FlexibleReLU_a21679485637bdec3078ec74d71572980}} 
\index{mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}!Alpha@{Alpha}}
\index{Alpha@{Alpha}!mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}}
\subsubsection{Alpha()\hspace{0.1cm}{\footnotesize\ttfamily [1/2]}}
{\footnotesize\ttfamily double const\& Alpha (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}}



Get the parameter controlling the range of the relu function. 



Definition at line 134 of file flexible\+\_\+relu.\+hpp.

\mbox{\label{classmlpack_1_1ann_1_1FlexibleReLU_acbb0e4747a3a307bee88bad71e5eeaf1}} 
\index{mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}!Alpha@{Alpha}}
\index{Alpha@{Alpha}!mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}}
\subsubsection{Alpha()\hspace{0.1cm}{\footnotesize\ttfamily [2/2]}}
{\footnotesize\ttfamily double\& Alpha (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}}



Modify the parameter controlling the range of the relu function. 



Definition at line 136 of file flexible\+\_\+relu.\+hpp.



References Flexible\+Re\+L\+U$<$ Input\+Data\+Type, Output\+Data\+Type $>$\+::serialize().

\mbox{\label{classmlpack_1_1ann_1_1FlexibleReLU_aef8c56f1f8624bd006afec8b3bcda9d6}} 
\index{mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}!Backward@{Backward}}
\index{Backward@{Backward}!mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}}
\subsubsection{Backward()}
{\footnotesize\ttfamily void Backward (\begin{DoxyParamCaption}\item[{const Data\+Type \&}]{input,  }\item[{const Data\+Type \&}]{gy,  }\item[{Data\+Type \&}]{g }\end{DoxyParamCaption})}



Ordinary feed backward pass of a neural network, calculating the function f(x) by propagating x backwards through f. 

Using the results from the feed forward pass.


\begin{DoxyParams}{Parameters}
{\em input} & The propagated input activation. \\
\hline
{\em gy} & The backpropagated error. \\
\hline
{\em g} & The calculated gradient. \\
\hline
\end{DoxyParams}
\mbox{\label{classmlpack_1_1ann_1_1FlexibleReLU_a797f7edb44dd081e5e2b3cc316eef6bd}} 
\index{mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}!Delta@{Delta}}
\index{Delta@{Delta}!mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}}
\subsubsection{Delta()\hspace{0.1cm}{\footnotesize\ttfamily [1/2]}}
{\footnotesize\ttfamily Output\+Data\+Type const\& Delta (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}}



Get the delta. 



Definition at line 124 of file flexible\+\_\+relu.\+hpp.

\mbox{\label{classmlpack_1_1ann_1_1FlexibleReLU_ad6601342d560219ce951d554e69e5e87}} 
\index{mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}!Delta@{Delta}}
\index{Delta@{Delta}!mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}}
\subsubsection{Delta()\hspace{0.1cm}{\footnotesize\ttfamily [2/2]}}
{\footnotesize\ttfamily Output\+Data\+Type\& Delta (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}}



Modify the delta. 



Definition at line 126 of file flexible\+\_\+relu.\+hpp.

\mbox{\label{classmlpack_1_1ann_1_1FlexibleReLU_a09440df0a90bdcc766e56e097d91205b}} 
\index{mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}!Forward@{Forward}}
\index{Forward@{Forward}!mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}}
\subsubsection{Forward()}
{\footnotesize\ttfamily void Forward (\begin{DoxyParamCaption}\item[{const Input\+Type \&}]{input,  }\item[{Output\+Type \&}]{output }\end{DoxyParamCaption})}



Ordinary feed forward pass of a neural network, evaluating the function f(x) by propagating the activity forward through f. 


\begin{DoxyParams}{Parameters}
{\em input} & Input data used for evaluating the specified function. \\
\hline
{\em output} & Resulting output activation. \\
\hline
\end{DoxyParams}
\mbox{\label{classmlpack_1_1ann_1_1FlexibleReLU_aaf577db350e2130754490d8486fba215}} 
\index{mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}!Gradient@{Gradient}}
\index{Gradient@{Gradient}!mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}}
\subsubsection{Gradient()\hspace{0.1cm}{\footnotesize\ttfamily [1/3]}}
{\footnotesize\ttfamily void Gradient (\begin{DoxyParamCaption}\item[{const arma\+::\+Mat$<$ eT $>$ \&}]{input,  }\item[{const arma\+::\+Mat$<$ eT $>$ \&}]{error,  }\item[{arma\+::\+Mat$<$ eT $>$ \&}]{gradient }\end{DoxyParamCaption})}



Calculate the gradient using the output delta and the input activation. 


\begin{DoxyParams}{Parameters}
{\em input} & The input parameter used for calculating the gradient. \\
\hline
{\em error} & The calculated error. \\
\hline
{\em gradient} & The calculated gradient. \\
\hline
\end{DoxyParams}
\mbox{\label{classmlpack_1_1ann_1_1FlexibleReLU_a0f1f4e6d93472d83852731a96c8c3f59}} 
\index{mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}!Gradient@{Gradient}}
\index{Gradient@{Gradient}!mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}}
\subsubsection{Gradient()\hspace{0.1cm}{\footnotesize\ttfamily [2/3]}}
{\footnotesize\ttfamily Output\+Data\+Type const\& Gradient (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}}



Get the gradient. 



Definition at line 129 of file flexible\+\_\+relu.\+hpp.

\mbox{\label{classmlpack_1_1ann_1_1FlexibleReLU_a19abce4739c3b0b658b612537e21956a}} 
\index{mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}!Gradient@{Gradient}}
\index{Gradient@{Gradient}!mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}}
\subsubsection{Gradient()\hspace{0.1cm}{\footnotesize\ttfamily [3/3]}}
{\footnotesize\ttfamily Output\+Data\+Type\& Gradient (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}}



Modify the gradient. 



Definition at line 131 of file flexible\+\_\+relu.\+hpp.

\mbox{\label{classmlpack_1_1ann_1_1FlexibleReLU_a0ee21c2a36e5abad1e7a9d5dd00849f9}} 
\index{mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}!Output\+Parameter@{Output\+Parameter}}
\index{Output\+Parameter@{Output\+Parameter}!mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}}
\subsubsection{Output\+Parameter()\hspace{0.1cm}{\footnotesize\ttfamily [1/2]}}
{\footnotesize\ttfamily Output\+Data\+Type const\& Output\+Parameter (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}}



Get the output parameter. 



Definition at line 119 of file flexible\+\_\+relu.\+hpp.

\mbox{\label{classmlpack_1_1ann_1_1FlexibleReLU_a21d5f745f02c709625a4ee0907f004a5}} 
\index{mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}!Output\+Parameter@{Output\+Parameter}}
\index{Output\+Parameter@{Output\+Parameter}!mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}}
\subsubsection{Output\+Parameter()\hspace{0.1cm}{\footnotesize\ttfamily [2/2]}}
{\footnotesize\ttfamily Output\+Data\+Type\& Output\+Parameter (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}}



Modify the output parameter. 



Definition at line 121 of file flexible\+\_\+relu.\+hpp.

\mbox{\label{classmlpack_1_1ann_1_1FlexibleReLU_aa530552c7ef915c952fbacc77b965c90}} 
\index{mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}!Parameters@{Parameters}}
\index{Parameters@{Parameters}!mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}}
\subsubsection{Parameters()\hspace{0.1cm}{\footnotesize\ttfamily [1/2]}}
{\footnotesize\ttfamily Output\+Data\+Type const\& Parameters (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}}



Get the parameters. 



Definition at line 114 of file flexible\+\_\+relu.\+hpp.

\mbox{\label{classmlpack_1_1ann_1_1FlexibleReLU_a9c5c5900772a689d5a6b59778ec67120}} 
\index{mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}!Parameters@{Parameters}}
\index{Parameters@{Parameters}!mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}}
\subsubsection{Parameters()\hspace{0.1cm}{\footnotesize\ttfamily [2/2]}}
{\footnotesize\ttfamily Output\+Data\+Type\& Parameters (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}}



Modify the parameters. 



Definition at line 116 of file flexible\+\_\+relu.\+hpp.

\mbox{\label{classmlpack_1_1ann_1_1FlexibleReLU_a372de693ad40b3f42839c8ec6ac845f4}} 
\index{mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}!Reset@{Reset}}
\index{Reset@{Reset}!mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}}
\subsubsection{Reset()}
{\footnotesize\ttfamily void Reset (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption})}



Reset the layer parameter. 

\mbox{\label{classmlpack_1_1ann_1_1FlexibleReLU_af0dd9205158ccf7bcfcd8ff81f79c927}} 
\index{mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}!serialize@{serialize}}
\index{serialize@{serialize}!mlpack\+::ann\+::\+Flexible\+Re\+LU@{mlpack\+::ann\+::\+Flexible\+Re\+LU}}
\subsubsection{serialize()}
{\footnotesize\ttfamily void serialize (\begin{DoxyParamCaption}\item[{Archive \&}]{ar,  }\item[{const unsigned}]{int }\end{DoxyParamCaption})}



Serialize the layer. 



Referenced by Flexible\+Re\+L\+U$<$ Input\+Data\+Type, Output\+Data\+Type $>$\+::\+Alpha().



The documentation for this class was generated from the following file\+:\begin{DoxyCompactItemize}
\item 
/var/www/mlpack.\+ratml.\+org/mlpack.\+org/\+\_\+src/mlpack-\/3.\+3.\+1/src/mlpack/methods/ann/layer/\textbf{ flexible\+\_\+relu.\+hpp}\end{DoxyCompactItemize}
