\section{Hyper\+Parameter\+Tuner$<$ M\+L\+Algorithm, Metric, CV, Optimizer\+Type, Mat\+Type, Predictions\+Type, Weights\+Type $>$ Class Template Reference} \label{classmlpack_1_1hpt_1_1HyperParameterTuner}\index{Hyper\+Parameter\+Tuner$<$ M\+L\+Algorithm, Metric, C\+V, Optimizer\+Type, Mat\+Type, Predictions\+Type, Weights\+Type $>$@{Hyper\+Parameter\+Tuner$<$ M\+L\+Algorithm, Metric, C\+V, Optimizer\+Type, Mat\+Type, Predictions\+Type, Weights\+Type $>$}} The class \doxyref{Hyper\+Parameter\+Tuner}{p.}{classmlpack_1_1hpt_1_1HyperParameterTuner} for the given M\+L\+Algorithm utilizes the provided Optimizer to find the values of hyper-\/parameters that optimize the value of the given Metric. \subsection*{Public Member Functions} \begin{DoxyCompactItemize} \item {\footnotesize template$<$typename... C\+V\+Args$>$ }\\\textbf{ Hyper\+Parameter\+Tuner} (const C\+V\+Args \&...args) \begin{DoxyCompactList}\small\item\em Create a \doxyref{Hyper\+Parameter\+Tuner}{p.}{classmlpack_1_1hpt_1_1HyperParameterTuner} object by passing constructor arguments for the given cross-\/validation strategy (the CV class). \end{DoxyCompactList}\item const M\+L\+Algorithm \& \textbf{ Best\+Model} () const \begin{DoxyCompactList}\small\item\em Get the best model from the last run. \end{DoxyCompactList}\item M\+L\+Algorithm \& \textbf{ Best\+Model} () \begin{DoxyCompactList}\small\item\em Modify the best model from the last run. \end{DoxyCompactList}\item double \textbf{ Best\+Objective} () const \begin{DoxyCompactList}\small\item\em Get the performance measurement of the best model from the last run. \end{DoxyCompactList}\item double \textbf{ Min\+Delta} () const \begin{DoxyCompactList}\small\item\em Get minimum increase of arguments for calculation of partial derivatives (by the definition) in gradient-\/based optimization. \end{DoxyCompactList}\item double \& \textbf{ Min\+Delta} () \begin{DoxyCompactList}\small\item\em Modify minimum increase of arguments for calculation of partial derivatives (by the definition) in gradient-\/based optimization. \end{DoxyCompactList}\item {\footnotesize template$<$typename... Args$>$ }\\\textbf{ Tuple\+Of\+Hyper\+Parameters}$<$ Args... $>$ \textbf{ Optimize} (const Args \&... args) \begin{DoxyCompactList}\small\item\em Find the best hyper-\/parameters by using the given Optimizer. \end{DoxyCompactList}\item Optimizer\+Type \& \textbf{ Optimizer} () \begin{DoxyCompactList}\small\item\em Access and modify the optimizer. \end{DoxyCompactList}\item double \textbf{ Relative\+Delta} () const \begin{DoxyCompactList}\small\item\em Get relative increase of arguments for calculation of partial derivatives (by the definition) in gradient-\/based optimization. \end{DoxyCompactList}\item double \& \textbf{ Relative\+Delta} () \begin{DoxyCompactList}\small\item\em Modify relative increase of arguments for calculation of partial derivatives (by the definition) in gradient-\/based optimization. \end{DoxyCompactList}\end{DoxyCompactItemize} \subsection{Detailed Description} \subsubsection*{template$<$typename M\+L\+Algorithm, typename Metric, template$<$ typename, typename, typename, typename, typename $>$ class CV, typename Optimizer\+Type = ens\+::\+Grid\+Search, typename Mat\+Type = arma\+::mat, typename Predictions\+Type = typename cv\+::\+Meta\+Info\+Extractor$<$\+M\+L\+Algorithm, Mat\+Type$>$\+::\+Predictions\+Type, typename Weights\+Type = typename cv\+::\+Meta\+Info\+Extractor$<$\+M\+L\+Algorithm, Mat\+Type, Predictions\+Type$>$\+::\+Weights\+Type$>$\newline class mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner$<$ M\+L\+Algorithm, Metric, C\+V, Optimizer\+Type, Mat\+Type, Predictions\+Type, Weights\+Type $>$} The class \doxyref{Hyper\+Parameter\+Tuner}{p.}{classmlpack_1_1hpt_1_1HyperParameterTuner} for the given M\+L\+Algorithm utilizes the provided Optimizer to find the values of hyper-\/parameters that optimize the value of the given Metric. The value of the Metric is calculated by performing cross-\/validation with the provided cross-\/validation strategy. To construct a \doxyref{Hyper\+Parameter\+Tuner}{p.}{classmlpack_1_1hpt_1_1HyperParameterTuner} object you need to pass the same arguments as for construction of an object of the given CV class. For example, we can use the following code to try to find a good lambda value for Linear\+Regression. \begin{DoxyCode} \textcolor{comment}{// 100-point 5-dimensional random dataset.} arma::mat data = arma::randu(5, 100); \textcolor{comment}{// Noisy responses retrieved by a random linear transformation of data.} arma::rowvec responses = arma::randu(5) * data + 0.1 * arma::randn(100); \textcolor{comment}{// Using 80% of data for training and remaining 20% for assessing MSE.} \textcolor{keywordtype}{double} validationSize = 0.2; HyperParameterTuner hpt(validationSize, data, responses); \textcolor{comment}{// Finding the best value for lambda from the values 0.0, 0.001, 0.01, 0.1,} \textcolor{comment}{// and 1.0.} arma::vec lambdas\{0.0, 0.001, 0.01, 0.1, 1.0\}; \textcolor{keywordtype}{double} bestLambda; std::tie(bestLambda) = hpt.Optimize(lambdas); \end{DoxyCode} When some hyper-\/parameters should not be optimized, you can specify values for them with the Fixed function as in the following example of finding good lambda1 and lambda2 values for L\+A\+RS. \begin{DoxyCode} HyperParameterTuner hpt2(validationSize, data, responses); \textcolor{keywordtype}{bool} transposeData = \textcolor{keyword}{true}; \textcolor{keywordtype}{bool} useCholesky = \textcolor{keyword}{false}; arma::vec lambda1Set\{0.0, 0.001, 0.01, 0.1, 1.0\}; arma::vec lambda2Set\{0.0, 0.002, 0.02, 0.2, 2.0\}; \textcolor{keywordtype}{double} bestLambda1, bestLambda2; std::tie(bestLambda1, bestLambda2) = hpt2.Optimize(Fixed(transposeData), Fixed(useCholesky), lambda1Set, lambda2Set); \end{DoxyCode} \begin{DoxyTemplParams}{Template Parameters} {\em M\+L\+Algorithm} & A machine learning algorithm. \\ \hline {\em Metric} & A metric to assess the quality of a trained model. \\ \hline {\em CV} & A cross-\/validation strategy used to assess a set of hyper-\/parameters. \\ \hline {\em Optimizer\+Type} & An optimization strategy (Grid\+Search and Gradient\+Descent are supported). \\ \hline {\em Mat\+Type} & The type of data. \\ \hline {\em Predictions\+Type} & The type of predictions (should be passed when the predictions type is a template parameter in Train methods of the given M\+L\+Algorithm; arma\+::\+Row$<$size\+\_\+t$>$ will be used otherwise). \\ \hline {\em Weights\+Type} & The type of weights (should be passed when weighted learning is supported, and the weights type is a template parameter in Train methods of the given M\+L\+Algorithm; arma\+::vec will be used otherwise). \\ \hline \end{DoxyTemplParams} Definition at line 96 of file hpt.\+hpp. \subsection{Constructor \& Destructor Documentation} \mbox{\label{classmlpack_1_1hpt_1_1HyperParameterTuner_a7fd1a69d0568cda10d907e1113fd189d}} \index{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner@{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner}!Hyper\+Parameter\+Tuner@{Hyper\+Parameter\+Tuner}} \index{Hyper\+Parameter\+Tuner@{Hyper\+Parameter\+Tuner}!mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner@{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner}} \subsubsection{Hyper\+Parameter\+Tuner()} {\footnotesize\ttfamily \textbf{ Hyper\+Parameter\+Tuner} (\begin{DoxyParamCaption}\item[{const C\+V\+Args \&...}]{args }\end{DoxyParamCaption})} Create a \doxyref{Hyper\+Parameter\+Tuner}{p.}{classmlpack_1_1hpt_1_1HyperParameterTuner} object by passing constructor arguments for the given cross-\/validation strategy (the CV class). \begin{DoxyParams}{Parameters} {\em args} & Constructor arguments for the given cross-\/validation strategy (the CV class). \\ \hline \end{DoxyParams} \subsection{Member Function Documentation} \mbox{\label{classmlpack_1_1hpt_1_1HyperParameterTuner_add74660238332755f6e83e85f78b15c5}} \index{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner@{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner}!Best\+Model@{Best\+Model}} \index{Best\+Model@{Best\+Model}!mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner@{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner}} \subsubsection{Best\+Model()\hspace{0.1cm}{\footnotesize\ttfamily [1/2]}} {\footnotesize\ttfamily const M\+L\+Algorithm\& Best\+Model (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}} Get the best model from the last run. Definition at line 184 of file hpt.\+hpp. \mbox{\label{classmlpack_1_1hpt_1_1HyperParameterTuner_a0c2bf016556a87be5f6f9e86f6b37dd9}} \index{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner@{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner}!Best\+Model@{Best\+Model}} \index{Best\+Model@{Best\+Model}!mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner@{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner}} \subsubsection{Best\+Model()\hspace{0.1cm}{\footnotesize\ttfamily [2/2]}} {\footnotesize\ttfamily M\+L\+Algorithm\& Best\+Model (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Modify the best model from the last run. Definition at line 187 of file hpt.\+hpp. \mbox{\label{classmlpack_1_1hpt_1_1HyperParameterTuner_adf93208d264cc5138f52a9c13b46c088}} \index{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner@{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner}!Best\+Objective@{Best\+Objective}} \index{Best\+Objective@{Best\+Objective}!mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner@{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner}} \subsubsection{Best\+Objective()} {\footnotesize\ttfamily double Best\+Objective (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}} Get the performance measurement of the best model from the last run. Definition at line 181 of file hpt.\+hpp. \mbox{\label{classmlpack_1_1hpt_1_1HyperParameterTuner_afc866ee105432ff61c071bfa43a08e8c}} \index{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner@{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner}!Min\+Delta@{Min\+Delta}} \index{Min\+Delta@{Min\+Delta}!mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner@{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner}} \subsubsection{Min\+Delta()\hspace{0.1cm}{\footnotesize\ttfamily [1/2]}} {\footnotesize\ttfamily double Min\+Delta (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}} Get minimum increase of arguments for calculation of partial derivatives (by the definition) in gradient-\/based optimization. This value is going to be used when it is greater than the increase calculated with the rules described in the documentation for \doxyref{Relative\+Delta()}{p.}{classmlpack_1_1hpt_1_1HyperParameterTuner_ac23b9b3760c584f1d276ed867e7722aa}. The default value is 1e-\/10. Definition at line 142 of file hpt.\+hpp. \mbox{\label{classmlpack_1_1hpt_1_1HyperParameterTuner_acf2793015fe5e3acb58b87a0e0f13813}} \index{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner@{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner}!Min\+Delta@{Min\+Delta}} \index{Min\+Delta@{Min\+Delta}!mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner@{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner}} \subsubsection{Min\+Delta()\hspace{0.1cm}{\footnotesize\ttfamily [2/2]}} {\footnotesize\ttfamily double\& Min\+Delta (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Modify minimum increase of arguments for calculation of partial derivatives (by the definition) in gradient-\/based optimization. This value is going to be used when it is greater than the increase calculated with the rules described in the documentation for \doxyref{Relative\+Delta()}{p.}{classmlpack_1_1hpt_1_1HyperParameterTuner_ac23b9b3760c584f1d276ed867e7722aa}. The default value is 1e-\/10. Definition at line 152 of file hpt.\+hpp. References Hyper\+Parameter\+Tuner$<$ M\+L\+Algorithm, Metric, C\+V, Optimizer\+Type, Mat\+Type, Predictions\+Type, Weights\+Type $>$\+::\+Optimize(). \mbox{\label{classmlpack_1_1hpt_1_1HyperParameterTuner_a4e04da235ec0434d69613c547b20dbea}} \index{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner@{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner}!Optimize@{Optimize}} \index{Optimize@{Optimize}!mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner@{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner}} \subsubsection{Optimize()} {\footnotesize\ttfamily \textbf{ Tuple\+Of\+Hyper\+Parameters}$<$Args...$>$ Optimize (\begin{DoxyParamCaption}\item[{const Args \&...}]{args }\end{DoxyParamCaption})} Find the best hyper-\/parameters by using the given Optimizer. For each hyper-\/parameter one of the following should be passed as an argument. \begin{DoxyEnumerate} \item A set of values to choose from (when using Grid\+Search as an optimizer). The set of values should be an S\+T\+L-\/compatible container (it should provide begin() and end() methods returning iterators). \item A starting value (when using any other optimizer than Grid\+Search). \item A value fixed by using the function \doxyref{mlpack\+::hpt\+::\+Fixed}{p.}{namespacemlpack_1_1hpt_ad773f4d1def8deb412ffbf37bdf289ec}. In this case the hyper-\/parameter will not be optimized. \end{DoxyEnumerate} All arguments should be passed in the same order as if the corresponding hyper-\/parameters would be passed into the Evaluate method of the given CV class (in the order as they appear in the constructor(s) of the given M\+L\+Algorithm). Also, arguments for all required hyper-\/parameters (ones that don\textquotesingle{}t have default values in the corresponding M\+L\+Algorithm constructor) should be provided. The method returns a tuple of values for hyper-\/parameters that haven\textquotesingle{}t been fixed. \begin{DoxyParams}{Parameters} {\em args} & Arguments corresponding to hyper-\/parameters (see the method description for more information). \\ \hline \end{DoxyParams} Referenced by Hyper\+Parameter\+Tuner$<$ M\+L\+Algorithm, Metric, C\+V, Optimizer\+Type, Mat\+Type, Predictions\+Type, Weights\+Type $>$\+::\+Min\+Delta(). \mbox{\label{classmlpack_1_1hpt_1_1HyperParameterTuner_a213f61b8936eadf8a162e45b4c6188d0}} \index{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner@{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner}!Optimizer@{Optimizer}} \index{Optimizer@{Optimizer}!mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner@{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner}} \subsubsection{Optimizer()} {\footnotesize\ttfamily Optimizer\+Type\& Optimizer (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Access and modify the optimizer. Definition at line 110 of file hpt.\+hpp. \mbox{\label{classmlpack_1_1hpt_1_1HyperParameterTuner_ab4d547696f7f5e4be93f9cc2a2dbfc9b}} \index{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner@{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner}!Relative\+Delta@{Relative\+Delta}} \index{Relative\+Delta@{Relative\+Delta}!mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner@{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner}} \subsubsection{Relative\+Delta()\hspace{0.1cm}{\footnotesize\ttfamily [1/2]}} {\footnotesize\ttfamily double Relative\+Delta (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}} Get relative increase of arguments for calculation of partial derivatives (by the definition) in gradient-\/based optimization. The exact increase for some particular argument is equal to the absolute value of the argument multiplied by the relative increase (see also the documentation for \doxyref{Min\+Delta()}{p.}{classmlpack_1_1hpt_1_1HyperParameterTuner_acf2793015fe5e3acb58b87a0e0f13813}). The default value is 0.\+01. Definition at line 121 of file hpt.\+hpp. \mbox{\label{classmlpack_1_1hpt_1_1HyperParameterTuner_ac23b9b3760c584f1d276ed867e7722aa}} \index{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner@{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner}!Relative\+Delta@{Relative\+Delta}} \index{Relative\+Delta@{Relative\+Delta}!mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner@{mlpack\+::hpt\+::\+Hyper\+Parameter\+Tuner}} \subsubsection{Relative\+Delta()\hspace{0.1cm}{\footnotesize\ttfamily [2/2]}} {\footnotesize\ttfamily double\& Relative\+Delta (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption})\hspace{0.3cm}{\ttfamily [inline]}} Modify relative increase of arguments for calculation of partial derivatives (by the definition) in gradient-\/based optimization. The exact increase for some particular argument is equal to the absolute value of the argument multiplied by the relative increase (see also the documentation for \doxyref{Min\+Delta()}{p.}{classmlpack_1_1hpt_1_1HyperParameterTuner_acf2793015fe5e3acb58b87a0e0f13813}). The default value is 0.\+01. Definition at line 132 of file hpt.\+hpp. The documentation for this class was generated from the following file\+:\begin{DoxyCompactItemize} \item /var/www/mlpack.\+ratml.\+org/mlpack.\+org/\+\_\+src/mlpack-\/3.\+3.\+0/src/mlpack/core/hpt/\textbf{ hpt.\+hpp}\end{DoxyCompactItemize}