\section{Bayesian\+Linear\+Regression Class Reference} \label{classmlpack_1_1regression_1_1BayesianLinearRegression}\index{Bayesian\+Linear\+Regression@{Bayesian\+Linear\+Regression}} A Bayesian approach to the maximum likelihood estimation of the parameters $ \omega $ of the linear regression model. \subsection*{Public Member Functions} \begin{DoxyCompactItemize} \item \textbf{ Bayesian\+Linear\+Regression} (const bool center\+Data=true, const bool scale\+Data=false, const size\+\_\+t n\+Iter\+Max=50, const double tol=1e-\/4) \begin{DoxyCompactList}\small\item\em Set the parameters of Bayesian Ridge regression object. \end{DoxyCompactList}\item double \textbf{ Alpha} () const \begin{DoxyCompactList}\small\item\em Get the precision (or inverse variance) of the gaussian prior. \end{DoxyCompactList}\item double \textbf{ Beta} () const \begin{DoxyCompactList}\small\item\em Get the precision (or inverse variance) beta of the model. \end{DoxyCompactList}\item const arma\+::colvec \& \textbf{ Data\+Offset} () const \begin{DoxyCompactList}\small\item\em Get the mean vector computed on the features over the training points. \end{DoxyCompactList}\item const arma\+::colvec \& \textbf{ Data\+Scale} () const \begin{DoxyCompactList}\small\item\em Get the vector of standard deviations computed on the features over the training points. \end{DoxyCompactList}\item const arma\+::colvec \& \textbf{ Omega} () const \begin{DoxyCompactList}\small\item\em Get the solution vector. \end{DoxyCompactList}\item void \textbf{ Predict} (const arma\+::mat \&points, arma\+::rowvec \&predictions) const \begin{DoxyCompactList}\small\item\em Predict $y_{i}$ for each data point in the given data matrix using the currently-\/trained Bayesian Ridge model. \end{DoxyCompactList}\item void \textbf{ Predict} (const arma\+::mat \&points, arma\+::rowvec \&predictions, arma\+::rowvec \&std) const \begin{DoxyCompactList}\small\item\em Predict $y_{i}$ and the standard deviation of the predictive posterior distribution for each data point in the given data matrix, using the currently-\/trained Bayesian Ridge estimator. \end{DoxyCompactList}\item double \textbf{ Responses\+Offset} () const \begin{DoxyCompactList}\small\item\em Get the mean value of the train responses. \end{DoxyCompactList}\item double \textbf{ R\+M\+SE} (const arma\+::mat \&data, const arma\+::rowvec \&responses) const \begin{DoxyCompactList}\small\item\em Compute the Root Mean Square Error between the predictions returned by the model and the true responses. \end{DoxyCompactList}\item {\footnotesize template$<$typename Archive $>$ }\\void \textbf{ serialize} (Archive \&ar, const unsigned int) \begin{DoxyCompactList}\small\item\em Serialize the \doxyref{Bayesian\+Linear\+Regression}{p.}{classmlpack_1_1regression_1_1BayesianLinearRegression} model. \end{DoxyCompactList}\item double \textbf{ Train} (const arma\+::mat \&data, const arma\+::rowvec \&responses) \begin{DoxyCompactList}\small\item\em Run \doxyref{Bayesian\+Linear\+Regression}{p.}{classmlpack_1_1regression_1_1BayesianLinearRegression}. \end{DoxyCompactList}\item double \textbf{ Variance} () const \begin{DoxyCompactList}\small\item\em Get the estimated variance. \end{DoxyCompactList}\end{DoxyCompactItemize} \subsection{Detailed Description} A Bayesian approach to the maximum likelihood estimation of the parameters $ \omega $ of the linear regression model. The Complexity is governed by the addition of a gaussian isotropic prior of precision $ \alpha $ over $ \omega $\+: \[ p(\omega|\alpha) = \mathcal{N}(\omega|0, \alpha^{-1}I) \] The optimization procedure calculates the posterior distribution of $ \omega $ knowing the data by maximizing an approximation of the log marginal likelihood derived from a type II maximum likelihood approximation. The determination of $ alpha $ and of the noise precision $ beta $ is part of the optimization process, leading to an automatic determination of w. The model being entirely based on probabilty distributions, uncertainties are available and easly computed for both the parameters and the predictions. The advantage over linear regression and ridge regression is that the regularization is determined from all the training data alone without any require to an hold out method. The code below is an implementation of the maximization of the evidence function described in the section 3.\+5.\+2 of the C.\+Bishop book, Pattern Recognition and Machine Learning. \begin{DoxyCode} @article\{MacKay91bayesianinterpolation, author = \{David J.C. MacKay\}, title = \{Bayesian Interpolation\}, journal = \{NEURAL COMPUTATION\}, year = \{1991\}, volume = \{4\}, pages = \{415--447\} \} \end{DoxyCode} \begin{DoxyCode} @book\{Bishop:2006:PRM:1162264, author = \{Bishop, Christopher M.\}, title = \{Pattern Recognition and Machine Learning (Information Science and Statistics)\}, chapter = \{3\} year = \{2006\}, isbn = \{0387310738\}, publisher = \{Springer-Verlag\}, address = \{Berlin, Heidelberg\}, \} \end{DoxyCode} Example of use\+: \begin{DoxyCode} arma::mat xTrain; \textcolor{comment}{// Train data matrix. Column-major.} arma::rowvec yTrain; \textcolor{comment}{// Train target values.} \textcolor{comment}{// Train the model. Regularization strength is optimally tunned with the} \textcolor{comment}{// training data alone by applying the Train method.} BayesianLinearRegression estimator(); \textcolor{comment}{// Instanciate the estimator with default option.} estimator.Train(xTrain, yTrain); \textcolor{comment}{// Prediction on test points.} arma::mat xTest; \textcolor{comment}{// Test data matrix. Column-major.} arma::rowvec predictions; estimator.Predict(xTest, prediction); arma::rowvec yTest; \textcolor{comment}{// Test target values.} estimator.RMSE(xTest, yTest); \textcolor{comment}{// Evaluate using the RMSE score.} \textcolor{comment}{// Compute the standard deviations of the predictions.} arma::rowvec stds; estimator.Predict(xTest, responses, stds) \end{DoxyCode} Definition at line 93 of file bayesian\+\_\+linear\+\_\+regression.\+hpp. \subsection{Constructor \& Destructor Documentation} \mbox{\label{classmlpack_1_1regression_1_1BayesianLinearRegression_a77974acf4cbc46c9a4d6792229134719}} \index{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}!Bayesian\+Linear\+Regression@{Bayesian\+Linear\+Regression}} \index{Bayesian\+Linear\+Regression@{Bayesian\+Linear\+Regression}!mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}} \subsubsection{Bayesian\+Linear\+Regression()} {\footnotesize\ttfamily \textbf{ Bayesian\+Linear\+Regression} (\begin{DoxyParamCaption}\item[{const bool}]{center\+Data = {\ttfamily true}, }\item[{const bool}]{scale\+Data = {\ttfamily false}, }\item[{const size\+\_\+t}]{n\+Iter\+Max = {\ttfamily 50}, }\item[{const double}]{tol = {\ttfamily 1e-\/4} }\end{DoxyParamCaption})} Set the parameters of Bayesian Ridge regression object. The regularization parameter is automatically set to its optimal value by maximization of the marginal likelihood. \begin{DoxyParams}{Parameters} {\em center\+Data} & Whether or not center the data according to the examples. \\ \hline {\em scale\+Data} & Whether or not scale the data according to the standard deviation of each feature. \\ \hline {\em n\+Iter\+Max} & Maximum number of iterations for convergency. \\ \hline {\em tol} & Level from which the solution is considered sufficientlly stable. \\ \hline \end{DoxyParams} \subsection{Member Function Documentation} \mbox{\label{classmlpack_1_1regression_1_1BayesianLinearRegression_a500ecd077d5cc5fdbf6ceb095d8de9e1}} \index{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}!Alpha@{Alpha}} \index{Alpha@{Alpha}!mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}} \subsubsection{Alpha()} {\footnotesize\ttfamily double Alpha (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}} Get the precision (or inverse variance) of the gaussian prior. \doxyref{Train()}{p.}{classmlpack_1_1regression_1_1BayesianLinearRegression_acdffd9c2f2e55eb1d2378fd38897b898} must be called before. \begin{DoxyReturn}{Returns} $ \alpha $ \end{DoxyReturn} Definition at line 173 of file bayesian\+\_\+linear\+\_\+regression.\+hpp. \mbox{\label{classmlpack_1_1regression_1_1BayesianLinearRegression_ad1b9206255af52171cb88dfb7c326576}} \index{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}!Beta@{Beta}} \index{Beta@{Beta}!mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}} \subsubsection{Beta()} {\footnotesize\ttfamily double Beta (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}} Get the precision (or inverse variance) beta of the model. \doxyref{Train()}{p.}{classmlpack_1_1regression_1_1BayesianLinearRegression_acdffd9c2f2e55eb1d2378fd38897b898} must be called before. \begin{DoxyReturn}{Returns} $ \beta $ \end{DoxyReturn} Definition at line 181 of file bayesian\+\_\+linear\+\_\+regression.\+hpp. Referenced by Bayesian\+Linear\+Regression\+::\+Variance(). \mbox{\label{classmlpack_1_1regression_1_1BayesianLinearRegression_a2ceb93c7c05b1b54ef3e97b6f16ae714}} \index{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}!Data\+Offset@{Data\+Offset}} \index{Data\+Offset@{Data\+Offset}!mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}} \subsubsection{Data\+Offset()} {\footnotesize\ttfamily const arma\+::colvec\& Data\+Offset (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}} Get the mean vector computed on the features over the training points. \begin{DoxyReturn}{Returns} responses\+Offset \end{DoxyReturn} Definition at line 195 of file bayesian\+\_\+linear\+\_\+regression.\+hpp. \mbox{\label{classmlpack_1_1regression_1_1BayesianLinearRegression_acf80db81f970ed2b4d3b7d030955f6ca}} \index{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}!Data\+Scale@{Data\+Scale}} \index{Data\+Scale@{Data\+Scale}!mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}} \subsubsection{Data\+Scale()} {\footnotesize\ttfamily const arma\+::colvec\& Data\+Scale (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}} Get the vector of standard deviations computed on the features over the training points. \begin{DoxyReturn}{Returns} data\+Offset \end{DoxyReturn} Definition at line 203 of file bayesian\+\_\+linear\+\_\+regression.\+hpp. \mbox{\label{classmlpack_1_1regression_1_1BayesianLinearRegression_af0373053c0d05086efdab33c0b2a4f61}} \index{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}!Omega@{Omega}} \index{Omega@{Omega}!mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}} \subsubsection{Omega()} {\footnotesize\ttfamily const arma\+::colvec\& Omega (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}} Get the solution vector. \begin{DoxyReturn}{Returns} omega Solution vector. \end{DoxyReturn} Definition at line 165 of file bayesian\+\_\+linear\+\_\+regression.\+hpp. \mbox{\label{classmlpack_1_1regression_1_1BayesianLinearRegression_a71e5e21b1d92756979b32face7b3c573}} \index{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}!Predict@{Predict}} \index{Predict@{Predict}!mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}} \subsubsection{Predict()\hspace{0.1cm}{\footnotesize\ttfamily [1/2]}} {\footnotesize\ttfamily void Predict (\begin{DoxyParamCaption}\item[{const arma\+::mat \&}]{points, }\item[{arma\+::rowvec \&}]{predictions }\end{DoxyParamCaption}) const} Predict $y_{i}$ for each data point in the given data matrix using the currently-\/trained Bayesian Ridge model. \begin{DoxyParams}{Parameters} {\em points} & The data points to apply the model. \\ \hline {\em predictions} & y, Contains the predicted values on completion. \\ \hline \end{DoxyParams} \begin{DoxyReturn}{Returns} Root mean squared error computed on the train set. \end{DoxyReturn} \mbox{\label{classmlpack_1_1regression_1_1BayesianLinearRegression_aae8d40ce5c5241fea6ddad8e18296470}} \index{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}!Predict@{Predict}} \index{Predict@{Predict}!mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}} \subsubsection{Predict()\hspace{0.1cm}{\footnotesize\ttfamily [2/2]}} {\footnotesize\ttfamily void Predict (\begin{DoxyParamCaption}\item[{const arma\+::mat \&}]{points, }\item[{arma\+::rowvec \&}]{predictions, }\item[{arma\+::rowvec \&}]{std }\end{DoxyParamCaption}) const} Predict $y_{i}$ and the standard deviation of the predictive posterior distribution for each data point in the given data matrix, using the currently-\/trained Bayesian Ridge estimator. \begin{DoxyParams}{Parameters} {\em points} & The data point to apply the model. \\ \hline {\em predictions} & Vector which will contain calculated values on completion. \\ \hline {\em std} & Standard deviations of the predictions. \\ \hline \end{DoxyParams} \mbox{\label{classmlpack_1_1regression_1_1BayesianLinearRegression_abca2546910dca0c0567999bfe773a484}} \index{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}!Responses\+Offset@{Responses\+Offset}} \index{Responses\+Offset@{Responses\+Offset}!mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}} \subsubsection{Responses\+Offset()} {\footnotesize\ttfamily double Responses\+Offset (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}} Get the mean value of the train responses. \begin{DoxyReturn}{Returns} responses\+Offset \end{DoxyReturn} Definition at line 210 of file bayesian\+\_\+linear\+\_\+regression.\+hpp. References Bayesian\+Linear\+Regression\+::serialize(). \mbox{\label{classmlpack_1_1regression_1_1BayesianLinearRegression_a9f0a0b8578197c4ea00e8120bcf97df0}} \index{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}!R\+M\+SE@{R\+M\+SE}} \index{R\+M\+SE@{R\+M\+SE}!mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}} \subsubsection{R\+M\+S\+E()} {\footnotesize\ttfamily double R\+M\+SE (\begin{DoxyParamCaption}\item[{const arma\+::mat \&}]{data, }\item[{const arma\+::rowvec \&}]{responses }\end{DoxyParamCaption}) const} Compute the Root Mean Square Error between the predictions returned by the model and the true responses. \begin{DoxyParams}{Parameters} {\em data} & Data points to predict \\ \hline {\em responses} & A vector of targets. \\ \hline \end{DoxyParams} \begin{DoxyReturn}{Returns} Root mean squared error. \end{DoxyReturn} \mbox{\label{classmlpack_1_1regression_1_1BayesianLinearRegression_af0dd9205158ccf7bcfcd8ff81f79c927}} \index{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}!serialize@{serialize}} \index{serialize@{serialize}!mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}} \subsubsection{serialize()} {\footnotesize\ttfamily void serialize (\begin{DoxyParamCaption}\item[{Archive \&}]{ar, }\item[{const unsigned}]{int }\end{DoxyParamCaption})} Serialize the \doxyref{Bayesian\+Linear\+Regression}{p.}{classmlpack_1_1regression_1_1BayesianLinearRegression} model. Referenced by Bayesian\+Linear\+Regression\+::\+Responses\+Offset(). \mbox{\label{classmlpack_1_1regression_1_1BayesianLinearRegression_acdffd9c2f2e55eb1d2378fd38897b898}} \index{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}!Train@{Train}} \index{Train@{Train}!mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}} \subsubsection{Train()} {\footnotesize\ttfamily double Train (\begin{DoxyParamCaption}\item[{const arma\+::mat \&}]{data, }\item[{const arma\+::rowvec \&}]{responses }\end{DoxyParamCaption})} Run \doxyref{Bayesian\+Linear\+Regression}{p.}{classmlpack_1_1regression_1_1BayesianLinearRegression}. The input matrix (like all mlpack matrices) should be column-\/major -- each column is an observation and each row is a dimension. \begin{DoxyParams}{Parameters} {\em data} & Column-\/major input data, dim(\+P, N). \\ \hline {\em responses} & A vector of targets, dim(\+N). \\ \hline \end{DoxyParams} \begin{DoxyReturn}{Returns} Root mean squared error. \end{DoxyReturn} \mbox{\label{classmlpack_1_1regression_1_1BayesianLinearRegression_a27d69dc28c0234af0508036a6a8442c5}} \index{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}!Variance@{Variance}} \index{Variance@{Variance}!mlpack\+::regression\+::\+Bayesian\+Linear\+Regression@{mlpack\+::regression\+::\+Bayesian\+Linear\+Regression}} \subsubsection{Variance()} {\footnotesize\ttfamily double Variance (\begin{DoxyParamCaption}{ }\end{DoxyParamCaption}) const\hspace{0.3cm}{\ttfamily [inline]}} Get the estimated variance. \doxyref{Train()}{p.}{classmlpack_1_1regression_1_1BayesianLinearRegression_acdffd9c2f2e55eb1d2378fd38897b898} must be called before. \begin{DoxyReturn}{Returns} 1.\+0 / $ \beta $ \end{DoxyReturn} Definition at line 188 of file bayesian\+\_\+linear\+\_\+regression.\+hpp. References Bayesian\+Linear\+Regression\+::\+Beta(). The documentation for this class was generated from the following file\+:\begin{DoxyCompactItemize} \item /var/www/mlpack.\+ratml.\+org/mlpack.\+org/\+\_\+src/mlpack-\/git/src/mlpack/methods/bayesian\+\_\+linear\+\_\+regression/\textbf{ bayesian\+\_\+linear\+\_\+regression.\+hpp}\end{DoxyCompactItemize}