Multi-layer perceptron. More...

#include <MLP.hpp>

Collaboration diagram for MLP:

Public Types
enum	ActivationFunction { SIGMOID, TANH }

enum	WeightInitialization { NORMAL, UNIFORM, GLOROT }

enum	OutputFormat { ACTIVATION, SOFTMAX, ONEHOT, SUMMARY }

Public Member Functions
	MLP ()

void	fit (MatrixD X, MatrixD y, vector< size_t > hiddenConfig, int maxIters, size_t batchSize=0, double learningRate=0.01, double errorThreshold=0.0001, double regularization=0, ActivationFunction func=SIGMOID, WeightInitialization weightInit=UNIFORM, bool adaptiveLR=false, bool standardize=true, bool verbose=true)
	Train a multiplayer perceptron. More...

void	fit (MatrixD X, MatrixD y, vector< MatrixD > hiddenLayers, unsigned int maxIters, size_t batchSize=0, double learningRate=0.01, double errorThreshold=0.0001, double regularization=0, ActivationFunction func=SIGMOID, bool adaptiveLR=false, bool standardize=true, bool verbose=true)
	Train a multiplayer perceptron. More...

MatrixD	predict (MatrixD X, OutputFormat of=ACTIVATION)
	Predict the classes of a data set. More...

Private Member Functions
MatrixD	summarize (MatrixD m)

Static Private Member Functions
static double	pow2 (double x)

static double	sigmoid (double x)

static double	sigmoidDerivative (double x)

static double	tanh (double x)

static double	tanhDerivative (double x)

static MatrixD	initNormal (size_t in, size_t out)
	Initialize a matrix according to a normal distribution N(0; 1) More...

static MatrixD	initNormal (size_t in, size_t out, double mean, double stddev)
	Initialize a matrix according to a normal distribution N(mean; stddev) More...

static MatrixD	initUniform (size_t in, size_t out)
	Initialize a matrix according to the uniform distribution U(0;1) More...

static MatrixD	initUniform (size_t in, size_t out, double min, double max)
	Initialize a matrix according to the uniform distribution U(min; max) More...

static MatrixD	binarize (MatrixD m)

static MatrixD	softmax (MatrixD m)

Private Attributes
MatrixD	data

MatrixD	dataMean

MatrixD	dataDev

MatrixD	classes

MatrixD	originalClasses

vector< MatrixD >	W

Detailed Description

Multi-layer perceptron.

Definition at line 23 of file MLP.hpp.

Member Enumeration Documentation

◆ ActivationFunction

enum MLP::ActivationFunction

Enumerator
SIGMOID
TANH

Definition at line 150 of file MLP.hpp.

◆ OutputFormat

enum MLP::OutputFormat

Enumerator
ACTIVATION
SOFTMAX
ONEHOT
SUMMARY

Definition at line 152 of file MLP.hpp.

◆ WeightInitialization

enum MLP::WeightInitialization

Enumerator
NORMAL
UNIFORM
GLOROT

Definition at line 151 of file MLP.hpp.

Constructor & Destructor Documentation

◆ MLP()

MLP::MLP ( )

inline

Definition at line 154 of file MLP.hpp.

Member Function Documentation

◆ binarize()

static MatrixD MLP::binarize ( MatrixD m )

inlinestaticprivate

Definition at line 107 of file MLP.hpp.

Here is the caller graph for this function:

◆ fit() [1/2]

void MLP::fit	(	MatrixD	X,
		MatrixD	y,
		vector< size_t >	hiddenConfig,
		int	maxIters,
		size_t	batchSize = `0`,
		double	learningRate = `0.01`,
		double	errorThreshold = `0.0001`,
		double	regularization = `0`,
		ActivationFunction	func = `SIGMOID`,
		WeightInitialization	weightInit = `UNIFORM`,
		bool	adaptiveLR = `false`,
		bool	standardize = `true`,
		bool	verbose = `true`
	)

inline

Train a multiplayer perceptron.

Parameters

X	Input data, with rows representing examples and columns representing features
y	Input labels as a column vector
hiddenConfig	vector containing the number of neurons in each hidden layer
maxIters	maximum number of training iterations
batchSize	size of the batch. If <= 0, the whole data is used in every iteration
learningRate	learning rate
errorThreshold	minimum error for early stopping
regularization	the regularization parameter. 0 indicates no regularization.
func	activation function
weightInit	weight initilization procedure
adaptiveLR	if true, the learning rate linearly decreases according to the number of iterations
standardize	if true, data is standardized according to its mean and standard deviation
verbose	output training summary at each iteration

Definition at line 171 of file MLP.hpp.

◆ fit() [2/2]

void MLP::fit	(	MatrixD	X,
		MatrixD	y,
		vector< MatrixD >	hiddenLayers,
		unsigned int	maxIters,
		size_t	batchSize = `0`,
		double	learningRate = `0.01`,
		double	errorThreshold = `0.0001`,
		double	regularization = `0`,
		ActivationFunction	func = `SIGMOID`,
		bool	adaptiveLR = `false`,
		bool	standardize = `true`,
		bool	verbose = `true`
	)

inline

Train a multiplayer perceptron.

Parameters

X	Input data, with rows representing examples and columns representing features
y	Input labels as a column vector
hiddenLayers	a vector of matrices, each one containing the weights for a hidden layer
maxIters	maximum number of training iterations
batchSize	size of the batch. If <= 0, the whole data is used in every iteration
learningRate	learning rate
errorThreshold	minimum error for early stopping
regularization	the regularization parameter. 0 indicates no regularization.
func	activation function
adaptiveLR	if true, the learning rate linearly decreases according to the number of iterations
standardize	if true, data is standardized according to its mean and standard deviation
verbose	output training summary at each iteration

Definition at line 239 of file MLP.hpp.

Here is the call graph for this function:

◆ initNormal() [1/2]

static MatrixD MLP::initNormal	(	size_t	in,
		size_t	out
	)

inlinestaticprivate

Initialize a matrix according to a normal distribution N(0; 1)

Parameters

in	number of rows
out	number of columns

Returns: a matrix of doubles, initialized according to the distribution

Definition at line 67 of file MLP.hpp.

◆ initNormal() [2/2]

static MatrixD MLP::initNormal	(	size_t	in,
		size_t	out,
		double	mean,
		double	stddev
	)

inlinestaticprivate

Initialize a matrix according to a normal distribution N(mean; stddev)

Parameters

in	number of rows
out	number of columns
mean	mean of the normal distribution
stddev	standard deviation of the normal distribution

Returns: a matrix of doubles, initialized according to the distribution

Definition at line 79 of file MLP.hpp.

◆ initUniform() [1/2]

static MatrixD MLP::initUniform	(	size_t	in,
		size_t	out
	)

inlinestaticprivate

Initialize a matrix according to the uniform distribution U(0;1)

Parameters

in	number of rows
out	number of columns

Returns: a matrix of doubles, initialized according to the distribution

Definition at line 89 of file MLP.hpp.

◆ initUniform() [2/2]

static MatrixD MLP::initUniform	(	size_t	in,
		size_t	out,
		double	min,
		double	max
	)

inlinestaticprivate

Initialize a matrix according to the uniform distribution U(min; max)

Parameters

in	number of rows
out	number of columns
min	minimum value
max	maximum value

Returns: a matrix of doubles, initialized according to the distribution

Definition at line 101 of file MLP.hpp.

◆ pow2()

static double MLP::pow2 ( double x )

inlinestaticprivate

Parameters

x

Returns: x to the power of 2

Definition at line 32 of file MLP.hpp.

◆ predict()

MatrixD MLP::predict	(	MatrixD	X,
		OutputFormat	of = `ACTIVATION`
	)

inline

Predict the classes of a data set.

Parameters

X	Input data to be classified
of	output format of the method

Returns: a matrix, each row containing the output of the network for an example of X

Definition at line 410 of file MLP.hpp.

Here is the call graph for this function:

◆ sigmoid()

static double MLP::sigmoid ( double x )

inlinestaticprivate

Parameters

x

Returns: Sigmoid of x

Definition at line 38 of file MLP.hpp.

◆ sigmoidDerivative()

static double MLP::sigmoidDerivative ( double x )

inlinestaticprivate

Parameters

x

Returns: Derivative of the sigmoid of x

Definition at line 44 of file MLP.hpp.

◆ softmax()

static MatrixD MLP::softmax ( MatrixD m )

inlinestaticprivate

Definition at line 134 of file MLP.hpp.

Here is the caller graph for this function:

◆ summarize()

MatrixD MLP::summarize ( MatrixD m )

inlineprivate

Definition at line 120 of file MLP.hpp.

Here is the caller graph for this function:

◆ tanh()

static double MLP::tanh ( double x )

inlinestaticprivate

Parameters

x

Returns: Hyperbolic tangent of x

Definition at line 51 of file MLP.hpp.

◆ tanhDerivative()

static double MLP::tanhDerivative ( double x )

inlinestaticprivate

Parameters

x

Returns: Derivative of the hyperbolic tangent of x

Definition at line 57 of file MLP.hpp.