Mlpack. mlpack is a free, open-source and header-only software library for machine learning and artificial intelligence written in C++, built on top of the Armadillo library and the ensmallen numerical optimization library. mlpack has an emphasis on scalability, speed, and ease-of-use. Its aim is to make machine learning possible for novice users by means of a simple, consistent API, while simultaneously exploiting C++ language features to provide maximum performance and maximum flexibility for expert users. mlpack has also a light deployment infrastructure with minimum dependencies, making it perfect for embedded systems and low resource devices. Its intended target users are scientists and engineers.

It is open-source software distributed under the BSD license, making it useful for developing both open source and proprietary software. Releases 1.0.11 and before were released under the LGPL license.  The project is supported by the Georgia Institute of Technology and contributions from around the world.

Features

Classical machine learning algorithms 
mlpack contains a wide range of algorithms that are used to solved real problems from classification and regression in the Supervised learning paradigm to clustering and dimension reduction algorithms. In the following, a non exhaustive list of algorithms and models that mlpack supports:

 Collaborative Filtering
 Decision stumps (one-level decision trees)
 Density Estimation Trees
 Euclidean minimum spanning trees
 Gaussian Mixture Models (GMMs)
 Hidden Markov Models (HMMs)
 Kernel density estimation (KDE)
 Kernel Principal Component Analysis (KPCA)
 K-Means Clustering
 Least-Angle Regression (LARS/LASSO)
 Linear Regression
 Bayesian Linear Regression
 Local Coordinate Coding
 Locality-Sensitive Hashing (LSH)
 Logistic regression
 Max-Kernel Search
 Naive Bayes Classifier
 Nearest neighbor search with dual-tree algorithms
 Neighbourhood Components Analysis (NCA)
 Non-negative Matrix Factorization (NMF)
 Principal Components Analysis (PCA)
 Independent component analysis (ICA)
 Rank-Approximate Nearest Neighbor (RANN)
 Simple Least-Squares Linear Regression (and Ridge Regression)
 Sparse Coding, Sparse dictionary learning
 Tree-based Neighbor Search (all-k-nearest-neighbors, all-k-furthest-neighbors), using either kd-trees or cover trees
 Tree-based Range Search
Class templates for GRU, LSTM structures are available, thus the library also supports Recurrent Neural Networks.

Bindings 
There are bindings to R, Go, Julia, Python, and also to Command Line Interface (CLI) using terminal. Its binding system is extensible to other languages.

Reinforcement learning 
mlpack contains several Reinforcement Learning (RL) algorithms implemented in C++ with a set of examples as well, these algorithms can be tuned per examples and combined with external simulators.  Currently mlpack supports the following:

 Q-learning 
 Deep Deterministic Policy Gradient
 Soft Actor-Critic 
 Twin Delayed DDPG (TD3)

Design features 
mlpack includes a range of design features that make it particularly well-suited for specialized applications, especially in the Edge AI and IoT domains. Its C++ codebase allows for seamless integration with sensors, facilitating direct data extraction and on-device preprocessing at the Edge. Below, we outline a specific set of design features that highlight mlpack's capabilities in these environments:

Low number of dependencies 
mlpack is low dependencies library which makes it perfect for easy deployment of software. mlpack binaries can be linked statically and deployed to any system with minimal effort. The usage of Docker container is not necessary and even discouraged. This makes it suitable for low resource devices, as it requires only the ensmallen and Armadillo or Bandicoot depending on the type of hardware we are planning to deploy to. mlpack uses Cereal library for serialization of the models. Other dependencies are also header-only and part of the library itself.

Low binary footprint 
In terms of binary size, mlpack methods have a significantly smaller footprint compared to other popular libraries. Below, we present a comparison of deployable binary sizes between mlpack, PyTorch, and scikit-learn. To ensure consistency, the same application, along with all its dependencies, was packaged within a single Docker container for this comparison.

Other libraries exist such as Tensorflow Lite, However, these libraries are usually specific for one method such as neural network inference or training.

Example 
The following shows a simple example how to train a decision tree model using mlpack, and to use it for the classification. Of course you can ingest your own dataset using the Load function, but for now we are showing the API:// Train a decision tree on random numeric data and predict labels on test data:

// All data and labels are uniform random; 10 dimensional data, 5 classes.
// Replace with a data::Load() call or similar for a real application.
arma::mat dataset(10, 1000, arma::fill::randu); // 1000 points.
arma::Row<size_t> labels =
    arma::randi<arma::Row<size_t>>(1000, arma::distr_param(0, 4));
arma::mat testDataset(10, 500, arma::fill::randu); // 500 test points.

mlpack::DecisionTree tree;               // Step 1: create model.
tree.Train(dataset, labels, 5);          // Step 2: train model.
arma::Row<size_t> predictions;
tree.Classify(testDataset, predictions); // Step 3: classify points.

// Print some information about the test predictions.
std::cout << arma::accu(predictions == 2) << " test points classified as class "
    << "2." << std::endl;The above example demonstrate the simplicity behind the API design, which makes it similar to popular Python based machine learning kit (scikit-learn). Our objective is to simplify for the user the API and the main machine learning functions such as Classify and Predict. More complex examples are located in the examples repository, including documentations for the methods

Backend

Armadillo 
Armadillo is the default linear algebra library that is used by mlpack, it provide matrix manipulation and operation necessary for machine learning algorithms. Armadillo is known for its efficiency and simplicity. it can also be used in header-only mode, and the only library we need to link against are either OpenBLAS, IntelMKL or LAPACK.

Bandicoot 
Bandicoot is a C++ Linear Algebra library designed for scientific computing, it has the an identical API to Armadillo with objective to execute the computation on Graphics Processing Unit (GPU), the purpose of this library is to facilitate the transition between CPU and GPU by making a minor changes to the source code, (e.g. changing the namespace, and the linking library). mlpack currently supports partially Bandicoot with objective to provide neural network training on the GPU.  The following examples shows two code blocks executing an identical operation. The first one is Armadillo code and it is running on the CPU, while the second one can runs on OpenCL supported GPU or NVIDIA GPU (with CUDA backend)using namespace arma;
        
mat X, Y;
X.randu(10, 15);
Y.randu(10, 10);
mat Z = 2 * norm(Y) * (X * X.t() - Y);using namespace coot;
        
mat X, Y;
X.randu(10, 15);
Y.randu(10, 10);
mat Z = 2 * norm(Y) * (X * X.t() - Y);

ensmallen 
ensmallen is a high quality C++ library for non linear numerical optimizer, it uses Armadillo or bandicoot for linear algebra and it is used by mlpack to provide optimizer for training machine learning algorithms. Similar to mlpack, ensmallen is a header-only library and supports custom behavior using callbacks functions allowing the users to extend the functionalities for any optimizer. In addition ensmallen is published under the BSD license.

ensmallen contains a diverse range of optimizer classified based on the function type (differentiable, partially differentiable, categorical, constrained, etc). In the following we list a small set of optimizer that available in ensmallen. For the full list please check this documentation website.

 Limited memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS)
 GradientDescent
 FrankWolfe
 Covariance matrix adaptation evolution strategy  (CMA-ES)
 AdaBelief
 AdaBound
 AdaDelta
 AdaGrad
 AdaSqrt
 Adam
 AdaMax
 AMSBound
 AMSGrad
 Big Batch SGD
 Eve
 FTML
 IQN
 Katyusha
 Lookahead
 Momentum SGD
 Nadam
 NadaMax
 NesterovMomentumSGD
 OptimisticAdam
 QHAdam
 QHSGD
 RMSProp
 SARAH/SARAH+
 Stochastic Gradient Descent SGD
 Stochastic Gradient Descent with Restarts (SGDR)
 Snapshot SGDR
 SMORMS3
 SPALeRA
 SWATS
 SVRG
 WNGrad

Support 
mlpack is fiscally sponsored and supported by NumFOCUS, Consider making a tax-deductible donation to help the developers of the project. In addition mlpack team participates each year Google Summer of Code program and mentors several students.

See also

 Armadillo (C++ library)
 List of numerical analysis software
 List of numerical libraries
 Numerical linear algebra
 Scientific computing

References

External links
 
 

C++ libraries
Data mining and machine learning software
Free computer libraries
Free mathematics software
Free science software
Free software programmed in C++
Free statistical software