Package: numpy-ml
numpy-mlEver wish you had an inefficient but somewhat legible collection of machine
learning algorithms implemented exclusively in NumPy? No?InstallationFor rapid experimentationTo use this code as a starting point for ML prototyping / experimentation, just clone the repository, create a newvirtualenv, and start hacking:$gitclonehttps://github.com/ddbourgin/numpy-ml.git
$cdnumpy-ml&&virtualenvnpml&&sourcenpml/bin/activate
$pip3install-rrequirements-dev.txtAs a packageIf you don't plan to modify the source, you can also install numpy-ml as a
Python package:pip3 install -u numpy_ml.The reinforcement learning agents train on environments defined in theOpenAI
gym. To install these alongside numpy-ml, you
can usepip3 install -u 'numpy_ml[rl]'.DocumentationFor more details on the available models, see theproject documentation.Available modelsGaussian mixture modelEM trainingHidden Markov modelViterbi decodingLikelihood computationMLE parameter estimation via Baum-Welch/forward-backward algorithmLatent Dirichlet allocation(topic model)Standard model with MLE parameter estimation via variational EMSmoothed model with MAP parameter estimation via MCMCNeural networksLayers / Layer-wise opsAddFlattenMultiplySoftmaxFully-connected/DenseSparse evolutionary connectionsLSTMElman-style RNNMax + average poolingDot-product attentionEmbedding layerRestricted Boltzmann machine (w. CD-n training)2D deconvolution (w. padding and stride)2D convolution (w. padding, dilation, and stride)1D convolution (w. padding, dilation, stride, and causality)ModulesBidirectional LSTMResNet-style residual blocks (identity and convolution)WaveNet-style residual blocks with dilated causal convolutionsTransformer-style multi-headed scaled dot product attentionRegularizersDropoutNormalizationBatch normalization (spatial and temporal)Layer normalization (spatial and temporal)OptimizersSGD w/ momentumAdaGradRMSPropAdamLearning Rate SchedulersConstantExponentialNoam/TransformerDlib schedulerWeight InitializersGlorot/Xavier uniform and normalHe/Kaiming uniform and normalStandard and truncated normalLossesCross entropySquared errorBernoulli VAE lossWasserstein loss with gradient penaltyNoise contrastive estimation lossActivationsReLUTanhAffineSigmoidLeaky ReLUELUSELUExponentialHard SigmoidSoftplusModelsBernoulli variational autoencoderWasserstein GAN with gradient penaltyword2vec encoder with skip-gram and CBOW architecturesUtilitiescol2im(MATLAB port)im2col(MATLAB port)conv1Dconv2Ddeconv2DminibatchTree-based modelsDecision trees (CART)[Bagging] Random forests[Boosting] Gradient-boosted decision treesLinear modelsRidge regressionLogistic regressionOrdinary least squaresBayesian linear regression w/ conjugate priorsUnknown mean, known variance (Gaussian prior)Unknown mean, unknown variance (Normal-Gamma / Normal-Inverse-Wishart prior)n-Gram sequence modelsMaximum likelihood scoresAdditive/Lidstone smoothingSimple Good-Turing smoothingMulti-armed bandit modelsUCB1LinUCBEpsilon-greedyThompson sampling w/ conjugate priorsBeta-Bernoulli samplerLinUCBReinforcement learning modelsCross-entropy method agentFirst visit on-policy Monte Carlo agentWeighted incremental importance sampling Monte Carlo agentExpected SARSA agentTD-0 Q-learning agentDyna-Q / Dyna-Q+ with prioritized sweepingNonparameteric modelsNadaraya-Watson kernel regressionk-Nearest neighbors classification and regressionGaussian process regressionMatrix factorizationRegularized alternating least-squaresNon-negative matrix factorizationPreprocessingDiscrete Fourier transform (1D signals)Discrete cosine transform (type-II) (1D signals)Bilinear interpolation (2D signals)Nearest neighbor interpolation (1D and 2D signals)Autocorrelation (1D signals)Signal windowingText tokenizationFeature hashingFeature standardizationOne-hot encoding / decodingHuffman coding / decodingTerm frequency-inverse document frequency (TF-IDF) encodingMFCC encodingUtilitiesSimilarity kernelsDistance metricsPriority queueBall treeDiscrete samplerGraph processing and generatorsContributingAm I missing your favorite model? Is there something that could be cleaner /
less confusing? Did I mess something up? Submit a PR! The only requirement is
that your models are written with just thePython standard
libraryandNumPy. TheSciPy libraryis also permitted under special
circumstances ;)See full contributing guidelineshere.