The simplest way to train a Neural Network in Python | by Roman Orac | Towards Data Science

scikit-learn is my first choice when it comes to classic Machine Learning algorithms in Python. It has many algorithms, supports sparse datasets, is fast and has many utility functions, like cross-validation, grid search, etc.
When it comes to advanced modeling, scikit-learn many times falls shorts. If you need Boosting, Neural Networks or t-SNE, it’s better to avoid scikit-learn.
scikit-learn has two basic implementations for Neural Nets. There’s MLPClassifier for classification and MLPRegressor for regression.
While MLPClassifier and MLPRegressor have a rich set of arguments, there’s no option to customize layers of a Neural Network (beyond setting the number of hidden units for each layer) and there’s no GPU support.
scikit-neuralnetwork addresses the issues with scikit-learn mentioned above. While there are already superior libraries available like PyTorch or Tensorflow, scikit-neuralnetwork may be a good choice for those coming from a scikit-learn ecosystem.
From developers of scikit-neuralnetwork:
scikit-neuralnetwork is a deep neural network implementation without the learning cliff! This library implements multi-layer perceptrons as a wrapper for the powerful pylearn2 library that’s compatible with scikit-learn for a more user-friendly and Pythonic interface.
To install scikit-neuralnetwork (sknn) is as simple as installing any other Python package:
pip install scikit-neuralnetwork
Let’s define X_train and y_train from the Iris dataset to run the examples below:
from sklearn.datasets import load_irisdata = load_iris()X_train = data['data']y_train = data["target"]
sknn offers a simple way to make a custom Neural Net. scikit-learn users will feel at home with a familiar API:
from sknn.mlp import Classifier, Layernn = Classifier(    layers=[        Layer("Maxout", units=100, pieces=2),        Layer("Softmax")],    learning_rate=0.001,    n_iter=25)nn.fit(X_train, y_train)
X_train, y_train variables are numpy arrays, so you can directly replace your scikit-learn model with a Neural Net from sknn. It even supports sparse datasets.
sknn has support for Convolutional Neural Nets. Finally, you will be able to achieve a state-of-the-art score on MNIST in a scikit-learn ecosystem.
from sknn.mlp import Classifier, Convolution, Layernn = Classifier(    layers=[        Convolution("Rectifier", channels=8, kernel_shape=(3,3)),        Layer("Softmax")],    learning_rate=0.02,    n_iter=5)nn.fit(X_train, y_train)
What about RNNs, like Long Short Term Memory (LTSM) or Gated Recurrent Unit (GRU)? RNNs are usually used for modeling sequences, like time series or textual data.
By going through the documentation, it seems there isn’t direct support for RNNs. There is support for native and custom layers, which should make the implementation of RNN possible.
From documentation:
You can use this feature to implement recurrent layers like LSTM or GRU, and any other features not directly supported. Keep in mind that this may affect compatibility in future releases, and also may expose edge cases in the code (e.g. serialization, determinism).
If you plan to work with RNNs I would recommend learning PyTorch or TensorFlow. If you need a quick start guide, I wrote an article about it a while ago:
towardsdatascience.com
scikit-learn has pipelines, which wrap feature transformation with modeling into a single pipeline.
Pipelines reduce the chance of overfitting and generally reduce the chances of various mistakes. They are also very useful when making cross-validation or grid search.
Many Machine Learning libraries don’t support scikit-learn pipelines so we need to implement it by ourselves. The great thing about scikit-neuralnetwork is that it fully supports scikit-learn pipelines.
Below is an example of a pipeline that scales features and trains a simple Neural Net.
from sknn.mlp import Classifier, Layerfrom sklearn.pipeline import Pipelinefrom sklearn.preprocessing import MinMaxScalerpipeline = Pipeline([        ('min/max scaler', MinMaxScaler(feature_range=(0.0, 1.0))),        ('neural network', Classifier(layers=[Layer("Softmax")], n_iter=25))])pipeline.fit(X_train, y_train)
Check the Churn prediction article for the advanced usage of a pipeline. I show how to process categorical and numerical features separately:
towardsdatascience.com
Unlike scikit-learn, scikit-neuralnetwork has support for GPUs as it is based on the Lasagne library. Note, GPU support requires an NVIDIA GPU with CUDA support. If you have a MacBook you most probably have a Radeon GPU, which doesn’t support CUDA.
From Lasagne documentation:
Thanks to Theano, Lasagne transparently supports training your networks on a GPU, which may be 10 to 50 times faster than training them on a CPU. Currently, this requires an NVIDIA GPU with CUDA support, and some additional software for Theano to use it.
To use GPU Backend you just need to import:
# Use the GPU in 32-bit mode, falling back otherwise.from sknn.platform import gpu32
From documentation:
WARNING: This will only work if your program has not yet imported the theano module, due to the way that library is designed. If THEANO_FLAGS are set on the command-line, they are not overridden.
If you’re well versed with sklearn library, then scikit-neuralnetwork may be a good starting point to get familiar with Deep Learning.
scikit-neuralnetwork is also useful when we need a Neural Network that works as a drop-in replacement for a sklearn algorithm.
I would recommend learning PyTorch if you’re just starting with Deep Learning or you plan to be in the field for the long term.
- Deploy Your 1st Machine Learning Model in the Cloud [Ebook]- How To Create Date Series in SQL [Article]- Free skill tests for Data Scientists & Machine Learning Engineers- Become a Cloud Developer using Microsoft Azure [Course]- Master SQL, the core language for Big Data analysis [Course]
Some of the links above are affiliate links and if you go through them to make a purchase I’ll earn a commission. Keep in mind that I link courses because of their quality and not because of the commission I receive from your purchases.
Follow me on Twitter, where I regularly tweet about Data Science and Machine Learning.