Package: pynet-dl
PynetPynet is a Python library containing all of the deep learning building blocks.FeaturesTensor operationsTensor operations are included in thepynet.functionalnamespace. Pynet provides a couple of basic tensor operations such as matrix multiplication or tensor addition. New tensor operation can be created by implementing theFunctionabstract class. Tensor functions that are included in the library are:Tensor addition[link]Matrix multiplication[link]Element-wise maximum[link]Element-wise sigmoid[link]Neural network layers and activation functionsImplementation of neural network layers and activation functions are included in thepynet.nnnamespace. The namespace contains basic layer and activation function implementations and can be extended by implementing theModuleabstract class. List of layers and activation functions included in the Pynet library:Fully connected layer (Linear)[link]Sequential module (container for other modules)[link]Rectified linear unit (ReLU) activation function[link]Sigmoid activation function[link]Loss functionsLoss function implementations are included in thepynet.lossnamespace. Custom loss functions can be created by implementing theLossabstract class. Loss functions included in the Pynet library are:Binary cross entropy[link]Mean squared error[link]Optimization algorithmsOptimization algorithm implementations are included in thepynet.optimizersnamespace. New optimization algorithm can be created by implementing theOptimizerabstract class. Optimization algorithms included in the library:Stochastic Gradient Descent (SGD)[link]Data manipulationpynet.datanamespace provides abstraction over the datasets used for training the neural network.In-memorydataset implementation is already included in the library, however one can create their own dataset implementation by extending theDatasetabstract class.Neural network trainingFor the neural network's training/testing procedure, one can either use default implementation of these procedures, which can be found in theDefaultTrainerclass or implement their own training/testing logic by extending theTrainerabstract class. One can also omit both these options and implement its own logic from scratch (for example, see theDefaultTrainertrain and test function implementations).When using one of theTrainer's implementations, aCallbackclass was created to perform additional actions at various stages of training/testing procedure. There are a couple of callbacks implemented and ready to use, such as:Callback providing printing all the measured model's metrics onto the console (PrintCallback)Callback providing optimizer's learning rate scheduling (LrSchedule)One can also easily create their own callback by extending theCallbackabstract class.SummaryTable below summarizes the feature section. Every row of a table contains name of the namespace, namespace description (what functionality this namespace provides) and a way of extending this functionality (by implementing the abstract class).NamespaceFunctionalityAbstract classpynet.functionalTensor operationsFunctionpynet.nnNeural network layers and activation functionsModulepynet.lossLoss functionsLosspynet.optimizersOptimization algorithmsOptimizerpynet.dataData manipulationDatasetpynet.training.trainerNeural network training/testing procedureTrainerpynet.training.callbacksCallback functions during the training/testingCallbackInstallationUse the package managerpipto install pynet.pipinstallpynet-dlUsageHere is the simple example how to use the Pynet library. In this example we are going to train a very small neural network on the make_circles dataset from sklearn.dataset package.TrainingFirst, we need to import all the necessary stuff:importnumpyasnpfromsklearn.datasetsimportmake_circles# Tensor classfrompynet.tensorimportTensor# Neural network modulesfrompynet.nn.sequentialimportSequentialfrompynet.nn.linearimportLinearfrompynet.nn.reluimportReLUfrompynet.nn.sigmoidimportSigmoid# Datasetsfrompynet.data.in_memoryimportInMemoryDataset# Loss functionsfrompynet.loss.bceimportBinaryCrossEntropy# Optimizersfrompynet.optimizers.sgdimportSGD# Weight initializersfrompynet.initializers.he_normalimportHeNormal# Trainer and training/testing callbacksfrompynet.training.trainer.defaultimportDefaultTrainerfrompynet.training.callbacks.printimportPrintCallbackThen, load the dataset and do preprocessing. As the single input to the neural network (i. e. sample xi) is supposed to be of shape [n_features x 1] (i. e. column vector), we need to add one extra dimension to the X array:X,y=make_circles(n_samples=1000,noise=0.025)# inputs to neural net must be of shape [n, 1]X=np.expand_dims(X,axis=2)Then, we will create the network, dataset, optimizer, appropriate loss function, trainer and optionally also specify a list of callbacks:epochs=20model=Sequential([Linear(inputs=2,neurons=16,initializer=HeNormal()),ReLU(),Linear(inputs=16,neurons=1,initializer=HeNormal()),Sigmoid()])dataset=InMemoryDataset(X,y)loss_f=BinaryCrossEntropy()sgd=SGD(learning_rate=0.01,momentum=0.9)callbacks=[PrintCallback(),LrSchedule(optimizer=sgd,schedule=lr_schedule)]trainer=DefaultTrainer()And then run the training:trainer.train(model=model,train_dataset=dataset,val_dataset=None,loss_f=loss_f,optimizer=sgd,epochs=epochs,callbacks=callbacks)InferenceAfter training the neural network, we can use it for inference.# Make prediction for the i-th sample in the dataseti=0y_pred=model.forward(Tensor(X[i])).ndarray.item()ExamplesMore examples of the Pynet library usage can be found in thenotebooksdirectory.LicenseMIT