Package: dipple
dipple: deep but simple to build..what is it?Dippleis a Python package that simplifies the process of creating neural network architectures, particularly for beginners in data science. It offers a simple and easy-to-use interface for building Linear, Logistic, Shallow, and Deep L-layer Neural networks using only a few lines of code.It currently supports Multi-layer Perceptron, with various regularization options such as L2 and Dropout, as well as optimizers and weight initialization techniques to improve training and avoid saddle points. The name "Dipple" is a combination of the words "deep" and "simple", and it reflects the package's goal of making building deep neural networks easy and accessible to beginners.The project, Dipple, was initiated in 2022 by Saran Pannasuriyaporn as a means of self-study in the field of deep learning. The author chose to learn by writing code from scratch, as a way of gaining a deeper understanding of the concepts. This package is not intended to replace existing libraries such as Tensorflow or Pytorch, but rather to provide an opportunity for aspiring learners to not only learn about deep learning, but also advanced concepts such as object-oriented programming by examining the code samples of Dipple Repository available onGitHub.RequirementPython 3.7 +InstallationpipinstalldippleQuick Start1. Import the datasetGet started quickly by exploring the Crescent dataset, a popular toy dataset for binary-class classification tasks and a benchmark for machine learning models. With Dipple, loading and using the Crescent dataset is easy and straightforwardfromdipple.import_datasetimportDataset_Librarydlib_borrow=Dataset_Library('crescent')df_train,df_test=dlib_borrow.get_2df()# Preview the first 3 rows of train setdisplay(df_train.head(3))x1x2y0-0.2168701.015449010.805050-0.557973120.711275-0.4100601#splitting predictor and classX_train=df_train[['x1','x2']]Y_train=df_train[['y']]2. Define the Multilayer Perceptron ArchitectureIf you wish to build a multi-layer perceptron with 2 hidden layers, containing 5 and 2 units respectively, you can define the details in thehyperparam_settingdictionary. In addition, you can also specify the activation function for both hidden and output layer as relu and sigmoid respectivelyhyperparam_setting={"layer_dims":[2,5,2,1],"hidden_activation_function":'relu',"output_activation_function":'sigmoid',}The Dipple's MLP implementation expects thehyperparam_settingto have specific keys named "layer_dims", "hidden_activation_function", and "output_activation_function" respectively, in order to define the architecture of the multi-layer perceptron.The available activation functions for both of hidden and output layers are as the following :linearsigmoidtanhreluleakyrelusoftmaxOnce choose, you can define the model object by inputhyperparam_settingfromdipple.mlpimport*model=MultilayerPerceptron.initiate_by_hyperparam_dict(hyperparam_setting)3. Configuring Gradient DescentThis step is used to specify the method for updating the parameters via gradient descent.If you want to set the gradient descent with loss function binary_cross_entropy_loss, learning rate 0.0001, weight initialization as he, regularization as L2 with lambda value of 0.001, and optimizer adam with beta1 = 0.9, beta2 = 0.99, eps = 10e-8, you can use the following code:model.compiles(loss='binary_cross_entropy_loss',lr=1e-3,initialization='he',regularization="L2",lambd=1e-2,optimizer='adam',beta1=0.9,beta2=0.99,eps=10e-8)The details of argument setting for method compile are shown as the following:loss: ['binary_cross_entropy_loss','cross_entropy_loss','MSE']learning rateweight initialization techniques: ['zero','random','he','xavier']regularization: ['dropout','L2']optimizer: : ['momentum','adagrad','rmsprop','adam']Note that if you wish to use dropout instead of L2, the keyword argumentlambdmust be replaced by tuplekeep_prob_sequence, which indicate the keep probability of a sequence of layer respectively4. Fit Data to our modelOnce the model is configured, we can use it to fit our data using mini-batch gradient descent with a batch size of 32 for 27000 epochs. To track the progress of the model and report the cost function every 1000 epochs. we can use the following codemodel.fit(X_train,Y_train,Epochs=27000,batch_size=32,report_cost=True,evry_report_epoch=1000)Output:========>Epoch1000/27000:===Cost===:0.4433844520553876
>Epoch2000/27000:===Cost===:0.3674708272179111
>Epoch3000/27000:===Cost===:0.34272523427485757...
Epoch26000/27000:===Cost===:0.1516102412308588
Epoch27000/27000:===Cost===:0.15146985031429971Once the model is trained, we can access the updated parameters (weights and biases) for each layer by using the following code:model.paramOutput======={'W1':array([[1.66269748,0.18280045],[0.98504132,1.58239975],[1.23171595,0.07314983],[1.56213207,0.05702136],[-0.39345288,0.88787371]]),..'W3':array([[-0.51395741,4.60415329]]),'b3':array([[-4.84589836]])}5. PredictThe trained model can be used to make predictions using the predict method. There are two options for the output: probability and cut-off value.If you want the probability, you can directly use the following code:model.predict(X_train)Output=======array([[0.12525224,0.96623857,0.96625601,0.99820462,0.00779925,....]])However, if you prefer the cut-off value with a threshold of 0.5, you can use the following code:model.predict(X_train,predict_proba=False,threshold=0.5)Output=======array([[0,1,1,1,0,....]])It's worth noting that when predict_proba is set to False, the threshold parameter is not required for multi-class classification (with softmax as the output activation function) as the class with the highest probability will be selected automatically.6. Evaluate and Interpret the resultIf there are 2 predictors, it is worth-try to plot 2D decision boundary. In this crescent datasetfromdipple.interpretimportplot_decision_boundary_2D
fromdipple.metricsimportaccuracy_scorethreshold=0.5

plot_decision_boundary_2D(model=model,X=X,y=Y,threshold=threshold)Y_pred=model.predict(X,threshold=threshold,predict_proba=False)print(f'Accuracy on train set : {accuracy_score(Y_pred,Y)}')Accuracy on train set : 0.94Dependenciesour package dipple implements by these packages with the following versionsDependencyVersionnumpy1.21.6pandas1.3.5matplotlib3.2.2LicenseMIT