Package: deepforge
deepforgeA python object oriented library to model deep neural networks based on Keras/Tensorflow (>=2.14).VersionThe current version is 0.0.7.Changelog16.11.2023 - Version 0.0.7: Adding type annotations, Docstrings and implementation of raise conditions.08.11.2023 - Version 0.0.6: Refactoring of the deepforge library.13.10.2023 - Version 0.0.5: Integration of a fully functional Denoising AutoEncoder class.12.10.2023 - Version 0.0.4: Integration of a fully functional Convolutional Recurrent Neural Network class.11.10.2023 - Version 0.0.3: Integration of a fully functional Convolutional Neural Network and Recurrent Neural Network classes.10.10.2023 - Version 0.0.2: Integration of a fully functional multivariate Deep Neural Network class.Overviewdeepforge is a Python object-oriented library built on top of Keras and TensorFlow for simplifying the creation and training of deep neural networks. It provides a user-friendly interface for designing, configuring, and training neural network models, making it easier for developers and researchers to work with deep learning. Next versions of the library should include support for other deep learning frameworks (such as PyTorch, Theano, Caffe, etc.).FeaturesHigh-level API for defining neural network architectures.Modular and extensible design for easy customization.Integration with Keras and TensorFlow for efficient training and deployment.Support for a variety of neural network types, including feed-forward, convolutional, recurrent, and more.Built-in utilities for data preprocessing, evaluation, and visualization.InstallationYou can install your library using pip:pipinstalldeepforgeThis will install the library with full support for tensorflow-gpu.Quick StartIn the following sections, we will refer to the Deep Neural Network schemes reported inThe mostly complete chart of neural networks explainedby Fjodor van Veen. The reference chart is reported here:Multivariate Deep Neural Network (multivariateDNN)The generic scheme of a Multivariate Deep Neural Network is:Here's a simple example of how to deepforge a simple Deep Neural Network via themultivariateDNNclass:importnumpyasnp# Import the deepforge libraryimportdeepforgeasdf# Initialize the environmentdf.initialize(CPU=20,GPU=1,VERBOSE='2',NPARRAYS=True)# Make an instance of a multivariate DNNmDnn=df.multivariateDNN(name="Simple DNN",inputN=1)# Set inputs, inner layers and out layersmDnn.setInputs([{'shape':(2,),'name':'Input layer'}])mDnn.setLayers([[{'units':16,'activation':'elu'},{'units':16,'activation':'elu'},{'units':16,'activation':'elu'},{'units':3,'activation':'linear'}]])mDnn.setOutLayers([{'units':1,'activation':'linear'}])# Configure the modelmDnn.setModelConfiguration(optimizer='adam',loss='mse')# Build the model and print the summarymDnn.build()mDnn.summary()# Train the modelx1=np.array([2,3,5,6,7],dtype=np.float32)x2=np.array([1,2,4,5,6],dtype=np.float32)X1=np.array([x1,x2],dtype=np.float32).Ty=np.array([3,4,6,7,8],dtype=np.float32)mDnn.fit(x=[X1],y=y,epochs=20,shuffle=True,verbose=0)# Save the modelmDnn.save('simpleDNN',tflite=False)# Make a prediction with the modelx1=np.array([8],dtype=np.float32)x2=np.array([7],dtype=np.float32)X1=np.array([x1,x2],dtype=np.float32).Ty=mDnn.predict([X1])print(y.numpy())# Load the modelmDnnCopy=df.multivariateDNN(name="simple DNN 2")mDnnCopy.load("simpleDNN")The output of the previous snippet is:[DF] Building model...
[DF] Model built!
Model: "SimpleDNN"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 Input layer (InputLayer)    [(None, 2)]               0         
                                                                 
 dense_20 (Dense)            (None, 16)                48        
                                                                 
 dense_21 (Dense)            (None, 16)                272       
                                                                 
 dense_22 (Dense)            (None, 16)                272       
                                                                 
 dense_23 (Dense)            (None, 3)                 51        
                                                                 
 dense_24 (Dense)            (None, 1)                 4         
                                                                 
=================================================================
Total params: 647
Trainable params: 647
Non-trainable params: 0
_________________________________________________________________
[DF] Saving model...
[DF] Model saved!
[[1.0306408]]
[DF] Loading model...
[DF] Loaded!The samemultivariateDNNclass can be used to build a more complex DNN model with a custom loss function:# DNN with 2 input layers and custom loss function exampleimportnumpyasnpfromkeras.lossesimportMeanSquaredError# Import the deepforge libraryimportdeepforgeasdf# Initialize the environmentdf.initialize(CPU=20,GPU=1,VERBOSE='2',NPARRAYS=True)# Define a custom loss functiondefcustom_loss(y_true,y_pred):mse=MeanSquaredError()returnmse(y_true,y_pred)# Make an instance of a multivariate DNNmDnn2=df.multivariateDNN(name="multivariate DNN",inputN=2)# Set inputs, inner layers and out layersmDnn2.setInputs([{'shape':(2,),'name':'inputLayer1'},{'shape':(2,),'name':'inputLayer2'}])innerLayers=[[{'units':32,'activation':'elu'},{'units':16,'activation':'elu'},{'units':8,'activation':'elu'},{'units':3,'activation':'linear'}]]innerLayers.append([{'units':32,'activation':'elu'},{'units':16,'activation':'elu'},{'units':8,'activation':'elu'},{'units':3,'activation':'linear'}])mDnn2.setLayers(innerLayers)outputLayers=[{'units':32,'activation':'elu'},{'units':1,'activation':'linear'}]mDnn2.setOutLayers(outputLayers)# Configure the modelmDnn2.setModelConfiguration(optimizer='adam',loss=custom_loss)# Build the model and print the summarymDnn2.build()mDnn2.summary()# Train the modelx1=np.array([0.2,0.3,0.5,0.6,0.7],dtype=np.float32)x2=np.array([1.1,1.2,1.4,1.5,1.6],dtype=np.float32)X1=np.array([x1,x2],dtype=np.float32).Tx3=np.array([0.2,0.3,0.5,0.6,0.7],dtype=np.float32)x4=np.array([1.1,1.2,1.4,1.5,1.6],dtype=np.float32)X2=np.array([x3,x4],dtype=np.float32).Ty=np.array([3.0,4.0,6.0,7.0,8.0],dtype=np.float32)mDnn2.fit(x=[X1,X2],y=y,epochs=50,shuffle=True,verbose=0)# Save the modelmDnn2.save('multivariateDNN',tflite=False)# Make a prediction with the modelx1=np.array([0.4],dtype=np.float32)x2=np.array([1.3],dtype=np.float32)X1=np.array([x1,x2],dtype=np.float32).Tx3=np.array([0.4],dtype=np.float32)x4=np.array([1.3],dtype=np.float32)X2=np.array([x3,x4],dtype=np.float32).Ty=mDnn2.predict([X1,X2])print(y.numpy())# Load the model with the custom loss functionmDnn2Copy=df.multivariateDNN(name="multivariate DNN 2")mDnn2Copy.load("multivariateDNN",custom_objects={'custom_loss':custom_loss})The output of the previous snippet is:[DF] Building model...
[DF] Model built!
Model: "multivariateDNN"
__________________________________________________________________________________________________
 Layer (type)                   Output Shape         Param #     Connected to                     
==================================================================================================
 inputLayer1 (InputLayer)       [(None, 2)]          0           []                               
                                                                                                  
 inputLayer2 (InputLayer)       [(None, 2)]          0           []                               
                                                                                                  
 dense_25 (Dense)               (None, 32)           96          ['inputLayer1[0][0]']            
                                                                                                  
 dense_29 (Dense)               (None, 32)           96          ['inputLayer2[0][0]']            
                                                                                                  
 dense_26 (Dense)               (None, 16)           528         ['dense_25[0][0]']               
                                                                                                  
 dense_30 (Dense)               (None, 16)           528         ['dense_29[0][0]']               
                                                                                                  
 dense_27 (Dense)               (None, 8)            136         ['dense_26[0][0]']               
                                                                                                  
 dense_31 (Dense)               (None, 8)            136         ['dense_30[0][0]']               
                                                                                                  
 dense_28 (Dense)               (None, 3)            27          ['dense_27[0][0]']               
                                                                                                  
 dense_32 (Dense)               (None, 3)            27          ['dense_31[0][0]']               
                                                                                                  
 concatenate_1 (Concatenate)    (None, 6)            0           ['dense_28[0][0]',               
                                                                  'dense_32[0][0]']               
                                                                                                  
 dense_33 (Dense)               (None, 32)           224         ['concatenate_1[0][0]']          
                                                                                                  
 dense_34 (Dense)               (None, 1)            33          ['dense_33[0][0]']               
                                                                                                  
==================================================================================================
Total params: 1,831
Trainable params: 1,831
Non-trainable params: 0
__________________________________________________________________________________________________
[DF] Saving model...
[DF] Model saved!
[[6.0725203]]
[DF] Loading model...
[DF] Loaded!For more detailed usage and examples, please refer to the documentation.Convolutional Neural Network (CNN)The generic scheme of a Convolutional Neural Network is reported here:Here's a simple example of how to deepforge a simple Convolutional Neural Network via theCNNclass and fitting the model with the MNIST dataset:# Convolutional Neural Network example with MNIST dataset training and validationimportnumpyasnp# Import the deepforge libraryimportdeepforgeasdffromkeras.datasetsimportmnistfromkeras.utilsimportto_categorical# Initialize the environmentdf.initialize(CPU=20,GPU=1,VERBOSE='2',NPARRAYS=True)# Make an instance of a CNNcnn=df.CNN(name="Simple CNN",inputN=1)# Set inputs, inner layers and out layerscnn.setInputs([{'shape':(28,28,1),'name':'Input Layer'}])cnn.setConvLayers([[{'filters':32,'kernel_size':(3,3),'activation':'relu'},{'filters':64,'kernel_size':(3,3),'activation':'relu'},{'filters':64,'kernel_size':(3,3),'activation':'relu'}]])cnn.setPoolLayers([[{'pool_size':(2,2)},{'pool_size':(2,2)}]])cnn.setOutLayers([{'units':64,'activation':'relu'},{'units':10,'activation':'softmax'}])# Configure the modelcnn.setModelConfiguration(optimizer='adam',loss='categorical_crossentropy',metrics=['accuracy'])# Build the model and print the summarycnn.build()cnn.summary()# Load the MNIST dataset and preprocess it(train_images,train_labels),(test_images,test_labels)=mnist.load_data()train_images=train_images.reshape((60000,28,28,1))test_images=test_images.reshape((10000,28,28,1))train_images=train_images.astype('float32')/255test_images=test_images.astype('float32')/255train_labels=to_categorical(train_labels)test_labels=to_categorical(test_labels)# Fit the modelcnn.fit(x=train_images,y=train_labels,epochs=5,batch_size=64)# Save the modelcnn.save('CNN',tflite=False)# Get the Keras model and run a test to evaluate accuracycnnModel=cnn.getModel()test_loss,test_acc=cnnModel.evaluate(test_images,test_labels)print("Test accuracy:",test_acc)The output of the previous code snippet is reported here:[DF] Building model...
[DF] Model built!
Model: "SimpleCNN"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 Input Layer (InputLayer)    [(None, 28, 28, 1)]       0         
                                                                 
 conv2d_6 (Conv2D)           (None, 26, 26, 32)        320       
                                                                 
 max_pooling2d_4 (MaxPooling  (None, 13, 13, 32)       0         
 2D)                                                             
                                                                 
 conv2d_7 (Conv2D)           (None, 11, 11, 64)        18496     
                                                                 
 max_pooling2d_5 (MaxPooling  (None, 5, 5, 64)         0         
 2D)                                                             
                                                                 
 conv2d_8 (Conv2D)           (None, 3, 3, 64)          36928     
                                                                 
 flatten_2 (Flatten)         (None, 576)               0         
                                                                 
 dense_4 (Dense)             (None, 64)                36928     
                                                                 
 dense_5 (Dense)             (None, 10)                650       
                                                                 
=================================================================
Total params: 93,322
Trainable params: 93,322
Non-trainable params: 0
_________________________________________________________________
Epoch 1/5
938/938 [==============================] - 6s 5ms/step - loss: 0.1780 - accuracy: 0.9445
Epoch 2/5
938/938 [==============================] - 6s 6ms/step - loss: 0.0501 - accuracy: 0.9843
Epoch 3/5
938/938 [==============================] - 5s 6ms/step - loss: 0.0365 - accuracy: 0.9886
Epoch 4/5
938/938 [==============================] - 6s 6ms/step - loss: 0.0273 - accuracy: 0.9916
Epoch 5/5
938/938 [==============================] - 6s 6ms/step - loss: 0.0226 - accuracy: 0.9930
313/313 [==============================] - 1s 4ms/step - loss: 0.0281 - accuracy: 0.9911
Test accuracy: 0.991100013256073
[DF] Saving model...
[DF] Model saved!Recurrent Neural Network (RNN)The generic scheme of a Recurrent Neural Network is reported here:Here's a simple example of how to deepforge a simple Recurrent Neural Network via theRNNclass:# Simple Recurrent Neural Network example# Make an instance of a RNNrnn=df.RNN(name="Simple RNN",inputN=1)# Set inputs, inner layers and out layersrnn.setInputs([{'shape':(1,2),'name':'Input layer'}])rnn.setRecurrentLayers([[{'units':500}]])rnn.setOutLayers([{'units':1,'activation':'linear'}])# Configure the modelrnn.setModelConfiguration(optimizer='adam',loss='categorical_crossentropy',metrics=['accuracy'])# Build the model and print the summaryrnn.build()rnn.summary()################################################################################################################################################# Another RNN example with 3 stacked LSTM layersrnn2=df.RNN(name="Stacked RNN")# Set inputs, inner layers and out layersrnn2.setInputs([{'shape':(6,2),'name':'Input layer'}])rnn2.setRecurrentLayers([[{'units':500,'return_sequences':True},{'units':500,'return_sequences':True},{'units':500}]])rnn2.setOutLayers([{'units':1,'activation':'linear'}])# Configure the modelrnn2.setModelConfiguration(optimizer='adam',loss='categorical_crossentropy',metrics=['accuracy'])# Build the model and print the summaryrnn2.build()rnn2.summary()The output for the previous code is:[DF] Building model...
[DF] Model built!
Model: "SimpleRNN"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 Input layer (InputLayer)    [(None, 1, 2)]            0         
                                                                 
 cu_dnnlstm_4 (CuDNNLSTM)    (None, 500)               1008000   
                                                                 
 dense_2 (Dense)             (None, 1)                 501       
                                                                 
=================================================================
Total params: 1,008,501
Trainable params: 1,008,501
Non-trainable params: 0
_________________________________________________________________
[DF] Building model...
[DF] Model built!


Model: "StackedRNN"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 Input layer (InputLayer)    [(None, 6, 2)]            0         
                                                                 
 cu_dnnlstm_5 (CuDNNLSTM)    (None, 6, 500)            1008000   
                                                                 
 cu_dnnlstm_6 (CuDNNLSTM)    (None, 6, 500)            2004000   
                                                                 
 cu_dnnlstm_7 (CuDNNLSTM)    (None, 500)               2004000   
                                                                 
 dense_3 (Dense)             (None, 1)                 501       
                                                                 
=================================================================
Total params: 5,016,501
Trainable params: 5,016,501
Non-trainable params: 0
_________________________________________________________________Convolutional Recurrent Neural Network (CRNN)The generic scheme of a Convolutional Recurrent Neural Network is reported here:Here's a simple example of how to deepforge a simple Convolutional Recurrent Neural Network via theCRNNclass:# Convolutional Recurrent Neural Network example# Make an instance of a CRNNcrnn=df.CRNN(name="Simple CRNN",inputN=1)# Set inputs, inner layers and out layerscrnn.setInputs([{'shape':(3,3,1),'name':'Input Layer'}])crnn.setConvLayers([[{'filters':32,'kernel_size':(2,2),'activation':'relu'},{'filters':64,'kernel_size':(2,2),'activation':'relu','padding':'same'}]])crnn.setPoolLayers([[{'pool_size':(2,2)},{'pool_size':(2,2)}]])crnn.setRecurrentLayers([{'units':500}])crnn.setOutLayers([{'units':1,'activation':'linear'}])# Configure the modelcrnn.setModelConfiguration(optimizer='adam',loss='categorical_crossentropy',metrics=['accuracy'])# Build the model and print the summarycrnn.build()crnn.summary()# Save the modelcrnn.save('CRNN')The output of the previous code snippet is reported here:[DF] Building model...
[DF] Model built!
Model: "SimpleCRNN"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 Input Layer (InputLayer)    [(None, 3, 3, 1)]         0         
                                                                 
 conv2d_2 (Conv2D)           (None, 2, 2, 32)          160       
                                                                 
 max_pooling2d_1 (MaxPooling  (None, 1, 1, 32)         0         
 2D)                                                             
                                                                 
 conv2d_3 (Conv2D)           (None, 1, 1, 64)          8256      
                                                                 
 flatten_1 (Flatten)         (None, 64)                0         
                                                                 
 reshape_1 (Reshape)         (None, 64, 1)             0         
                                                                 
 cu_dnnlstm_1 (CuDNNLSTM)    (None, 500)               1006000   
                                                                 
 dense_1 (Dense)             (None, 1)                 501       
                                                                 
=================================================================
Total params: 1,014,917
Trainable params: 1,014,917
Non-trainable params: 0
_________________________________________________________________
[DF] Saving model...
[DF] Model saved!Denoising AutoEncoder (DAE)The generic scheme of a Denoising AutoEncoder is reported here:Here's a simple example of how to deepforge a simple Denoising AutoEncoder via theDAEclass:# Denoising AutoEncoder exampleimportnumpyasnpimportmatplotlib.pyplotasplt# Make an instance of a DAEdae=df.DAE(name="Simple DAE")# Set the number of inputs/outputsinOutNum=100# Set inputs, inner layers and out layersdae.setEncoderLayer({'shape':(inOutNum,)})dae.setHiddenLayers([{'units':round(inOutNum/2),'activation':'relu'}])dae.setDecoderLayer({'units':inOutNum,'activation':'linear'})# # Configure the modeldae.setModelConfiguration(optimizer='adam',loss='mse')# Build the model and print the summarydae.build()dae.summary()# Generate some noisy datadefgenerate_noisy_data():y=np.linspace(0,1,inOutNum)x=y+np.random.normal(0,0.1,inOutNum)returnx,yx,y=generate_noisy_data()x=x.reshape(1,inOutNum)y=y.reshape(1,inOutNum)# Train the modeldae.fit(x=[x],y=y,epochs=100,verbose=0)# Test the denoising autoencoderyHat=dae.predict(x)# Visualize the resultsplt.figure(figsize=(10,4))plt.plot(y.reshape(inOutNum,),label='Clean Data')plt.plot(x.reshape(inOutNum,),label='Noisy Data')plt.plot(yHat.reshape(inOutNum,),label='Denoised Data')plt.legend()plt.show()# Save the modeldae.save('DAE')The output of the previous code snippet is reported here:[DF] Building model...
[DF] Model built!
Model: "SimpleDAE"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_3 (InputLayer)        [(None, 100)]             0         
                                                                 
 dense_4 (Dense)             (None, 50)                5050      
                                                                 
 dense_5 (Dense)             (None, 100)               5100      
                                                                 
=================================================================
Total params: 10,150
Trainable params: 10,150
Non-trainable params: 0
_________________________________________________________________[DF] Saving model...
[DF] Model saved!DocumentationCheck out the full documentation forKerasandTensorflowfor in-depth information on how to use the library.LicenseThis project is licensed under the GNU GPL License - see theLICENSEfile for details.ContactIf you have any questions or feedback, feel free to reach out tome.