Package: simplified-keras
simplified kerasCommon used actions in kerasTable of contentsGeneral infoLibrariesSetupDocumentationPyPiTODODevelopmentStatusContactGeneral infoThis package is a set of common used actions in keras. At this moment includes (may be outdated):Main packageDefault callbacksRestore callbacksGeneratorsPlotsAccuracy and Loss plotPredictions with image plotHistogram with CDF and image plotConfusion matrixMetricsConfusion matrixModel StatisticsFolder StatisticsModel memory usageTransformationsConvert predictions to classes arrayConvert one hot encoding to sparseUnfreeze modelStretch histogramNormalize histogram claheReplace activationsLayersLibrariesMatplotlib - version 3.3.3NumPy - version 1.19.4Tensorflow - version 2.4.1Pandas - version 1.1.5Seaborn - version 0.11.1SetupInstall from PyPi:pip install simplified-kerasDocumentationMain packageGeneratorsdefault generatorsfromkeras.preprocessing.imageimportImageDataGeneratorfromsimplified_keras.generatorsimportget_train_val_generators,get_val_test_generatorsimg_size=(48,48)img_datagen=ImageDataGenerator(rescale=1/255)# same for get_val_test_generators# default: data_dir='../data', color_mode='rgb', batch_size=128, class_mode='categorical'train_generator,validation_generator=get_train_val_generators(img_datagen,data_dir='../data/normal',color_mode='grayscale',target_size=img_size)val_generator,test_generator=get_val_test_generators(img_datagen,batch_size=32)numpy_memmap_generatorfromsimplified_keras.generatorsimportnumpy_memmap_generator# default batch_size=128, shuffle_array=Truetrain_gen=numpy_memmap_generator('imgs.npy','labels.npy',batch_size=64,shuffle_array=False)Default callbacksfromsimplified_keras.callbacksimportget_default_callbackscallbacks=get_default_callbacks('models/vgg16_calssifier.h5',monitor='val_loss',verbose=0)hist=model.fit(train_generator,steps_per_epoch=train_steps,validation_data=validation_generator,validation_steps=valid_steps,epochs=100,callbacks=callbacks,verbose=2)Signature:defget_default_callbacks(model_path,monitor='val_acc',base_patience=3,lr_reduce_factor=0.5,min_lr=1e-7,verbose=1):return[ReduceLROnPlateau(monitor=monitor,factor=lr_reduce_factor,min_lr=min_lr,patience=base_patience,verbose=verbose),EarlyStopping(monitor=monitor,patience=(2*base_patience+1),verbose=verbose),ModelCheckpoint(monitor=monitor,filepath=model_path,save_best_only=True,verbose=verbose)]Restore callbacksUsed to restore callback after paused learning. Model should come from last checkpoint.fromsimplified_keras.callbacksimportrestore_callbacks,get_default_callbackscallbacks=get_default_callbacks('models/vgg16_calssifier.h5',monitor='val_loss',verbose=0)acc,loss=model.evaluate(val_ds)# acc or loss depending on the compiled model metricsrestore_callbacks(callbacks,acc)PlotsAccuracy and Loss plotfromsimplified_keras.plots.history_plotsimportplot_acc_and_losshistory=model.fit(train_gen,teps_per_epoch=train_steps,epochs=5,validation_data=val_gen,validation_steps=val_steps,callbacks=callbacks)fig=plot_acc_and_loss(history)Result:Predictions with image plotfromkeras.modelsimportload_modelfromkeras.preprocessing.imageimportImageDataGeneratorfromsimplified_keras.generatorsimportget_train_val_generatorsfromsimplified_keras.plotsimportplot_predictions_with_imgimg_size=(48,48)img_datagen=ImageDataGenerator(rescale=1/255)_,validation_generator=get_train_val_generators(img_datagen,data_dir='../data/normal',color_mode='grayscale',target_size=img_size)model=load_model('../models/standard_model.h5')batch,labels=validation_generator.next()preds=model.predict(batch)named_labels=['a','b','c','d','e','f','g']fig=plot_predictions_with_img(1,preds,labels,batch,named_labels,grayscale=True)Result:Histogram with CDF and image plotimportcv2fromsimplified_keras.plotsimportplot_gray_img_with_histogramimg=cv2.imread(f'{src_train_path}/0/241.png',0)fig1=plot_gray_img_with_histogram(img)img2=stretch_histogram(img)fig2=plot_gray_img_with_histogram(img2)Result:Confusion matrix plotfromsimplified_keras.transformationsimportpredictions_to_classes,one_hot_to_sparsefromsimplified_keras.metricsimportget_confusion_matrixesfromsimplified_keras.plotsimportplot_confusion_matrixpredictions=model.predict(validation_images)predicted_classes=predictions_to_classes(predictions)sparse_labels=one_hot_to_sparse(validation_labels)cm,cm_normalized=get_confusion_matrixes(predicted_classes,sparse_labels)classes=['a','b','c','d','e','f','g']f1=plot_confusion_matrix(cm,classes)f2=plot_confusion_matrix(cm_normalized,classes,figsize=(10,8))Results:MetricsConfusion matrixfromsimplified_keras.transformationsimportpredictions_to_classes,one_hot_to_sparsefromsimplified_keras.metricsimportget_confusion_matrixespredictions=model.predict(validation_images)predicted_classes=predictions_to_classes(predictions)sparse_labels=one_hot_to_sparse(validation_labels)# Returns two numpy arrays: standard and normalizedcm,cm_normalized=get_confusion_matrixes(predicted_classes,sparse_labels)Model StatisticsCalculates:FPFNTPTNTPR # Sensitivity/true positive rateTNR # Specificity/true negative ratePPV # Precision/positive predictive valueNPV # Negative predictive valueFPR # Fall out or false positive rateFNR # False negative rateFDR # False discovery rateACC # Overall accuracy for each classMuch more and still increasingfromsimplified_keras.transformationsimportpredictions_to_classes,one_hot_to_sparsefromsimplified_keras.metricsimportget_confusion_matrixesfromsimplified_keras.metricsimportget_model_statisticspredictions=model.predict(validation_images)predicted_classes=predictions_to_classes(predictions)sparse_labels=one_hot_to_sparse(validation_labels)cm,cm_normalized=get_confusion_matrixes(predicted_classes,sparse_labels)stats=get_model_statistics(cm)classes=['a','b','c','d','e','f','g']fig=stats.visualize(classes)print(stats.TN)#[2890 3530 2874 2361 2591 3000 2661]Visualization:Folder statisticsfromsimplified_keras.metricsimportget_folders_statisticsstat=get_folders_statistics('../data/normal/train')print(stat.nr_of_elements,stat.info)# 28709 {'0': 3995, '1': 436, '2': 4097, '3': 7215, '4': 4830, '5': 3171, '6': 4965}fig=stat.bar_plot()Result:Model memory usagefromsimplified_keras.metricsimportget_model_memory_usagebatch_size=64# outputs usage in GBusage=get_model_memory_usage(batch_size,model)print(usage,'GB')# 8.34 GBTransformationsConvert predictions to classes arrayfromsimplified_keras.transformationsimportpredictions_to_classespredictions=model.predict(validation_images)predicted_classes=predictions_to_classes(predictions)print(pedicted_classes)#[6 3 3 ... 6 2 0]Convert one hot encoding to sparsefromsimplified_keras.transformationsimportone_hot_to_sparsesparse_labels=one_hot_to_sparse(validation_labels)print(sprase_labels)#[6 6 6 ... 6 2 0]Stretch histogramfromsimplified_keras.transformationsimportstretch_histogram# default color_mode='rgb'stretch_histogram(image,color_mode='grayscale')Unfreeze modelfromsimplified_keras.transformationsimportunfreeze_modelfromtensorflow.keras.optimizersimportAdam# default params: optimizer=Adam(learning_rate=1e-5), metrics="acc"unfreeze_model(model,optimizer=Adam(learning_rate=1e-4),metrics="loss")Normalize histogram clahefromsimplified_keras.transformationsimportnormalize_histogram_clahe# default clip_limit=2.0, tile_grid_size=(8, 8), color_mode='rgb'normalize_histogram_clahe(image)Replace activationsReplaces all activation functions in given modelfromsimplified_keras.transformationsimportreplace_activationsfromtensorflow.keras.layersimportLeakyReLUl_relu=LeakyReLU()replace_activations(model,l_relu)LayersAugumentation layers build on tensorflow image operationsfromsimplified_keras.transformationsimportRandomSaturation,RandomHue,RandomBrightnessfromtensorflow.kerasimportSequential# for more informations about parameters see tf.image docsaugument_layers=Sequential([RandomSaturation(0.5,1.5),# must be [0 - 0,5]RandomHue(0.2),RandomBrightness(0.2)])PyPisimplified-kerasTODOAdd tests and a lot of features :)DevelopmentWant to contribute? Great!To fix a bug or enhance an existing module, follow these steps:Fork the repoCreate a new branch (git checkout -b improve-feature)Make the appropriate changes in the filesVerify if they are correctAdd changes to reflect the changes madeCommit changesPush to the branch (git push origin improve-feature)Create a Pull RequestStatusLibrary is:in progressContactalbert.lis.1996@gmail.com- feel free to contact me!