Package: pytorch-common
pytorch-commonAPypi modulewith pytorch common tools like:Build releaseStep 1: Increase version into next files:pytorch_common/__init__.py
pyproject.tomlStep 2: Build release.$poetrybuild✔Buildingpytorch-common(0.2.3)-Buildingsdist-Builtpytorch-common-0.2.3.tar.gz-Buildingwheel-Builtpytorch_common-0.2.3-py3-none-any.whlStep 3: Publish release to PyPI repository.$poetrypublish✔Publishingpytorch-common(0.2.3)toPyPI-Uploadingpytorch-common-0.2.3.tar.gz100%-Uploadingpytorch_common-0.2.3-py3-none-any.whl100%FeaturesCallbacks(keras style)Validation: Model validation.ReduceLROnPlateau:Reduce learning rate when a metric has stopped improving.Models often benefit from reducing the learning rate by a factor
of 2-10 once learning stagnates. This scheduler reads a metrics
quantity and if no improvement is seen for a 'patience' number
of epochs, the learning rate is reduced.EarlyStop:Stop training when model has stopped improving a specified metric.SaveBestModel:Save model weights to file while model validation metric improve.Logger:Logs context properties.In general is used to log performance metrics every n epochs.MetricsPlotter:Plot evaluation metrics.This graph is updated every n epochs during training process.Allow save each plot into a file.CallbackandOutputCallback:Base classes.CallbackManager:Simplify callbacks support to fit custom models.StratifiedKFoldCV:Support parallel fold processing on CPU.MixinsFiMixinfit(data_loader, loss_fn, epochs, optimizer, callbacks, verbose, extra_ctx, train_fn)CommonMixinparams(): Get model params.Get associateddevice.PredictMixinevaluate(data_loader)evaluate_score(data_loader, score_fn)predict(features)PersistentMixinsave(path)load(path)Utilsdevice managementStopwatchdata splitosmodelLoggerBuilderDict UtilsWeightsFileResolver: Resolver best model weights file path using a given metric likemineva_loss,maxeval_acc, etc...PlotPlot primitives likeplot_loss.ExamplesDevice managementimportpytorch_common.utilaspu# Setup prefered device.pu.set_device_name('gpu')# / 'cpu'# Setup GPU memory fraction for a process (%).pu.set_device_memory('gpu'# / 'cpu',process_memory_fraction=0.5)# Get prefered device.# Note: In case the preferred device is not found, it returns CPU as fallback.device=pu.get_device()Loggingimportloggingimportpytorch_common.utilaspu## Default loggin in console...pu.LoggerBuilder()\.on_console()\.build()## Setup format and level...pu.LoggerBuilder()\.level(logging.ERROR)\.on_console('%(asctime)s-%(levelname)s-%(message)s')\.build()Stopwatchimportloggingimportpytorch_common.utilaspusw=pu.Stopwatch()# Call any demanding process...# Get resposne time.resposne_time=sw.elapsed_time()# Log resposne time.logging.info(sw.to_str())Dataset splitimportpytorch_common.utilaspudataset=...# <-- Torch.utils.data.Datasettrain_subset,test_subset=pu.train_val_split(dataset,train_percent=.7)train_subset,val_subset,test_subset=pu.train_val_test_split(dataset,train_percent=.7,val_percent=.15)Kfoldingimportloggingfrompytorch_common.kfoldcvimportStratifiedKFoldCV,\ParallelKFoldCVStrategy,\NonParallelKFoldCVStrategy# Call your model under this function..deftrain_fold_fn(dataset,train_idx,val_idx,params,fold):pass# Get dataset labelsdefget_y_values_fn(dataset):passcv=StratifiedKFoldCV(train_fold_fn,get_y_values_fn,strategy=NonParallelKFoldCVStrategy()# or ParallelKFoldCVStrategy()k_fold=5)# Model hyperparams...params={'seed':42,'lr':0.01,'epochs':50,'batch_size':4000,...}# Train model...result=cv.train(dataset,params)logging.info('CV results:{}'.format(result))Assertionsfrompytorch_common.errorimportAssertions,Checker# Check functions and construtor params usign assertions..param_value=-1# Raise an exception with 404103 eror code when the condition is not metAssertions.positive_int(404103,param_value,'param name')Assertions.positive_float(404103,param_value,'param name')# Other optionsAssertions.is_class(404205,param_value,'param name',aClass)Assertions.is_tensor(404401,param_value,'param name')Assertions.has_shape(404401,param_value,(3,4),'param name')# Assertions was impelemented using a Checker builder:Checker(error_code,value,name)\.is_not_none()\.is_int()\.is_positive()\.check()# Other checker options..#   .is_not_none()#   .is_int()#   .is_float()#   .is_positive()#   .is_a(aclass)#   .is_tensor()#   .has_shape(shape)Callbacksfrompytorch_common.callbacksimportCallbackManagerfrompytorch_common.modulesimportFitContextFactoryfrompytorch_common.callbacksimportEarlyStop,\ReduceLROnPlateau,\Validationfrompytorch_common.callbacks.outputimportLogger,\MetricsPlotterdeftrain_method(model,epochs,optimizer,loss_fn,callbacks):callback_manager=CallbackManager(ctx=FitContextFactory.create(model,loss_fn,epochs,optimizer),callbacks=callbacks)forepochinrange(epochs):callback_manager.on_epoch_start(epoch)# train model...callback_manager.on_epoch_end(train_loss)ifcallback_manager.break_training():breakreturncallback_manager.ctxmodel=# Create my model...optimizer=# My optimizer...loss_fn=# my lost functioncallbacks=[# Log context variables after each epoch...Logger(['fold','time','epoch','lr','train_loss','val_loss',...]),EarlyStop(metric='val_auc',mode='max',patience=3),ReduceLROnPlateau(metric='val_auc'),Validation(val_set,metrics={'my_metric_name':lambday_pred,y_true:# calculate validation metic,...},each_n_epochs=5),SaveBestModel(metric='val_loss'),MetricsPlotter(metrics=['train_loss','val_loss'])]train_method(model,epochs=100,optimizer,loss_fn,callbacks)UtilsWeightsFileResolver$ls./wegiths2023-08-21_15-17-49--gfm--epoch_2--val_loss_1.877971887588501.pt2023-08-21_15-13-09--gfm--epoch_3--val_loss_1.8183038234710693.pt2023-08-19_20-00-19--gfm--epoch_10--val_loss_0.9969356060028076.pt2023-08-19_19-59-39--gfm--epoch_4--val_loss_1.4990438222885132.ptimportpytorch_common.utilaspuresolver=pu.WeightsFileResolver('./weights')file_path=resolver(experiment='gfm',metric='val_loss',min_value=True)print(file_path)'./weights/2023-08-19_20-00-19--gfm--epoch_10--val_loss_0.9969356060028076.pt'Go to next projects to see funcional code examples:https://github.com/adrianmarino/deep-fmhttps://github.com/adrianmarino/attention