Package: reloading
reloadingA Python utility to reload a loop body from source on each iteration without
losing stateUseful for editing source code during training of deep learning models. This lets
you e.g. add logging, print statistics or save the model without restarting the
training and, therefore, without losing the training progress.Installpip install reloadingUsageTo reload the body of aforloop from source before each iteration, simply
wrap the iterator withreloading, e.g.fromreloadingimportreloadingforiinreloading(range(10)):# this code will be reloaded before each iterationprint(i)To reload a function from source before each execution, decorate the function
definition with@reloading, e.g.fromreloadingimportreloading@reloadingdefsome_function():# this code will be reloaded before each invocationpassAdditional OptionsPass the keyword argumenteveryto reload only on every n-th invocation or iteration. E.g.foriinreloading(range(1000),every=10):# this code will only be reloaded before every 10th iteration# this can help to speed-up tight loopspass@reloading(every=10)defsome_function():# this code with only be reloaded before every 10th invocationpassPassforever=Trueinstead of an iterable to create an endless reloading loop, e.g.foriinreloading(forever=True):# this code will loop forever and reload from source before each iterationpassExamplesHere are the short snippets of how to use reloading with your favourite library.
For complete examples, check out theexamples folder.PyTorchforepochinreloading(range(NB_EPOCHS)):# the code inside this outer loop will be reloaded before each epochforimages,targetsindataloader:optimiser.zero_grad()predictions=model(images)loss=F.cross_entropy(predictions,targets)loss.backward()optimiser.step()Hereis a full PyTorch example.fastai@reloadingdefupdate_learner(learner):# this function will be reloaded from source before each epoch so that you# can make changes to the learner while the training is runningpassclassLearnerUpdater(LearnerCallback):defon_epoch_begin(self,**kwargs):update_learner(self.learn)path=untar_data(URLs.MNIST_SAMPLE)data=ImageDataBunch.from_folder(path)learn=cnn_learner(data,models.resnet18,metrics=accuracy,callback_fns=[LearnerUpdater])learn.fit(10)Hereis a full fastai example.Keras@reloadingdefupdate_model(model):# this function will be reloaded from source before each epoch so that you# can make changes to the model while the training is running using# K.set_value()passclassModelUpdater(Callback):defon_epoch_begin(self,epoch,logs=None):update_model(self.model)model=Sequential()model.add(Dense(64,activation='relu',input_dim=20))model.add(Dense(10,activation='softmax'))sgd=SGD(lr=0.01,decay=1e-6,momentum=0.9,nesterov=True)model.compile(loss='categorical_crossentropy',optimizer=sgd,metrics=['accuracy'])model.fit(x_train,y_train,epochs=200,batch_size=128,callbacks=[ModelUpdater()])Hereis a full Keras example.TensorFlowforepochinreloading(range(NB_EPOCHS)):# the code inside this outer loop will be reloaded from source# before each epoch so that you can change it during trainingtrain_loss.reset_states()train_accuracy.reset_states()test_loss.reset_states()test_accuracy.reset_states()forimages,labelsintqdm(train_ds):train_step(images,labels)fortest_images,test_labelsintqdm(test_ds):test_step(test_images,test_labels)Hereis a full TensorFlow example.TestingMake sure you havepythonandpython3available in your path, then run:$ python3 reloading/test_reloading.py