Package: notifyker
NotifyKerNotifier and manager bot for ML libraries: Keras and ChainerFeatures:Get status of your training: epoch, metricsTemporarily suspend training and then continue (for example, to free up the C(G)PU for another task (not the memory))Interrupt training immediatelyGet updates per epoch/batch with metricsSet verbose level:0 - only completion message;1 - execution status per epoch2 - execution status per epoch with on-line metrics (like Keras default output)Easy to build your own notifier and callbackHow to useSet TOKEN and PROXY while creating the instance of NotifierTelegramMenu or NotifierTelegramChainerfromnotifykerimportNotifierTelegramMenufromnotifyker.chainerimportExtensionNotifierReportTOKEN='xxxx:yyy'PROXY={'proxy_url':'socks5h://ip:port','urllib3_proxy_kwargs':{'username':'username','password':'password'}}# use NotifierTelegram in order to work without keyboard menunfk=NotifierTelegramMenu(TOKEN=TOKEN,PROXY=PROXY)chainer_not=ExtensionNotifierReport(notifier=nfk)...trainer.extend(chainer_not)trainer.run()KerasfromnotifykerimportNotifierTelegramMenufromnotifyker.kerasimportCallbackSimpleTOKEN='xxxx:yyy'PROXY={'proxy_url':'socks5h://ip:port','urllib3_proxy_kwargs':{'username':'username','password':'password'}}# use NotifierTelegram in order to work without keyboard menunfk=NotifierTelegramMenu(TOKEN=TOKEN,PROXY=PROXY)callback=CallbackSimple(notifier=nfk)model.fit(...callbacks=[callback])Set PROXY = None if not requiredStartEnter/startcommand to your telegram bot. Now it started and you receive update messages.MenuEnter/menuto open keyboard menu with following options:Status - get the status of the last completed epochVerbose - change verbose level or leave unchangedLevel 0:Level 2:Pause - suspend training process (with option block on_batch_end callback method)Continue - continue suspended training processInterrupt - stop training and stop bot updatesCommandsAlso, you can use these commands directly by typing:/status/verbose - only get the current level of verbose (TODO: opportunity to change)/pause/continue/interrupt/help - get the description of commands|optionsYour own notifierimport base class (NotifierBase or NotifierTelegram)fromnotifykerimportNotifierTelegramThen create your own class with inheritance and implement/reimplement methods. Note: if you add some new command - add it to handlerfromtelegram.extimportCommandHandler# if requiredclassMyOwnNotifyker(NotifierTelegram):defmy_own_hello(self,bot,update):"""Send hello in reply"""self.updater.reply_text('Hello!')defhandlers(self):super().handlers()self.updater.dispatcher.add_handler(CommandHandler('hello',self.my_own_hello))That's it!Your own callbackAlso, you can use your own callback in the same way. Let's say you want only to know when the training ends.importtimefromnotifykerimportCallbackBaseclassMyOwnCallback(CallbackBase):defon_training_end(self):end_time=time.ctime(int(time.time()))self.notifier.message('Training completed{}in{}'.format(tag,end_time))Register your own telegram botIn order to create your own telegram bot - talk to t.me/botfather - this is official bot to... create bots.Also, you'll get a TOKEN of your bot, that will be used in the Notifier latter. Click on the name of your bot and the chat will open with your bot.Also, BotFather allows to manage bot profile image, description, privacy options, etc.Official introduction is here:https://core.telegram.org/botsNotifier for TFIt is possible to use this for tensorflow, but you should add necessary functions in your tf code. Just create:nfk=NotifierTelegramMenu(TOKEN=TOKEN,PROXY=PROXY)For example, in order to implement pause, after your epoch add:if'p'innfk.flags_batch:self.notifier.flags_batch.remove('p')while'c'notinnfk.flags_batchand's'notinnfk.flags_batch:time.sleep(10)You can look at callbackNK/callback_simple.py to find more information about implementation.