Package: zenkai
ZenkaiZenkai is a framework built on Pytorch for researchers to more easily explore a wider variety of machine architectures for deep learning (or just learning with hidden layers). It is fundamentally based on the concepts of target propagation, where a target is propagated backward. As backpropagation with gradient descent can be viewed as a form of target propagation, it extends what one can do with Pytorch to a much larger class of machines. It aims to allow for much more freedom and control over the learning process while minimizing the added complexity.InstallationpipinstallzenkaiBrief OverviewZenkai consists of several packages to more flexibly define and train deep learning machines beyond what is easy to do with Pytorch.zenkai: The core package. It contains all modules necessary for defining a learning machine.zenkai.utils: Utils contains a variety of utility functions that are . For example, utilities for ensemble learning and getting retrieving model parameters.zenkai.kikai: Kikai contains different types of learning machines : Hill Climbing, Scikit-learn wrappers, Gradient based machines, etc.zenkai.tansaku: Package for adding more exploration to learning. Contains framework for defining and creating population-based optimizers.Further documentation is available athttps://zenkai.readthedocs.ioUsageZenkai's primary feature is the "LearningMachine" which aims to make defining learning machines flexible. The design is similar to Torch, in that there is a forward method, a parameter update method similar to accGradParameters(), and a backpropagation method similar to updateGradInputs(). So the primary usage will be to implement them.Here is a (non-working) exampleclassMyLearner(zenkai.LearningMachine):"""A LearningMachine couples the learning mechanics for the machine with its internal mechanics."""def__init__(self,module:nn.Module,step_theta:zenkai.StepTheta,step_x:StepX,loss:zenkai.Loss):super().__init__()self.module=module# step_theta is used to update the parameters of the# moduleself._step_theta=step_theta# step_x is used to update the inputs to the moduleself._step_x=step_xself.loss=lossdefassess_y(self,y:IO,t:IO,reduction_override:str=None)->zenkai.AssessmentDict:# assess_y evaluates the output of the learning machinereturnself.loss.assess_dict(x,t,reduction_override)defstep(self,x:IO,t:IO,state:State):# use to update the parameters of the machine# x (IO): The input to update with# t (IO): the target to update# outputs for a connection of two machinesreturnself._step_theta(x,t,state)defstep_x(self,x:IO,t:IO,state:State)->IO:# use to update the target for the machine# step_x is analogous to updateGradInputs in Torch except# it calculates "new targets" for the incoming layerreturnself._step_x(x,t,state)defforward(self,x:zenkai.IO,state:State,release:bool=False)->zenkai.IO:y=self.module(x.f)returny.out(release=release)my_learner=MyLearner(...)forx,tindataloader:state=State()assessment=my_learner.learn(x,t,state=state)# outputs the logs stored by the learnerprint(state.logs)Learning machines can be stacked by making use of step_x in the training process.classMyMultilayerLearner(LearningMachine):"""A LearningMachine couples the learning mechanics for the machine with its internal mechanics."""def__init__(self,layer1:LearningMachine,layer2:LearningMachine):super().__init__()self.layer1=layer1self.layer2=layer2# use these hooks to indicate a dependency on another methodself.add_step(StepXDep(self,'t1',use_x=True))self.add_step_x(ForwardDep(self,'y1',use_x=True))defassess_y(self,y:IO,t:IO,reduction_override:str=None)->zenkai.AssessmentDict:# assess_y evaluates the output of the learning machinereturnself.layer2.assess_y(y,t)defstep(self,x:IO,t:IO,state:State):# use to update the parameters of the machine# x (IO): The input to update with# t (IO): the target to update# outputs for a connection of two machinesmy_state=state.mine((self,x))self.layer2.step(my_state['y2'],my_state['t1'])self.layer1.step(my_state['y1'],t1)defstep_x(self,x:IO,t:IO,state:State)->IO:# use to update the target for the machine# it calculates "new targets" for the incoming layermy_state=state.mine((self,x))t1=my_state['t1']=self.layer2.step_x(my_state['y2'],t)returnself.layer1.step_x(my_state['y1'],t1)defforward(self,x:zenkai.IO,state:State,release:bool=True)->zenkai.IO:# define the state to be for the self, input pairmy_state=state.mine((self,x))x=my_state['y1']=self.layer1(x,state)x=my_state['y2']=self.layer2(x,state,release=release)returnxmy_learner=MyLearner(...)forx,tindataloader:state=State()assessment=my_learner.learn(x,t,state=state)# outputs the logs stored by the learnerprint(state.logs)ContributingTo contribute to the projectFork the projectCreate your feature branchCommit your changesPush to the branchOpen a pull requestLicenseThis project is licensed under the MIT License - see the LICENSE.md file for details.Citing this SoftwareIf you use this software in your research, we request you cite it. We have provided aCITATION.cfffile in the root of the repository. Here is an example of how you might use it in BibTeX: