Package: neuralogic
Documentation|Examples|PapersPyNeuraLogic lets you use Python to writeDifferentiable Logic ProgramsAboutLogic programming is a declarative coding paradigm in which you declare your logicalvariablesandrelationsbetween them. These can be further composed into so-calledrulesthat drive the computation. Such a rule set then forms alogic program, and its execution is equivalent to performing logic inference with the rules.PyNeuralogic, through itsNeuraLogicbackend, then makes this inference processdifferentiablewhich, in turn, makes it equivalent to forward propagation in deep learning. This lets you learn numeric parameters that can be associated with the rules, just like you learn weights in neural networks.What is this good for?Many things! For instance - ever heard ofGraph Neural Networks(GNNs)? Well, agraphhappens to be a special case of a logical relation - a binary one to be more exact. Now, at the heart of any GNN model there is a so-calledpropagation rulefor passing 'messages' between the neighboring nodes. Particularly, the representation ('message') of a nodeXis calculated by aggregating the previous representations of adjacent nodesY, i.e. those with anedgebetweenXandY.Or, a bit more 'formally':Relation.message2(Var.X)<=(Relation.message1(Var.Y),Relation.edge(Var.Y,Var.X))...and that's the actualcode! Now for a classic learnable GNN layer, you'll want to add some weights, such asRelation.message2(Var.X)[5,10]<=(Relation.message1(Var.Y)[10,20],Relation.edge(Var.Y,Var.X))to project your[20,1]input node embeddings ('message1') through a learnable[10,20]layer before the aggregation, and subsequently a[5,10]layer after the aggregation.If you don't like the default settings, you can of coursespecifyvarious additional details, such as the particular aggregation and activation functions(R.message2(V.X)[5,10]<=(R.message1(V.Y)[10,20],R.edge(V.Y,V.X))) | [Transformation.RELU,Aggregation.AVG]to instantiate the classic GCN layer specification, which you can directly train now!How is it different from other GNN frameworks?Naturally, PyNeuralogic is by no means limited to GNN models, as the expressiveness ofrelationallogic goes much further beyond graphs. Hence, nothing stops you from playing directly with:multiple relations and object typeshypergraphs, nested graphs, relational databasesrelational pattern matching, various subgraph GNNsalternative propagation schemesinclusion of logical background knowledgeand more...InPyNeuraLogic, all these ideas take the same form of simple small logic programs. These are commonly highly transparent and easy to understand, thanks to their declarative nature. Consequently, there is no need to design a zoo of blackbox class names for each small modification of the GNN rule - you code directly at the level of the logical principles here!Thebackend enginethen creates the underlying differentiable computation (inference) graphs in a fully automated and dynamic fashion, hence you don't have to care about aligning everything into some static (tensor) operations.How does it perform?While PyNeuraLogic allows you to easily declare highly expressive models with capabilities farbeyond the common GNNs, it does not come at the cost of performance for the basic GNNs either. On the contrary, for a range of common GNN models and applications, such as learning with molecules, PyNeuraLogic is actuallyconsiderablyfaster than the popular GNN frameworks, as demonstrated in ourbenchmarks.We hope you'll find the framework useful in designingyour owndeeprelationallearning ideas beyond the GNNs!
Please let us know if you need some guidance or would like to cooperate!💡 Getting startedInstallationTo install PyNeuraLogic's latest release from the PyPI repository, use the following command:$ pip install neuralogicPrerequisitesTo use PyNeuraLogic, you need to install the following prerequisites:Python >= 3.8
Java >= 1.8In case you want to use visualization provided in the library, it is required to haveGraphvizinstalled.🔬 ExamplesSimple XOR exampleMolecular GNNsRecursive XOR generalizationVisualizationSubgraph PatternsDistinguishing k-regular graphsDistinguishing non-regular graphs📦 Predefined ModulesPyNeuraLogic has a set of predefined modules to get you quickly started with your experimenting!
It contains, for example, predefined modules for:Graph Neural Networks (GNNConv, SAGEConv, GINConv, RGCNConv, ...)Meta graphs and meta paths (MetaConv, MAGNN, ...)Transformer, LSTM, GRU, RNN,...and more!📝 PapersBeyond Graph Neural Networks with Lifted Relational Neural NetworksMachine Learning Journal, 2021Lifted Relational Neural NetworksJournal of Artificial Intelligence Research, 2018Lossless compression of structured convolutional models via liftingICLR, 2021📘 ArticlesWhat is Relational Machine Learning?What is Neural-Symbolic Integration?From Graph ML to Deep Relational LearningBeyond Graph Neural Networks with PyNeuraLogicTowards Deep Learning for Relational DatabasesBeyond Transformers with PyNeuraLogic🎥 VideosBeyond Graph Neural Networks with Lifted Relational Neural Networks