DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning
Authors: Aaditya Naik
Jason Liu
Claire Wang
Amish Sethi
Saikat Dutta
Mayur Naik
Eric Wong
Venue: ICML 2025 poster (2025)
Keywords: neurosymbolic learning
scalability
vectorization
differentiable reasoning
Primary area: deep_learning->everything_else

Abstract: Neurosymbolic learning enables the integration of symbolic reasoning with deep learning but faces significant challenges in scaling to complex symbolic programs, large datasets, or both. We introduce DOLPHIN, a framework that tackles these challenges by supporting neurosymbolic programs in Python, executing complex symbolic reasoning on the CPU while vectorizing probabilistic computations and gradient propagation on the GPU. Across 13 benchmarks spanning tasks over text, image, and video data, with symbolic reasoning features like recursion and blackbox functions, DOLPHIN converges to state-of-the-art accuracies on the more complex benchmarks while existing frameworks such as Scallop, ISED, and IndeCateR+ fail to converge within the time limit. On simpler benchmarks, DOLPHIN matches their performance, while achieving these results 1.71x to 62x faster than the baselines. Overall, DOLPHIN advances the scalability of neurosymbolic frameworks, achieving state-of-the-art efficiency and convergence on difficult benchmarks where existing frameworks struggle. The code is published at https://github.com/Dolphin-NeSy/Dolphin.

DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning
Aaditya Naik Jason Liu
Claire Wang'
Amish Sethi'
Saikat Dutta
Mayur Naik
Eric Wong'
Abstract
Passage
Question
Rich's daughter Christine made dinner for her sister Kim. Beth went to her brother Rich's birthday party. Anne went shopping with sister Kim.
Neurosymbolic learning enables the integration of symbolic reasoning with deep learning but faces significant challenges scaling to com- plex symbolic programs, large datasets, or both. We introduce DOLPHIN, framework that tack- les these challenges by supporting neurosymbolic programs in Python, executing complex symbolic reasoning on the CPU while vectorizing proba- bilistic computations and gradient propagation on the GPU. Across 13 benchmarks spanning tasks over text, image, and video data, with sym- bolic reasoning features like recursion and black- box functions, DOLPHIN converges to state-of- the-art accuracies on the more complex bench- marks while existing frameworks such as Scallop, ISED, and I fail to converge within the time limit. On simpler benchmarks, DOL- PHIN matches their performance, while achieving these results 1.71x to 62x faster than the baselines. Overall, DOLPHIN advances the scalability of neu- rosymbolic frameworks, achieving state-of-the art efficiency and convergence on difficult bench- marks where existing frameworks struggle. The code is published at https: / /github. com/ Do
How are Rich and Anne related?
Rich
Anne
Relationship Extractor
Symbolic Program
Backpropupation
Sevater
beather
X
Brashor
atater
beuther
Par
6.00
(n(Ru, My,8)
(A Xaysy)
XAx
brother
CtB. Aele, Ro.dill
PAx
0:00
Rysher A.
f fwoe
Xxa
brothei
jister
vife
Py Hill Pilkit in
0.03
Pboa Bull by
PlotR Rells Po1,,, R..
Output
faather
Figure I: Illustration of the execution of neurosymbolic program for the kinship reasoning task CLUTRR. While existing neurosymbolic frameworks run the neural models on the GPU, they run the symbolic program entirely on either the CPU (e.g. Scallop) or the GPU (e.g. Logic Tensor Networks), rendering them inefficient in terms of compute and memory, respectively. In DOLPHIN, both neural models and symbolic programs specified as PyTorch modules, but only probabilistic computations (P) are vectorized on GPU whereas symbolic computations (C) execute on CPU.
1. Introduction
hampati et al.). Neurosymbolic programming (Chaudhuri et al., 2021) has emerged as a promising paradigm to in- corporate symbolic reasoning into deep learning models, providing the best of both worlds.
Deep learning has made great strides in tasks such as im- age classification, speech recognition, and natural language processing. With the emergence of foundation models like GPT-4 and SAM, deep learning is increasingly applied to more complex tasks. Despite significant strides, these mod- els remain limited in their ability to reliably perform reason- ing required for tasks involving structure, logic, and plan- ning, where symbolic approaches traditionally excel (Kamb-
Various frameworks have been developed to improve the programmability and accessibility of neurosymbolic applica- tions (Manhaeve et al., 2018; Li et al., 2023; Solko-Breslin et al., 2024). These frameworks support complex symbolic reasoning features like recursion and black-box functions, implement efficient differentiable reasoning algorithms, and provide bindings for deep learning frameworks like PyTorch. However, they incur significant overhead during training.
'Department of Computer and Information Science, Uni- versity of Pennsylvania 'Department of Computer Science, Cornell University. Correspondence to: Aaditya Naik <as- naik @ seas upenn.euu.........
Figure/ shows an example of a kinship reasoning task called CLUTRR whose goal is to infer the relationship be- tween two people based on a passage describing interactions
Proceedings of the 4nnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning
and relationships within a family. A natural neurosymbolic formulation splits this task's computation into a neural com- ponent, which extracts relationships from the input passage, and a symbolic component which infers new relationships to obtain the final answer. The latter involves specifying complex manipulations over symbols (C), c.g., multi-hop kinship reasoning between pairs of family members, and performing probabilistic computations (P) to track the prob- abilities of the symbols derived using L. In general, as the complexity of the symbolic program increases, the number of possible results and their associated weights also grows exponentially, leading to combinatorial explosion in the number of required computations. This issue is exacer- bated by larger datasets usually found in deep learning tasks. Deep learning frameworks typically address this challenge by batching computations across multiple data samples.
putations over these representations, while allowing support for black-box Python functions that simplify the writing of complex symbolic programs. Third, we develop set of vectorized provenance semirings (Green et al., 2007) that are easily pluggable into DOIPHIN and enable to efficiently compute symbolic gradients.
Together, these components enable DOLPHIN to construct computation graph that integrates both neural and prob- abilistic computations (P), ensuring high parallelism and end-to-end differentiability on GPU. At the same time, it runs L over discrete symbols on CPU, allowing flexible manipulation over arbitrary Python objects. This allows DOLPHIN to scale effectively to complex problems such as CLUTRR-N (Table 2) where N denotes the max length of the reasoning chain in the training dataset. In the case of Scallop, the length of the reasoning chain increases, the gap between convergence times dramatically widens. In contrast, for N = 3. DOLPHIN takes around 13 minutes to converge, about 5x faster than Scallop, while for N = 4, DOLPHIN takes around 15 minutes, about 8.5x faster than Scallop. Finally, DOL.PHIN is implemented as a library inte- grated with PyTorch, allowing users to easily incorporate it into their existing deep learning pipelines.
Neurosymbolic frameworks like LYRICS (Marra et al., 2019) and Logic Tensor Networks (LTN) (Badreddine et al., 2022) also batch the computations of both L and P on the GPU. LTN grounds all logits and discrete symbols as ten- sors, and the aforementioned computations (L and P)) are specified in differentiable first-order logic as operations over those tensors. These programs output value quantifying the satisfiability of model outputs with respect to logic con- straints. This approach is highly performant for smaller tasks, such as MNIST SumN, where the goal is to predict the sum of N MNIST images. As we see in Table in 8 4.3, LTN takes around 90 seconds to converge for N = 5. However, when the complexity increases to N = 10, LTN runs out of memory on consumer-grade GPUs (here, with capacity of 11GBs), due to the combinatorial explosion of required symbols (from 10" to 10 and their probabilities that require to be grounded on the GPU.
We evaluate DOLPHIN on a diverse set of neurosymbolic tasks involving text, image, and video, using rich reasoning features like recursion and black-box Python functions. On simpler problems, neurosymbolic programs written using DOLPHIN match the accuracy of state-of-the-ee the-rtt methods, while achieving these results 47x, 62x, 8x, and 1.7x faster than baselines like Scallop, sampling- baseed frameworks like ISED and IndeCateR+, and solely GPU based methods like LTN respectively. We also observe that DOLPHIN efficiently scales to more complex benchmarks and larger datasets, achieving state-of-the--a accuracies. While baselines fail to converge on 5 out of 8 such benchmarks within 10 hours, DOLPHIN requires 5.5 hours in the worst case.
On the other hand, neurosymbolic frameworks like Deep- ProbLog (Manhaeve et al., 2018) and Scallop (Li et al., 2023) run neural models on the GPU but use separate CPU-based backend for executing both L and P. This avoids issues of memory consumption on the GPU, but the lack of batched computations on CPU results in slowdowns as the problem complexity increases. We see this in MNIST SumN (Table 2). Scallop requires around 15 minutes to converge for N = 5, but needs around and 2 hours to converge for N 10 and N 15, respectively.
We make the following contributions in this work:
We propose DOLPHIN, a novel neurosymbolic program- ming framework for end-to-end diffcrentiable symbolic reasoning in a scalable manner ($3).
We develop novel Pythonic abstractions and primitives to enable writing complex symbolic manipulations for neurosymbolic programs ($3.1).
In this paper, we propose DOLPHIN solution for scaling neurosymbolic learning. In DOLPHIN, we build three key components that effectively tackle scalability challenges with existing neurosymbolic frameworks. First, we develop a unified representation that efficiently captures the relation- ships between neural network outputs as PyTorch tensors on GPU and associated discrete symbols as Python objects on CPU. Second, we introduce set of primitives to enable writing symbolic manipulations that can be mapped to com-
We design DOLPHIN to be extendable to new provenances and develop vectorized provenances that can be plugged into DOLPHIN for efficient computation of symbolic gra- dients on parallelizable hardware such as GPU ($3.2).
We evaluate DOLPHIN on a diverse range of 13 challeng- ing neurosymbolic tasks across different domains and show that it effectively scales with increasing problem complexity and dataset size ($4).
2

---

mers express logic without handling underlying complexi- ties. Lines 8 and 10 of Figure 2 show how MNTSTNet'S out- objects. Each DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning
class SumNWet www.ech.nn.Modd..e.
2.
def _intt (self):
super (SumNNot, fell)._nnii...._____ (nir__)))
4
selr.ONN - MNISTNet ()
put can be captured within Distribut Distribution associates a single collection of digits with the corresponding batch of logits produced by MN STNet, along with any gradients and associated metadata.
6 der (selard i
7 4 (ange(10)
D_res Distsibezion (self. www.ee.e...eeee...............ee CNN (imgs (01),
9
for i in range (1, Len (imga)):
D_t nistribbr but fon (se f. (1
The programmer can now express the symbolic program in terms of operations over Distributions. For instance, in line 11, the apply function is used, taking two Distributions as arguments. along with a lambda function that specifies the addition operation. Under the hood, apply combinatorially explores all possible sums of the symbols from D_res and D.i and calculates their associated probabilities. The result of app y a new iii iitribbtion on over the calculated sums, and stored back into D.res. This is repeated iteratively until all the outputs of the CNN are summed appropriately.
11
D_rea - apply (D_res, D_i, Lambda Kx x y)
return get_logits (D_res)
Figure DOLPHIN code for the MNIST SumN task,
D
D;
D
4TP%
DOL provides additional primitives to support more complex symbolic programs. Figure 3b shows the com- putation graph for the PathFinder task (Tay et al., 2021), which involves recursively building paths to identify if two points in maze are connected. The union primitive is used to support the recursive nature of this program. Since Distribut objects associate symbols with the batched logits themselves, probabilistic computations are vectorized and directly operate over PyTorch tensors.
(a) MNIST Sum-N.
TAgeENN
FoothEEN
Dage
DeriPs
apply concat
Dus
Pussi D'
aeply SoCornected
Dya
Pike
D' Puu
Expeiin
Tiur
D DeantBath
This deep integration of DOLPHIN into PyTorch allows pro- grammers to write symbolic programs as symbolic layers that interact with PyTorch neural layers within a neurosym- bolic model. DOLPHIN can thus leverage the hardware acceleration supported by PyTorch. This contrasts with sys- tems like Scallop, where tensors are converted into Scallop- friendly tags transferred to process outside the Python environment with CPU-bound probability computations, re- stricting scalability.
(b) PathFinder.
Figure 3: Computation graphs for two neurosymbolic pro-
grams written using DOLPHIN.
2. Overview
We illustrate DOLPHIN using the MNIST SumN task (De Smet et al., 2024), where the goal is to add N MNIST digit images. The task grows exponentially difficult, with 1 possible states and only 9.V + 1 labels, making super- vision sparse. Figure 2 shows the code for this task using DOLPHIN with PyTorch. The neural module MNISTNet is PyTorch model classifying a batch of images into one of 10 classes representing the digits 0-9, This is done for each of the N batches of images in the tuple imgs. The logits pro- duced by MNISTNet, representing probability distributions over the digits, are then passed as inputs to the symbolic program. Lines 8-11 depict a symbolic program written in Python using DOLPHIN primitives.
3. The DOLPHIN Framework
We based DOLPHIN'S design on four core principles. First is flexible programmability, to allow writing complex sym- bolic manipulations (C) with Python's 's rich and expressive language features. Second, probabilistic computations (P) must allow end-to-end differentiability on the GPU. Third, DOLPHIN must be scalable tasks with large data and problem complexity. Finally, it must be tunable, allowing developers to define and choose provenances, treating them like deep learning hyperparameters.
To support training, the symbolic program must track digit probabilities, compute probability distributions over all pos- sibilities (0 to 9NN, and propagate gradients for backpropa- gation. Batched computations further complicate this, mak- ing native PyTorch implementations cumbersome.
Together, these principles help address the challenges of scaling neurosymbolic frameworks. Flexible programmabil- ity and tunability allow us to write complex neurosymbolic programs, while GPU differentiability and scalability work towards tackling problem and data complexity. We now de- scribe DOLPHIN and show how we realize these principles.
DOLPHIN abstracts symbolic computation, letting program-
3

---

DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning
decoupled, operations over symbols can be run on the CPU, allowing the support for arbitrary Python objects and func- tions in the symbolic program, even while the tag computa- tions are performed on GPU.
Symbol Tag
8 E S. (objects)
t E T (tensors)
Distribution
12
E D S + T
APPLY
D x (S* + 5) +D
Example 3.1. Consider MNIST images I1, Iz for the SumN lask discussed in Section 2. Let SCNN be the neural model which classifies image 1 into one of 10 classes represent- ing digits O to 9. Let fcww(T) = {0.00, 0.90. 0.01} and fcow( [2) = {0.78, 0.09. 0.003. We thus define the following Distributions:
FILTER
D x (S + BB- +D
APPLYIF
D ((5* +5) (S* - B) D
UNION
DXD DD
GETPROBS
D 10+10
Figure 4: Formal definition of DOLPHIN's programming abstractions (top) and primitives (bottom).
D1 = {0- - 0.00, 1 - 0.90, ... - 0.013
2. D2 10 + 0.78, 1 0.09. ...., ,9 0.01}
3.1. The DOLPHIN Syntax
To allow flexible programmability, DOLPHIN provides an interface that developers can use to express symbolic programs in Pythonic manner.
3.1.2. OPERATIONS
DOLPHIN provides five operations to allow the expression of complex neurosymbolic programs in conjunction with user-defined functions, shown in Figure 4.
3.1.1. ABSTRACTIONS
This is the primary operation that can be used to manipulate Distributions. It takes as inputs K > 1 Dis- tributions, along with function f of the same arity. This function defines operations over the symbols of K distri- butions. APPLY then computes the results of f over all possible combinations of arguments sourced from the sym- bols of the Distributions as well as their associated tags, and returns a new Distribution with these results and tags. This occurs in two stages akin to the popular map-reduce pattern. In the map stage, APPLY computes the results of over the symbols of the input Distributions and conjuncts their tags:
APPLY.
DOLPHIN provides three main abstractions for expressing symbolic programs, shown in Figure 4. Symbols S represent symbolic entities relevant to the program. These entities can be any Pythonic object, such as hand-written digits in MNIST-SumN or coordinates of points in PathFinder. Tags T are tensors that represent their likelihoods. Typically, tags for symbols are derived from the outputs of machine learning models, such as the logits produced by the CNN classifier in MNIST-SumN. Finally, Distribution D maps collection of symbols to their corresponding tags.
Distributions serve as the fundamental datatype of DOL- PHIN program and act as its main interface with a PyTorch model. As seen in the following code snippet from Figure 2:
R = { (f(#1, 82, 8t). (f1 & 12 & @ fx)) Di(a:) = f, = 1,....k)
(1)
Here, the tag of each result symbol f(1, 82, ......, 3k) is the conjunction @ of the tags (4,,62..... (a) of the input sym- bols it was derived from. The function f is executed se- quentially on the CPU for each combination of symbols as function f can be any user-defined Python function, includ- ing complex control flows and operations like regex parsing, image processing, or Python's 's eval (). It may also be many-to-one function and the tags shared by a resulting symbol must be aggregated to form the final tags of the output Distribution. We, therefore, shuffle the results from the map stage to compute function M from each symbol to tags from R associated with it:
Decs 1 iistrribu bution on (self f. CNN (imgs [0]), d)
the logits output by the model are directly passed to the Distribution object, effectively acting as an input to the sym- bolic program. These logits form the batched tags within a Distribution object which also maintains the set of cor- responding symbols d. The symbolic manipulations in a DOLPHIN program occur over the discrete symbols, while the probabilistic computations occur over tags stored as Py- Torch tensors. This enables a seamless integration between the PyTorch model and the symbolic program.
This has several advantages. First, it preserves the gradients of the model output throughout the symbolic program, en- abling end-to and-to-end differentiability via PyTorch 's autograd, addressing the second core principle of DOLPHIN. Second, allows DOLPHIN to perform operations over an entire batch of tags as per the principle of scalability, leveraging the vectorized operations provided by Py PoTch. DOLPHIN can thus operate efficiently on specialized hardware like GPUs, allowing the symbolic program to scale effectively. Third, since symbol and tag computations are effectively
M A { (s,t)
(2)
We then proceed to the reduce stage, where we aggregate the tags of each symbol in M using disjunction @ to produce the final Distribution Dres:
Dws u. (
(3)
Since the tags here are Py ToTch tensors representing proba- bilities, the implementations of the conjunction and disjunc- tion operations are dictated by the underlying provenance specified by the program, detailed in Section 3.2.
4

---

DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning
Example 3.2. Continuing from Example 3.1, let function [(r, = x + y be applied to D1 and D2 to produce new Distribution D' = apply(D1, D2, f). D' thus represents a Distribution over the sum of the symbols from D1 and D2:
3.1.3. WRITING COMPLEX DOLPHIN PROGRAMS
Some neurosymbolic tasks require the writing of programs containing complex control flows and recursion (e.g. the PathFinder task. 5 4.1). This can be done in one of two ways. The simplest way is to specify any control flow operations within the user-defined functions supplied to the DOLPIIIN operations. Alternatively, one can specify branches of con- trol flow separately and merge their results via UNION, as shown in Figure (Appendix for more details).
= - 0.00, 0.70, 18 + 0.003
Consider the tag of (1) I can be result of D2 being 0 and D2 being 1, or of D2 being I and D2 being 0:
(1(1)= (Pr (0) 6 D2(1)) @ (Di (1) @ D2(0))
In some cases, even though DOLPHIN limits the effect of combinatorial explosion in terms of efficiency and memory usage, the number of combinations may still be excessive. In such cases, DOLPHIN allows developers to sample sub- sets of symbols from Distribution objects specified within the symbolic program, effectively limiting the number of symbols processed in each operation.
(0.00 & 0.09) @ (0.90 @ 0.78)
This expression's 's value (e.g. 0.70) depends on the prove- nance specified (e.g. DAMP), discussed in Section 3.2.
FILTER. The FILTER operation is used to filter out symbols from Distribution. It takes in single Distribution, along with a user-defined function that returns boolean value. This operation then returns a new Distribution that contains only symbols that satisfy the condition with their tags.
3.2. DOLPHIN Provenances
The DOLPHIN primitives discussed above define how to conjunct or disjunct tags corresponding to the symbol ma- nipulations, e.g. Equations (1) and (3). These tag operations are achieved by using mathematical framework called provenance semirings (Green et al., 2007). Provenance semirings provide generalized algebraic structure to propa- gate probabilities over tagged data.
Example 3.3. Continuing from Example 3.1, assume we want a Distribution over just the even symbols of Dj. We can consider a filtering function fxx = (x mod 2 = 0). The resulting Distribution will have all the numbered symbols completely removed:
D' = fi fi.iterDDi.f) {0 0.00, 2 -.0
Designing and implementing provenances can be challeng- ing since they must be accurate enough to capture the se- mantics of the symbolic program, while at the same time being coarse enough to maintain computational feasibility, Furthermore, the provenances must be differentiable.
APPLYIF. This operation is a conditional version of AP- PLY. It takes in K Distributions and functions faply and feond of the same arity. For each combination of symbols from the K Distributions, APPLYIF computes Suppy and its associated tags only if the condition feond is satisfied over that combination of symbols. The operation then returns a new Distribution with these results and tags.
While neurosymbolic frameworks like Scallop (Li et al., 2023) implement differentiable provenances, they are not designed to leverage hardware accelerations or batched op- timizations due to the CPU-bound nature of their imple- mentations. Frameworks like LTN use t-norms that more amenable to vectorization, but lack support for more complex provenances such as Differentiable Top-k Proofs (DTKP) (Huang et al., 2021). We thus design differentiable, vectorized provenances in DOLPHIN to enable GPU support.
UNION. The UNION operation takes in two Distributions and returns new Distribution containing the union of the input symbols, along with their tags. Any symbols common to both input Distributions have their tags disjuncted.
Example 3.4. Consider Distributions D1 {0
We simplify the definition of provenances as a 5-tuple: (T, 0, 1, @ @). Here, T is the tag space, @ T x T T is the conjunction operator with identity 0, and @ T x T + T is the disjunction operator with identity 1. We then imple- ment two differentiable provenances in DOLPHIN: Differen- tiable Add-Mult Probabilities (DAMP) and Differentiable Top-K Proofs (DTKP). Table summarizes the operations of these provenances. While building the neurosymbolic program, the developer may specify which provenance to use, satisfying the core principle of tunability.
0.01,1 + 0.24] and D2- {0- + 0.63, 4 + 0.37J. The union will be:
union(Ui,D)) 10- 0.64. 1 + 0.24, 4 0.37}
GETPROBS. The GETPROBS operation extracts the proba- bilities from the tags of a Distribution. This is used mainly once the symbolic program has been executed to extract the final probabilities of the symbols in the output Distribution. These probabilities can then be used to compute the loss function for training the neural model.
Differentiable Add-Mult Probabilities. Differentiable
5

---

DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning
Table I: DOLPIINN provenances implemented in Py ToTohh.
Provenance
Domain
0
1
t 4 t' clamp (t + t') 1 top (cat(t, t²)) top (Iminn((aal It/D (4,, ty) Etxt])
8 8 t'
DAMP
[0
0
1
t-t'
DTKP-AM
[0, 1] U (00)-
0,5 000 i
Add-Mult Probabilities (DAMP) a popular technique that uses the probability space as its tag space: T = [0, 1]. Its conjunction operation @ is defined as the product of prob- abilities, clamped at 1, and its disjunction operation @ is defined as the sum of probabilities. The main assumption un- derlying the DAMP operations is that the input Distributions are mutually exclusive and independent. This assumption al- lows DAMP compute probabilities extremely efficiently, as the operations are simple and can be easily vectorized.
in and t', In each case, the top, operation retains only up to k proofs with the highest probabilities.
These definitions thus allow us to take advantage of the benefits of the DTKP provenance while enabling efficient computation on the GPU. To calculate the probability of the entire tag, DTKP-AM adds the probabilities of the in- dividual proofs and clamps it 1. We provide detailed discussion of DTKP-AM in Appendix A.
Differentiable Top-k Proofs. Differentiable Top-k Proofs (DTKP) (Huang et al., 2021) was proposed to overcome the shortcomings of DAMP. This provenance tracks set of up to k proofs for each symbol. Each proof denotes the set of input symbols necessary to derive the output symbol. These proofs are then used compute the probabilities of the output symbols. In Scallop, DTKP tags converted into probabilities via differentiable weighted model counting (WMC). This form of DTKP, which we call DTKP-WMC, is computationally hard and is by nature difficult to vec- torize due to the varying sizes of proof sets and the WMC procedure. We hence design a novel vectorized approxi- mation of DTKP-WMC. we term DTKP-AM (DTKP with Add-Mult), that can be efficiently computed the GPU.
3.3. Building the DOLPHIN Program
The programmer specifies the neurosymbolic task using a Python program P. which integrates neural components with symbolic operations via DOLPHIN'S 's interface. Given dataset D and one or more neural networks MI...., Mr.
DOLPHIN constructs a computation graph where symbolic transformations occur on the CPU, and probabilistic compu- tations, including neural network inference, are efficiently executed on the GPU. All computations leverage distribu- tion objects D., enabling end-to-end differentiability and scalability. Training optimizes the objective function
(0)) min Σ L(P(M)())). (2.0)ED
We first define the structure of tags in DTKP-AM conform to the constraints of PyTorch tensors. Each tag t for a symbol is a 2-dimensional tensor of shape (k,, [T). where is the maximum number of proofs to be retained and is an ordered list of all input symbols (symbols that are present in the input Distributions). Each row & of corresponds to one of the tag's k proofs. Each element tay thus represents the probability of the jth input symbol in the ith proof:
where is the loss function (e.g.. binary cross entropy).
4. Experiments
We evaluate DOLPIIIN on a set of 13 benchmarks of varying complexity and scale across 5 neurosymbolic tasks. Our evaluation addresses the following research questions:
RQ1: Scalability. Can DOLPHIN scale to tasks and datasets beyond the scope of existing SOTA frameworks?
Pj the jth symbol is present the ith proof 0.s otherwise
tgg
RQ2: Accuracy. Do models written in DOLPHINN con- verge to SOTA accuracies in less training time?
where Pj the probability of the jth input symbol. The probability of each proof is then computed by taking the product of the normal:
RQ3: Provenance Comparisons. Which provenances are most effective for each benchmark?
Pr(l.)= II norm((g)) where norm(Ly)
0
tu
4.1. Benchmarks
otherwise
We describe the benchmarks used to evaluate DOLPHIN and give additional information about the experiment setup and DOLPHIN code for each benchmark in Appendix D.
We next define the operations of DTKP-AM in Table 1. The @ operation is defined as the union of two tag tensors t and while @ is defined as the element-wiie wise minimum of the normalized elements of all possible combinations of proofs
MNIST SumN. The MNIST SumN (or briefly, SumN) task from (De Smet al., 2024) takes as inputs N handwritten
6

---

DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning
digits from the MNIST dataset and returns their sum. We consider three versions of this task: SumN-5 (N = 5), SumN-10 (N 10), and 15mN-15 (N- 15)..
Table 2:
Training times (in seconds) for DOLPIIIN and Scallop on all benchmarks. Training times more than 10 hours are highlighted in red. The scaling factor a is the ratio of the total training times of Scallop DOLPHIN.
Hand- Wertd-r Formula (HWF). The HWF task from (Li et al., 2020) takes as input a set of images of handwritten digits and arithmetic operators representing a formula. The task is to evaluate the formula and return the result. We consider three versions based on formula length: HWF-7 (up to 7), HWF-15 (up to 15), and HWF-19 (up to 19).
Task
DOLPHIN
Scallop
T Tinal
Timal
SumN-5
53.86
923.78
17.15
SumN-10
104.91
3.42c3
32.56
SumN-15
157.05
7.41e3
47.18
PathFinder. PathFinder (or Path) (Tay et al., 2021) tests the ability of an agent to reason over long-range dependencies within an image of two dots and a sequence of curved and dashed lines. The task is to identify whether the two dots are connected via the lines. We consider three versions based on the image size in pixels: Path-32 (32 32), Path-128 (128 x 128), and Path-256 (256 x 256).
HWF-7
2.45c3
9.99c3
4.08
HWF-15
9.78e3
1.66e5
16.97
HWF-19
1,63e4
1.82e5
11.16
Path-32
1.29c4
2.2c4
1.71
Path-128
1,67e4
4.17e4
2.49
Path-256
1.97e4
1.14e5
5.78
CLUTRR-3
807.12
4.29c3
5.32
CLUTRR-4
923.86
7.83e3
8.48
CLUTRR. In this task from (Sinha et al., 2019), given some text containing information about several individuals and some of their relationships, the model must infer the relationship between two given individuals, which is not explicitly provided in the input. We consider two versions, where the training data contains relation chains of lengths up to 3 (CLUTRR-3) or 4 (CLUTRR-4).
Mugen- IK
2.39e3
6.71e3
2.81
Mugen-5K
1.15e4
3,59e4
3.12
Breslin et al., 2024) and IndeCateR+ (De Smet al., 2024). We also include Logic Tensor Networks (LTN) (Serafini & Garcez, 2016), which combines first-order logic with con- tinuous optimization by compiling logical constraints into a computation graph on the GPU. We compare DOLPHIN against Scallop on all benchmarks, and against ISED and IndeCateR+ on SumN and HWF. We compare SumN with LTN, but were unable to write HWF in LTN (explained in Appendix H.1). Since LTN also runs out of memory for simpler benchmark like SumN-10, we do not com- pare against it for the more complex benchmarks of Path, CLUTRR, and Mugen. Similarly, we do not evaluate ISED and IndeCateR+ on these benchmarks, as ISED already fails to scale for simpler tasks like S1 and HWF-7, while IndeCateR+ does not support their recursive structures.
Mugen. In this task from (Hayes et al., 2022), given 3.2 second long video of gameplay footage and text captioning the video, the goal is to measure how aligned the text is with the video. There are two variants: Mugen-TVR, where the model retrieves the video that best aligns with the text, and Mugen-VTR, where the model retrieves the text that best aligns with the video. We consider two versions of this task: 1K and 5K comprising 1000 and 5000 training samples.
4.2. Experimental Setup and Baselines
Setup. All experiments, except CLUTRR, were run ma- chines with two 20-core Intel Xeon Gold 6248 CPUs, four NVIDIA GeForce RTX 2080 Ti (11 GB) GPUs, and 768 GB RAM. Since CLUTRR demands more GPU memory due to running the RoBERTa model with a standard batch size of 16, all programs for this benchmark were run with NVIDIA 1100 40GB GPU. We ran each tool thrice until convergence until a soft timeout of 10 hours was reached and report the average best accuracy and training time. For HWF/MNIST, we use the same CNN architecture as Scal- lop (Appendix D). For CLUTRR, we use Scallop's Roberta configuration: pretrained model (roberta-base) finetuned while training the classification head.
4.3. RQI: Scalability
Table 2 presents the total training times (Tomal) till conver- gence in seconds for DOLPHIN and Scallop across all bench- marks, alongside the scaling factor a (the ratio of the total training times of the baselines to DOLPHIN), Table 3 shows the same for the remaining baselines over SumN and HWF. We set a soft timeout of 10 hours, though we still report the training times that run over highlighted in red. The results demonstrate that DOLPHIN advances the state-of-the- in neurosymbolic learning by scaling to more complex prob- lems, e.g., larger versions of HWF, Path, and Mugen, that are beyond the reach of the other baselines which time out within 10 hours. Even for the other benchmarks where base- lines do not time out, DOLPHIN achieves a scalability factor of up to 47x, 62x, and 3.49x against Scallop, ISED, and IndeCateR+ with an average speed up of 13.95x across all baselines for all benchmarks.
Baselines. We select Scallop (Li et al., 2023), contempo- rary state-of-the-art neurosymbolic framework supporting differentiable programming optimized to run on the CPU in parallel using multiple cores. We also choose two sampling- based gradient approximation methods, ISED (Solko-
7

---

DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning
Table 3: Training time (in seconds) for DOLPIII, LTN, ISED, and IndeCateR+ on SumN and HWF. Training times more than 10 hours are highlighted in red, a is the ratio of training times of the baselines to DOLPHIN. OOM occurred on an NVIDIA GeForce RTX 2080 Ti (11 GB).
Task
DOLPHIN
LTN
ISED
IndeCateR+
T Tuull
Twul
Q
Tioal
Cx
Tona
a
SumN-5
53.86
92.54
1.72
299.63
5.56
416.78
7.74
SumN-10
104.91
OOM
2.17c3
20.64
385.65
3.68
SumN-15
157.05
OOM
9.8c3
62.41
548.28
3.49
HWF-7
2.45e3
N.A.
1.58e4
6.45
1,35e4
5.51
HWF-15
9.78c3
5,99e4
6.13
2,51e4
2.57
HWF-19
1.63e4
9.34e4
5.73
6.27e4
3.85
We also use the training times per epoch to calculate a scalability factor Ctepach (Table 5, Appendix D). We see that among cases where baselines timeout, Eepoch is up to 280x for HWF-19 and is 40.6x faster on average. This results in DOLPHIIN effectively training for more epochs in less time compared with the other baselines, which also allows it to converge to higher accuracies, as we see in Figure 5. We expand on these results in the next RQ.
4.5. RQ3: Provenance Comparisons
We perform ablation studies to compare the effectiveness of the DAMP and DTKP-AM provenances for each benchmark. We share the graphs in Figure 9 (Appendix E). In all cases, training with the DAMP provenance takes around 24.19 seconds per epoch less than with DTKP-AM on average.
However, the effectiveness of each provenance varies across benchmarks. For all variations of Path, CLUTRR, and Mu- gen, both provenances achieve comparable accuracies, with DTKP-AM having slight edge. For SumN, DAMP prove- nance is more effective than the DTKP-AM by 72% points on average, since the top-k proofs cannot capture all the possible ways in which sums of digits can be computed.
4.4. RQ2: Accuracy
Figure 5 presents the accuracy of DOLPHIN and the base- lines on the different benchmarks trained for up to 10 hours. DOLPHIN accuracies are marked in blue. In all cases, for DOLPHIN, we report the accuracies of the best-performing provenance. We use the DAMP provenance for MNIST, CLUTRR, and Mugen benchmarks, and the DTKP-AM provenance for the HWF and PathFinder benchmarks.
In contrast, for HWF, DTKP-AM is more effective than DAMP by an average of 42.2% points. Each step of the HWF program, shown in Appendix H, involves both a con- catenation operation and partial parsing operation before the final expression is evaluated produce a result. As such, it is difficult for the tags in DAMP to capture the semantics of the symbolic program. In the case of DTKP-AM, each tag is a collection proofs over input symbols correspond- ing to logits derived from the neural model. Therefore, any calculated gradients can be directly backpropagated the logits that most influenced the output, making this more effective provenance for this task.
We observe that in all cases, DOLPHIN achieves state-of-the- art accuracy among general-purpose neurosymbolic frame- works, except in CLUTRR, where DOIP'H accuracy is slightly lower than Scallop's. ss. As we scale up to larger versions of the benchmarks, DOLPHIN achieves better accu- racy, because the baselines either report lower accuracy due to the complexity of the benchmark (e.g., black-box sam- pling techniques such as ISED on HWF) or fail to converge within 10 hours due to slower per-epoch train time (e.g., Scallop PathFinder-256, IndeCateR+ on HWF-19). As result, DOLPHIN significantly outperforms the second-best benchmark on the largest dataset versions, achieving up to 20% gain on HWF and 33% on Path. In some cases, given no timeout, Scallop and IndeCateR+ eventually converge to accuracies comparable to DOLPHIN, as we show Ap- pendix D.2. However, doing so requires significantly more training time, as discussed earlier in Section 4.3.
5. Discussion
Dolphin can be used for any task where the output of a model can be cast as distribution over probabilities. This abstraction naturally encompasses wide range of discrimi- native models in machine learning, such as classifiers, struc- tured prediction systems, and vision-languag models. By associating each symbolic object with a probability dis- tribution over its possible values (tags), Dolphin enables downstream symbolic reasoning over uncertain predictions made by neural networks.
These results show that not only do DOLPHIN'S 's scalability improvements not come at the cost of accuracy, but Dot.- PHIN enables SOTA accuracy when previously unattainable.
For instance, consider an autonomous driving scenario where a standard object detector such as Faster R-CNN
8

---

DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning
Accuacy (%)
100
Dolphin
Scallop
50
LTN
ISED
IndeCateR+
Sumiy 5
Sun
Sun
15 HWF
HWF
15 HWF
CLUTRR
CLUTRR
Path
Path
Path
Mugon-T TVR
Magen- TVR
Sk Mugan- VTR
Magenn VTR
10
19
32
128
256
3
4
Uk
5K
Figure 5: Accuracy of DOLPHIN and baselines trained for up to 10 hours across all benchmarks.
outputs bounding boxes and class probabilities. One can represent each detection as a symbolic object whose at- tributes include coordinates and Distribution over class probabilities:
proof search using heuristics. SLASH (Skryagin et al., 2024) integrates neural networks and probabilistic circuits with Answer Set Programming. achieving scalability by dynamically pruning stochastically insignificant parts of programs. A-NESI (van Krieken al., 2023) uses learned neural models to approximate the exact probabilistic seman- tics of WMC, boosting scalability. However, these methods are often task-specific and lack generalizability to broader neurosymbolic learning, or they struggle to scale due to memory constraints when grounding symbolic computa- tions on GPU.
CLASSES = ('cor', 'person', 'truck', ...]
class DelectedDaject:
def _intt_ (self, coards, scora, class_logitan self.coords coords
self. - score
self.distr vistribution (CLASSES, class_logits)
Now, consider checking whether detected person is inside a detected car using symbolic reasoning:
Specialized neurosymbolic solutions. There are many specialized solutions for various neurosymbolic tasks. For instance, NGS (Li et al., 2020) uses a hand-coded syntax to specify the structure of mathematical expressions for HWF. More general solutions, such as NS-CL (Mao et al., 2019) in- cludes a framework for visual question answering that learns symbolic representations for text and images. NeRd (Chen et al., 2021) transforms questions in natural language into ex- ecutable programs based symbolic information extracted from text. (Orvieto et al., 2023) proposes a recurrent neural network architecture that achieves 95% accuracy on Path-32 and 94% on 128. In contrast, DOLPHIN is a general system that tries to scale diverse neurosymbolic programs.
def ts_inss :oord
peraon_ insion_nnniee___eee caz apply
0l. disdi d2.distr, Lambda el, c2: el == "parson" and c2 == "car" and ia_inside coorrda, o2.cnords)
This computation yields a Distribution over True and False, representing the likelihood that two detected objects constitute a person inside a car. This paradigm can generalize to many domains where neural predictions must be interpreted symbolically-Be example, relationship inference in vision scenes, symbolic post-processing over structured outputs, or probabilistic parsing in NLP. Dolphin thus offers principled and composable way to integrate symbolic logic over deep model outputs, opening the door for broader real-world applicability.
7. Conclusion and Limitations
We proposed DOLPHIN, a framework for scaling neurosym- bolic learning. DOLPHIN provides abstractions for writing symbolic programs along with pluggable vectorized prove- nances to compute symbolic gradients. This allows users to write differentiable symbolic programs in Python within Py- Torch pipelines that can scale to complex programs and large datasets. We show that DOLPHIN scales significantly better than existing neurosymbolic frameworks while achieving state-of slute-of-the-art performance on variety of tasks.
6. Related Work
Neurosymbolic programming frameworks. Apart from existing frameworks like Scallop (Li et al., 2023), Deep- ProbLog (Manhaeve et al., 2018), and ISED (Solko-Breslin et al.. 2024), there exist domain-speciff tools such as NeurASP (Yang et al., 2021) for phrase alignment. These ap- proaches often suffer from inefficiencies due to CPU-bound symbolic reasoning.
A limitation of DOLPHIN is that it needs the user to write programs a batched manner. This a common pattern within deep learning but may be restrictive to users new to batched programming. Also, while DOLPHIN works well with most models, the representation needed by generative models (e.g., Causal I.LMs) has not been investigated yet. DOLPHIN also lacks support for non-deterministic symbolic programs. We leave these for future work.
Scaling techniques. Various methods exist scale differen- tiable reasoning. LYRICS (Marra et al., 2019), Logic Tensor Networks (Badreddine et al., 2022), and Tensorlog (Cohen et al., 2020) compile first-order logic constraints into GPU- compatible computation graphs. Other techniques, such as Greedy NTP (Minervini et al., 2020a) and the condi- tional theorem prover (Minervini et al., 2020b), optimize

---

DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning
Acknowledgements
catlog-derivative trick. Advances in Neural Information Processing Systems, 36, 2024.
We thank the reviewers for their insightful feedback that helped to improve this paper. This research was supported by the ARPA-H program on Safe and Explainable Al under the award D24AC00253-00, the NSF award #2313010, and Google PhD Fellowship.
Green, T. J., Karvounarakis, G., and Tannen, Prove- nance semirings. In Proceedings of the twenty-sixth ACM SIGMOD-SIGACT
symposiun Principles of database systems, pp. 31-40, 2007.
Hayes, T., Zhang, S., Yin, X., Pang, G., Sheng. S., Yang. H., Ge, S., Hu, Q., and Parikh, D. Mugen: playground for vidco-audio-tex multimodal understanding and genera- tion. In European Conference Computer Vision, pp. 431 191-449. Springer, 2022.
Impact Statement
This paper presents work whose goal to advance the field of Machine Learning. There are many potential societal consequences of our work, none which feel must be specifically highlighted here.
Huang, J., Li, Z.,, Chen, B., Samel, K., Naik, M., Song, I.., and Si, X. Scallop: From probabilistic deductive databases to scalable differentiable reasoning. Advances in Neural Information Processing Systems, 34:25134- 25145, 2021.
References
Badreddine, S.. Avila Garcez, A., Serafini, L., and Spranger, M. Logic tensor networks. Artificial Intelligence, 303:103649, 2022. ISSN 0004-3702. doi: htss://di..og/11000111///.......
Kambhampati, S., Valmeekam, K., Guan, L., Verma, M., Stechly, K., Bhambri, S., Saldyt, L. P., and Murthy, A. B, Position: Lims can't plan, but can help planning in Ilm- modulo frameworks. In Forty-first International Confer- ence on Machine Learning.
URL betps://ww..cclee http://wwwww..ccceeeeeeeeeecceet /iliane//rriccl////////002222222
Chaudhuri, S., Ellis, K., Polozov, O., Singh, R., Solar- Lezama, A., Yue, Y., et al. Neurosymbolic programming. Foundations and Trends@ in Programming Languages, (3):158-443 2021.
Li, Q.. Huang, S., Hong, Y., Chen, Y., Wu, Y. N., and Zhu, S.-C. Closed loop neural-symbolic learning via integrating neural perception, grammar parsing, and sym- bolic reasoning. In International Conference on Machine Learning, pp. 5884-5894, PMLR, 2020.
Chen, 7., Gao, Q. and Moss, L.. S. Neurallog: Natural language inference with joint neural and logical reason- ing. In Ku, L.-W., Nastase, V., and Vulić, I. (eds.), Pro- ceedings of #SEM 2021: The Tenth Joint Conference on Lexical and Computational Semantics, pp. 78 88, Online, August 2021. Association for Computational Linguis- tics. doi: 10. 122116633/122222ttarraeea 1.7. URL https: //aclanthologe www.hhhllo...ooooo.e..ooooo http:2221...taareeee
Li, Z., Huang, J., and Naik, M. Scallop: A language for neurosymbolic programming. Proceedings of the ACM on Programming Languages, 7(PLDI): 1463-16444440 1487, 2023.
Liu, Y. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv: /692 2019.
Manhaeve, R., Dumancic, S., Kimmig, A., Demeester, T. and De Raedt, L. Deepproblog: Neural probabilistic logic programming. Advances in neural information procoesssioproccssiiig systems, 31, 2018.
Cohen, W. W., Yang. F. and Mazaitis, K. Tensor- log: A probabilistic database implemented using deep- learning infrastructure. J. Artif. Intell. Res., 67:285-325, 2020. URL https:/ httt::::/ap..eemattiieeeeee..eee org/ 211263674.
Mao, J., Gan, C., Kohli, P., Tenenbaum, J. B., and Wu, J. The neuro-symbolic concept learner: Interpreting scenes, words, and sentences from natural supervision. In International Conference on Learning Representations, 2019. URL /ttpttts:://ppeeeeeet hwtp///wpewwwwweeeee/peeee.e..wp net ferum?? i id=rJgMlhRetm.
Dang, M., Khosravi, P., Liang, Y., Vergari, A., and den Broeck, G. V. Juice: A julia package for logic and probabilistic circuits. In AAAI Confer- ence on Artificial Intelligence, 2021. URL https: //api. www.nticscholee......... wrw.aa.pog///oooo/// 235363700.
Marra, G., Giannini, F., Diligenti, M., and Gori, M. Lyrics: A general interface layer to integrate logic inference and deep learning. In Machine Leurning and Knowledge Discovery in Databases: European Conference, ECML PKDD 2019, Würzburg, Germany, September 16-20, 2019, Proceedings, Part II. pp. 283-298, Berlin, Heidel- berg, 2019. Springer-Verlag. ISBN 778-3-030-46146-4
Darwiche, A.
An advance on variable elimination with applications to tensor-based computation. In ECAI 2020, pp. 2568. IOS Press, 2020.
De Smet, L., Sansone, E., and Zuidberg Dos Martires, P. Dif- ferentiable sampling of categorical distributions using the
10

---

DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning
doi: 10. 101100779788-00000441-----111. URL URL // 5ol.corg/10.1000//98---333
Solko-Breslin, A., Choi, S., Li, Z.,, Velingker, N., Alur, R., Naik, M., and Wong, E. Data-efficient learning with neural programs, arXiv preprint wrXiv:2406.062444 2024.
Minervini, P., Bošnjak, M., Rocktäschel, T. Riedel, S., and Grefenstette, E. Differentiable reasoning on large knowledge bases and natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pp. 5182-5190, 2020a.
Tay, Y., Dehghani, M., Abnar, S., Shen, Y., Bahri, D., Pham, P., Rao, J., Yang, L., Ruder, S., and Metzler, D. Long range arena A benchmark for efficient transformers. In International Conference Learning Representations, 2021. URL https: i id=qVyew-grC22
http://ss///////eeee//ee/ee /oppnrevieeeeeeeeeeeeeeeeeeeeeee net. forum?
Minervini, P., Riedel, S., Stenetorp, P., Grefenstette, E., and Rocktäschel, T. Learning reasoning strategies in end-to- end differentiable proving. In International Conference on Machine Learning, pp. 6938-6949, PMLR, 2020b.
van Krieken, E., Thanapalasingam, T., Tomezak, J., van Harmelen, F., and Ten Teije, A. A-nesi: A scalable approximate method for probabilistic neurosymbolic in- ference. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances Neural Information Processing Systems, volume 36, pp. 24586- 24609. Curran Associates, Inc., 2023.
Naik, A., Stein, A., Wu, Y., Naik, M., and Wong, E. Torchql: A programming framework for integrity straints in machine learning, Proc. ACM Program. Lang., 8(00PSLAI), April 2024. doi: 10.1 10.1145/3649841, URL atttps://dol org/ /1/.///.... 11e.11455//0 01144///444444 9841.
Yang, Z., Ishay, A., and Lee, J. Neurasp: embracing neural networks into answer set programming. In Proceedings of the Twenty-Ninte International Joint Conference on Artifi- cial Intelligence, ICCAT 2021. ISBN 97809992411665
Orvieto, A., Smith, S. L., Gu, A., Fernando, A., Gulcehre, C., Pascanu, R., and De, S. Resurrecting recurrent neu- ral networks for long sequences. In Proceedings of the 40th International Conference on Machine Learning, ICML'23, JMLR.org, 2023.
Sanh, V. Distilbert, distilled version of bert: Smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108 2019.
Scallop Language Group. Scallop and neuro-symbolic programming: Tags, instrumentation, and prove- nance. Eleventh Summer School on Formal Techniques, 2022. URL htttps://wwa/ wwwpwwwwwwwwlp........w.... lang.org/ 2sftt2/lectures///
pdf.
Serafini, I.. and Garcez, A. Logic tensor networks: Deep learning and logical reasoning from data and knowledge. arXiv preprint arXiv:1606 1606..4422, 2016.
Sinha, K., Sodhani, S., Dong, J., Pineau, J., and Hamilton, W. L. CLUTRR: A diagnostic benchmark for inductive reasoning from text. In Inui, K., Jiang, J., Ng, V., and Wan, X. (eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lan- guage Processing (EMNLP-IJCNLP), pp. 4506-4515 Hong Kong, China, November 2019. Association for Computational Linguistics. Lingutttionalngnguiisssssss, doi: 10.18653/v1/D19-114 URL hwtps:ttp//.m http://aaaaaalllaaaaaaaa org/D19 -4458
Skryagin, A., Ochs, D., Dhami, D. S., and Kersting, K. Scalable
answer set programming. J. Artif. Int. Res., 78, January 2024. ISSN 1076-9757. doi: 10. 10.1613/jair..115000 URL https:://o...oo////// 6613 jair 1..1027..

---

DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning
A. DTKP-AM Provenance
We clarify and expand on some aspects of the DTKP-AM provenance.
A.1. WMC approximation
In this section, we emphasize that DTKP-AM does not perform precise weighted model counting (WMC) and address possible shortcomings that could arise. A hardware-efficient vectorization of exact WMC beyond the scope of this paper, and is itself an active area of research. Instead, we use the following add-mut mult approximation of WMC:
Pr(() = Pr(tt) = EII norm(t;y)
We note that this approximation upper bounds the result from DTKP-W MMC the coarseness arises from the summation, which may double count models that satisfy more than one of the proofs. However, add-mult achieves significant computational speedup since it simplifies the exponential enumeration over all possible models into a linear pass over the tag's elements.
We further claim that this approximation does not destroy all the semantics from DTKP-WS OTKP-WMC due to DTKP-AM's faithful implementation of the semiring operations @ and @ for tracking top-k proofs. DTKP-AM tags therefore remain similar to DTKP-WMC tags at every intermediate symbolic reasoning step. By contrast, the imprecise add-mult is a one-time transformation of the final tags into probabilities, performed only after the tags have been propagated through the entire symbolic program. Crucially, we show there exists information that uniquely captured by top-k tag operations, and not lost when fuzzily converting the tags to probabilities.
As simple illustrative example, consider using APPLY with the following toy function:
T
a a=b
f(a, b) =
IF
otherwise
For any distribution D of mutually exclusive input symbols (e.g. the digit classification of CNN), we intuitively would like the distribution f fD, D, D) to assign probability of I to symbol T and a probability of 0 to symbol F. According to our semantics, the tag for T is actually given by:
((D, = (D(1)) D())
However, if we were to use DAMP to compute the tags for (DD the provenance treats the two input distributions as independent when they are the exact same distribution! Thus, the probability assigned to T by f(D, D) incorrectly calculated as:
((D, = =
On the other hand, consider any top-k provenance that satisfies:
1. 1& 1' top UI,
where x is the set Cartesian product. Note that DTKP-AM does satisfy this condition, where the set union is implemented with an element-wise minimum. Now assuming D(i) is initialized in the natural way (i.e. tag consisting of a single proof containing just the input symbol i), then D(i) @ (t) = D()) and therefore:
(T = + D(i)
Under both DTKP- WMC and DTKP-AM, the probability of T is:
E1l (aom(D(1)) = Σ D(i = Σ Pr(i)= =1 3
for any normalized D with at most l symbols. Even the number of symbols exceeds kt we note that the distributions we seek to learn are often skewed (an accurate model should assign probability to the ground truth that significantly outweighs
12

---

DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning
the other symbols). For such distributions, DTKP and DTKP-AM would still yield the same probability for T,, and it is much closer to 1 than the sum of squares result from DAMP.
While this example may seem contrived, it still suggests the smaller role a "correct" WMC can have on the final answer compared to @ and @ implemented with proper set-based semantics. We even hypothesize that in most cases, the add-mult approximation does not meaningfully affect the final result compared to DTKP-WMC This is empirically demonstrated by our benchmark results, which shows DTKP-AM achieving similar accuracy to Scallop's implementation of DTKP-WWM. In fact, DAMP can be considered as a sort of ablation, where both the WMC and semiring operations use fuzzy add-mul: semantics instead of set-based one, and indeed, its accuracy often performs worse than both DTKP-WMC and AMK.-AM.
A.2. Role of too and 000
In this section, we motivate the use of too and -oo in DTKP-AM's tensor representation of tags. Because tensors I are rectangular where every proof and symbol j must have entry lig, we require a way denote the absence of an input symbol from proof, and the absence of a proof from tag. Importantly, an absent symbol should not influence the probability of proof (i.e. its normalized value should contribute I to the probability's product), and an absent proof should not influence the probability of a tag (i.e. it should contribute O to the sum during add-mult WMC), Indeed this is captured by our definition of norm, which clamps +00 to (representing absent symbols) and -oo to 0 (representing absent proofs) during any probability calculation. While this introduces clamping operations, PyTorch's implementation of clamp backpropagation ensures a gradient of everywhere, even on the clamp boundaries (source: https: / /ggtthb..... com/ wwwptch/tooppppppptt http:ppttttttttotoooooopptp http://tooooccooot//ee//e
Since Ô corresponds to the tag consisting of no proofs (i.e. tag with probability 0), we initialize it to be a tensor where every proof is absent (all -o)). Likewise, since i corresponds to the tag consisting of a single empty proof (i.e. tag with probability 1), we initialize it to be a tensor where every symbol is absent from the first row / proof (all +o0), while the remaining rows / proofs are absent (all -00).
A.3. Further reading
For more in-depth explanation of provenances in general, including the formalization of DTKP semantics with Boolcan formulae, see Section 4 of (Li et al., 2023). For worked examples of provenance computation with comparisons of top-k provenances to DAMP, we refer the reader to (Scallop Language Group, 2022).
B. Control Flows and Recursion in DOLPHIN
Table 4: Time taken by the symbolic program for the HWF task split by the time spent the CPU and GPU. UDFs refer user-defined functions where control flows reside for HWF. The times annotated with C and G indicate time spent on the CPU and GPU, respectively.
Config
Time for UDF (s)
Time for Tag Computations (s)
Total Time (s)
No Parallelism Parallelized Tag Computations
36.24 (C)
461.02 02 (C)
497.26
14.13 (C)
75.125 (G)
89.25
In this section, we provide a more detailed explanation of how DOLPHIN handles control flows and recursion. In DOLPHIN, control flows largely exist within the lambda functions supplied to the 'Apply'. 'ApplyIf', and 'Filter' operations, which can be arbitrary Python functions over the symbols in the Distributions. As discussed in Section 3.1.2, these functions can include complex operations like if-then-else branches, loops, and even recursions. We do assume that divergent control flows are resolved within the lambda function itself. The nature of these functions means that they cannot be parallelized over the GPU. Instead, they are executed sequentially on the CPU, while the associated tags are computed parallely on the GPU. We optimize the design of the Distribution class so that there is one set of CaU-based computations for the entire batch of samples rather than one set of computations for each sample, which is typical of other neurosymbolic frameworks. This allows DOLPHHIN to maintain the benefits of parallelism even while the user-defined functions are executed sequentially.
13

---

DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning
Coord namedtuple Coord'', ['x', 'y'1) 1)
def compute_paths (paths, edges):
new_paths - apply_if (paths, edges, lambda pl, p2: (p1.x, p2.y), lambda pl, p2: pl.
y == p2.x)
merged unien (paths, nex_paths)
checking for convergence via fix-point
if merged. -- symbels, return merged
else:
10
return compute_patiie (merged, edges)
11
12 edges Distribution ion (model (img), , points)
13
paths compute_pace (edges, edges)
Figure 6: Example of transitive closure computation in DOLPHIN.
B.1. Control Flow in HWF
We demonstrate this by showing the time taken by the symbolic program for the HWF task split by the time spent on the CPU and GPU in Table 4. The first row shows the time taken when the neurosymbolic model is run sequentially the CPU with no parallelism. The second row shows the time taken when tag computations are parallelized on the GPU over batches of 64 samples each. The times annotated with C and G indicate time spent on the CPU and GPU, respectively, We only show the time taken in the forward pass in the table.
Observe that the time, both for UDF computation and for Tag computation, decreases as we move from sequential CPU evaluation the batched evaluation. Due DOLPHIN's design, increases in batch size result fewer CPU operations, since the set of CPU operations is shared for the entire batch, while parallelizing more tag computations the entire batch.
B.2. Recursion
In order to write recursive computations in DOLPHIN, one has two choices: either supply recursive user-defined function to the DOLPHIN primitives, write more fine-grate fine-grained program in Python that uses DOLPHIN primitives in the base case as well the recursive case, set to terminate once condition is met. Here, the diverging control flows can be merged using the UNION primitive. We follow the latter approach for tasks involving recursion, such as Path and CLUTRR. The crux of those programs involves performing a transitive closure computation over a graph, represented by a set of edges for Path or relations for CLUTRR. We show an example of transitive closure computation in Figure 6.
Here, lets say that model is a neural model that predicts the edges between each pair of points in a graph, represented by points. The compute-paths function computes the transitive closure of the graph by iteratively applying the edges the paths. The APPLYIF function applies the edges to the paths if the end of the first path is the same as the start of the second path. The UNION function merges the new paths with the existing paths. The function compute.paths is called recursively until a fixpoint is reached, specifically until no new paths can be added. This is simple example of a recursive computation in DOLPHIN, and also forms the core program needed for the PathFinder task. We perform a similar recursive computation for the CLUTRR task, where we find the transitive closure of a graph representing relations between people in a passage.
C. On the Language and Semantics
C.1. Language
To develop the operations provided by DOLPHIN, studied several neurosymbolic tasks determine the most common operations needed for these tasks. We found that the main operation needed in most programs is to apply function to symbols from different input models and relations. This is primarily achieved via the join operation in Datalog, but we introduce the Apply or ApplyIf primitives for a more Pythonic approach. Filters are used to remove symbols violating conditions, similar to Datalog selections, while Union mimics the disjunction operation in Datalog, typically needed for writing recursive programs as described in Appendix B.
14

---

DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning
C.2. Semantics
We designed DOLPHIN to general-purpose neurosymbolic framework able support various semantics, as long as they can be expressed as operations over tags tracked via the Distribution class. DOLPHIN assumes that the provenance supplied to it offers both the conjunction and disjunction operations that operate over combinations of tags from input symbols, as well as a way to translate tags to probabilities. As long as these assumptions are satisfied, the primitives of DOLPHIN preserve the semantics offered by the provenances.
As such, supplying the DAMP provenance to the DOLPHIN program introduces basic fuzzy semantics which are preserved by DOLPHIN. However, there are cases where the independence assumptions may not hold and fuzzy semantics may not be appropriate.
The DTKP-AM provenance, on the other hand, offers alternative without the assumption of variable independence, except on the input variables. At each step of the program, each symbol is associated with the tags of the input symbols that produce it via the proofs. Again, since DTKP-AM satisfies the aforementioned assumptions, the top-k semantics of the provenance are preserved.
These tags are then translated into probabilities by performing an add-mult operation over the proofs. This approximation of the WMC operation more complex and results in more precise translation of tags to probabilities. However, we see in the experiments where Scallop uses DTKP-WMG the accuracies achieved by DTKP-AM and DTKP-WMC are comparable.
D. DOLPHIN Experiment Details for Benchmarks
D.1. Comparison of Per Epoch Training Times
Table 5: Comparison of training times (in seconds) taken by each baseline. The Timeout (TO) is set at 10 hours, is the scaling factor, which is the ratio of the per epoch training times of the baselines and DOL.HIN.
Task
DOLPHIN
Scallop
LTN
ISED
IndeCateR+
Tepech
Tepoch
Olepoch
Tepach
Ospech
Tepoch
@xppcch
Tepoch
Crgockk
SumN.5 5
10.77
184.76
17.16
4.63
0.43
59.93
5.56
59.54
5.52
SumN-10
32.56
216.54
32.14
SumN-15
10.47
493.87
47.17
00M
653.39
62.41
23.84
2.28
HWF.7
152.87
499.57
3:27
790.04
5.17
540.26
3.53
HWF-15
858.8
1.4984
17.35
N.A.
6.84c3
7.97
2.51c3
2.93
HWF-
1.4e3
3.92e5
280
1.05e4
7.5
4.18c3
2.99
Path-32
1,29c3
2.2c3
1.71
Path 128
1.67e3
4.18e3
2.5
N.A.
Path-256
1.97c3
1.1364
5.74
CLUTRR-3
152.21
N.A.
CLUTRR-4
165.13
1
4.74
Mugen IK
165.74
133.68
0.81
Mugen-SK
826.31
634.86
0.77
The following are more details on the experiment setup. For each experimental trial, report the highest evaluation accuracy over all epochs. Unless otherwise noted, each trial was run on a machine with Intel Xeon Gold 6248 (2.50 GHz) CPUs and NVIDIA GeForce RTX 2080 Ti (11 GB) GPUs. Additionally, for Scallop, the accuracy of the best-performing provenance is reported. Table 5 shows the training time per epoch for all of the baselines in each of the benchmarks.
D.2. Accuracy without Timeouts
While Figure 5 shows the accuracy of baselines run until a timeout of 10 hours, Figure 7 shows their accuracy when run until convergence or until the test-time accuracy plateaus. We see that only IndeCateR+ for HWF-19 and Scallop for Path-256 are able to match DOLPHIN's 's accuracies. Scallop and ISED are unable to converge for the larger versions of HWF, while ISED unable to converge for the larger versions of SumN.
15

---

10.21 DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning
Monnyy
L7N
Figure 7: Accuracy of DOLPHIIN and baselines across all benchmarks without any timeout.
D.3. Effect of K on Accuracy and Runtime
We report preliminary results on the effect of the top-K value in DTKP-AM across different tasks. As shown Tables and 7, increasing K generally has little effect final accuracy per-epoch training time. HWF-19 is the only task where increasing K offers noticeable gains. For all other benchmarks, accuracy remains stable and runtime scales sub-linearly due to DTKP-AM's vectorized implementation.
Table 6: Accuracy (%) of DTKP-AM with different values of K across five benchmarks.
Benchmark
Sum-15
9.61
10.81
10.51
HWF-19
8.94
99.15
96.89
95.75 80.86
Path-256
81.39
82.14
82.38
CLUTRR-4
Mugen-5K
(94.1 (95.7)
(95.4 /95.7)
(95.3 /5.4)
(95.4/95.)
Table 7: Training time per epoch (T/ep in seconds) for DTKP-AM with different values of K.
Benchmark
Sam-15
37.21
47.70
53.52
58.54
HWF-19
1.21c3
1.40c3
1.33c3
1.46c3
Path-256
1.97c3
2.34c3
2.10c3
2.12c3
CLUTRR-4
Mugen 5K
460.38
464.68
470.05
465.10
D.4. MNIST Sum-N
For this task, the base neural network model is standard CNN (a 3-layer convolutional network with ReLU activations) classifying each image into 10 classes of digits (0, 1...., 9). The symbolic module sums the Distribution objects over the logits output by the neural model for each image.
Each of the MNIST Sum-N tasks had batch train size of 64 samples, a learning rate of 0.001, and top-k value of 1. Each of the tasks were trained on a dataset size of the original MNIST dataset divided by N Sum-N. Sum5's dataset consisted of 12000 train samples and 2000 test samples. Sum10's dataset consisted of 6000 train samples and 1000 test samples. Sum15's dataset consisted 4000 train samples and 666 test samples. Figure is a high-level overview of the Sum-N model's architecture.
D.5. Hand-Written Formula
For Hand-W Wrrrt-rr Formula, the perception model again a standard CNN that classifies images into 14 classes: digits (0, 1, ...... 9), and 4 operations (+, -, x, and /). The DOLPHIN program for this task builds strings of formulae from the outputs of the neural model and evaluates them using Python's eval function, demonstrating the ability of DOLPHIN support black-box functions.
We trained each task with a batch train size of 64 samples. The learning rate was 0.0001, the global sampling value was 7, and top-k value was 3. Length 7's dataset consisted of 9600 samples for training, 2400 samples for testing. Length 15 consisted of 24000 training samples and 6000 testing samples. Length 19 consisted of 32000 training samples and 8000 testing samples.
16

---

DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning
3
q
Tuples of N MNIST Images
MNISTNet
Logits L
Logits L
Logits NN
Distribution D
Distribution D
Distribution D
L
apply,
apply,
Logits
Distribution D DRes
Res
0
9N
L
9N
I
ON
Figure 8: Components of the SumNNet model written in Figure 2.
D.6. PathFinder
For this task, the perception model is also CNN, but it predicts edges between pairs of nodes (denoted by dashes) well as the end points depicted in the image of the maze. The DOLPHIN program for this task is recursive since must search for paths between the two dots.
For each of the PathFinder tasks, we used a batch train size of 64 samples, a learning rate of 0.0001, and top-k value 1. Each task's dataset consisted of 539459 images for training and 59940 images for testing. Each task had own dataset of images with dimensions of the task's pixel size.
D.7. CLUTRR
For each CLUTRR task, we used a single A100 GPU (40 GB), with learning rate of 0.00001 and use batch size of 16. The length of the training dataset for CLUTRR (Small) was 11,093 and that of the test set was 1146. The training set for CLUTRR (Medium) contained 15,083 samples and the test set contained 1048 samples.
The DOLPHIN program for CLUTRR receives as inputs pairs of entities from the input paragraph along with the logits for each pair over 21 possible relations produced by the classification head of the Roberta-bass (Liu, 2019) model. The program then recursively derives relations over the graphs these pairs represent until no new relations can be derived. After that, it returns the Distribution over relations for the target pair of entities.
D.8. Mugen
For each Mugen task, we use batch size and a learning rate of 0.0001. From the full Mugen dataset, we sample a training set of 5000 examples for Mugen (Medium), and from that set, we sample a training set of 1000 for Mugen (Small), Both Small and Medium are evaluated on fixed holdout set of 1000 samples. We train and evaluate for up 100 epochs.
17

---

DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning
We use combination of DistilBert (Sanh, 2019) and S3D (De Smet et al., 2024) as the perception model for the text and video inputs respectively. The DOLPHIN program for both Mugen tasks computes the temporal alignment of given text-video pair. The inputs (extracted from the text and video by neural components) are pairs of IDs and actions, where the ID order corresponds to the action sequence (e.g. the IDs for video actions are the frame numbers). The program finds the Distribution of all valid mappings between text event IDs and video frame IDs that preserve the order of actions.
E. Graph Results of RQ3
100
Accuney (%)
75
DAMP DTKP-AM
50
25
SumA
SumN
SumN 15
HWF
HWT 15
HWF
CLUTRR
CLUTRR
Patii
Path 128
Pali 256
Mugen TVR IK
Mugen- TVR 5K
Magen- TR IK Mugen VTR SK.
10
19
Average Training Time per Epoch
10a
DAMP DTKP-AM
10a
10
10"
SumN 5
SumN 10
Sumii
HWF
HWF 15
HWF
CLUTRR
CLUTRR 4
Path 32
Path 128
Path 256
Mugenn IK
Mugen
15
19
SK
3
Figure 9; Accuracy and average training time per epoch for DAMP and DTKP-AM.
We show the results of the provenance comparison experiments (RQ3) in Figure 9. The graph the top shows the accuracies achieved by each provenance over all tasks, while the bottom graph shows the average training time per epoch required for each provenance over all tasks.
F. Comparison with basso Neurosymbolic Frameworks
Systems like LYRICS (Marra et al., 2019), Logic Tensor Networks (LTNs) (Badreddine et al., 2022), and Tensorlog (Cohen et al., 2020) all have limited expressivity, which is of the obstacles DOLPHIN aims to overcome. Specifically, they restrict the symbolic programs to first order logic and require users specify low-level information such how variables are grounded and what their domains are. They also restrict the symbols to be in the form of tensors and the user defined functions to consist of TensorFlow operations. These restrictions allow such systems to use TensorFlow to compile these programs into highly efficient computational graphs, but at the cost of expressivity. These frameworks also exclusively support simpler provenances and t-norms which are not sufficient for complex neurosymbolic programs.
On the other hand, DOLPHIN allows the user to track tags for symbols which can be arbitrary Pythonic objects. DOLPHIN programs further allow the user to manipulate Distributions over such symbols using arbitrarily complex code which may not necessarily translate to computational graph. As such, there is fine balance between the probabilistic computations, that happen over a GPU, and the symbolic computations, that take place on CPU, all while maintaining a mapping between the two. This fundamental design choice is also what allows DOLPHIN to be more expressive and flexible than existing systems. We also design DOLPHIN to be modular so that users can casily extend it to support new provenances and t-norms. As such, the t-norms used in LYRICS and LTN can be trivially added in vectorized manner to DOLPHIN.
18

---

DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning
For instance, assume the case of MNIST Sum-2, where 'model' the neural model. This is how it needs to be expressed in
LTN:
### Predicates
Digit = Sen.Predicate FromLogits (medel, activvation__fmnolooo=sffffff
#FF Variables
d1 ltn.Variable ("digitsi", range )10))
d2 Itn.Variable ("aiiiale"diigit22"" ("digits2", range (10))
### Operaters
Not 1tn. htt.:h/appee..eoomeeeeeeeeetttot And oat.apappa...ntttttttt.t.....a.. Or itn. ortpppppe...cceee.......o.oa.aap 10 Implies Ltn.Krapper T
11 Forall Itn. Exists wtt.
() semantics="forallly
() - semantics="exiataa
13
14
15 mask
16 add Fen.Fuuctiinnii Lambda (lambda inputs: inputs (innttttss
17 equals 1tn. Pre.iccccctte Lambda (lambda inputs: inputs == inputs (1])
18
19 ### Axioms
20 @t. fwn.fuuciiion
def axioms (i magges_x,, images_y, labels_2, p_schedu l_.schedule-ff.consccee )2.)):
22
images_x 1tn. Van.Variable(""", images_x)
23
images y Itn. Variialaal images_yl
24
labels_2 z = Ltn.Variable
labels_2
axiom = Forall(
1tn. itn. (images_,, images_y, labels_2),
27
Exists(
28
(d1,
29
(Digit [images_xx d1]), dig), [images_y, d2))),
mask=equals [add ([dl, d2]), labels_z] _
p-p_schedule
32
)
33
p=2
34
)
resuit_logita axiom.tense. return result_logits
16
Note that the FOL semantics of the Real Logic language used in LTN requires the user to specify the tracking of the probabilities with the symbols denoted by the 'digits*' variables.
On the other hand, DOLPHIN's design allows the user to write the same program in a more intuitive way:
d1 Distributien (model (img [0]), range (10))
d2 Distribution (model (img [1]), range (10)
desult_logita GetProbs (Apply (d1, d2, lmmbda X, y: y)
F.1. Optimizing Probabilistic Computations
Other works such (Dang et al., 2021) and (Darwiche, 2020), focus on solely on probabilistic computations rather than neurosymbolic frameworks. For instance, Juice (Dang al., 2021) is Julia package for logic and probabilistic circuits, which is not designed to be integrated with deep learning frameworks. On the other hand, (Darwiche, 2020) focuses on variable elimination with applications to optimize tensor-based computation. It will be interesting see how DOL,HIN can be integrated with such systems to further improve the scalability and efficiency of neurosymbolic learning, and will include a discussion on this in the revised manuscript. However, we still believe that DOLPHIN's 's novelty lies in its design that allows for the seamless integration of general purpose neurosymbolic programs within deep learning frameworks, which is not addressed by the existing systems.
19

---

DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning
Table 8: MNIST ProductN Training Epoch Times in Seconds.
N
B = 64
B = 256
Time per Epoch (s)
Accuracy
Time per Epoch (s)
Accuracy
11.42
0.96
8.92
0.97
8
12.55
0.95
9.15
0.95
16
27.45
0.94
15.71
0.89
20
36.59
0.92
18.73
0.85
G. On Combinatorial Explosions
The nature of the APPLY and APPLYIF primitives imply the possibility of combinatorial ballooning of computations in cases where either the number of symbols is large or where there are several distributions over which the function applied. This is indeed a fundamental challenge neurosymbolic frameworks as a whole. DOLPHIN mitigates this by leveraging the Distribution class, which condenses symbols into a single collection stored CPU RAM while maintaining tags as GPU tensor (b x N T, where is the batch size, N is the number of symbols and T is the shape of the tag). As shown in Figure 8, this approach reduces symbolic overhead by avoiding redundant evaluations for each batch sample, unlike frameworks like Scallop, where each sample in batch is independently evaluated. While tag evaluations still involve all combinations across all samples in batch, they are computed in vectorized manner on the GPU.
To see the effect of such computations even on larger experiments, we consider MNIST ProductN, where we multiply digits classified by MNIST CNN as opposed adding them in SumN. We show the per epoch training times in Table 8 for batch sizes of 64 and 256. In both cases, the DOLPHIN program able to achieve high accuracies even for N=20 while running in reasonable amounts of time. The scaling gets even better for larger batch sizes (e.g. 256) since it increases the number of parallelized operations executed at any given time.
H. The HWF Model
We show the ncurosymbolic model written in DOLPHIN for the HWF task along with the base neural model. the HWF task, the neural model simply classifies each input image into 14 symbols: 10 digits and 4 operations.
class SymbolNet (nn. Module)
def sosiit((seei))))
super (SymbclNet, self) initt inii_(
self convv Co.von (1, 32, stride paddi ng 1) self.convz convod (32, 64, 3, stride 1, padding 1)
self.fcl nn Linear (30976, 128)
self. bn nn. nn. Bac.cattoooomdd1129 self.fc2 nn Linear (128, 14)
10
def forward (sclf, x):
11
sclf.cenvi (x)
12
(.)elu(x)
13
self.conv2 (x)
14
x mmax_poo122dxx, 2)
15
X F.dropout (x, p=0.25, training=self. trainiig))
16
torch.flatten 1)
self.
18
fel
19
X F.relu(x)
20
X
F.dropout (x, p-0.5, training=self.traannii sc1f.
return F.softmax dim=1)
This neural model is then used in the DOLPHIN program as follows:
class HWFNet (nn. :odule)
def _init_____ (self) super (HWFNet, self) .nniniitt init_
20

---

DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning
Symbol embedding
self walff.symell_cnn SymbolNet ()
self. operatres = ("+*, ) ("-", ), ("*", ), ("/", )]
self.symbold. (str(i),) for in range (10) self, operators
10
self.db db =
11
def (sewf, img_seq, img_seq_len);
13
batch_size, formula_ length,
img_seq shape
14
length [1. Wt.item () for in ing_seq_een
15
inp meg.eq. Flatten (start_dim-O, end_dim-1) symbol self sylbol enn (imp) view (batch size, -1, 14)
17
symbol
18
19
def aval_formulatt (s)
20
try:
return eval ("" (**.joinss))
22
except: return math.nam nan
23
24
def concat_symbol (formula, symbol)- return formula
if formulal-1] == "";
27
28
else:
29
if not isinstance (symbol, tuple):
symbel (symbol,)
formula symbol
32
(formula))) if [ormulal-2] in ["*", "/"]:
and 1en (formula) 1:
33
34
haa_rresult_o (eval (eral_forrful formula (formula(33:))
formula formula[=-3] (oval_result,)
36
return formula
37
38
def infer_expression (length, +syrbols):
res symbols
for r1nge len (symboll)):
41
res Apply symbolsli], concat_symbol
42
x = (Apply (res, eval formula), ) return
43
45
def pecrgosyyboll lengths)
46
distrs []
47
for i in range (symbol. (tymmbol..sapee11)) shape
48
if i lengths:
distrs. apppedd (pstrrnitiiiiittnniissoso (symollss [i, :] vilveewws (-1, 14), self self.symbols)
50
if
51
distrs(-1) distre(-1). (1ambda s : s not in
52
else:
distrs[-1] else: distrs.append (Distribut (torch. (1, device-device), ((",),, 11)
(1ambda in self self.operators)
55
56
res (lengths, vdistrs)
return
59
60
Berchql.qeeryii " base="symbols") join ("lengths") \
61
project (lambda symbols, lengths: reorg (symbols, lengths)) \
project (infer_expression, batch_sizo-batth_ssiee
64
res q (db, ("osorss("sybbol:: symbol, "lengths": length), disable=True) rows
65
66
stacked = Distribution. stack (res)
return Peobsobs (stacked)
21

---

DOLPHIN: A Programmable Framework for Scalable Neurosymbolic Learning
The HWFNet class is the neurosymbolic model. It takes in sequence of images, img_seq, and their lengths, img.seq. Note that within a single batch there may be image sequences of varying lengths. The neural model, symbol cnn, is used to classify each image in the sequence into one of the 14 symbols. Since we know that each number in the expression is single digit, the reorg function is used to filter out relevant symbols based on their position in the sequence (operators in even places, digits in odd places). This function also pads sequences of smaller lengths with empty strings, written as Distributions with single element and a probability of 1. Once reorganized, the wwfffrr.xpresssoc. function is used to infer the expression from the symbols. It does so by first concatenating Distributions using the concat.symbol function, which also performs partial evaluations whenever possible. Once all the symbols are concatenated, the expression evaluated using the eval_formula function. The final expression is then returned as a Distribution. As sidenote, while optional, we use the TorchQL (Naik et al., 2024) library to help write certain parts of the program. This shows the ease with which Distributions can be used with existing machine learning frameworks.
For such a complex DOLPHIN program, using a simple provenance like DAMP proves insufficient for longer sequences since the tags of all possible combinations of symbols are collated into a single number. On the other hand, DTKP-AM is able to track the top-k proofs for each symbol, pruning out the less probable proofs. Furthermore, since each proof is a collection of input symbols leading to a specific output, once the loss calculated, gradients can be backpropogated directly to the input symbols that had the most influence the output. On the other hand, the gradients may be distributed across all symbols in DAMP as it backpropogates through each intermediate computation regardless of their role in the computation of the output, resulting in slower convergence.
H.1. Writing HWF using LTNS
The crux of the HWF program written in DOLPHIIN relies on string concatenations and using Python's 's eval function. In order to write HWF using LTNs, we need a different approach, since LTNs require all symbols to be grounded as tensors and all functions to be operations over those tensors. While one can ground all the possible strings (representing 0, .,...,9 , and operators +. -, x, /) as real-value tensor encodings, one cannot execute functions like eval over those encodings since cannot be compiled onto the tensorflow computation graph.
This leaves us with the option of trying write the HWF program as a probabilistic parser, as seen in Scallop. However, such a parser needs to be able to recursively parse arbitrarily long expressions, since HWF expressions not of fixed length even within the same task. This means it needs the ability to evaluate subtrees of the expression AST and produce intermediate results as new constants that can be populated within relations and operated over.
Based on our understanding of LTNs, the LTN paper does not provide any information on the dynamic creation of new constants or adding new domain elements at inference time, and instcad focuses on examples and experiments where the domains are static and grounded prior to the evaluation of the logic formulae. We also spent considerable time perusing the official LTN repository, but could not find examples that could guide us towards writing an implementation of HWF.
We try to simulate the creation of a constant at inference time. In the following code, we attempt to write the program for MNIST Sum-3 by adding the first two digits in one LTN function, and then adding the result to the third digit, as follows (where the MNIST model is represented as model:
### Predicates
Digit- = 1tn. Predicate. Seaitte.Froomooog FromLogits ts (medel, activation wortefffnctiioooo " "softmax")
### Variables
dl 1tn. Variablee ("digitsi", range )10))
5 d2 itn.varialle (a ("digits2", range )10))
d3 Vtn.varialle (""ariaaleediiiits range (10))
### Operators
Not
And-
10 Or Stn.Wrapper:
11 Implies = Itn.
())
12 Forall 1tn. NrapNerrpprrr_rr htg.pppp...uattoooooo.o...oaaorr http://....aaaoooooooooooooooorr
http:e/etw...taaooooooooeeeaeeee () scmantics="oxlsta""
www.reg () semantics="foral")
13 Exists
14
15 # mask
16 add = 1tn. Can.Function Lambda (lambda inputs: inputs [i]+
17 equals Itn. Predi cate. Lamoda (lambda inputs: inputs [0] == inputs [1])
18
22

---

DOLPHIN: A Programmable Framework for Scalable Neurosymbolie Learning
19
MFF Axioms
20
@t function
21
def axioms (images_x, images_y, images_M, labels_2, D_schedulerrf.consttart )2.)):
images_x = ltn, Var.aaaabble""" images_x)
images_y 1tn. Variable ("y"" images_y)
24
images_s 1tn, itn.Variable"".. ("w", images_M)
25
labele_z 1tn. Variable labels_z)
26
diagonal = dtn. (images_X, x, images_y, images_x, labels_z)
formula (nd
28
Digit imagees__, d1]),
29
And
10
d2]),
( d3])
33
34
exists Exists(
35
d21,d2, d3), formula,
37
mask=oquals [add ([add( [dl, d2]), d31), labels_z)) s_2)),
38
p-p_schedule
39
axiom = Forall(
diagenal,
42
exists, p=2
43
44
sat axicm. tensor
return sat
Here, rather than adding all three digits d1, d2, and d3, we write the following mask: equals ( ([add [add [dl, d2]) d3]). However, this line results in a typecheck error, since functions can only take terms, but add ( [[d1, d2]) returns a formula.
Given the lack of resources on such functionalities in LTN both in the paper, tutorials, and the examples their repository, we were not able write an implementation of HWF evaluate DOLPHIN against.
23


Reviews:
[{"review_id": "VL3UsFgTXs", "reviewer": "Reviewer_jjdh", "summary": "This paper brings enhanced scalability to neurosymbolic learning.  The authors introduce a framework which allows symbolic computation to be conducted on the CPU, and allows vectorized computation of probabilities on the GPU.  The authors introduce several Pythonic programming primitives to facilitate writing neurosymbolic programs .  The tool allows pluggable support for different vectorized provenances to compute symbolic gradients.  The empirical results demonstrates that, in general, DOLPHIN is much more efficient than SoTA tools, and achieves comparable, if not better, accuracy on various neurosymbolic programming tasks.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their suggestions. We will add the unit of time in the caption of Table 2 and fix the grammatical errors they pointed out.\n\nWe indicate the provenance used for each benchmark in Section 4.4 (RQ2: Accuracy) and compare both provenances in Section 4.5 (RQ3: Provenance Comparisons).\n\nThe Dolphin CLUTRR program is designed for batched computations, unlike Scallop, which evaluates each sample independently. Moreover, the Scallop programs for CLUTRR-3 and CLUTRR-4 exhibit higher accuracy variance, indicating possible minor numerical issues or nondeterminism. We believe these factors account for the small (~2% pts) accuracy difference.", "claims_and_evidence": "The claims are well-supported by the methodology and the experiments.", "methods_and_evaluation_criteria": "The proposed method and evaluation makes sense for the task at hand.", "theoretical_claims": "N/A – There were no proofs to check.", "experimental_designs_or_analyses": "I did not see any issues with the soundness of the experimental designs or analyses.", "relation_to_broader_scientific_literature": "The authors present a solution to a known issue (the scalability issue) in neurosymbolic programming, demonstrating superior performance to SoTA techniques in the literature.", "essential_references_not_discussed": "Not that I am aware of.", "other_strengths_and_weaknesses": "Strengths:\n- The paper addresses an important problem – making neurosymbolic learning more scalable.  The framework is elegant and clean, and is - based on effective and impactful programming paradigms.\n- The experimental results are thorough, and demonstrate superior performance of DOLPHIN over SoTA work.\n- The paper provides fertile ground for new work and acknowledges certain limitations of their framework.\n- The paper is very well-written.  Even as someone who is not as familiar with neurosymbolic programming, I was able to easily grasp the - -concepts presented in the paper.  The appendices provide useful background information for the reader.\n\nWeaknesses\n- I do not see any major weaknesses in the paper within the scope of the work presented.", "other_comments_or_suggestions": "Minor Comments/Questions\n1. Table 2 – it would be helpful to add the unit of time in the caption, or in the main body of the text.\n\n2. Section 4.4 – The authors mention they “use” a certain provenance depending on the benchmark, but given the preceding sentence, does this mean that they report the corresponding provenance’s result for each dataset?\n\n3. Do the authors have any conjectures about why DOLPHIN’s accuracy is slightly lower than Scallop in the CLUTTR benchmark?\n\n4. Grammatical error (lines 107-108) – “enable to efficiently compute symbolic gradients” → “enable the efficient computation of symbolic gradients”.", "questions_for_authors": "I have no major questions that will affect my review.  I have a few minor questions, which I wrote in \"Other Comments Or Suggestions\".", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8a6YtXRqQO", "reviewer": "Reviewer_dh9r", "summary": "This work presents DOLPHIN, a Python library that allows for efficient training of traditional neurosymbolic methods on CPUs and GPUs. The key idea is to accelerate probabilistic symbolic manipulations on GPUs, where other solutions (e.g., Scallop) rely on manipulations on CPUs. \n\nIn addition to the efforts in defining a Python library with abstractions and operations, DOLPHIN’s main conceptual contribution is a new provenance semiring, Differentiable Top-k Proofs with Add-Mult (DTKP-AM), which is a GPU-friendly vectorized approximation of the weighted model counting (WMC) version of DTKP. \n\nExperimental results confirm that performing probabilistic symbolic manipulations on GPUs is indeed beneficial. Moreover, the new DTKP-AM provenance can improve the accuracy on one benchmark (HWF), while it performs on par (Path, CLUTRR, and Mugen) or worse (SumN) than a baseline provenance (DAMP).", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for suggestions on experiment design and additional literature. We will add the results from additional experiments discussed below in the paper and expand the related work.\n# Scallop’s Performance on Mugen\nScallop’s Mugen results were obtained from their PLDI’23 artifact. Despite significant efforts, we were unable to reproduce those results, thus, we reported the numbers we observed. We reached out to the Scallop authors, who provided us with a different symbolic program that reproduced their results using DTKP-WMC (K=5). We also redesigned the Dolphin program for Mugen to be more batchable across samples, which we run with DTKP-AM (K=5):\nVersion|Framework|VTR|TVR|Total Time|Time per Epoch\n-|-|-|-|-|-\n1k|Dolphin|89.43|91.83|2.39e3|92.3\n||Scallop|86.26|89.57|6.71e3|314.68\n5k|Dolphin|95.03|95.36|1.15e4|470.05\n||Scallop|91.26|94.03|3.59e4|1.58e3\n\nWhile Scallop now converges, Dolphin still achieves higher accuracies and trains ~3.3x faster than Scallop.\n# Experiment Results without a Strict Timeout\nWe ran the experiments until Scallop converged and report its accuracies. We will do the same for the other baselines in the revised version.\nBenchmark|Dolphin Total Time|Dolphin Accuracy|Scallop Total Time|Scallop Accuracy\n-|-|-|-|-\nHWF-15|9.78e3|92.13|1.66e5|39.58\nHWF-19|1.63e4|86.18|1.82e5|7.02\nPath-256|1.94e4|82.62|1.14e5|83.01\nPath-128|1.78e4|84.03|4.17e4|84.90\n\nFor Path-128 and 256, Scallop converges to Dolphin’s accuracies in \\~11.6 and \\~31.7 hours (\\~2.4x and \\~6x slower), respectively. For HWF-15, Scallop reaches 100% on only 2 seeds out of 6, but stays below 12.5% on the rest even after \\~46 hours of training on average. For HWF-19, none of the 6 runs were able to converge after ~50 hours of training on average.\n# DTKP-AM with Different Values of K\nWe report preliminary results of this experiment. As a general trend, with the exception of HWF-19, the value of K doesn’t have much impact on the final accuracy (Acc). Increasing K also does not significantly impact the per epoch time (T/ep) due to DTKP-AM's vectorizations.\nBenchmark|K=1 Acc|K=1 T/ep|K=3 Acc|K=3 T/ep|K=5 Acc|K=5 T/ep|K=7 Acc|K=7 T/ep\n-|-|-|-|-|-|-|-|-\nSum-15|9.61|37.21|10.81|47.70|10.51|53.52|10.21|58.54\nHWF-19|8.94|1.21e3|99.15|1.40e3|96.89|1.33e3|95.75|1.46e3\nPath-256|81.39|1.97e3|82.14|2.34e3|80.86|2.10e3|82.38|2.12e3\nCLUTRR-4|53.62|240.50|48.52|257.89|50.35|261.31|48.17|257.99\nMugen-5K|(94.1/95.7)|460.38|(95.4/95.7)|464.68|(95.3/95.4)|470.05|(95.4/95.4)|465.10\n# Benefits of DTKP-AM\nEmpirically, DTKP-AM offers an advantage over DAMP of ~2% pts on average across the complex versions of all tasks. It significantly outperforms DAMP by ~50% pts on HWF-15 and ~75% pts on HWF-19, while also yielding up to ~5% pts higher accuracy on Mugen and ~3% pts improvement on PathFinder.\n# Model Details\nFor HWF/MNIST, we use the same CNN architecture as Scallop (will be added in Appendix D). For CLUTRR, we use Scallop’s Roberta configuration: a pretrained model (roberta-base) finetuned while training the classification head.\n# Symbolic Computations on the GPU\nPrecompiling symbolic computations on the GPU poses a few challenges. First, it restricts symbolic programs to PyTorch tensor operations, a small subset of the Python functions Dolphin currently supports. Second, since we perform one set of CPU computations per batch (~16% of the train time per batch), there is limited scope for improving the training time. The parallelism will only occur across combinations of symbols, not over samples in a batch. Third, enumerating all possible combinations of symbols on the GPU could pose memory consumption issues on complex tasks, as seen with LTN.\n\nOne could potentially compile symbolic computations on the GPU by representing symbols as GPU tensors and implementing user-defined functions as tensor operations since Dolphin supports arbitrary Python objects. We tested this strategy on Sum-5, but it did not show any improvements in training time, and was 2 seconds slower to train per epoch. We consider developing efficient strategies for compiling symbolic computations directly onto GPUs an exciting direction for future research.\n# Related Work\nThank you for highlighting these related papers. KLAY (ICLR’25) lacks released code, preventing direct comparison. It compiles symbolic and probabilistic computations into GPU operations by transforming arithmetic circuits from Boolean logic into layered tensor structures. In contrast, Dolphin separates CPU-based symbolic reasoning from GPU-based probabilistic computations. The paper also omits comparisons against LTN or Scallop on Dolphin’s benchmarks.\nDeepSoftLog improves NTP by using probabilistic semantics instead of fuzzy semantics, enabling non-redundant proofs, well-defined proof scores, and non-sparse gradients. A-NESI uses learned neural models to approximate the exact probabilistic semantics of WMC, boosting scalability. We will include these works in the related work section.", "claims_and_evidence": "**Experimental design with tight timeout:**\nI am skeptical concerning the comparison of accuracy and training time (until convergence) for different methods, given the very tight training time budget constraint (10 hours) on a consumer-grade GPU (NVIDIA GeForce RTX 2080 Ti). The accuracy comparison in Figure 5 can be seen as unfair since some of the methods have not been trained until convergence (e.g., Scallop on HWF-15/19). Disentangling training time from accuracy would give better insights, e.g., by comparing the time per epoch, the time to convergence without a hard timeout constraint, and the accuracy at convergence. \n\n**No clear global benefit for DTKP-AM**\nOne of the main contributions of this paper, DTKP-AM, does not have a clear benefit across different benchmarks. Indeed, the results presented in the main paper primarily show another provenance semiring (DAMP). DTKP-AM is only beneficial in the HWF benchmark (see Figure 10). This could be due to the very low top-k used (k=1 for all benchmarks). The use of such low k might have been chosen to achieve competitive timing results compared to DAMP.", "methods_and_evaluation_criteria": "See comments on “Claims and Evidence” regarding **Experimental design with tight timeout.**", "theoretical_claims": "This paper does not contain any proofs or theoretical claims.", "experimental_designs_or_analyses": "See comments on “Claims and Evidence” regarding **Experimental design with tight timeout**", "relation_to_broader_scientific_literature": "Accelerating probabilistic manipulations in neurosymbolic models is an important problem to make these methods usable in practice. At least the introduction of DOLPHIN as a Python framework for performing probabilistic manipulations on GPUs is of value. However, the second contribution (DTKP-AM) does not have a clear benefit in practice (see my comments on “Claims and Evidence”).", "essential_references_not_discussed": "The following work accelerates arithmetic circuits (i.e., WMC) on GPUs. Hence, it should be discussed and possibly compared in this paper: \n\nJaron Maene and Vincent Derkinderen and Pedro Zuidberg Dos Martires, “KLay: Accelerating Arithmetic Circuits for Neurosymbolic AI,” ICLR, 2025. \n\nMoreover, there are other works that introduce approximations for probabilistic manipulations: \n\nJaron Maene and Luc De Raedt, “Soft-Unification in Deep Probabilistic Logic,” NeurIPS 2023. \nEmile van Krieken et al., “A-NESI: A Scalable Approximate Method for Probabilistic Neurosymbolic Inference,” NeurIPS 2023.", "other_strengths_and_weaknesses": "The introduction of the general “apply” function is appreciated, as goes beyond simple additive relations usually used in neurosymbolic benchmarking.", "other_comments_or_suggestions": "Missing the date in the reference «Kambhampati, S., Valmeekam, K., Guan, L., Verma, M., Stechly, K., Bhambri, S., Saldyt, L. P., and Murthy, A. B. Position: Llms can’t plan, but can help planning in llm modulo frameworks. In Forty-first International Conference on Machine Learning.» \n\nIt would be good to add a reference when introducing Differentiable Add-Mult Probabilities. \n\nThe caption of Table 2 should mention that the presented numbers are in seconds. \n\nPlease provide more details on the chosen networks in Appendix D. E.g., the exact CNN architecture is not mentioned in D.2. Also, does the work use a pretrained network? Moreover, it is not clear what is trainable in D.5 regarding Roberta-base: do you train only the classification head? Do you use a pretrained network?", "questions_for_authors": "More experimental results without strict timeout constraints for a fair accuracy comparison would be appreciated (see comments on “Claims and Evidence” regarding **Experimental design with tight timeout**)\n\nHow does the accuracy and timing change for DTKP-AM when increasing k? \n\nIt seems that still a fair share of the compute time is spent on the CPU to perform symbolic manipulations (once for each batch). Wouldn’t it be possible to precompile the symbolic program and run everything on the GPU? \n\nThe results of the Scallop baseline on Mugen reported in Figure 5 (and Figures 8&9 in the appendix) are significantly worse than what the original work (Li et al. 2023) show in their paper. How serious were the efforts in reproducing that work? I would appreciate comments about the reason for this large discrepancy in the reproduction, beyond what is mentioned in Appendix D.7.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SPaBSQ0phJ", "reviewer": "Reviewer_Vu2p", "summary": "This paper introduces Dolphin, a novel neurosymbolic learning framework. Dolphin provides three key abstractions: symbolic objects, tags (which associate symbols with tensors), and distributions (which map symbols to probabilities). Leveraging this abstraction, Dolphin decouples symbolic and probabilistic computations, enabling vectorized probabilistic computation on GPUs. This allows Dolphin to efficiently handle large-scale batched data. To ensure end-to-end differentiability, Dolphin employs vectorized provenance semirings, enabling parallel gradient computation on GPUs. It also provides two customizable provenance mechanisms, DAMP and DTKP, allowing users to fine-tune symbolic differentiation. Dolphin introduces five core operations that facilitate batched data processing, complex control flows, and recursion. Furthermore, its seamless integration with Python and PyTorch allows users to easily develop and deploy complex neurosymbolic programs with flexibility. Experimental results across 13 neurosymbolic tasks demonstrate that Dolphin significantly improves computational efficiency while maintaining state-of-the-art accuracy, outperforming existing methods in scalability and performance.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their suggestions and will add the discussions to the revised paper.\n\n# Discussion on other uses of Dolphin\n\nDolphin can be used for any task where the output of a model can be cast as a distribution over probabilities, including discriminative models. Consider an example of autonomous driving, where an object detector is used to detect obstacles on the road. Commonly used models like Faster R-CNN (cite) output bounding boxes, class probabilities, and confidence scores for multiple objects in an image. One can create a custom Python class that associates the coordinates of each bounding box with a Distribution object over the classes and their probabilities output by the model:\n\n```python\n\nCLASSES = [‘car’, ‘person’, ‘shirt’, …]\nclass DetectedObject\n    def __init__(self, coords, score, class_logits):\n        self.coords = coords\n        self.score = score\n        self.distr = Distribution(CLASSES, class_logits)\n```\n\nOne can then derive the probability that given a pair of objects, it represents a person inside a car:\n\n```python\n# function to check if one set of coordinates is inside another\ndef is_inside(coord_a, coord_b):\n    …\n\ndef person_inside_car = apply(o1.distr, o2.distr,\n    lambda c1, c2: c1 == “person” and c2 == “car” and is_inside(o1.coords, o2.coords))\n```\n\nHere, `person_inside_car` will be a distribution over “True” and “False”, giving the probability that detected objects `o1` and `o2` represent a person inside a car.\n\n# Explanation of Operations\n\nWe describe the Filter and Union operations in more detail:\n\n**Filter.** In the Filter operation, the symbols that do not satisfy the filtering condition are completely removed from the returned distribution. So consider a Distribution $$D :: \\\\{ 0 \\rightarrow t_1, 1 \\rightarrow t_2, 2 \\rightarrow t_3\\\\},$$ with a filtering condition `lambda x: x % 2 == 0` which removes odd numbers from the distribution. The returned Distribution will be $$D_\\text{filtered} :: \\\\{ 0 \\rightarrow t_1, 2 \\rightarrow t_3 \\\\}.$$\n\n**Union.** In the Union operation, a new Distribution is returned that contains the symbols from both input Distributions. A Union can occur over any pair of Distributions regardless of the types of symbols. For instance, consider as inputs Distributions $$ D :: \\\\{0 \\rightarrow t_1, 1 \\rightarrow t_2, 2 \\rightarrow t_3\\\\} \\text{ and } D' :: \\\\{1 \\rightarrow t\\'\\_1 , 4 \\rightarrow t\\'\\_2\\\\}.$$\nThe output Distribution will be\n$$D_\\text{union} :: \\\\{0 \\rightarrow t_1, 1 \\rightarrow t_2 \\oplus t'_1, 2 \\rightarrow t_3, 4 \\rightarrow t'_2\\\\}.$$\nNote here that the tags for common symbols are disjuncted.\n\n# Restricted to Semiring Structures Over Tags\nWhile symbolic operations in Dolphin are indeed defined using semiring structures over tags—similar to existing frameworks such as Scallop—this abstraction is primarily introduced to facilitate efficient computations, including GPU acceleration, and has been shown to be quite flexible for solving neurosymbolic tasks.", "claims_and_evidence": "Yes, the claims are well supported.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "There are no theoretical claims in this paper.", "experimental_designs_or_analyses": "Yes.", "relation_to_broader_scientific_literature": "This paper is related to the neurosymbolic programming research. The batch process of tags is largely related to MapReduce. The authors already provide detailed discussion of these backgrounds in the paper.", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strengths\n1. Dolphin provides a practical and user-friendly solution for neurosymbolic programming. The GPU acceleration addresses a key limitation of previous methods like Scallop, offering a computationally efficient approach with just five core operations. Its seamless integration with Python and PyTorch significantly enhances usability and flexibility, making it more accessible for practitioners. Additionally, the provenance semiring framework provides a strong and elegant theoretical foundation for the proposed method.\n2. The paper is well-written, easy to follow, and presents a thorough background analysis. It clearly explains the limitations of previous approaches and justifies the need for Dolphin, making it an engaging and informative read.\n\nWeaknesses\n1. The symbolic operations are restricted to semiring structures over tags, which may limit the expressiveness of symbolic programs. While this abstraction enables efficient GPU computation, it could potentially restrict the range of symbolic reasoning tasks that can be expressed within the framework. \n2. The benchmarks primarily focus on relatively simple tasks and do not demonstrate Dolphin’s applicability to real-world, complex scenarios. The paper does not explore how Dolphin could be scaled to more sophisticated domains, such as robotics or autonomous driving, which may involve complex logical reasoning and control modules, as well as unstructured outputs from deep models (e.g., object detectors with noisy or non-standard outputs), which is not thoroughly discussed.\n\nOverall, this is an excellent paper that presents an elegant solution to the key challenges of previous methods, particularly in efficiency and complexity. It brings neurosymbolic programming much closer to practical usability, a promising paradigm with the potential for significant impact on machine learning. Its seamless integration with Python and PyTorch further enhances its compatibility with existing deep learning pipelines. Thus, it would be exciting if the authors could further illustrate its potential to be really applied for large-scale applications.", "other_comments_or_suggestions": "1. It would be good to provide a simple example and more details about how the data are changing for each of the 5 operations. For example, in Union, what is the exact format of the returned data? Does it produce a list of tuples with tagged tensors? In Filter, what happens to the symbols that are filtered out? Do they become empty placeholders, removed entirely, or assigned a default value?", "questions_for_authors": "1. The limitation parts have discussed that Dolphin can only deal with discriminative models. However, there are many non-standard discriminative models with multiple heads and complicated outputs, such as object detectors which are common in application domains. It would be good if the authors could provide a discussion or case studies on how Dolphin can be used with popular discriminative models in typical application domains. It will be very useful for practitioner who wants to use Dolphin in real-world scenarios.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper introduces Dolphin, a programmable framework designed to enhance the scalability of neurosymbolic learning by integrating symbolic reasoning with deep learning models. Dolphin addresses the limitations of existing frameworks by enabling neurosymbolic programs to be written in Python and executed efficiently using PyTorch. It achieves this by mapping both forward symbolic reasoning and backward gradient propagation to vectorized computations, allowing for end-to-end differentiable symbolic programs that can leverage GPU acceleration. All reviewers agree that this is interesting an valuable work. I fully agree. Since the paper is about scalability and Scallop is one of the baselines, the authors should also discuss

Arseny Skryagin, Daniel Ochs, Devendra Singh Dhami, Kristian Kersting. Scalable Neural-Probabilistic Answer Set Programming. Journal of Artificial Intelligence Research (JAIR) 78:579–617, 2023

in the camera-ready version.
Decision: Accept (poster)