Dolphin: A Programmable Framework for Scalable Neurosymbolic Learning

## Abstract

Abstract Neurosymbolic learning has emerged as a promising paradigm to incorporate
symbolic reasoning into deep learning models.
However, existing frameworks are limited in scalability with respect to both
the training data and the complexity of symbolic programs.
We propose Dolphin , a framework to scale neurosymbolic learning at a fundamental level by mapping both forward chaining and backward gradient propagation in symbolic programs
to vectorized computations.
For this purpose, Dolphin introduces a set of abstractions and primitives
built directly on top of a high-performance deep learning framework like
PyTorch, effectively enabling symbolic programs to be written as PyTorch modules.
It thereby enables neurosymbolic programs to be written in a language like Python that is familiar to developers and compile them to computation graphs that are amenable to end-to-end differentiation on GPUs.
We evaluate Dolphin on a suite of 13 benchmarks across 5 neurosymbolic tasks that combine deep learning models for
text, image, or video processing with symbolic programs that involve multi-hop
reasoning, recursion, and even black-box functions like Python eval() . Dolphin only takes 0.33 0.33 0.33 0.33 %- 37.17 37.17 37.17 37.17 % of the time (and 2.77 2.77 2.77 2.77 % on average) to train these models on the largest input per task compared to baselines Scallop, ISED, and IndeCateR+, which time out on most of these inputs.
Models written in Dolphin also achieve state-of-the-art accuracies even on the largest benchmarks.

## 1 Introduction

Deep learning has achieved remarkable success on a wide range of tasks such as image classification, speech
recognition, and natural language processing. With the recent emergence of foundation models like GPT-4 and CLIP, deep learning models are now increasingly being used in more challenging tasks. While such models work well for prediction and generation tasks, they are limited in their ability to perform reasoning required for tasks involving structure, logic, and planning, where symbolic reasoning traditionally excels (Kambhampati et al., ).
Neurosymbolic programming (Chaudhuri et al., ) has emerged as a promising paradigm to incorporate symbolic reasoning into deep learning models, providing the best of both worlds.

Various frameworks have been developed to improve the programmability and accessibility of neurosymbolic applications (Manhaeve et al., ; Li et al., ; Solko-Breslin et al., ).
These frameworks support complex symbolic reasoning features like recursion and black-box functions, implement efficient differentiable reasoning algorithms, and provide bindings for deep learning frameworks like PyTorch.
Despite significant strides, however, these frameworks incur significant overhead
when training neurosymbolic programs.

Consider a typical workflow of such a framework depicted in Figure 1(a).
In this case, we have a supervised learning task with labeled data $(x,y)$, a neural network $M_{\theta}$ that processes input $x$, and a symbolic program $P_{\tt symbolic}$ that takes the network’s output $r$ and produces final output $y$.
Existing frameworks, such as Scallop (Li et al., ), execute the neural model on GPU but use a separate CPU-based backend (implemented in Rust in Scallop’s case) for the
symbolic program. Moreover, such systems suffer from slow inter-process
data transfers between the two environments.

Figure: (a) Typical neurosymbolic framework (e.g. Scallop).
Refer to caption: x1.png

Together, these issues hinder the scalability of neurosymbolic learning with respect to problem complexity
and data complexity.
First, the symbolic computation engine must derive a set of all possible results and their associated probabilities in a manner that is differentiable with respect to the network’s parameters $\theta$.
As the complexity of the symbolic program increases, the number of possible results and their associated weights also grows exponentially, leading to a combinatorial explosion in the number of required computations.
However, the symbolic computations are discrete and not easily parallelizable on modern hardware like GPUs.
Second, larger datasets also compound the computational cost of neurosymbolic learning. Deep learning typically addresses this challenge by batching computations across multiple data samples.
However, in neurosymbolic learning, the computations may differ across data samples, making it difficult to batch them effectively.

To address these challenges, we need to fundamentally rethink the design of a neurosymbolic framework. One approach is to develop specialized and low-level primitives that scale specific benchmarks but make it time-intensive for developers to write neurosymbolic programs tailored to particular tasks. Alternatively, providing high-level primitives—such as a logic programming language like Scallop (Li et al., ) or DeepProbLog (Manhaeve et al., )—simplifies the development of symbolic programs but limits the fine-grained control needed to scale specific applications. Finally, to truly democratize neurosymbolic programming, it is crucial to develop a framework that seamlessly integrates into the everyday deep learning workflows that developers already use.

In this work, we propose Dolphin, a novel framework for scalable neurosymbolic learning. In Dolphin, we build three key components that effectively tackle the scalability and programmability challenges described above. First, we develop a general symbolic representation that efficiently captures the relationships between neural network outputs and associated discrete symbols.
Second, we introduce a set of primitives to map forward chaining in symbolic programs to vectorized computations over these representations.
Third, we develop a set of vectorized provenance semirings (Green et al., ) that are easily pluggable into Dolphin and enable to efficiently compute symbolic gradients.
As illustrated in Figure 1(b), these components together allow Dolphin to build a computation graph that spans both symbolic and neural operations, is highly parallelizable, and end-to-end differentiable on GPUs. Finally, Dolphin is implemented as a library that is integrated with PyTorch, allowing users to easily incorporate it into their
existing deep learning pipelines.

We evaluate Dolphin on a diverse set of neurosymbolic tasks that involve text, image, video, and multi-modal data, and use rich reasoning features such as recursion and black-box Python functions.
Neurosymbolic programs written using Dolphin only require $0.33$%-$37.17$% ($2.77$% on average) of the time to train compared to state-of-the-art baselines including differentiable reasoning frameworks like Scallop, and sampling-based frameworks like ISED and IndeCateR+ while maintaining similar levels of accuracy.
We also observe that Dolphin efficiently scales to
more complex benchmarks and larger datasets whereas the baselines either time out after 10 hours or fail to converge.

We make the following contributions in this work:

- •
We propose Dolphin, a novel neurosymbolic programming framework for end-to-end differentiable symbolic reasoning in a scalable manner.
- •
We develop novel abstractions to represent symbolic and neural computations and introduce vectorized primitives for neurosymbolic programs.
- •
We develop vectorized provenances that can be plugged into Dolphin for efficient computation of symbolic
gradients on parallelizable hardware such as GPUs.
- •
We evaluate Dolphin on a diverse range of challenging neurosymbolic tasks across different domains and show that it effectively scales with increasing problem complexity and dataset size.

## 2 Overview

We illustrate our approach using the MNIST Sum-$N$ task from (De Smet et al., ).
The goal is to train a model that takes as input $N$ images of MNIST digits and returns the sum of the digits represented by the images.
During learning, supervision is provided only on the sum instead of the labels of the digits. The difficulty of the problem scales exponentially as there are $10^{N}$ states in the input space. Further, there are only $9N+1$ possible labels, resulting in very sparse supervision.

Figure: Figure 3: Formal definition of Dolphin’s programming abstractions and primitives.
Refer to caption: x3.png

## 3 The Dolphin Framework

### 3.1 Dolphin Core Design Principles

We based Dolphin’s framework design on the following core principles:

- •
Flexible programmability: The framework should allow developers to write neurosymbolic applications in Python with minimal effort, providing intuitive primitives that seamlessly integrate with Python’s rich and expressive language features.
- •
End-to-end differentiability on GPUs: The framework should allow any neurosymbolic program to be end-to-end differentiable on GPUs irrespective of the task characteristics.
- •
Scalable: The framework should easily scale with greater problem and data complexity.
- •
Tunable: Similar to hyperparameters in deep learning, the framework should provide a simple interface for developers to choose provenances (and their configurations) or define new ones.

Next, we show how we realize these principles by describing the key components of Dolphin.

### 3.2 The Dolphin Syntax

Dolphin provides a programming interface that developers can use to express symbolic programs in a Pythonic manner.
Dolphin maps each operation of the symbolic program to PyTorch which enables end-to-end GPU-accelerated differentiable reasoning.
Figure 3 presents Dolphin’s programming interface including the symbolic abstractions and operations over them.

Figure: Figure 3: Formal definition of Dolphin’s programming abstractions and primitives.

#### 3.2.1 Abstractions

The three main abstractions provided by Dolphin for expressing differentiable symbolic programs are shown on the left of Figure 3.
*Symbols* $S$ represent symbolic entities relevant to the program.
These entities can be any Pythonic object, such as hand-written digits in MNIST-SumN or coordinates of points in PathFinder.
*Tags* $T$ are tensors that represent relative likelihoods.
Typically, tags for symbols are derived from the outputs of machine learning models, such as the probability distribution over digits produced by the CNN classifier in MNIST-SumN.
Finally, *Distribution* $D$ represents the likelihood of an input being classified as one of the pre-defined symbols.

Distributions serve as the fundamental datatype of a Dolphin program and act as its main interface with a machine learning model.
For instance, when the developer instantiates a Distribution object, such as in the following code snippet from Figure LABEL:fig:mnist_sum_n_code:

$$ $\texttt{D}_{\text{res}}$ = Distribution(self.CNN(imgs[0]), digits) $$

the output of the CNN model is directly passed to the Distribution object, effectively acting as an input to the symbolic program.
The Distribution object itself, as shown in Figure LABEL:fig:mnist_sum_breakdown, contains batches of tags extracted from the model outputs, and maintains the set of corresponding symbols.

To enable such a seamless integration between the PyTorch model and the symbolic program, Distributions are designed to
operate directly over PyTorch tensors.
This has two main advantages.
First, it preserves the gradients of the model output throughout the symbolic program, enabling end-to-end differentiability.
Second, it allows Dolphin to perform operations over an entire batch of tags, leveraging the vectorized operations provided by PyTorch.
Dolphin can thus operate efficiently on specialized hardware like GPUs, allowing the symbolic program to scale effectively.

#### 3.2.2 Operations

Figure 3 shows the five operations supported by Dolphin that developers can use to manipulate Distributions and express complex symbolic programs. We now expand on these operations.

##### Apply .

This is the primary operation that developers can use to manipulate Distributions.
It takes as inputs $K$ Distributions, where $K\geq 1$, along with a function $f$ of the same arity.
This function defines operations over the symbols of $K$ distributions.
Apply then computes the results of $f$ over all possible combinations of arguments sourced from the symbols of
the Distributions as well as their associated tags, and returns a new Distribution with these results and tags.

This operation occurs in two stages akin to the popular map-reduce pattern.
In the map stage, Apply computes the results of $f$ over the symbols of the input Distributions and their associated tags:

$$ $R\ =\ \{\ (f(s_{1},s_{2},\ldots,s_{k}),(t_{1}\otimes t_{2}\otimes\ldots\otimes t _{k}))\ |\ D_{i}(s_{i})=t_{i},i=1,\ldots,k\ \}$ (1) $$

Here, the tag of each result symbol $f(s_{1},s_{2},\ldots,s_{k})$ is the conjunction $\otimes$ of the tags $(t_{1},t_{2},\ldots,t_{k})$ of the input symbols it was derived from.

Note that the function $f$ can be any user-defined Python function, including complex functions like regex parsing, image processing, or Python’s eval().
As a result, $f$ may be a many-to-one function and the tags shared by a resulting symbol must be aggregated
to form the final tags of the output Distribution.
We, therefore, shuffle the results from the map stage to compute a function $M$ from each symbol to tags from $R$ associated with it:

$$ $M=\lambda\ s\ .\ \{\ t\ |\ (s,t)\in R\ \}$ (2) $$

We then proceed to the reduce stage, where we aggregate the tags of each symbol in $M$ using disjunction $\oplus$ to produce the final Distribution $D_{\text{res}}$:

$$ $D_{\text{res}}=\lambda\ s\ .\ \bigoplus\ \{\ t\ |\ t\in M(s)\ \}$ (3) $$

Since the tags here are PyTorch tensors representing probabilities, the implementations of the conjunction and disjunction
operations are dictated by the underlying provenance used by the program.
A more detailed explanation of the provenances is provided in Section 3.3.

Filter.
The Filter operation is used to filter out symbols from a Distribution based on some condition.
It takes in a single Distribution, along with a user-defined function that returns a boolean value, which acts as the condition.
This operation then returns a new Distribution that contains only the symbols that satisfy the condition, along with their tags.

ApplyIf.
The ApplyIf operation is a conditional version of Apply.
It takes in $K$ Distributions and two functions $f_{\textit{apply}}$ and $f_{\textit{cond}}$ of the same arity.
For each combination of symbols from the $K$ Distributions, ApplyIf computes the results of $f_{\textit{apply}}$ and
its associated tags only if the condition $f_{\textit{cond}}$ is satisfied over that combination of symbols.
The operation then returns a new Distribution with these results and tags.

Union.
The Union operation is used to combine two Distributions.
It takes in two Distributions and returns a new Distribution that contains the union of the symbols from the two input Distributions, along with their tags.
Any symbols common to both input Distributions have their tags merged via a disjunction operation.
Union is especially useful when writing recursive programs in Dolphin that require combining the results of multiple recursive calls.

GetProbs.
The GetProbs operation is used to extract the probabilities from the tags of a Distribution.
This is used mainly once the symbolic program has been executed to extract the final probabilities of the symbols in the output Distribution.
These probabilities can then be used to compute the loss function for training the neural components of the neurosymbolic model.
The actual extraction of the probabilities from the tags depends on the specific provenance used in the program, which we discuss next.

### 3.3 Dolphin Provenances

As discussed earlier, each symbol in a distribution is associated with a batch of one or more tags.
The Dolphin primitives define how to manipulate certain tags.
For instance, Equations (1) and (3) specify the tags to be conjuncted or disjuncted.
We now define the semantics of these operations.

The goal of such operations is to approximate the probabilities of the symbols in the output distribution as accurately as possible.
This is achieved by using a mathematical framework called provenance semirings (Green et al., ).
Provenance semirings provide generalized algebraic structure to propagate tags when computing over tagged data.
In the case of Dolphin distributions, we can view the tags as representing the probabilities, and the data as the distribution’s symbols.

Designing and implementing provenances can be challenging since they must be accurate enough to capture the semantics
of the symbolic program, while at the same time being coarse enough to maintain computational feasibility.
Furthermore, the provenances must be differentiable to enable end-to-end training for neurosymbolic tasks.
While neurosymbolic frameworks like Scallop (Li et al., ) implement differentiable provenances, they
are not designed to leverage hardware accelerations or batched optimizations due to the CPU-bound nature of their implementations.
We thus design vectorized provenances in Dolphin that are differentiable and enable GPU computations.

We simplify the definition of provenances from Scallop as a 5-tuple: $(T,\mathbf{0},\mathbf{1},\otimes,\oplus)$.
Here, $T$ is the tag space, $\otimes:T\times T\to T$ is the conjunction operator with identity $\mathbf{0}$,
and $\oplus:T\times T\to T$ is the disjunction operator with identity $\mathbf{1}$.
We then implement two differentiable provenances in Dolphin: Differentiable Add-Mult Probabilities (DAMP) and
Differentiable Top-K Proofs (DTKP).
Table 1 summarizes the operations of these provenances.

**Table 1: Dolphin provenances implemented in PyTorch.**
| Provenance | Domain | $\mathbf{0}$ | $\mathbf{1}$ | $t\oplus t^{\prime}$ | $t\otimes t^{\prime}$ |
| --- | --- | --- | --- | --- | --- |
| DAMP | $[0,1]$ | 0 | 1 | $\text{clamp}_{0}^{1}(t+t^{\prime})$ | $t\cdot t^{\prime}$ |
| DTKP-AM | $[0,1]\cup\{\infty,-\infty\}$ | $\mathbf{\hat{0}}_{ij}=\begin{cases}\infty&i=1\\<br>-\infty&i>1\end{cases}$ | $\mathbf{\hat{1}}_{ij}=\infty$ | $\text{top}_{k}(\text{cat}(t,t^{\prime}))$ | $\text{top}_{k}([\min(|t_{i}|,|t^{\prime}_{j}|)\mid(t_{i},t^{\prime}_{j})\in t<br>\times t^{\prime}])$ |

Differentiable Add-Mult Probabilities.
Differentiable Add-Mult Probabilities (DAMP) is a popular technique that uses the probability space as its tag space: $T=[0,1]$.
Its conjunction operation $\otimes$ is defined as the addition of probabilities, clamped at $\mathbf{1}$,
and its disjunction operation $\oplus$ is defined as the multiplication of probabilities.
The main assumption underlying the DAMP operations is that the input Distributions are mutually exclusive and
independent.
This assumption allows DAMP to compute probabilities extremely efficiently, as the operations are simple and can be easily
vectorized.

Differentiable Top-$k$ Proofs.
Differentiable Top-$k$ Proofs (DTKP) (Huang et al., ) was proposed to overcome the shortcomings of DAMP.
This provenance tracks a set of up to $k$ proofs for each symbol.
Each proof, in turn, denotes the set of input symbols necessary to derive the output symbol.
These proofs are then used to compute the probabilities of the output symbols.

In Scallop, DTKP tags are converted into probabilities via differentiable weighted model counting (WMC).
This form of DTKP is computationally hard and is by nature difficult
to vectorize due to the varying sizes of proof sets and the WMC procedure.
We therefore design a vectorized version of DTKP, called DTKP-AM (DTKP with Add-Mult), that can be efficiently computed on GPUs.

We first define the structure of tags in DTKP-AM in a manner that conforms to the constraints of PyTorch tensors.
Each tag $t$ for a symbol $s$ is a 2-dimensional tensor of shape $(k,|I|)$, where $k$ is the maximum number of proofs to be
retained and $I$ is an ordered list of all input symbols (symbols that are present in the input Distributions).
Each row $t_{i}$ of $t$ corresponds to one of the tag’s $k$ proofs.
Each element $t_{ij}$ thus represents the probability of the $j$th input symbol in the $i$th proof:

$$ $t_{ij}=\begin{cases}p_{j}&\text{if the $j$th symbol is present in the $i$th proof}\\ \mathbf{\hat{0}}_{ij}&\text{otherwise}\end{cases}$ $$

where $p_{j}$ is the probability of the $j$th input symbol.
The probability of each proof is then computed by taking the product of the normalized elements of that proof:

$$ $\Pr(t_{i})=\prod_{j}\text{norm}(t_{ij})\quad\text{where}\quad\text{norm}(t_{ij })=\begin{cases}1&t_{ij}=+\infty\\ 0&t_{ij}=-\infty\\ t_{ij}&\text{otherwise}\end{cases}$ $$

We next define the operations of DTKP-AM in Table 1.
The $\oplus$ operation is defined as the union of two tag tensors $t$ and $t^{\prime}$ while $\otimes$ is defined as
the element-wise minimum of the absolute values of all possible combinations of proofs in $t$ and $t^{\prime}$.
In each case, the $\text{top}_{k}$ operation retains only upto $k$ proofs with the highest probabilities.

These definitions thus allow us to take advantage of the benefits of the DTKP provenance while enabling efficient computation
on GPUs.
To calculate the probability of the entire tag, DTKP-AM adds the probabilities of the individual proofs and clamps it at 1.

### 3.4 Building the Dolphin Program

The programmer specifies the neurosymbolic task using a Python program $P$ that uses Dolphin’s programming interface
to connect the neural components (e.g., neural networks) with the symbolic components and operations. We call $P$ the *symbolic program*. Because $P$ is a Python program and Dolphin interfaces with PyTorch, Dolphin supports
any PyTorch-based neural network(s), most Python language features, and custom user-defined functions. This feature
enables greater flexibility and expressiveness in neurosymbolic programs than existing frameworks.

In addition to $P$, the programmer provides one or more neural networks $M_{1},\ldots,M_{k}$, and a dataset $\mathcal{D}$. Given these inputs, Dolphin extracts the computation graph that encodes how the neural network outputs are transformed using symbolic operations to produce a final result $D_{\text{res}}$. All computations in Dolphin are expressed using distribution objects $D_{i}$. Each Dolphin primitive (e.g., Apply) takes one or more distribution objects as inputs and applies a transformation to produce another distribution object. Because each distribution object $D_{i}$ only contains vectors of tags (or probabilities), the entire computation graph (including the neural network(s))
can be ported to a GPU for faster execution.
During training, Dolphin optimizes over the standard objective function:

$$ $\phi(\theta)=\min_{\theta}\sum_{(x,y)\in\mathcal{D}}\mathcal{L}(\text{$P$}(M_{ \theta}(x)),y)$ (4) $$

Here $\mathcal{L}$ is the loss function, such as binary cross entropy. While Dolphin allows $P$ to take multiple neural networks as inputs, we show only one neural network model $M$ here for simplicity.

## 4 Experiments

We evaluate Dolphin on a set of 13 benchmarks of varying complexity and scale across 5 neurosymbolic tasks. Our evaluation addresses the following research questions:

- •
RQ1: Scalability. How does Dolphin scale to complex problems and large datasets compared to existing baselines?
- •
RQ2: Accuracy. Do models written in Dolphin converge to SOTA accuracies?
- •
RQ3: Provenance Comparisons. Which provenances are most effective for each benchmark?

### 4.1 Benchmarks

MNIST-SumN.
The MNIST-SumN (or briefly, SumN) task from (De Smet et al., ) takes as inputs $N$ handwritten digits from the MNIST dataset
and returns their sum.
We consider three versions of this task: small ($N=5$), medium ($N=10$), and large ($N=15$).
The perception model for this task is a standard CNN while the symbolic program sums the outputs of this model.

Hand-Written Formula (HWF).
The HWF task from Li et al. () takes as input a set of images of handwritten digits and arithmetic operators representing
a formula.
The task is to evaluate the formula and return the result.
We consider three versions of HWF: small (formulas of length up to 7), medium (formulas of length up to 15),
and large (formulas of length up to 19).
Similar to MNIST-SumN, the perception model for this task is a standard CNN.
The Dolphin program for this task builds strings of formulae from the outputs of the neural model and evaluates them
using Python’s eval function, demonstrating the ability of Dolphin to support black-box functions.

PathFinder.
PathFinder (or Path) from  Tay et al. () tests the ability of an agent to reason over long-range dependencies within an input image. The image consisting of two dots and a sequence of curved and dashed lines. The task is to identify whether the two dots are connected via the lines.
We consider three versions of this task based on the image size in pixels: small (32 x 32), medium
(128 x 128), and large (256 x 256).
The perception model for this task is also a standard CNN.
However, the Dolphin program for this task is recursive, since it must search for paths between the two dots.

CLUTRR.
In this task from Sinha et al. (), the input is a passage of text containing some information about several individuals and some of their relationships. The task is then to infer the relationship between two given individuals, which is not explicitly provided in the input. Hence, the Dolphin program for this task needs to reason over long chains of relationships recursively. We consider two versions of this task, where the training data contains relation chains of lengths up to 3 (small) or 4 (medium). We use Roberta-base (Liu, ) as the language model for this task.

Mugen.
In this task from Hayes et al. (), the input is a video of gameplay footage that is 3.2 seconds long and a natural language passage captioning the video.
The goal is to measure how aligned the text is with the video.
This task therefore has two variants: Mugen-TVR, where the model must retrieve the video that best aligns with the
text, and Mugen-VTR, where the model must retrieve the text that best aligns with the video.
This benchmark tests the ability of the model to reason over multimodal data. We consider two versions of this task: small, with 1000 training samples, and medium, with 5000 training samples.
We use a combination of DistilBert (Sanh, ) and S3D (De Smet et al., ) as the perception model for the text and video inputs respectively.

### 4.2 Experimental Setup and Baselines

Setup.
All experiments, with the exception of the CLUTRR benchmark, were run on machines with two 20-core Intel Xeon Gold 6248
CPUs, four NVIDIA GeForce RTX 4352 GPUs, and 768 GB RAM.
Since the CLUTRR benchmark requires a larger amount of GPU memory, it was run on a machine with 8 NVIDIA A100 40GB GPUs
instead.
We ran each tool thrice until convergence or until a timeout of 10 hours was reached and report the average best accuracy and training time.
The code of Dolphin and the benchmarks is provided in the supplementary material.

Baselines.
We select Scallop (Li et al., ), a contemporary state-of-the-art neurosymbolic framework supporting differentiable programming optimized to run on the CPU and use multiple cores to parallelize its computations.
We also choose two sampling-based gradient approximation methods, ISED (Solko-Breslin et al., ) and IndeCateR+ (De Smet et al., ).
We compare Dolphin against Scallop on all benchmarks, and against ISED and IndeCateR on MNIST-SumN and HWF.

### 4.3 RQ1: Scalability

**Table 2: Comparison of training times taken by each baseline. The Timeout (TO) is set at 10 hours. $\alpha$ is the scaling factor, which is the ratio of the per epoch training times of the baselines and Dolphin.**
| Task | Dolphin | Scallop | ISED | IndeCateR+ |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
|  | $\text{T}_{\text{total}}$ | $\text{T}_{\text{epoch}}$ | $\text{T}_{\text{total}}$ | $\text{T}_{\text{epoch}}$ | $\alpha$ | $\text{T}_{\text{total}}$ | $\text{T}_{\text{epoch}}$ | $\alpha$ | $\text{T}_{\text{total}}$ | $\text{T}_{\text{epoch}}$ | $\alpha$ |
| SumN (S) | 78.33 | 15.67 | 923.78 | 184.76 | 11.80 | 299.63 | 59.93 | 3.82 | 3.16e3 | 632.52 | 40.36 |
| SumN (M) | 144.92 | 14.49 | 3.41e3 | 341.57 | 23.57 | 2.16e3 | 216.54 | 14.94 | 2.63e4 | 2.63e3 | 181.88 |
| SumN (L) | 220.47 | 14.70 | 7.41e3 | 493.87 | 33.60 | 9.8e3 | 653.39 | 44.45 | TO | 2.27e3 | 154.37 |
| HWF (S) | 3.17e3 | 158.79 | 9.99e3 | 499.57 | 3.15 | 1.58e4 | 790.04 | 4.97 | 1.76e4 | 877.92 | 5.52 |
| HWF (M) | 1.46e4 | 731.67 | TO | 1.49e4 | 20.41 | TO | 6.83e3 | 9.34 | TO | 3.6e3 | 4.93 |
| HWF (L) | 2.42e4 | 1.21e3 | TO | 3.92e5 | 323.21 | TO | 1.05e4 | 8.61 | TO | 5.59e3 | 4.61 |
| Path (S) | 1.08e4 | 1.08e3 | 2.2e4 | 2.2e3 | 2.03 | N.A. |  |  |  |  |  |
| Path (M) | 1.79e4 | 1.79e3 | TO | 4.17e3 | 2.32 |  |  |  |  |  |  |
| Path (L) | 1.94e4 | 1.94e3 | TO | 1.12e4 | 5.81 |  |  |  |  |  |  |
| CLUTRR (S) | 1.54e3 | 154.85 | 4.29e3 | 429.97 | 2.77 | N.A. |  |  |  |  |  |
| CLUTRR (M) | 2.91e3 | 291.36 | 7.83e3 | 783.11 | 2.69 |  |  |  |  |  |  |
| Mugen (S) | 3.62e3 | 180.80 | 1.34e4 | 133.68 | 0.74 | N.A. |  |  |  |  |  |
| Mugen (M) | 1.78e3 | 890.34 | TO | 634.86 | 0.71 |  |  |  |  |  |  |

Table 2 presents the training times for Dolphin and baselines on all benchmarks, along with the
total training time ($T_{\textit{total}}$), time per epoch ($T_{\textit{epoch}}$), and the scaling factor $\alpha$.
The scaling factor is the ratio of the per epoch times of the baselines to Dolphin.
We observe that in almost all cases, Dolphin shows significant improvements in training times, training up to about 300x faster than Scallop, 44x faster than ISED, and 180x faster than IndeCateR+.
On average, Dolphin reports a speedup of 36x times over all baselines.
Further, Dolphin converges on all benchmarks, while the baselines time out on most of the Medium and Large versions of the benchmarks.
These results thus demonstrate that, unlike existing tools, Dolphin can scale to complex problems and large datasets.

Dolphin trains slightly slower than Scallop on both versions of the Mugen benchmark.
This is because the Dolphin program written for Mugen uses Python objects and operations that are not fully batchable across samples.
In contrast, the Scallop program, which is written in a compiled and optimized language, runs around 1.3x faster than Dolphin on average per iteration.
However, Dolphin requires only 20 epochs to converge whereas Scallop requires almost 1000 epochs (Li et al., ).
As a result, Dolphin’s total training time is still significantly lower than Scallop’s (3̃.7x for Mugen(S)). We show the accuracy curves for Mugen in Appendix B.

### 4.4 RQ2: Accuracy

Figure 4 presents the accuracy of Dolphin and the baselines on the different benchmarks.
Dolphin accuracies are marked in blue.
In all cases, for Dolphin, we report the accuracies of the best-performing provenance.
We use the DAMP provenance for MNIST, CLUTRR, and Mugen benchmarks, and the DTKP-AM provenance for the HWF and PathFinder benchmarks.

We observe that in all cases, Dolphin achieves state-of-the-art accuracy, except in CLUTRR, where
Dolphin’s accuracy is slightly lower than that of Scallop.
For most Medium and Large versions of the benchmarks, Dolphin achieves better accuracy, whereas the baselines either report lower accuracy due to the complexity of the benchmark (e.g., HWF) or fail to converge within the time limit (e.g., Scallop on PathFinder-Large).
Most importantly, these results show that Dolphin’s scalability improvements do not come at the cost of accuracy.

Figure: Figure 4: Accuracy of Dolphin and baselines across all benchmarks.
Refer to caption: x3.png

### 4.5 RQ3: Provenance Comparisons

Figure: Figure 5: Accuracy and average training time per epoch for DAMP and DTKP-AM.
Refer to caption: x4.png

We perform ablation studies to compare the effectiveness of the DAMP and DTKP-AM provenances for each benchmark.
Figure 5 presents the accuracy of Dolphin using DAMP and DTKP-AM in the top graph, and the average training time per epoch in the bottom.
We observe that the relative effectiveness of the provenances differ across benchmarks.
For instance, in the MNIST-SumN benchmark, the DAMP provenance is more effective than the DTKP-AM provenance,
since the top-k proofs are not able to capture all the possible ways in which the sum of the digits can be computed.
In contrast, for HWF, the DTKP-AM provenance is more effective than DAMP since the tags in DAMP provenance lack the structure needed to capture the semantics of the symbolic program.
Otherwise, DTKP-AM mostly has a slight edge over DAMP.
On the other hand, DAMP is faster than DTKP-AM for all benchmarks, since it does not track individual proofs.
We thus see that selecting provenances is a trade-off between accuracy and training time.

## 5 Related Work

Neurosymbolic programming frameworks. Frameworks like Scallop (Li et al., ), DeepProbLog (Manhaeve et al., ), and ISED (Solko-Breslin et al., ) provide a simple interface for neurosymbolic programming.
There are also domain-specific tools like NeurASP Yang et al. () for answer set programming and NeuralLog Chen et al. () for phrase alignment in NLP.
While these frameworks provide intuitive abstractions, they are bottle-necked due to expensive data transfers between symbolic computations done on CPU versus neural computations that execute on GPU, making neurosymbolic learning hard to scale.
In contrast, Dolphin provides a deeper integration of the two worlds by building a Python-based API on top of PyTorch, which scales better.

Scalability improving techniques. Several optimization techniques have been proposed to improve the scalability of differentiable reasoning algorithms. For instance, Greedy NTP (Minervini et al., ) reduces the computation cost of Neural Theorem Prover (NTP) (Rocktäschel & Riedel, ) by tracking only a subset of proof states using nearest neighbor search. Likewise, the conditional theorem prover (Minervini et al., ) employs a machine learning-based proof selection technique. However, unlike Dolphin, these methods are point solutions that do not fundamentally address the scalability challenge for neurosymbolic learning.

Specialized neurosymbolic solutions.
There are many specialized solutions for various neurosymbolic tasks. For instance, NGS (Li et al., ) uses a hand-coded syntax to specify the structure of mathematical expressions for HWF. More general solutions, such as NS-CL (Mao et al., ) includes a framework for visual question answering that learns symbolic representations for text and images. NeRd (Chen et al., ) transforms questions in natural language into executable programs based on symbolic information extracted from text. In this work, we rather focus on building a general programming framework that improves the scalability of diverse neurosymbolic programs.

## 6 Conclusion and Limitations

We proposed Dolphin, a programmable framework for scaling neurosymbolic learning.
Dolphin provides abstractions for writing symbolic programs along with pluggable vectorized provenances to compute symbolic gradients.
This allows users to write differentiable symbolic programs in Python within PyTorch pipelines that can scale
to complex programs and large datasets.
We show that Dolphin scales significantly better than existing neurosymbolic frameworks while achieving state-of-the-art performance on a variety of tasks.

A limitation of Dolphin is that it requires the user to express symbolic programs
in a batched manner. While this is a common pattern within deep learning, it may be restrictive to users new to batched programming.
Another limitation is that while Dolphin works well with most models, the representation of Distributions needed
by generative models (e.g., Causal LLMs like Llama) has not yet been investigated. We leave this to future work.

## Appendix A Dolphin Experiment Details for Benchmarks

The following are more details on the experiment setup.
For each experimental trial, we report the highest evaluation accuracy over all epochs.
Unless otherwise noted, each trial was run on a machine with Intel Xeon Gold 6248 (2.50 GHz) CPUs and NVIDIA GeForce RTX 2080 Ti (11 GB) GPUs.

### A.1 MNIST Sum-N

Each of the MNIST Sum-N tasks had a batch train size of 64 samples, a learning rate of 0.001, and a top-k value of 1. Each of the tasks were trained on a dataset size of the original MNIST dataset divided by N of Sum-N. Sum5’s dataset consisted of 12000 train samples and 2000 test samples. Sum10’s dataset consisted of 6000 train samples and 1000 test samples. Sum15’s dataset consisted of 4000 train samples and 666 test samples.

### A.2 Hand-Written Formula

For Hand-Written Formula we trained each task with a batch train size of 64 samples. The learning rate was 0.0001, sample-k was 7, and top-k value was 3. Length 7’s dataset consisted of 9600 samples for training, 2400 samples for testing. Length 15 consisted of 24000 training samples and 6000 testing samples. Length 19 consisted of 32000 training samples and 8000 testing samples.

### A.3 PathFinder

For each of the PathFinder tasks, we used a batch train size of 64 samples, a learning rate of 0.0001, and a top-k value of 3. Each task’s dataset consisted of 539459 images for training and 59940 images for testing. Each task had its own dataset of images with dimensions of the task’s pixel size.

### A.4 CLUTRR

For each CLUTRR task, we used a single A100 GPU (40 GB), with a learning rate of 0.00001 and use a batch size of 16.
The length of the training dataset for CLUTRR (Small) was 11,093 and that of the test set was 1146.
The training set for CLUTRR (Medium) contained 15,083 samples and the test set contained 1048 samples.

### A.5 Mugen

For each Mugen task, we use a batch size of 3 and a learning rate of 0.0001. From the full Mugen dataset, we sample a training set of 5000 examples for Mugen (Medium), and from that set, we sample a training set of 1000 for Mugen (Small). Both Small and Medium are evaluated on a fixed holdout set of 1000 samples.
We train and evaluate for up to 100 epochs.

## Appendix B Accuracy Curves for Mugen

Figure: Figure 6: Test accuracies for each epoch of Dolphin and Scallop for Mugen (Small). The accuracies for TVR are on the left and VTR are on the right.
Refer to caption: extracted/5901627/figures/mugen_tvr_1k.png

Figure: Figure 7: Test accuracies for each epoch of Dolphin and Scallop for Mugen (Medium). The accuracies for TVR are on the left and VTR are on the right.
Refer to caption: extracted/5901627/figures/mugen_tvr_5k.png

Figures 6 and 7 show the test accuracies for each epoch of Dolphin and Scallop for Mugen (Small) and Mugen (Medium) respectively.
The accuracies for TVR are on the left and VTR are on the right.
Note that Dolphin converges within 20 epochs in all cases, while Scallop does not converge within 100 epochs.

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 8
presentation: 3
questions: Please refer to weaknesses 1.

Other questions:

1. What is the limitation of the proposed DOLPHIN?
2. Are lambda functions fast enough? Do we require doing some acceleration for the proposed operations, such as designing some specific CUDA kernel or triton functions?
soundness: 3
strengths: 1. The authors propose an end-to-end neurosymbolic framework that makes all progress differentiable.
2. With a glance at the provided code, the DOLPHIN is a lightweight implementation of the framework integrated with PyTorch, having a potential opportunity to support the neurosymbolic community.
3.  The evaluation on 13 benchmarks across 5 neurosymbolic tasks show the advantage of the proposed DOLPHIN.
summary: The paper proposes DOLPHIN, a scalable neurosymbolic learning framework that integrates symbolic reasoning efficiently within deep learning models. Unlike existing frameworks that struggle with CPU-GPU data transfer bottlenecks, DOLPHIN enables symbolic programs to operate as a set of inheritanted PyTorch nn.module, allowing efficient GPU-based differentiation.
weaknesses: 1. It seems DOLPHIN only supports neurosymbolic programs with deterministic symbolic processes. For example, if HWF task requires the neural part to predict both numbers and operators (+,-,*,/), the symbolic part cannot be programmed with the Apply function. How DOLPHIN deal with this situation?

---

confidence: 4
contribution: 2
initial_rating: 6
presentation: 3
questions: Please see "weaknesses" above -- in particular my confusion with the DTKP provenance.
soundness: 3
strengths: The paper is very well written, and describes the basic execution and batching mechanism clearly, at least for the DAMP provenance.  The Dolphin framework does seem to be an improvement over SOTA in terms of basic usability for certain classes of neuro-symbolic programs.
summary: The authors introduce Dolphin, which is a pytorch-friendly framework for performing neuro-symbolic computations.  The neural component of a computation is assumed to be a model, such as an MNIST digit classifier, which outputs a discrete set of symbols (e.g the digits 0-9), with a probability attached to each of them.  The symbolic component is a python program which runs a computation over the symbols.  The result of symbolic execution is a pytorch tensor, representing final output probabilities for each possible result, which is end-to-end differentiable with the neural components.  The authors apply Dolphin to several of neuro-symbolic benchmarks, and show that it is faster than competing frameworks.

Dolphin essentially works by running the symbolic program for every possible combination of input symbols, and tracking the probability of each combination.  The symbolic program is executed on CPU, but Dolphin evaluation will merge different traces of the program which have the same output into batches.  The probabilities can then be computed using batch operations that are GPU-friendly, as well as being end-to-end differentiable with pytorch.

The authors also provide two different mechanisms, which they call provenances, for tracking probabilities.  The DAMP provenance tracks all probabilities, while DTKP tracks only the top-K proofs.
weaknesses: The authors spend a lot of time talking about the easy parts of the problem, and fail to adequately discuss the hard parts.  As a result, they gloss over two glaring weaknesses that I see with using this approach to solve anything other than cherry-picked trivial problems.  

The first issue is the combinatorics.  When evaluating a function f(A,B), where A and B are distributions over symbols, evaluation must evaluate f(a,b) for every possible combination of symbols { a | a \in A }, and { b | b\ in B }.  Depending on the exact problem, this can easily lead to an combinatorial explosion in the number of possible outputs.  The authors test their code on the "sum of MNIST digits" problem, where the combinatorics are reasonable; even given 20 digits, there are at most 181 possible answers.  If they were to instead try the "product of MNIST digits", which is a tiny change to the code, then the number of possible outputs would balloon, and the technique would likely fail.  

The second issue is control flow.  As a symbolic computation, the "sum of digits" has no loops or branches, and thus is trivially easy to batch.  The authors mention that they support recursive computations, but those generally require a branch to terminate the recursion, and often have divergent control flow.  In the presence of branches, different traces of the program take different paths, and no longer cleanly batch together.  

The usual solution (e.g. in CUDA programs) is that when evaluation encounters a branch, it splits the batch of traces into a then-branch and an else-branch, and then merges the traces together again afterwards.  Without merging, the traces will continue to diverge on subsequent branches, until each trace is operating independently at batch size 1, and the benefits of parallelism are lost.  

Merges happen at the join points in a control-flow graph, which requires the underlying library to build a control-flow graph.  Alternatively, since there are only two batched operations (conjunction and disjunction), the authors could first construct an (unbatched) DAG of operations, and then merge/batch together independent nodes of the DAG after the fact, in the style of Looks et al. "Deep learning with dynamic computation graphs," or Neubig et al. "On-the-fly operation batching in dynamic computation graphs."

However, the authors make no mention of any machinery to analyze control-flow, build control-flow graphs, or otherwise auto-batch in the presence of divergent control flow.  In fact, they do not even provide a discussion or examples of how to write recursive computations with their library at all, despite claiming that it is possible.  

My main objection with both of these issues is that the authors simply don't discuss these problems at all, when IMO they are very clearly major limitations that affect the kind of programs that Dolphin is able to run.   

A further weakness of the writing itself is that the authors do not do a good job of explaining the DTKP provenance, which seems like it's quite important.  I have several criticisms here.  First, it is possible that choosing only the top-K proofs after each operation will address the combinatorics issue, which would be a big deal.  However, I'm uncertain, because the authors gloss over combinatorics problem altogether without discussion.  Second, the authors claim that their mechanism for merging DTKP tags is equivalent to weighted model counting, but this claim is wholly unsubstantiated.  I didn't really understand the formula in Table 1 at all, including how infinities get into the tags.  At the very least, the authors should provide a detailed discussion of DKTP in the appendix, ideally with a proof of how it relates to WMC, if space within the paper itself is an issue.  

Finally, the authors mention that wrt. to the HWF task, "the DTKP-AM provenance is more effective than DAMP since the tags in DAMP provenance lack the structure needed to capture the semantics of the symbolic program."  This statement seems important, but really requires further explanation; I don't understand it at all.  Providing HWF as a worked example (perhaps in the appendix) would be valuable to anybody who actually wants to use Dolphin.  

Errors:

Line 310: "Its conjuction operation is defined as the addition of probabilities, and its disjunction is defined as the multiplication of probabilities."  Unless my understanding is way off base, shouldn't this be the other way around?  For independent observations, p(A and B) means multiplying p(A) and p(B)?  That's what the authors show in Table 1.

---

confidence: 3
contribution: 2
initial_rating: 5
presentation: 2
questions: - It would be better to provide a breakdown of the training time (e.g., according to Figure 1) to justify the major efficiency improvement of Dolphin.
- In Figure 5, for small tasks that all competitors could converge within the time limit, why Dolphin has a better accuracy?
- In Figure 6, there is a trade-off between accuracy and training time for different provenances (DAMP vs. DTKP-AM). How should we select the most suitable one in practice?
soundness: 3
strengths: - A new framework for neurosymbolic learning, namely Dolphin, is developed.
- Dolphin shows superior performance compared to existing frameworks.
summary: This work presents Dolphin, a brand new framework designed to enhance the scalability of neurosymbolic learning. Dolphin allows developers to write differentiable symbolic programs in Python, utilizing PyTorch for end-to-end GPU acceleration. The framework conveys flexible programmability, end-to-end differentiability, scalability, and tunability, which are essential to handling complex programs and large datasets effectively. Experimental results demonstrate that Dolphin significantly outperforms existing frameworks.
weaknesses: My major concern is that the paper is not well structured and hard to follow. In the introduction section, the authors criticized existing frameworks that they must use a separate CPU-based backend and suffer from the slow inter-process data transfers. For me, this is implementation-specific, and it does not drive me to the reasons why we should redesign the entire framework. Although the authors further discuss the challenges in lines 52-67, I find it rather irrelevant to the aforementioned limitation of slow inter-process data transfers.

Moreover, as a new framework, there lacks an overview to depict the layered structures. This prevents readers from having a general picture. It is hard to tell why the designs/implementations could realize the core principles. Additionally, I cannot map the core principles to the challenges discussed in the introduction, either.

---

confidence: 5
contribution: 2
initial_rating: 3
presentation: 3
questions: - Line 213: “Tags are tensors that represent relative likelihoods”. Relative with respect to what? Relative likelihoods (as I understand it) represent a fraction between two likelihoods.
- The experimental section doesn’t really explain how the CLUTRR and Mugen tasks are solved by Dolphin. E.g. what are the Dolphin programs used here? I think it could be useful to at least include these in the Appendix.
- I found the naming of “Distribution” unclear. Unless I misunderstand it, a “Distribution” isn’t a probability distribution? (E.g. the Filter operation can remove probability mass.)
- How did you perform hyperparameter tuning? Did you tune the baselines in a similar fashion? Given that Table 2 compares total training time, better hyperparameters also affect the reported runtime.
- Why is there such a pronounced accuracy difference between Dolphin and Scallop in some experiments? From what I understand, the provenances like DAMP are essentially inherited from Scallop, so similar accuracy in Scallop should be possible (although not with the same runtime of course).

Minor comments:

- The paper mentions that an “NVIDIA GeForce RTX 4352” was used for the experiments for all experiments (besides CLUTRR). Is this a typo? I’m not aware of the existence of this specific model, and could not find anything about it on the internet. In contrast, the Appendix mentions that a GeForce RTX 2080 Ti was used.
- For Table 2, what is the unit of time? I assume this is in seconds, but I couldn’t find this anywhere.
- For Table 2, what is the provenance used for Dolphin? I assume this is DTKP-AM, but I couldn’t find this anywhere.
- Line 518: “these methods are point solutions”. What do you mean by "point solution"?
- The brackets on the citation on line 107 are wrong.
 - Figure 6 bottom would be more clear with a log y-scale.
- Several citations refer to the arXiv preprint instead of the conference publication (e.g. for neurASP, CLUTRR, and NeuralLog).
soundness: 1
strengths: - The problem tackled in the paper (scalable and efficient neurosymbolic learning) is relevant and an important issue for the broader neurosymbolic community. 
- Overall, I feel that the paper is rather well-written and includes several useful figures and examples, resulting in a clear presentation of the ideas. 
- The design of Dolphin seems more accessible to a deep learning audience compared to many existing neurosymbolic systems. Notably, it does not require knowledge of e.g. ASP or Prolog (as is the case for NeurASP or DeepProbLog) and can be integrated easily into a deep learning library such as PyTorch or Tensorflow.
summary: Many neurosymbolic frameworks have been introduced in recent years, which typically execute their symbolic component on the CPU. This limits their scalability, due to the inferior CPU performance and data transfer latency between the CPU and GPU. The paper introduces Dolphin, a neurosymbolic framework that is fully implemented by parallel tensor operations, and hence can be run on the GPU using conventional deep learning libraries such as PyTorch. The experiments indicate that Dolphin exhibits considerable speed-ups compared to existing frameworks such as Scallop.
weaknesses: **Novelty**. The key contribution of the paper - speeding up a neurosymbolic framework by tensorizing it and running on GPUs - is certainly not a new idea. One example of prior work is the LYRICS framework by Marra et al. (2019), which also uses tensor operations to perform the symbolic operations in parallel on the GPU (see e.g. Figure 1 in their paper). Logic tensor networks (Badreddine et al., 2022) and Tensorlog (Cohen, 2020) are some additional examples. These frameworks often also support different provenances / semirings / t-norms. The parallelization of neurosymbolic methods with expressive probabilistic semantics is more challenging, but also here there is plenty of existing work (see e.g. Darwiche (2020) or Dang et al. (2021)). Unfortunately, the paper does not mention prior work on parallelized neurosymbolic learning, nor how it is different from these existing methods.

**Semantics**. It is not clear to me what exact semantics Dolphin aims to achieve. The first provenance (DAMP) is essentially fuzzy semantics (which already has been shown to be easily parallelizable, e.g. Badreddine et al. (2022)). On the other hand, “Apply” mostly brute-force enumerates models meaning the necessary independence assumptions for probabilistic sematics can often hold. (c.f. the MNIST-experiment). The second provenance is the top-k semiring, which is less trivial to parallelize. However, the proposed solution of adding the different proofs destroys the top-k semantics (lower-bounding the WMC). This also results in the introduction of clamp operations, which could lead to zero gradients.

**Language**. A distinction from existing methods is that Dolphin introduces its own set of programming primitives (apply, filter, etc.). Previous neurosymbolic frameworks have typically built on an existing language, e.g. Datalog for Scallop, ASP for NeurASP, ProbLog for DeepProbLog, etc. However, there is no justification for the choice of programming primitives. How does its expressivity relate to existing systems such as Scallop? Why wasn’t an existing language chosen? In my opinion, a lot of different choices could have been made (e.g. why do you need ApplyIf instead of just Apply + Filter?).

**Experiments**. I was surprised that the IndeCateR baseline achieved such low accuracy, given that the experiment seems to be the same as in the IndeCater paper, where the reported results are much better. I just tried out the original IndeCateR implementation myself, and I could replicate the MNIST-addition (L) on my machine in 2 minutes. In contrast, the paper reports a timeout after 10 hours. The accuracy also reaches 86.8%, as opposed to less than 10% in the paper (I'm not sure how the paper reports accuracy if it times out). As the code for the baselines is not included in the supplementary material, I hope the authors can clarify these discrepancies. There are additional issues in the experimental section, e.g. there is no mention of hyperparameter tuning, c.f. the questions section. 

Lastly, the performance of Dolphin is claimed to be state-of-the-art but I’ve seen several systems get better results on the considered benchmarks (the comparison is hard as actual numbers are not reported, and only bars). To give just some examples, Orvieto et al. (2023) report 94% for Pathfinder, and Manhaeve et al. (2021) report near-perfect accuracies for CLUTRR. I want to stress that I don’t think state-of-the-art results are necessary, but if they are claimed this should be properly supported.

In summary, the concerns about the novelty of the paper combined with the experimental evaluation issues unfortunately mean I cannot recommend acceptance.


**References**

Badreddine, S., Garcez, A. D. A., Serafini, L., & Spranger, M. (2022). Logic tensor networks. *Artificial Intelligence*.

Cohen, W., Yang, F., & Mazaitis, K. R. (2020). Tensorlog: A probabilistic database implemented using deep-learning infrastructure. *Journal of Artificial Intelligence Research*, *67*, 285-325.

Dang, M., Khosravi, P., Liang, Y., Vergari, A., & Van den Broeck, G. (2021). Juice: A julia 
package for logic and probabilistic circuits. In *Proceedings of the AAAI Conference on Artificial Intelligence*.

Darwiche, A. (2020). An advance on variable elimination with applications to tensor-based computation. In *ECAI.*

Manhaeve, R., Dumančić, S., Kimmig, A., Demeester, T., & De Raedt, L. (2021). Neural probabilistic logic programming in DeepProbLog. *Artificial Intelligence*, *298*, 103504.

Marra, G., Giannini, F., Diligenti,  M., & Gori, M. (2019). Lyrics: A general interface layer to integrate logic inference and deep learning. In *Joint European Conference on Machine Learning and Knowledge Discovery in Databases*.

Orvieto, A., Smith, S. L., Gu, A., Fernando, A., Gulcehre, C., Pascanu, R., & De, S. (2023, July). Resurrecting recurrent neural networks for long sequences. In *International Conference on Machine Learning* (pp. 26670-26698). PMLR.

Meta-review: The paper introduces Dolphin, a framework designed to enhance the scalability of neurosymbolic learning by integrating symbolic reasoning into deep learning models using PyTorch. Dolphin allows symbolic programs to be written as PyTorch modules, enabling end-to-end differentiation on GPUs. The reviewers are divided over this paper. On one hand, two reviewers believe that this is a new approach as mentioned in the abstract, while two more reviewers feel this is more nuanced case. The latter reviewers express that there is relevant work in neurosymbolic methods that are tensorized and those are not included in the baselines. This is a major drawback, since that would not place Dolphin appropriately in the literature and would significantly reduce the claimed novelty. 

This is a challenging case: the paper can add value to the community, but only once compared with appropriate frameworks and benchmarks, especially on more challenging cases where the brute force approach might not scale. Thus, at the moment, this is a borderline reject, based on the reviewers' comments.

Decision: Reject