Neural Arithmetic Unit by reusing many small pre-trained networks
Authors: Ammar Ahmad
Oneeb Babar
Murtaza Taj
Venue:  (2020)
Keywords: NALU
feed forward NN

Abstract: We propose a solution for evaluation of mathematical expression. However, instead of designing a single end-to-end model we propose a Lego bricks style architecture. In this architecture instead of training a complex end-to-end neural network, many small networks can be trained independently each accomplishing one specific operation and acting a single lego brick. More difficult or complex task can then be solved using a combination of these smaller network. In this work we first identify 8 fundamental operations that are commonly used to solve arithmetic operations (such as 1 digit multiplication, addition, subtraction, sign calculator etc). These fundamental operations are then learned using simple feed forward neural networks. We then shows that different operations can be designed simply by reusing these smaller networks. As an example we reuse these smaller networks to develop larger and a more complex network to solve n-digit multiplication, n-digit division, and cross product. This bottom-up strategy not only introduces reusability, we also show that it allows to generalize for computations involving n-digits and we show results for up to 7 digit numbers. Unlike existing methods, our solution also generalizes for both positive as well as negative numbers.

Under review as a conference paper at ICLR 2020
NEURAL ARITHMETIC UNIT BY REUSING MANY SMALL PRE-TRAINED NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
We propose solution for evaluation of mathematical expression. However, in- stead of designing single end-to-end model we propose a Lego bricks style ar- chitecture. In this architecture instead of training complex end-to-end neural network, many small networks can trained independently each accomplish- ing specific operation and acting single lego brick. More difficult or com- plex task can then be solved using a combination of these smaller network. this work first identify 8 fundamental operations that commonly used solve arithmetic operations (such as digit multiplication, addition, subtraction, sign calculator etc). These fundamental operations then learned using simple feed forward neural networks. We then shows that different operations can be de- signed simply by reusing these smaller networks. As example reuse these smaller networks to develop larger and more complex network to solve n-digit multiplication and n-digit division. This bottom-up strategy not only introduces reusability, we also show that it allows to generalize for computations involving n-digits and we show results for up to digit numbers. Unlike existing methods, our solution also generalizes for both positive as well as negative numbers.
INTRODUCTION
The success of feed-forward Artificial Neural Network (ANN) lies in their ability to learn that allow an arbitrarily connected network develop an internal structure appropriate for a particular task. This learning is dependent the data provided to the network during the training process. It has been commonly observed that almost all ANNs lack generalization and their performance drastically degrades unseen data. This includes degradation of performance categories but acquired under from a different setup (location, lighting, view point, size, ranges etc). Although there are techniques such Domain Adaptation to address these generalization issues, however this behaviour indicates that the learning process in neural network is primarily based on memorization and they lack understanding of inherent rules. Thus the decision making process in ANN is lacking quantitative reasoning, numerical extrapolation or systematic abstraction.
data containing the seen
However when observe other living species, numerical extrapolation and quantitative reason- ing is their fundamental capability what makes them intelligent beings. For e.g. if we observe the learning process among children, they can memorize single digit arithmetic operation and then ex- trapolate it to higher digits. More specifically our ability to +, x and + higher digit number is based on understanding how reuse the examples that we have memorized for single digits. This indicates that the key to generalization is in understanding to reuse what has been memorized. Fur- thermore, complex operations are usually combination of several simple function. Thus complex numerical extrapolation and quantitative reasoning among ANNs can be developed by identifying and learning the fundamental operations that can reused develop complex functions.
Inspired from the methodology of learning adopted by humans, in this work we first identify sev- eral fundamental operations that are commonly used to solve arithmetic operations (such as digit multiplication, addition, subtraction, merging of two number based their place value, learning to merge sign +/- etc). These fundamental operations are then learned using simple feed forward neural networks. We then reuse these smaller networks develop larger and more complex net- work to solve various problems like n-digit multiplication, n-digit division, cross product etc. To the best of our knowledge this is the first work that proposed generalized solution for these arith-

---

Under review as a conference paper at ICLR 2020
metic operations. Furthermore, unlike exiting methods (Horiik et al. (1989): Siu & Roychowdhury (1992); ;; Peddada (15amaaaamaa (2013); Trask et al. (2018) ours is also the first solution that works for both positive as well negative numbers.
2 RELATED WORK
Neural Network are known for their ability to approximate mathematical functions (see Hormik al.] (1989). Exploratory work has been done to approximate simple arithmetic operations which include multiplication and division (see Siu & Roychowdhury (1992)). Although, it might seem fairly intuitive task but the inability to generalize over unseen data makes the architecture of pro- posed networks complex in nature (sce|Cho@rr et al. (2019)). Recent works including Resnet (He al.) 2016) through identity mappings, highway networks (Srivastava et al.) 2015) through gating units, and dense networks Huang et al. (2017) through densely connected layers; have tried to train net- works which can generalize over minimal training data. To this effect, EqnMaster (see Peddada (2015) uses generative recurrent networks approximate arithmetic functions (addition, subtrac- tion and multiplication only). However, this model still does not generalize well as it is only able to perform these functions over 3-digit numbers. Most recent work in this regard is Neural Arithmetic Logic Unit (NALU) (see Trask et al. (2018)) which uses linear activations to represent numerical quantities and then using the learned gate operations predicts the output of arithmetic functions. The results of NALU (Trask et al. (2088) determine that extrapolation issues will always be a problem in end-to-end learning based tasks. simple Feed Forward Network (see SeeeFronco Cannas (1998) can determine solutions to arithmetic expressions including multiplication. This approach does not result the most efficient network architecture to solve even the simplest arithmetic expression (for example: 3-depth network for multiplication). Optimal Depth Networks (see (Siu & Roychowdhury (1992) through a deterministic approach using binary logic gates can perform simple arithmetic functions. Using digital circuits an inspiration for neural network architecture has been studied in various researches including Digital Sequential Circuits (se Ninomiya & Asai (1994)) and Simu- lation of Digital Circuits (see (Saamma (203)). Using digital circuitry as a point of reference neural networks can be designed to solve simple arithmetic problems. Our work establishes on the premise provided by Binary Multiplier Neural Networks (see Huang et al. 2013)) that perform arithmetic operations on binary digits. We propose a network that can deterministically predict the output of basic arithmetic functions over the set of both n-digit positive and negative decimal integers as opposed to the state-of- sate-off-the-as art model which only works on (limited digits) positive integers.
These existing methods that propose a single neural network that when trained independently for different tasks attempt to predict output for different functions. We instead propose to train several smaller networks to perform different sub tasks that are needed to complete difficult arithmetic operation such signed multiplication, division and cross product. We then use various combina- tion of these already trained smaller networks and design networks that can perform complex task. Similar, to LSTMs, also propose loop unrolling strategy that allows to generalize the solution from seen 1 digit arithmetic to unseen n-digit arithmetic operations.
FUNDAMENTAL OPERATION
Multiplication is often performed repeated addition similarly, division can be performed re- peated subtraction( O). These operations are implemented on digital circuits as shift and accumu- lator to sum each partial products. These digital circuits are know to perform accurate arithmetic operations and are commonly used in digital computing equipment. They can also be easily scaled by increasing the shifters and accumulators( O). Furthermore, some initial work has been done to shown that neural networks can be used to simulate digital circuits (Ninomiya & Asai (1944)).
Using this as inspiration we analyze that n-digit multiplication on paper and pencil involve 1-digit multiplication, addition, place value shifter, and sign calculator. Division operation intern requires subtraction operation in addition to those needed for multiplication. We thus designed 6 neural networks to perform 8 fundamental operations namely i) addition, ii) subtraction iii) 1-digit multi- plication iv) place value shifter, v) input sign calculator, vi) output sign calculator, vii) digit sign multiplication and viii) absolute. We then design several complex neural network one for each com-
2

---

Under review as a conference paper at ICLR 2020
Hidden Layer
Cutput Layer (62 neurore)
I0
10
100
(a) Addition
(b) Substraction
(6) Place Value Shifter
(c) 1-digit muttipiiction
Hidden Layer (10 neurons) Soft sign Activation
Layer neuroni Soft-sign activation
Absoluta Component
Maxpooling Layer
Xg
(e) Absolute
Input Sign Calculator
(g) Output Sign Calculator
(h) 1-digit sign multipiicstion
Figure 1: Fundamental operations
plex function such as multiplication and division and show that these fundamental blocks can be used to design several complex functions including arithmetic equation calculator.
3.1 ADDITION AND SUBTRACTION
The basic function of a neuron network is sum transform where each input is multiplied with weight, these weighted inputs are then passed through an activation function to produce the final output. This inherent summation property simplifies the task of creating a neural network addition of two numbers. Our addition module is thus implemented using single neuron (having linear activation) with two inputs by setting their weights to +1), Similarly, our subtraction module consist single neuron with two inputs by setting their weights to {+1,- -1} (see Fig. a-b)).
3.2 PLACE VALUE SHIFTER
This module is designed facilitate shift-and-add multiplication in which each digit of the multi- plier is multiplied one at time from right to left with each digit of the multiplicand. The output of these multiplications are then combined to create a single number. The place value shifter used to place the output the appropriate position followed by addition to obtain the final output. Similar to addition network, this module is also composed a single neuron with linear activation. Instead of taking only integer values, it can be unrolled for inputs each taking 1 digit number as input. This network uses fixed weights in the power of 10 for each preceding digit (see Fig. [c)).
3.3 1-DIGIT MULTIPLICATION
In 1-digit integer multiplication there are 82 possible outcomes (digits through 9 multiplied with each other and outcome of multiplication with 0). Thus proposed feed forward network to learn the results of single digit multiplication has two input neurons, hidden layer with 30 neurons and output layer of 82 neurons. The model takes input 1-digit integers and produces 82. The highest-ranked prediction is then selected as the output of single digit multiplication (see Fig. [d).
3.4 ABSOLUTE
This network takes single number and computes its absolute value using a neural network having 2 hidden layer. The first hidden layer performs the & + and xr operation, the second layer is

---

Under review as a conference paper at ICLR 2020
L-iggt
(a)
(b)
Sieecce
(c)
(d)
Figure 2: Multiplication and Division using fundamental operations. (a) and (b) are the high-level representation of signed multiplication and signed division operations respectively. (c) and (d) rep- resents the unrolled architecture for n-digit multiplication and n-digit division (respectively) of two positive integers.
maxpool layer and the final output layers subtracts the input from the output of maxpool layer. The entire process can be represented max(x x x) (see Fig. Ie).
3.5 INPUT/OUTPUT SIGN CALCULATOR
The input sign calculator is used to extract sign of an input number It is single neuron that uses the following activation function z/ (1 mod(()) (x) (see Fig. (t).
The output sign calculator computes the resultant from multiplication division of two numbers.
takes two inputs which are a combination of I and -1 (1, 1, 1, 11,-11-1 -1, 1. -1,-1) and generates neural network with 2 hidden layers. The first
-))
the resulting sign. The operation is modelled layer simply adds the two numbers, the second layer is a single neuron that uses modulus as its activation function and the final layer performs subtraction of from the output of previous layer. 'Tee process can be represented mathematically as mod(x:1 + x2) - (see Fig. gg)
3.6 1-DIGIT SIGN MULTIPLICATION
The network takes as input sign and magnitude which is passed on to hidden layer of 10 neurons. The output layer has a soft-sign activation predict the output of the sign multiplication. The module is used to assign a sign to the output of a complex operation such as multiplication and division (see Fig. h)))
4

---

Under review as a conference paper at ICLR 2020
4 MULTIPLICATION
We perform multiplication between n digit signed integers by using all the 8 different fundamental operations. The high-level diagram of signed multiplication is shown in Fig. aa.. First two numbers are converted into positive integers by using the absolute operation. The input and output sign calculators are also used here to extract the sign input numbers and to compute sign of resulting output value. The multiplication between the two positive integers obtained via absolute operation then performed using multiply sub module shown in Fig. C.c.
The multiply sub module tokenizes the inputs into single digits. The single digit multiplication oper- ation is then used to multiply each token of multiplicant with the 1-st token of multiplier. The output of each of the these operations is then added with the carry forward from the 1-digit multiplication with the previous token. Each of these results are then combined to form a single number. This process then repeats for the each token of the multiplier. The final output is then assigned sign using I-digit sign multiply (see also Algorithm 1.
Algorithm Multiplication model based on digital circuitry m-digit multiplicand and n-digit multiplier
Expected number of output digits for an n*m multiplication are n+m digits. Total elements produced by row-wise multiplication (m+1)*n Each row can have maximum of m+1 output digits.
list final Output =
current. node first row first output for output node: do
create new node current,.ode
next node mth node from current node
loop
create new_node nexttnode parallel to current.node
mede_node = mthnnode from current_node
if next nodeeeeossssslts in out-of-bound index then break
end
nezt_node meets the base case then
break
end
end loop
SUM = use Adder network to find the sum of all nodes in the list
(Ones, Tens) use Separator (look Figure (D)) to separate ones and tens
list [Tens]
append Ones to the final_Outppa
current_node is set to the next output node
end
5

---

Under review as a conference paper at ICLR 2020
5 DIVISION
The division model also separates the sign and magnitude during pre-processing. The architecture inspired from long division model where n-digit divisor controls the output computation. The n-digit divisor is multiplied with single digit multipliers (0 through 9) and subtracted from selected n-digit chunk dividend. From the given set of outputs we select smallest non-negative integer by introducing two additional layers for this specific purpose. The selected node represents the remainder and the quotient result of division for the selected chunk of n-digit dividend and n-digit divisor. The quotient is combined over all iteration and the remainder is knitted the next digit available in the divisor. Figure b)d) shows the architecture of multiplication network.
We can generate division model based on digital circuitry for decimal digits shown in Algo- rithm 2.
Algorithm 2 Division model based on digital circuitry
Inputs (divisor, dividend)
Quotient= =
Remainder =
length.o, divisor = dividend -index, larl =
dividend -index,ddideexeeeeee nd X Divisor. * 0, 2, 7,7,,,8,
current. dividend dividend (iwrided(diiideer (dividend inder start dividend index end 1] for do
Y1 = current.dividend X
Y2 Y1. 11110
Y =
for i: / to 10 do append max(Yli, Y2i) to Y end
(output, inder) = (min(Y) min,t(Y))
append index to Quotient
dividend indez,tart dividend index end +
Siridenddindez..a nd dévidend boniddendindereemee +
current = Joutput, diwiddeddddiiddddddddde...
dividend
if dividend index. is out of bounds then break end
end
Remainder = current.dividend
Merge Quotient

---

Under review as a conference paper at ICLR 2020
Table 1: Comparison of proposed model with existing deep networks showing classification aceu- racy in percentage.
D-RNN
D-GRU
BB
RNN
GRU
LSTM
R-RNN
Ours
+
89.2
95.8
71
86.5
97.7
95.1
100
93.4
95.6
51.4
82.8
96.6
93
94.6
100
x 92.6
81.3
36.5
13.5
51.1
40.9
41.3
100
Table 2:
Comparison with Neural Arithmetic and Logic Unit (NALU) (Trask et al. 2018). This comparison is only values between 0-30 as this is the range in which NALU is trained and operates.
Int
Exp
Int
Exp
int
Exp
Int
Exp
NALU
100
100
93.1
95
100
97.2
98.9
97.2
Ours
100
100
100
100
100
100
100
100
EXPERIMENTS
6.1
SETUP
We compare our model with the results provided by Peddada (2015) on their dataset. Since, they only implement addition, subtraction and multiplication so, we can not compare the results division architecture proposed in this paper. We do however, compare the result of signed arithmetic operations.
For our second comparison, we have used the implementation provided by Neural Arithmetic and Logic Unit (NALU) Trask et al. (2018) and train to match the results claimed by them. Once trained,
their prediction dataset (range: 0 to 30 uniform numbers) and round them integer values. We calculate the test results on this data for NALU and our implementation.
To support claim of generalization which are randomly generated. The dataset also includes negative integers to check the accuracy for signed arithmetic operations.
our model 2-digit integers up 7-digit integers
6.2 RESULTS
Experiment 1: This experiment shows that our model outperforms the recurrent and discriminative networks (Peddada 2015). Even within the testing range their dataset we get 100% accuracy. Furthermore, the signed multiplication is exclusive our model only. Table I shows the results of this comparison.
Experiment 2: We compares our results to the state-of-the-art model for arithmetic operations positive integer values called Neural Arithmetic and Logic Unit (NALU) (Trask et al.) 2018). This experiment also allows us compare results for the division architecture proposed our paper. The NALU network only trained between the range of 10 20 outside which it fails drastically. For fair comparison have used their generated test set in this experiment. Table shows the results of this comparison.
We also conducted experiments demonstrate that our model able to generalize to n-digits for signed arithmetic operations (see Table scales in size with the complexity of the inputs and performs exactly the same on all inputs. After the learning for the atomic components our model demonstrates output accuracy is data independent.
Experiment 3:

---

Under review as a conference paper at ICLR 2020
Table 3: Results showing generalization of proposed approach on higher digit integers
Stanford Dataset
Inhouse Dataset
X
X
digit
100
100
100
100
100
100
100
digit
100
100
100
100
100
100
100
digit
100
100
100
100
100
100
digit
100
100
100
100
100
100
digit
100
100
100
100
100
100
digit
100
100
100
100
100
100
7 CONCLUSION
In this paper we show that many complex tasks can be divided into smaller sub-tasks, furthermore many complex task share similar sub-tasks. Thus instead of training complex end-to-end neural network, many small networks can be trained independently each accomplishing one specific op- eration. More difficult complex task then be solved using combination of these smaller network. In this work we first identify several fundamental operations that are commonly used to solve arithmetic operations (such digit multiplication, addition, subtraction, place value shifter etc). These fundamental operations are then learned using simple feed forward neural networks. We then reuse these smaller networks to develop larger and a more complex network to solve various problems like n-digit multiplication and n-digit division.
One of the limitation of the proposed work is the use of float operation in the tokenizer which limits the end-to-end training of complex networks. However, since we are only using pre-trained smaller network representing fundamental operations, this does not creates any hinderance our current work. However, aim to resolve this issue in future. We have also designed a cross product network using the same strategy and we are currently testing its accuracy. As a future work we aim to develop point cloud segmentation algorithm by using larger number of identical smaller network (i.e. cross product) that can compute normal vector using 3 3D points as input.
REFERENCES
Sungjae Cho, Jaeseo Lim, Chris Hickey, Jung Ae Park, and Byoung-Tak Zhang Simulating prob- lem difficulty in arithmetic cognition through dynamic connectionist models. In International Conference on Cognitive Modelling, 2019.
Leonardo Franco and Sergio A. Cannas. Solving arithmetic problems using feed-forward neu- ral networks. Neurocomputing, 18(1):61 - 79, 1998. ISSN 0925-2312. doi: https://doi.or 101011680923-23122770000666
URL http: http://wwee wttp///wwww... www.sacceneeiieee.o wtw.::tttt...ccc..ceeeeemm
art.icle/p11/0002222222222
K. He, X. Zhang, and S. Ren J. Sun. Optimal depth neural networks for multiplication and related
problems. In ECCV, 2016.
Kurt Hornik, Maxwell Stinchcombe, and Halbert White. Multilayer feedforward networks
universal approximators. Neural Networks, 2(5):359 366, 1989. ISSN 0893-6080.
doi: http:://bbcoooo/o/ooooooooo/ooo0 com/ http:///canneeeett..eeea//////0.
URL http://www//w.... scw.scedeiedd..ct.
G. Huang, Z. Liu, L. v. d. Maaten, and K. Q. Weinberger. Densely connected convolutional net- works. In IEEE Conference Computer Vision and Pattern Recognition, pp. 2261-2269 July 2017.
Han Huang, Luxia Xu, and Fangyue Chen. Neural network binary multiplier. 2013.
H. Ninomiya and H. Asai. Design and simulation of neural network digital sequential circuits. IEICE Trans. on Fundamentals of Electronics, Communications and Computer Sciences, E77-A (6):968-976, 1994. ISSN 0925-2312.
8

---

Under review as a conference paper at ICLR 2020
Amani V. Peddada. Eqnmaster: Evaluating mathematical expressions with generative recurrent networks. 2015.
D. P. Sharma. Neural network simulation of digital circuits. International Journal of Computer Applications, 79(6). 2013. ISSN 0975-8887.
Kai- Yai-Yeung Siu and Vwani P. Roychowdhury. Optimal depth neural networks for multiplication and related problems. In NIPS, 1992.
Rupesh Kumar Srivastava, Klaus Greff, and Jürgen Schmidhuber. Highway networks. CoRR, abs/1505.00387, 2015. URL aattp://araae org/ 1505.003877777
Andrew Trask, Felix Hill, Scott E Reed, Jack Rae, Chris Dyer, and Phil Blunsom. Neu- ral arithmetic logic units. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa- Bianchi, and R. Garnett (eds.), Advances in Neural Information Processing Systems 31. pp. 8035-8044 Curran Associates, Inc., 2018. URL http::/papers nips. cc/paper/ - neural - ar i hmet ic- logic- units.pdf


Reviews:
[{"review_id": "H1luNTN3tS", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a method to design a NN based mathematical expression evaluation engine. I find that the paper could benefit a lot from some rewriting as it is not very clear and over claiming at points.\nThe introduction states that almost all ANNs lack generalization, this is in my opinion an overstatement. Domain shift and adaptation are techniques to cope with situations where the test data distribution is not coherent with the training data distribution. If this would be true in general we would have not seen such a resurgence and widespread use of ANN in the past years.\n\nThe paper lacks also proper citations to previous work and I find the background section and motivation rather weak. \n\nThe fundamental operations presented in section 3 do not involve any learning at all, contrary to the referenced work of Trask et al where parameters are actually learned (see relaxation of sign function with tanh etc.). I therefore find the use of ANN as basic constituent of the block to be wrong, each network has fixed hand-crafted weights. If I were to replace ANN with the ordinary corresponding function nothing would in the presented framework.\n\nMultiplication and division as explained in the algorithms do not require learning at all. I am afraid the ML contribution of this work is in my opinion almost non existent. I see every component as being scripted rather than learned from the data, which would be of course much more interesting.\n\nExperiments are not clear at all, setup and explanation of results are not sufficient and I find them not thoroughly executed. Table 1 mentioned classification but the task is never clearly explained. Experiment 2 compares to NALU but in the proposed work nothing is learned, unless I misunderstood the work entirely.\n\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "H1gznbUMqr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Writing\nOverall, the readability of this paper is far from the acceptance criteria of ICLR. there are just way too many grammatical errors or typos throughout the entire paper that prevent me from understanding this paper, such as\nSec.1\nAlthough…, however…\nIn neural network -> neural networks\nThey lack understanding -> it lacks understanding\nNumerical and quantitative reasoning is their r fundamental capability -> are .. capabilities…\n...\nJust too many to print all of them here. Please proofread your paper before submission.\n\nThe introduction is poorly written that I cannot get a full picture of what goals this paper tries/has achieved after reading it.\n\nAlgorithm 1 and 2 seem to be very poorly formatted but only illustrate minimal useful information.\n\n\nMotivation\nI don’t see clear usage nor convincing results based on the current shape of this paper -- what application could this work enable or what theoretical insights it reveals?\n\nMethod\nThis paper proposes to use neural nets to do arithmetic operations (though I don’t see convincing motivations to do so). A new idea the paper proposes is to train a few networks to first learn/fit basic operations, and then use these trained NNs to assemble large NNs which are supposed to form more complex arithmetic operations. Unfortunately, the writing of this paper prevents me from fully understanding the technical details of this paper.\n\nResults\nThe results in this paper are currently minimal. Many details about the experiment setup or how different methods are compared are not clear.\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I did not assess the derivations or theory."}, "extra_text": {}}, {"review_id": "BygZ_Hp9FS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes to use neural networks to evaluate the mathematical expressions by designing 8 small building blocks for 8 fundamental operations, e.g., addition, subtraction, etc. They then design multi-digit multiplication and division using these small blocks. \n\nThe motivation of this paper is not very clear to me, i.e., why do you want to mimic the arithmetic operations using the logic networks, what is the real use case here. In the introduction, the paper motivates by pointing out the limitation of neural networks, which is memorization based and they want to generalize by understanding the inherent rules. However, if you look at the way the fundamental building blocks are designed, and how the multiplication model works, the rules are injected based on human knowledge, e.g., the way signal digit multiplication extends to multi-digit multiplication, there is simply no understanding by the model itself. Besides, the whole process has no training, i.e., the weights of the small networks are fixed, and what is the trainable parts? \n\nThe whole paper has many spelling and grammar errors, which hinders the reading. And the writing needs to be significantly improved. ", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes to train and compose neural networks for the purposes of arithmetic operations. All reviewers agree that the motivation for such a work is unclear, and the general presentation in the paper can be significantly improved. As such, I cannot recommend this paper in its current state for publication.

Decision: Reject