From: ICLR.cc/2023/Conference/Program_Chairs
title: Paper Decision
decision: Reject
metareview:_summary,_strengths_and_weaknesses: This is a rather unique paper, proposing a variant of single-layer perceptrons modified to include recurrent connections, as an alternative to standard deep learning architectures. It’s an interesting idea, but it seems undeveloped and vague. For instance, the original submission is only 3 pages of text. 

All reviewers indicated that the paper is far too short and lacking in technical details. Having read the paper myself, I tend to agree. The figures especially are rather difficult to figure out, as they include only brief legends. The figures appear improved in the updated paper, but are still rather inscrutable. I agree with review joB1 that this paper is still a work in progress and would benefit from much more development, better presentation, and some sort of experimental evaluation of their proposed method. The reviewers, especially joB1, gave some great feedback and I urge authors to consider incorporating these in future submissions of this work.

Note that the authors added their rebuttals to the updated version of the paper itself, rather than via openreview, and hence the reviewers likely missed them initially.


justification_for_why_not_higher_score: This paper seems quite incomplete and lacking in technical detail. It’s presented in a confusing way and would benefit from more work and attention, and is thus not suitable for publication at this time.
justification_for_why_not_lower_score: N/A

From: ICLR.cc/2023/Conference/Paper5553/Authors
title: Thank you
comment: Your review was especially useful to help improve the manuscript. An updated version with a clearer Figure 3, a new Figure 4, some numerical justification is now provided. The link to Lawrence 2022b works now : https://openreview.net/forum?id=401LFvBGIb

From: ICLR.cc/2023/Conference/Paper5553/Reviewer_wULo
confidence: 2: You are willing to defend your assessment, but it is quite likely that you did not understand the central parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: The paper suggests an alternative network architecture, but lacks analytical and numerical evaluation.
strength_and_weaknesses: Strength:
* Out of the box

Weakness: 
* Lack of box
clarity,_quality,_novelty_and_reproducibility: Novel, but might profit from "a gifted mathematicians to make new connections or help neuroscientists in unravelling the mysteries of small-scale brains."

summary_of_the_review: Ingenious, but in lack of a theory and of numerics
correctness: 2: Several of the paper’s claims are incorrect or not well-supported.
technical_novelty_and_significance: 1: The contributions are neither significant nor novel.
empirical_novelty_and_significance: Not applicable
flag_for_ethics_review: NO.
recommendation: 3: reject, not good enough

From: ICLR.cc/2023/Conference/Paper5553/Reviewer_sByK
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: SLIM is intended to be a perceptron that involves a minimally connected recurrent network. The internal state variables have a nearest-neighbor interaction on a chain. The authors conjecture such networks could realize arbitrary N-bit Boolean functions, in contrast to the limitation of Rosenblatt perceptrons, as shown by Minsky and Papert.
strength_and_weaknesses: Strengths:
The paper explores alternatives to the deep learning architecture and hopes to have self-organized nanomaterial-based learning machines.

Weaknesses:
The key idea is not so novel. From a multitude of recurrent neural networks (including the reservoir computing framework, mentioned by the authors) to predictive coding networks, many approaches utilize the idea of a network with internal dynamics, the equilibrium point of which helps perform the task.

There is very little work done to substantiate the claim for the particular architecture proposed.

clarity,_quality,_novelty_and_reproducibility: The purpose of the paper is clear. Quality, novelty and reproducibility is hard to comment on since there is so little material.
summary_of_the_review: This paper would need much more work to rise to the level of being worthy of consideration for publication at ICLR. At this point, it is just a scheme, and not a particularly novel one.
correctness: 2: Several of the paper’s claims are incorrect or not well-supported.
technical_novelty_and_significance: 1: The contributions are neither significant nor novel.
empirical_novelty_and_significance: Not applicable
flag_for_ethics_review: NO.
recommendation: 1: strong reject

From: ICLR.cc/2023/Conference/Paper5553/Reviewer_h2U8
confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
summary_of_the_paper: The authors introduce a structural variant of single-layer perceptrons with added recurrent connections. The model seems to be mostly a standard RNN, with the different of a more general activation function.
No evaluation is provided, aside from a hand-designed example.
strength_and_weaknesses: Strengths:
- The paper is short, which makes it easy to read.

Weaknesses:
- The main problem tackled in the paper is not a particular concern since the early 80s and is largely solved.
- The structure of the paper is unorthodox and lacks most of the required content (concerning the theory and the model) and any practical evaluation. 
- Figure 3 is difficult to understand, as no explanation is given about the notation in the text nor in the caption.
- The proposed architecture seems to be a standard RNN with a more flexible activation function, which however the authors leave undefined.
- Related literature is severely lacking, both to justify the significance of the problem addressed and all the work done on it since the 80s.

clarity,_quality,_novelty_and_reproducibility: The paper is written poorly and in a non-standard way, and cannot be accepted in its present form.
The work presented does not seem novel, or it is only marginally novel.
Almost no implementation details are given, and the only example is designed by hand.

summary_of_the_review: N/A
correctness: 3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.
technical_novelty_and_significance: 1: The contributions are neither significant nor novel.
empirical_novelty_and_significance: 1: The contributions are neither significant nor novel.
flag_for_ethics_review: NO.
recommendation: 1: strong reject

From: ICLR.cc/2023/Conference/Paper5553/Reviewer_joB1
confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
summary_of_the_paper: Goals and contribution:
* The authors try to draw attention of the community to shallow recurrent neural networks trained by Hebbian learning in the context of cognitive material systems. 
* They argue that these models could compete with deep-learning systems in the future  - thanks to upcoming advancement of in-materio technologies.
* They demonstrate this idea on an illustrative example of the N-bit (2-bit) parity function realized by the new-proposed SLIM (Shallow learning in Materio) model.
* The authors also point out several crucial open problems of the proposed approach as topics for future research.


strength_and_weaknesses: Strengths: 
1. I agree with the authors that the shallow recurrent higher-order neural networks are a promising area of future research - due to the progress of in-materio technologies. 
2. The illustrative example is well-chosen. It demonstrates well how the proposed SLIM model could overcame the classical problem of n-bit parity (Minsky&Pappert,1988) - thanks to the higher-order units used instead of threshold logic gates.
3. The authors provide several open questions concerning their model as possible subjects of future research.

Limitations:
1. The paper is very brief  and vague in every aspect and its techniqual quality is imited.  
2. There is no theoretical justification of the method (and its convergence) and the empirical evaluation is limited to a toy problem.

Discussion of the limitations:
1. The paper is very brief  and vague in every aspect and its techniqual quality is imited.  
  * Mainly, the proposed model should be described in more detail, including training algorithm and analysis/discussion of the choice of the F_i functions.
  * The illustrative example also needs a more detailed description. How did you choose F_1,2 and why? Was the model trained by Hebbian learning? 
    Can you show the concrete final model (its F-functions and weights)? Figure 3 is not comprehensible - could you describe more clearly what the four images represent? 
2. There is no theoretical justification of the method (and its convergence) and the empirical evaluation is limited to a toy problem.
The paper addresses more limitations and problems of the proposed model than its advances.  Many important questions are not addressed: 
 * Is it possible to extend the 2-bit parity model to the n-bit one? Are you able to show a solution for n=3?
 * The training process of the model is not described/analyzed.
 * The choice of F-functions for different tasks is not described/analyzed.
 * From the paper it seems like it is difficult to successfully  apply the model to different tasks. Is it right?  
 
clarity,_quality,_novelty_and_reproducibility: Originality:
* The main idea of the proposed approach - to apply the recurrent shallow neural networks in the context of in-materio technologies (enabling to use higher-order units) - is novel and interesting. 
However, the realization is not very convincing (due to the lack of clear description, deeper theoretical or experimental evaluation or comparison to alternative approaches). 

Related works seem to be cited adequately, except:
- Based on Section 1, the paper closely follows (Lawrence, 2022b). However, (Lawrence, 2022b) seems to be unavailable (the link doesn't work and I was not able to find it elsewhere)
 - Because the authors concentrate on the n-bit parity function realized by shallow networks, they should compare their approach to some alternative/similar models and cite them (e.g., [1], [2]).
 


Clarity and reproducibility: 
* Both the proposed model and the experiment should be described in more detail. 
There are many open questions that hinder reproducibility (e.g., missing description of the training process, missing detailed description of the used F-functions,...). 
* Figure 3 is not well-described and thus harly comprehensible.

Quality:
* The technical quality of the paper is poor. The paper looks like an incomplete work in progress (see "Strength And Weaknesses" Section for details).

Minor:
* (Rumelhart et al., 1986) is cited as a reference to TPU-based architectures, isn't that an error?

[1] Aizenberg, I. Solving the XOR and parity N problems using a single universal binary neuron. Soft Comput 12, 215–222 (2008). https://doi.org/10.1007/s00500-007-0204-9

[2] Iyoda, E.M., Nobuhara, H. & Hirota, K. A Solution for the N-bit Parity Problem Using a Single Translated Multiplicative Neuron. Neural Processing Letters 18, 233–238 (2003). https://doi.org/10.1023/B:NEPL.0000011147.74207.8c



summary_of_the_review: Although the main idea of the paper is novel and interesting, its realization is poor. The proposed model is not well-described and it is not adequately empirically or theoretically justified. 
correctness: 2: Several of the paper’s claims are incorrect or not well-supported.
technical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
empirical_novelty_and_significance: 2: The contributions are only marginally significant or novel.
flag_for_ethics_review: NO.
recommendation: 3: reject, not good enough