Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 6: Weak Accept: Technically solid, moderate-to-high impact paper, with no major concerns with respect to evaluation, resources, reproducibility, ethical considerations.
review_content: summary: Neural circuits present in the fly mushroom body provide a nice biological analog of a simple feedforward artificial neural network. Such circuits also exist in cerebellum and have been studied both from the lens of supervised learning and unsupervised representations. This paper looks at the unsupervised aspect, studying how such networks can construct locality-sensitive embeddings/hashes into higher dimensions. The main contribution of this work is to study data-dependent ways for these circuits to embed. Three methods are presented and compared to some past methods. These data-dependent hashing methods are shown to offer improvements in some situations, and a few hypotheses as to when/why these methods do or do not work better are presented.
strengths_and_weaknesses: Strengths: The paper is overall pretty well-written and the methods are tested on multiple datasets. The motivation from neuroscience and computer science fits well into the (historical) focus of NeurIPS. 

Weaknesses: The main weakness is that code is not provided and the algorithm descriptions in the paper are confusing, especially the notation that was used. I do not think I could reproduce the paper's results without code. It is unclear whether the comparison of runtimes with other methods is entirely fair because the implementation details are not fully-described. Some claims of why some methods work or do not (in terms of "clustered" data) are not well-justified.

Specific critiques by line number:
36: parenthetical "(if the hash length...)" is awkward, rephrase

Fig 2 caption: NHL, ALH, APF abbreviations don't get used elswhere, could remove to make figure more readable

53: should cite Sophie Caron's latest work beside Eichler and Elkahlah

54: Litwin-Kumar et al studied random connections, so this theoretical study is not a good one to refer to here

90: corpora should be "corpus"

111: "and this form of network can be related to radial basis functions" is unclear

121: The summary of FlyLSH in this section is long and not the method which is presented. I recomment shortening this to allow a better description of the novel methods.

133: "should not be sensitive to this..." this claim is unclear, because you haven't presented your own methods yet. 

134: "most intensive pixels" unclear, why should we be thinking about pixels? Any kind of data could be used.

143: use \leq instead of <=

144: use \ll instead of <<

155: You need to spend much more space describing the novel methods. This is the most confusing part of the paper. 

Algorithm 2 line 4: I believe you lack and "and" between $\mathbf{y}' > \theta$ and $\exists$.

198-199: This should be earlier in Section 3.

200: I found the description of the datasets to be very good.

213: 1000 to 1,000 (be consistent throughout document, there are multiple instances of this)

214: Suggested rewording "For each query sample,"

215: capitalize "Hamming", strikeout "sample,"

217: remove ", that is,"

225: This paragraph seems unnecessary, out-of-place, and potentially confusing. I would put it elsewhere.

234: WTA not defined

245: mean average precision (mAP) not defined

Fig 3: It is unclear how the algorithms were implemented and how this might influence the runtimes

Fig 5: Define "magnitude variance"

285: change "MB inspired" to "MB-inspired"

Fig 6: use curly braces 10^{-2}

299: Method 1 doesn't depend on the original data order, so this isn't surprising.

302-304: "The threshold..." sentence needs more explanation

322: "... properties of the data" would benefit from supporting citations

questions: Can you provide code?

Algorithm 1/2: The notation needs to be explained. What is $\mathbf{i}'$ in particular its dimension? How do you determine the "top n values", by what metric? What range can $\beta$ cover? What is $\mathbf{y}'$ initialized to? What does it mean for $\mathbf{y}' > \theta$ if $\mathbf{y}'$ is a vector? What is the variable $u$? Unless these issues are all addressed, the algorithms are unreproducible.

Last paragraph of Section 4.2: The claim that CIFAR-10 is different due to differences in magnitudes suggests that standardization would fix this problem. I don't see how CIFAR-10 isn't clustered, there are 10 classes, which is stated in line 338. Not much evidence is presented for the claim (272) "LSH methods work best on datasets that are clustered and samples that lie on the hypersphere".... No clear definition of clustering is given. Also, sphereing the data by normalization or standardization is a simple thing to try to support your claim.


limitations: There needs to be more discussion of limitations of the methods proposed. Also, the support for many of the claims as to why the methods work in some cases and not in others is limited, and the authors shouldn't hide this.

The discussion of societal impact is almost non-existant and seems an afterthough. However, the authors mention "security camera video" as a potential data source (344). What kind of ethical issues could come with this? I suggest doing more here.
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This paper suggests a new architecture for nearest neighbor search. The paper uses a new algorithm to design 
a neural network that solves NN that is inspired by circuitry in the insect brain. In contrast to previous approaches that use random connections  the paper suggests a non random architecture. The new method outperforms previous neuro-inspired methods in several metrics such as search time and accuracy. 
strengths_and_weaknesses: The experimental results are nice and the algorithms is interesting. I was uncertain about the end goal of this paper. 
Is it to find the best network for NN? If yes much more comprehensive comparisons should be made. Is it to give new explanations as to how the insect brain solves NN? That connections are typically non random in biological neural networks is long known. E.g., 
"Highly Nonrandom Features of Synaptic Connectivity in Local Cortical Circuits"
questions: I would elaborate more on the primary goal of this paper. Also some mathematical proofs of what the algorithms achieves can strengthen the contribution. 
limitations: In contrast to Dasgupta, S., Stevens, C. F., & Navlakha, S. (2017) no provable guarantees are given regarding the performance of the algorithm.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper proposes a new locally-sensitive hashing scheme based on the FlyLSH – a prior work where the hashes were obtained using a neural network modeled after the fruit fly’s olfactory system. Whereas in the FlyLSH the network’s weights were random, here the authors propose to connect the strongest input dimensions to distinct output neurons for distinct stimuli. The authors test their model on a standard set of tasks (MNIST/CIFAR/GloVe/SIFT) where they report an increase in performance.
strengths_and_weaknesses: This work presents an important and timely contribution to the field of locally-sensitive hashing. After the introduction of the FlyLSH, where the network weights are random, the next logical step was to optimize these weights in a data-driven manner to achieve better performance. This work offers an elegant and computationally light way to meet this goal.

Among the strengths of this work are:

-The computational simplicity of the proposed method. The learning of the weights can be performed online as the new samples, substantially different from the already-learned ones, are assigned the new KCs on the output layer. The evaluation is also fast due to the fixed-length sparse code which allows finding the nearest neighbors in a small number of comparisons.

-The thorough testing of the method. The authors have tested their proposed model on four standard benchmarks in a way compatible with analogous testing in previous works; they show an increase in the model’s performance compared to the FlyLSH baseline. Besides, the differences in performance are discussed for the data types in datasets and the best use cases for the model are defined.

-The text is well-written and well-structured including a comprehensive introduction and a thorough description of the related work.

-The implications of this work for neuroscience may be interesting. Intuitively, the model may replicate: i) the difference in the projection densities of the ORs, ii) the lack of the “grandparent cells” among the KCs, i.e. the cells encoding individual odors, and iii) the “primacy” hypothesis stating that the odors may be encoded by the sets of the earliest-responding glomeruli.

For these reasons the paper appears to be a substantial contribution relevant to the NeurIPS community

questions: No questions at this time.
limitations: The limitations are discussed adequately.
review_title: null

Meta-review: metareview: This paper provides learning algorithm to learn connections in a biologically-motivated network for locality sensitive hashing. This contribution extends previous work where such connections were randomly chosen. I recommend acceptance based on the overall sentiment of the reviews.
confidence: Certain
Decision: Accept