Title: Antonymy-Synonymy Discrimination through the Repelling Parasiamese Neural Network
Authors: Mathias Etcheverry
Dina Wonsever
Venue: ICLR 2022
Keywords: antitransitivity
parasiamese network
antonymy-synonymy discrimination

Abstract: Antonymic and synonymic pairs may both occur nearby in word embeddings spaces because they have similar distributional information. Different methods have been used in order to distinguish antonyms from synonyms, making the antonymy-synonymy discrimination a popular NLP task. In this work, we propose the repelling parasiamese neural network, a model which considers a siamese network for synonymy and a parasiamese network for antonymy, both sharing the same base network. Relying in the antagonism between synoymy and antonymy, the model attempts to repell siamese and parasiamese outputs making use of the contrastive loss functions. We experimentally show that the repelling parasiamese network achieves state-of-the-art results on this task.


Review discussion:
I think there are some nice contributions here.  It is always nice to find a new dataset such as Fallows.  The triplet suggestion is also nice.
The repelling suggestion is a nice improvement over Etcheverry and Wonsever.
It would help the community to provide a github.
I found table 4 a bit hard to read.  I wonder if it would help to move table 4 closer to the discussion of triplets (3.1.2), and then make
it clear how the values in the table correspond to variables in the formula.
NO.
> I think there are some nice contributions here. It is always nice to find a new dataset such as Fallows. The triplet suggestion is also nice.
Thanks!
>The repelling suggestion is a nice improvement over Etcheverry and Wonsever.
Thanks!
> It would help the community to provide a github.
A github will be provided.
> I found table 4 a bit hard to read. I wonder if it would help to move table 4 closer to the discussion of triplets (3.1.2), and then make it clear how the values in the table correspond to variables in the formula.
Good suggestion.  A new version will be uploaded soon.  Thank you for your review!
While this paper proposes a new architecture for antonymy detection with provably strong results, I do not recommend this paper for acceptance due to several reasons, the chief among them being the lack of clarity in the paper.
- First, the motivation for the architecture is very hand-wavy, and while it "works", the justification provided is not solid. It is is also very hard to connect the various parts of the paper. Part of this is the language of the paper (see next section), but also the connections between sections are not well made.
- Second, the paper is couched in very unclear language which makes this further difficult to follow.
For instance, I do not understand this what this sentence means (Sec 2.2 -- "Antitransitivity is a drawback if a distance function is pretended to be considered to distinguish antonymy, "). Simplifying the language of the paper and improving the clarity will go a long way in improving the strength of the paper.
- Finally, the paper does not contextualize itself very well with respect to prior work.
Specifically, the work of Mrkšic´ et al. has been cited, but the concept of the repelling parasiamese network and the attract-repel framework of that paper have extremely similar philosophies. It would be useful to understand what is common with that work, and what is different (i understand the architecture is different, but the question is at a more conceptual label).
- I am also uncertain about the quality of the dictionary based dataset. Can the authors comment on the how they ensured that the ground truth in this dictionary is actually of high quality? Was there any human evaluation done on a subset of entries of this dataset?
NO.
Thanks for your review. Clarifications and improvements are being made based on your feedback.
The task and the idea are indeed interesting.
However, there are some moments that require clarification.
In Section 2.3.1, the expression (3) directly illustrates one part of the definition of antitransitivity relations. It is not clear why the special name “left anti-Euclidean” relation was introduced and why the Property occurs.
Moreover, the name “anti-Euclidean” does not seem to be proper. If there is a need for such a term, then the Proof is inconsistent. As, again, expression (3) shows only one side of antitransitivity definition, then the proof should also cover a part when aRb /\ bRc -> !(bRc).
It is not clear why in Section 4.1 three datasets are described, but the Table 3 with results consists only of two parts. Are Xie’s and Nguyen’s datasets incorporated in each other somehow?
Also, it would be better to show more details about datasets. It was stated that they consist of pairs of antonyms and synonyms relations.
However, one of the main stated goals was to show that antitransitivity relations could be learnt by the model and illustrated on the data. Table 4 shows such an example set.
But it is interesting to know if there are any such triangles in the train datasets originally and how the model behaves on such samples.
The work will also benefit from ablation study of the necessity of both types losses for model training. In Table 4 it is unclear what letter in brackets (T) and (P) are stand for. It can be guessed, but it requires direct description in the caption of the table.
So, more clarification how losses were included in training, both of them or separately, would make the work more understandable.
NO.
Thank you for your review.   Euclideanness related parts will be removed.
Some responses between lines:
> It is not clear why in Section 4.1 three datasets are described, but the Table 3 with results consists only of two parts. Are Xie’s and Nguyen’s datasets incorporated in each other somehow?
Ok. The rows below "Lexical Split" are from Xie's dataset. This part will be clarified. Thanks.
> Also, it would be better to show more details about datasets. It was stated that they consist of pairs of antonyms and synonyms relations.
However, one of the main stated goals was to show that antitransitivity relations could be learnt by the model and illustrated on the data. Table 4 shows such an example set.
But it is interesting to know if there are any such triangles in the train datasets originally and how the model behaves on such samples.
Thanks. Good idea!
> The work will also benefit from ablation study of the necessity of both types losses for model training. In Table 4 it is unclear what letter in brackets (T) and (P) are stand for. It can be guessed, but it requires direct description in the caption of the table.
So, more clarification how losses were included in training, both of them or separately, would make the work more understandable.
Thanks. This will be improved. A new version will be uploaded soon.

Meta-review:
role: PC
data: comment: The paper presents a new approach for distinguishing synonyms and antonyms via an extension of a parasiamese neural network, called "the repelling parasiamese network".  The strengths of the paper, as identified by reviewers, are a novel architecture for antonymy detection, a new dataset, and solid empirical results. However, there are major drawbacks identified by reviewers w5dj and hoTU. Specifically, there are clarity issues in writing, lack of a proper justification to the proposed architecture, insufficient details about the quality of datasets, insufficient contextualization in prior work. The scores are borderline, but unfortunately, the authors did not use the rebuttal opportunity to sufficiently address these questions/concerns raised by the reviewers. I thus recommend to reject the paper.

Decision: Reject