Antonymy-Synonymy Discrimination through the Repelling Parasiamese Neural Network

Abstract: Antonymic and synonymic pairs may both occur nearby in word embeddings spaces because they have similar distributional information. Different methods have been used in order to distinguish antonyms from synonyms, making the antonymy-synonymy discrimination a popular NLP task. In this work, we propose the repelling parasiamese neural network, a model which considers a siamese network for synonymy and a parasiamese network for antonymy, both sharing the same base network. Relying in the antagonism between synoymy and antonymy, the model attempts to repell siamese and parasiamese outputs making use of the contrastive loss functions. We experimentally show that the repelling parasiamese network achieves state-of-the-art results on this task.

Reviews and author responses:

 
> I think there are some nice contributions here. It is always nice to find a new dataset such as Fallows. The triplet suggestion is also nice.

Thanks!

>The repelling suggestion is a nice improvement over Etcheverry and Wonsever.

Thanks!

> It would help the community to provide a github.

A github will be provided.

> I found table 4 a bit hard to read. I wonder if it would help to move table 4 closer to the discussion of triplets (3.1.2), and then make it clear how the values in the table correspond to variables in the formula.

Good suggestion.  A new version will be uploaded soon.  Thank you for your review!


 
Thanks for your review. Clarifications and improvements are being made based on your feedback.


    

   

 Thank you for your review.   Euclideanness related parts will be removed. 

Some responses between lines:

> It is not clear why in Section 4.1 three datasets are described, but the Table 3 with results consists only of two parts. Are Xie’s and Nguyen’s datasets incorporated in each other somehow?

Ok. The rows below "Lexical Split" are from Xie's dataset. This part will be clarified. Thanks. 


> Also, it would be better to show more details about datasets. It was stated that they consist of pairs of antonyms and synonyms relations. However, one of the main stated goals was to show that antitransitivity relations could be learnt by the model and illustrated on the data. Table 4 shows such an example set. But it is interesting to know if there are any such triangles in the train datasets originally and how the model behaves on such samples.

Thanks. Good idea!


> The work will also benefit from ablation study of the necessity of both types losses for model training. In Table 4 it is unclear what letter in brackets (T) and (P) are stand for. It can be guessed, but it requires direct description in the caption of the table. So, more clarification how losses were included in training, both of them or separately, would make the work more understandable.

Thanks. This will be improved. A new version will be uploaded soon.  


The paper presents the new approach for distinguishing antonyms and synonyms. This task could be interesting for further applications such as paraphrasing and text entailment. The authors present proposed the repelling parasiamese neural network. The model with several setups was tested on three datasets of antonyms and synonyms pairs showing the improvement over the baselines.
 The task and the idea are indeed interesting. However, there are some moments that require clarification.

In Section 2.3.1, the expression (3) directly illustrates one part of the definition of antitransitivity relations. It is not clear why the special name “left anti-Euclidean” relation was introduced and why the Property occurs. Moreover, the name “anti-Euclidean” does not seem to be proper. If there is a need for such a term, then the Proof is inconsistent. As, again, expression (3) shows only one side of antitransitivity definition, then the proof should also cover a part when aRb /\ bRc -> !(bRc).

It is not clear why in Section 4.1 three datasets are described, but the Table 3 with results consists only of two parts. Are Xie’s and Nguyen’s datasets incorporated in each other somehow?

Also, it would be better to show more details about datasets. It was stated that they consist of pairs of antonyms and synonyms relations. However, one of the main stated goals was to show that antitransitivity relations could be learnt by the model and illustrated on the data. Table 4 shows such an example set. But it is interesting to know if there are any such triangles in the train datasets originally and how the model behaves on such samples. 

The work will also benefit from ablation study of the necessity of both types losses for model training. In Table 4 it is unclear what letter in brackets (T) and (P) are stand for. It can be guessed, but it requires direct description in the caption of the table. So, more clarification how losses were included in training, both of them or separately, would make the work more understandable.
 Overall, the paper presents an interesting approach. However, more attention should be dedicated to the details in theoretical part and more thorough testing of the proposed approaches.


This paper proposes a simple new architecture for the task of antonymy detection. The architecture is an extension of a previous architecture for the same task, known as the parasiamese network. The new architecture, called the repelling parasiamese network, essentially contains a copy of a siamese network and a parasiamese network tied together using a euclidean loss. Experimental results on several datasets demonstrate that this architecture can detect antonymy reliably and sometimes even beat the state of the art.  While this paper proposes a new architecture for antonymy detection with provably strong results, I do not recommend this paper for acceptance due to several reasons, the chief among them being the lack of clarity in the paper.

- First, the motivation for the architecture is very hand-wavy, and while it "works", the justification provided is not solid. It is is also very hard to connect the various parts of the paper. Part of this is the language of the paper (see next section), but also the connections between sections are not well made.
- Second, the paper is couched in very unclear language which makes this further difficult to follow.  For instance, I do not understand this what this sentence means (Sec 2.2 -- "Antitransitivity is a drawback if a distance function is pretended to be considered to distinguish antonymy, "). Simplifying the language of the paper and improving the clarity will go a long way in improving the strength of the paper.
- Finally, the paper does not contextualize itself very well with respect to prior work. Specifically, the work of Mrkšic´ et al. has been cited, but the concept of the repelling parasiamese network and the attract-repel framework of that paper have extremely similar philosophies. It would be useful to understand what is common with that work, and what is different (i understand the architecture is different, but the question is at a more conceptual label).
- I am also uncertain about the quality of the dictionary based dataset. Can the authors comment on the how they ensured that the ground truth in this dictionary is actually of high quality? Was there any human evaluation done on a subset of entries of this dataset? This paper is currently not suitable for acceptance as it is extremely unclear and hard to follow, and does not contextualize itself with respect to prior work clearly.

The task is a binary classification task.  The system is given a pair of words that are either synonyms or antonyms.  The task is to say which pairs are synonyms and which are antonyms.

Table 3 reports results on two datasets: (1) Nguyen's and (2) Fallows's.   The second dataset is a contribution.   This paper points out the usefulness of https://www.gutenberg.org/ebooks/51155

The paper proposes a new model, REPELLING PARASIAMESE NEURAL NETWORK.

Results in table 3 are impressive, especially on Fallows's

The suggestion of using triplets is nice. I think there are some nice contributions here.  It is always nice to find a new dataset such as Fallows.  The triplet suggestion is also nice.

The repelling suggestion is a nice improvement over Etcheverry and Wonsever.

It would help the community to provide a github.

I found table 4 a bit hard to read.  I wonder if it would help to move table 4 closer to the discussion of triplets (3.1.2), and then make
it clear how the values in the table correspond to variables in the formula.   There are some nice contributions here.  The main innovation of the paper is the repelling contribution to Etcheverry and Wonsever, though the Fallows dataset might have more impact on future papers by other researchers.

Meta-review: The paper presents a new approach for distinguishing synonyms and antonyms via an extension of a parasiamese neural network, called "the repelling parasiamese network".  The strengths of the paper, as identified by reviewers, are a novel architecture for antonymy detection, a new dataset, and solid empirical results. However, there are major drawbacks identified by reviewers w5dj and hoTU. Specifically, there are clarity issues in writing, lack of a proper justification to the proposed architecture, insufficient details about the quality of datasets, insufficient contextualization in prior work. The scores are borderline, but unfortunately, the authors did not use the rebuttal opportunity to sufficiently address these questions/concerns raised by the reviewers. I thus recommend to reject the paper.

Decision: Reject