instruction:
In this task, you are given a paper review. Based on the review, your job is to identify language and generate "en" if the review is in English or generate "es" if the review is in Spanish. Note that URLs in the text have been replaced with [Link].
question:
This paper presents an approach to tag word senses with temporal information (past, present, future or atemporal). They model the problem using a graph-based semi-supervised classification algorithm that allows to combine item specific information - such as the presence of some temporal indicators in the glosses - and the structure of Wordnet - that is semantic relations between synsets â, and to take into account unlabeled data. They perform a full annotation of Wordnet, based on a set of training data labeled in a previous work and using the rest of Wordnet as unlabeled data. Specifically, they take advantage of the structure of the label set by breaking the task into a binary formulation (temporal vs atemporal), then using the data labeled as temporal to perform a finer grained tagging (past, present or future). In order to intrinsically evaluate their approach, they annotate a subset of synsets in Wordnet using crowd-sourcing. They compare their system to the results obtained by a state-of-the-art time tagger (Stanford's SUTime) using an heuristic as a backup strategy, and to previous works. They obtain improvements around 11% in accuracy, and show that their approach allows performance higher than previous systems using only 400 labeled data. Finally, they perform an evaluation of their resource on an existing task (TempEval-3) and show improvements of about 10% in F1 on 4 labels.  This paper is well-constructed and generally clear, the approach seems sound and well justified. This work led to the development of a resource with fine grained temporal information at the word sense level that would be made available and could be used to improve various NLP tasks. I have a few remarks, especially concerning the settings of the experiments.  I think that more information should be given on the task performed in the extrinsic evaluation section. An example could be useful to understand what the system is trying to predict (the features describe âentity pairsâ but it has not been made clear before what are these pairs) and what are the features (especially, what are the entity attributes? What is the POS for a pair, is it one dimension or two? Are the lemmas obtained automatically?). The sentence describing the labels used is confusing, I'm not sure to understand what âevent to document creation timeâ and âevent to same sentence eventâ means, are they the kind of pairs considered? Are they relations (as they are described as relation at the beginning of p.8)? I find unclear the footnote about the 14 relations: why the other relations have to be ignored, what makes a mapping too âcomplexâ? Also, are the scores macro or micro averaged? Finally, the ablation study seems to indicate a possible redundancy between Lexica and Entity with quite close scores, any clue about this behavior?  I have also some questions about the use of the SVM.  For the extrinsic evaluation, the authors say that they optimized the parameters of the algorithm: what are these parameters?  And since a SVM is also used within the MinCut framework, is it optimized and how? Finally, if it's the LibSVM library that is used (Weka wrapper), I think a reference to LibSVM should be included.   Other remarks: - It would be interesting to have the number of examples per label in the gold data, the figures are given for coarse grained labels (127 temporal vs 271 atemporal), but not for the finer grained. - It would also be nice to have an idea of the number of words that are ambiguous at the temporal level, words like âpresentâ. - It is said in the caption of the table 3 that the results presented are âsignificantly betterâ but no significancy test is indicated, neither any p-value.  Minor remarks: - Related work: what kind of task was performed in (Filannino and Nenadic, 2014)? - Related work: ârequires a post-calibration procedureâ, needs a reference (and p.4 in 3.3 footnote it would be clearer to explain calibration) - Related work: âtheir model differ from oursâ, in what? - Table 3 is really too small: maybe, remove the parenthesis, put the â(p,r,f1)â in the caption and give only two scores, e.g. prec and f1. The caption should also be reduced. - Information in table 4 would be better represented using a graph. - Beginning of p.7: 1064 â 1264 - TempEval-3: reference ? - table 6: would be made clearer by ordering the scores for one column - p.5, paragraph 3: atemporal) â atemporal
answer:
en


question:
This paper delves into the mathematical properties of the skip-gram model, explaining the reason for its success on the analogy task and for the general superiority of additive composition models. It also establishes a link between skip-gram and Sufficient Dimensionality Reduction.  I liked the focus of this paper on explaining the properties of skip-gram, and generally found it inspiring to read. I very much appreciate the effort to understand the assumptions of the model, and the way it affects (or is affected by) the composition operations that it is used to perform. In that respect, I think it is a very worthwhile read for the community.  My main criticism is however that the paper is linguistically rather naive. The authors' use of 'compositionality' (as an operation that takes a set of words and returns another with the same meaning) is extremely strange. Two words can of course be composed and produce a vector that is a) far away from both; b) does not correspond to any other concept in the space; c) still has meaning (productivity wouldn't exist otherwise!) Compositionality in linguistic terms simply refers to the process of combining linguistic constituents to produce higher-level constructs. It does not assume any further constraint, apart from some vague (and debatable) notion of semantic transparency. The paper's implication (l254) that composition takes place over sets is also wrong: ordering matters hugely (e.g. 'sugar cane' is not 'cane sugar'). This is a well-known shortcoming of additive composition.   Another important aspect is that there are pragmatic factors that make humans prefer certain phrases to single words in particular contexts (and the opposite), naturally changing the underlying distribution of words in a large corpus. For instance, talking of a 'male royalty' rather than a 'king' or 'prince' usually has implications with regard to the intent of the speaker (here, perhaps highlighting a gender difference). This means that the equation in l258 (or for that matter the KL-divergence modification) does not hold, not because of noise in the data, but because of fundamental linguistic processes. This point may be addressed by the section on SDR, but I am not completely sure (see my comments below).  In a nutshell, I think the way that the authors present composition is flawed, but the paper convinces me that this is indeed what happens in skip-gram, and I think this is an interesting contribution.   The part about Sufficient Dimensionality Reduction seems a little disconnected from the previous argument as it stands. I'm afraid I wasn't able to fully follow the argument, and I would be grateful for some clarification in the authors' response. If I understand it well, the argument is that skip-gram produces a model where a word's neighbours follow some exponential parametrisation of a categorical distribution, but it is unclear whether this actually reflects the distribution of the corpus (as opposed to what happens in, say, a pure count-based model). The fact that skip-gram performs well despite not reflecting the data is that it implements some form of SDR, which does not need to make any assumption about the underlying form of the data. But then, is it fair to say that the resulting representations are optimised for tasks where geometrical regularities are important, regardless of the actual pattern of the data? I.e. there some kind of denoising going on?  Minor comments:  - The abstract is unusually long and could, I think, be shortened.  - para starting l71: I think it would be misconstrued to see circularity here. Firth observed that co-occurrence effects were correlated with similarity judgements, but those judgements are the very cognitive processes that we are trying to model with statistical methods. Co-occurrence effects and vector space word representations are in some sense 'the same thing', modelling an underlying linguistic process we do not have direct observations for. So pair-wise similarity is not there to break any circularity, it is there because it better models the kind of judgements humans known to make.  - l296: I think 'paraphrase' would be a better word than 'synonym' here, given that we are comparing a set of words with a unique lexical item.  - para starting l322: this is interesting, and actually, a lot of the zipfian distribution (the long tail) is fairly uniform.  - l336: it is probably worth pointing out that the analogy relation does not hold so well in practice and requires to 'ignore' the first returned neighbour of the analogy computation (which is usually one of the observed terms).  - para starting l343: I don't find it so intuitive to say that 'man' would be a synonym/paraphrase of anything involving 'woman'. The subtraction involved in the analogy computation is precisely not a straightforward composition operation, as it involves an implicit negation.   - A last, tiny general comment. It is usual to write p(w|c) to mean the probability of a word given a context, but in the paper 'w' is actually the context and 'c' the target word. It makes reading a little bit harder... Perhaps change the notation?  Literature:  The claim that Arora (2016) is the only work to try and understand vector composition is a bit strong. For instance, see the work by Paperno & Baroni on explaining the success of addition as a composition method over PMI-weighted vectors:  D. Paperno and M. Baroni. 2016. When the whole is less than the sum of its parts: How composition affects PMI values in distributional semantic vectors. Computational Linguistics 42(2): 345-350.  *** I thank the authors for their response and hope to see this paper accepted.
answer:
en


question:
This paper provides a new perspective to understanding the ResNet and Highway net. The new perspective assumes that the blocks inside the networks with residual or skip-connection are groups of successive layers with the same hidden size, which performs to iteratively refine their estimates of the same feature instead of generate new representations. Under this perspective, some contradictories with the traditional representation view induced by ResNet and Highway network and other paper can be well explained.  The pros of the paper are: 1. A novel perspective to understand the recent progress of neural network is proposed. 2. The paper provides a quantitatively experimentals to compare ResNet and Highway net, and shows contradict results with several claims from previous work. The authors also give discussions and explanations about the contradictories, which provides a good insight of the disadvantages and advantages between these two kind of networks.  The main cons of the paper is that the experiments are not sufficient. For example, since the main contribution of the paper is to propose the “unrolled iterative estimation" and the stage 4 of Figure 3 seems not follow the assumption of "unrolled iterative estimation" and the authors says: "We note that stage four (with three blocks) appears to be underestimating the representation values, indicating a probable weak link in the architecture.". Thus, it would be much better to do experiments to show that under some condition, the performance of stage 4 can follow the assumption.   Moreover, the paper should provide more experiments to show the evidence of "unrolled iterative estimation", not comparing ResNet with Highway Net. The lack of experiments on this point is the main concern from myself.
answer:

en