Venue: AKBC 2019 Conference

Reviews:

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: The paper presents innovative work towards learning numerical attributes in a KB, which the authors claim to be the first of its kind. The approach leverages KB embeddings to learn feature representations for predicting missing numerical attribute values. The assumption is that data points that are close to each other in a vector (embeddings) space have the same numerical attribute value. Evaluation of the approach is on a set of highest ranked 10 numerical attributes in a QA setting with questions that require a numerical answer.

The paper is well written and the approach is explained  in full detail. 

My only concern is the application of the method across numerical attributes of (very) different granularity and context. The authors briefly mention the granularity aspect in section 3 and point to a normalization step. However, this discussion is very brief and leaves perhaps some further questions open on this.
review_title: Solid work, convincing experiments and results

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: The paper reports on the prediction of numerical attributes in knowledge bases, a problem that has indeed
received too little attention. It lays out the problem rather clearly and defines datasets, a number of baselines
as well as a range of embedding-based models that I like because they include both simple pipeline models 
(learn embeddings first, predict numerical attributes later) and models that include the numerical attributes
into embedding learning. I also appreciate that the paper makes an attempt to draw together ideas from different
research directions.

Overall, I like the paper. It's very solidly done and can serve as excellent base for further studies. Of course, I also have comments/criticisms.

First, the authors state that one of the contributions of the paper is to "introduce the problem of predicting the value 
of entities’ numerical attributes in KB". This is unfortunately not true. There is relatively old work by Davidov and
Rappoport (ACL 2010) on learning numeric attributes from the web (albeit without a specific reference to KBs), and 
a more recent study specifically aimed at attributes from KBs (Gupta et al. EMNLP 2015) which proposed and modelled exactly the same task, including defining freely available FreeBase-derived datasets.
More generally speaking, the authors seem to be very up to date regarding approaches that learn embeddings directly
from the KB, but not regarding approaches that use text-based embeddings. This is unfortunate, since the
model that we defined is closely related to the LR model defined in the current paper, but no direct comparison is 
possible due to the differences in embeddings and dataset.

Second, I feel that not enough motivation is given for some of the models and their design decisions. For example,
the choice of linear regression seems rather questionable to me, since the assumptions of linear regression (normal distribution/homoscedasticity) are clearly violated by many KB attributes. If you predict, say, country populations, the 
error for China and India is orders of magnitude higher then the error for other countries, and the predictions are dominated by the fit to these outliers. This not only concerns the models but also the evaluation, because
MAE/RMSE also only make sense when you assume that the attributes scale linearly -- Gupta et al. 2015 use this
as motivation for using logistic regression and a rank-based evaluation. 
I realize that you comment on non-linear regression on p.12 and give a normalized evaluation on p.13: 
I appreciate that, even though I think that it only addresses the problem to an extent.

Similarly, I like the label propagation idea (p. 7) but I lack the intuition why LP should work on (all) numeric attributes.
If, say, two countries border each other, I would expect their lat/long to be similar, but why should their (absolute) GDP be similar? What is lacking here is a somewhat more substantial discussion of the assumptions that this (and the other) 
models make about the structure of the knowledge graph and the semantics of the attributes.

Smaller comment:
* Top of p.5, formalization: This looks like f is a general function, but I assume that one f is supposed to be learned
   for each attribute? Either it should be f_a, or f should have the signature E x A -> R.
   (p4: Why is N used as a separate type for numeric attributes if the function f is then supposed to map into reals anyway?)


review_title: Nice paper on prediction of numeric attributes in KBs.

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper presents a method to predict the values
of numerical properties for entities where these properties
are missing in the KB (e.g., population of cities or height of athletes).
To this end, the paper develops a suite of methods, using
regression and learned embeddings. Some of the techniques
resemble those used in state-of-the-art knowledge graph completion,
but there is novelty in adapting these techniques
to the case of numerical properties.
The paper includes a comprehensive experimental evaluation
of a variety of methods.

This is interesting work on an underexplored problem,
and it is carried out very neatly.

However, I am skeptical that it is practically viable.
The prediction errors are such that the predicted values
can hardly be entered into a high-quality knowledge base.
For example, for city population, the RMSE is in the order
of millions, and for person height the RMSE is above 0.05
(i.e., 5cm). So even on average, the predicted values are
way off. Moreover, average errors are not really the decisive
point for KB completion and curation.
Even if the RMSE were small, say 10K for cities, for some
cities the predictions could still be embarrassingly off.
So a knowledge engineer could not trust them and would 
hardly consider them for completing the missing values.

Specific comment:
The embeddings of two entities may be close for different
reasons, potentially losing too much information.
For example, two cities may have close embeddings because
they are in the same geo-region (but could have very different
populations) or because they have similar characteristics
(e.g., both being huge metropolitan areas). Likewise, two
athletes could be close because of similar origin and
similar success in Olympic Games, or because they played
in the same teams. 
It is not clear how the proposed methods can cope with
these confusing signals through embeddings or other techniques.
review_title: interesting paper with decent contribution, but doubts about practical viability

Meta-review: metareview: The authors consider the problem of predicting or imputing numerical attributes in knowledge bases. In contrast to simple local or global attribute prediction, they design a regression-based model that uses knowledge graph embeddings and extend the embedding computation to also use numerical attributes. Evaluation shows interesting preliminary results.

The critical consensus was that this paper is worthy of acceptance, but has several serious limitations that should be carefully explained. A critical issue is that the relationships used for propagating information do not necessarily correlate to similar numerical attribute values (e.g., geographical location is useful for predicting lat/long but not GDP or population), that a linear regression model is not sufficient to capture the full extent of relationships, that normalization of error values across different relationships skews the evaluation, several important and relevant related works were omitted, and the overall error rates are still somewhat high.
confidence: 4: The area chair is confident but not absolutely certain
Decision: Accept (Poster)