Venue: ESWC 2019 Workshop KGB

Reviews:

final_score: rating: 1: Strong rejection
review_content: The main problem with this paper is that a lot of topics are being tackled in a superficial way.
The introduction says that the following items will be discussed:
- the process of building knowledge graphs for molecular biology data
- a set of expert curated molecular biological knowledge bases
- the different possible applications for the knowledge graphs and their embedding models in learning different biological associations
- a new knowledge graph based on SwissProt knowledge base that is centred around proteins 
- a comparative evaluation for a set of knowledge graph embedding models on the proposed knowledge graph to predict protein associations to different biological entities

I discuss each of them in more details now.

# Process of building knowledge graphs for molecular biology data
The text contains one small paragraph about this. Considering that text claims that this is a study, this is not enough.

# Set of expert curated molecular biological knowledge bases
The text only talks about the most popular knowledge graphs, but (i) how do you determine what are popular knowledge graphs and (ii) why do you only take a subset of the knowledge graphs when you are conducting a study?
The text mentions that problems with current processes for building graphs will be identified. This does not happen. Furthermore, only association labelling and merging biological associations are mentioned as integration topics, but why only these two?

# Different possible applications for the knowledge graphs and their embedding models in learning different biological associations
The main problems here are that no reason is given why current solutions do not work and that no reason is given why knowledge graphs should be used in stead.

# New knowledge graph based on SwissProt knowledge base that is centred around proteins
Why do you create this knowledge graph? In the text it is said to be an experimental example, but an example of what? What is being proven here? What is the experiment?

# Comparative evaluation for a set of knowledge graph embedding models on the proposed knowledge graph to predict protein associations to different biological entities
It is not clear what you want to assess in the evaluation. Based on the rest of the paper I would expect to get an evaluation about the building process of knowledge graphs.

Overall, the text need to be updated to reflect a clear (single) problem that is being tackled. At the moment a lot of different topics are being touched upon, but this is done in a superficial way. So this does not qualify as a study. Furthermore, due to the lack of a clear problem statement, it is difficult to understand (i) what the paper is about, (ii) what readers will have gained after reading the paper and (iii) how the different sections are related to each other.
review_title: Lot of topics are tackled in superficial way

final_score: rating: 1: Strong rejection
review_content: The paper "On Molecular Biological Knowledge Graphs" lack of clear problem statement and overall there is lack of coherence in the description of the paper. After reading the paper it is unclear what were the main aims and contributions and in its current form cannot be accepted. I have the following specific comments:
- "biological knowledge bases became available in many forms including biological paper abstracts" sounds strange - the paper cited is about a text indexer, please rephrase
- Section 3.1 does not talk about processing of the data whereas Section 3.2 goes on to say "After processing..." - the section needs more details of the kind of processing
- "the data extracted from them is usually formatted as sets of pairs of entities with unlabelled associations." - is that true for all databases?
- Section 3.2 is very vague - the actual labeling and merging methods are not described in detail
- "During the process of extracting protein associations with other entities,..." is unclear
- "we label each association instance according to the type of the destination entity,..." are these "types" pre-defined?
- "Negative assertions for both testing and validation examples are generating using corrupting either the subject or the object of the associations using uniform randomly selected entities of the same type as the original entity." Please provide an example
- "random models as baseline" sounds very vague and please avoid use of random, that is not very scientific
- "We discussed the process of building knowledge graph from popular curated biological knowledge sources,..." this is not described in the paper
- Also, how your results tie back to the use cases is unclear
- " we have presented an example experimental pipeline of processing, building and using a knowledge graph of molecular biological data" - Is the pipeline published? Is it re-usable?
- Also there is no mention of related work and comparison of your methods with existing work along with your contributions and innovation

See also
- Evaluation of Knowledge Graph Embedding Approaches for Drug-Drug Interaction Prediction using Linked Open Data https://swat4hcls.figshare.com/articles/Evaluation_of_Knowledge_Graph_Embedding_Approaches_for_Drug-Drug_Interaction_Prediction_using_Linked_Open_Data/7325186
- eagle-i: biomedical research resource datasets http://www.semantic-web-journal.net/system/files/swj533.pdf

The paper is also poorly written. I have spotted these errors but the paper needs to be proof read again:
- In general, facts should be backed by references
- Please take care of capitalizing words in the references (e.g. ClinVar)
- "easiest for of association" - "easiest for assocation"
- "can belong different" - "can belong to different"
- "can intuitively fixed" - "can be intuitively fixed"
- "drug-target interactions [?]." - reference number missing
-  "we assess investigate" - "we investigate"
review_title: Lack of problem statement and lack of coherence

final_score: rating: 1: Strong rejection
review_content: This article suffers from an important problem, in that it does not contain a problem statement. As such, it is very hard for me to even summarize what the paper is about. To the best of my understanding, I can say that the paper does something with knowledge graphs and biological knowledge, and that the numbers at the end of the article should indicate that embedding models improve over the baseline model. Unfortunately, because of the way the article is written, I cannot determine a) the novelty of what is described, and b) the meaning of the supplied numbers. As such, I cannot recommend acceptance.

In what follows, I will try to provide the author with constructive feedback on how to approach writing down the discussed topics, such that its usefulness and applicability improves.

As written above, the main issue is a total lack of a problem statement. It is unclear from the title and the abstract what the research undertaken is, and what problem it addresses (and whether that was successful). The title itself is an indication of this issue: the article indeed does _something_ with biological knowledge graphs, but what that is, is never fully clear. The abstract and introduction should state a concrete problem: something is wrong or missing with biological knowledge graphs, and this article explains how the author has fixed that. One such problem could be, for instance, that there is no methodological study about the different phases of knowledge graphs in molecular biology (which appears to be one of the problems the article is tackling). That said, I am not sure whether this specific problem is in fact a problem that has not been solved. Another problem could be the effectiveness of knowledge graphs for the domain of molecular biology (which is what the evaluation seems to be addressing).

Several sections seem to lack novelty, or at least do not distinguish existing work from novel contributions. What is new about 3? Is it the systematic study? Because it concerns existing work, some of which are already relatively old. The same goes for 4. It seems that 2/3/4 are one long Related Work section, which should be much shorter for articles of this length.

The main contribution seems to be in 5, but I honestly did not understand it. What is the question we are finding an answer to here? Are we looking for the best model? What about this is not known in existing work?

Apart from that, the paper also suffers from some inaccurate definitions. For instance, the very first sentence that "knowledge graph became a popular means for modelling data of interconnected entities i.e. linked data" seems very weird, because Linked Data forms a knowledge graph. So I disagree that those concepts have this relation. I have a similar problem with the first sentence of the introduction, which seems to provide another definition of the same concepts.

Given these issues, I cannot recommend the article for acceptance. I encourage the author to study the creation of a research question/problem, and to clearly state that problem in the beginning. For now, the article is very confusing, and its contributions are unclear.
review_title: Missing clear problem statement, and clarity about contributions over existing work

Meta-review: This contribution is rejected
Decision: Reject