TASK DEFINITION: In this task, you are given a part of an article. Your task is to generate headline (title) for this text. Preferred headlines are under fifteen words.
PROBLEM: With the large volume of new information created every day, determining the validity of information in a knowledge graph and filling in its missing parts are crucial tasks for many researchers and practitioners. To address this challenge, a number of knowledge graph completion methods have been developed using low-dimensional graph embeddings. Although researchers continue to improve these models using an increasingly complex feature space, we show that simple changes in the architecture of the underlying model can outperform state-of-the-art models without the need for complex feature engineering. In this work, we present a shared variable neural network model called ProjE that fills-in missing information in a knowledge graph by learning joint embeddings of the knowledge graph’s entities and edges, and through subtle, but important, changes to the standard loss function. In doing so, ProjE has a parameter size that is smaller than 11 out of 15 existing methods while performing 37% better than the current-best method on standard datasets. We also show, via a new fact checking task, that ProjE is capable of accurately determining the veracity of many declarative statements. Knowledge Graphs (KGs) have become a crucial resource for many tasks in machine learning, data mining, and artificial intelligence applications including question answering [34], entity disambiguation [7], named entity linking [14], fact checking [32], and link prediction [28] to name a few. In our view, KGs are an example of a heterogeneous information network containing entity-nodes and relationship-edges corresponding to RDF-style triples 〈h, r, t〉 where h represents a head entity, and r is a relationship that connects h to a tail entity t. KGs are widely used for many practical tasks, however, their correctness and completeness are not guaranteed. Therefore, it is necessary to develop knowledge graph completion (KGC) methods to find missing or errant relationships with the goal of improving the general quality of KGs, which, in turn, can be used to improve or create interesting downstream applications. The KGC task can be divided into two non-mutually exclusive sub-tasks: (i) entity prediction and (ii) relationship prediction. The entity prediction task takes a partial triple 〈h, r, ?〉 as input and produces a ranked list of candidate entities as output: Definition 1. (Entity Ranking Problem) Given a Knowledge Graph G = {E,R} and an input triple 〈h, r, ?〉, the entity ranking problem attempts to find the optimal ordered list such that ∀ej∀ei ((ej ∈ E− ∧ ei ∈ E+)→ ei ≺ ej), where E+ = {e ∈ {e1, e2, . . . , el}|〈h, r, e〉 ∈ G} and E− = {e ∈ {el+1, el+2, . . . , e|E|}|〈h, r, e〉 / ∈ G}. Distinguishing between head and tail-entities is usually arbitrary, so we can easily substitute 〈h, r, ?〉 for 〈?, r, t〉. The relationship prediction task aims to find a ranked list of relationships that connect a head-entity with a tail-entity, i.e., 〈h, ?, t〉. When discussing the details of the present work, we focus specifically on the entity prediction task; however, it is straightforward to adapt the methodology to the relationship prediction task by changing the input. A number of KGC algorithms have been developed in recent years, and the most successful models all have one thing in common: they use low-dimensional embedding vectors to represent entities and relationships. Many embedding models, e.g., Unstructured [3], TransE [4], TransH [35], and TransR [25], use a margin-based pairwise ranking loss function, which measures the score of each possible result as the Ln-distance between h+ r and t. In these models the loss functions are all the same, so models differ in how they transform the 1 ar X iv :1 61 1. 05 42 5v 1 [ cs .A I] 1 6 N ov 2 01 6 entity embeddings h and t with respect to the relationship embeddings r. Instead of simply adding h + r, more expressive combination operators are learned by Knowledge Vault [8] and HolE [29] in order to predict the existence of 〈h, r, t〉 in the KG. Other models, such as the Neural Tensor Network (NTN) [33] and the Compositional Vector Space Model (CVSM) [27], incorporate a multilayer neural network solution into the existing models. Unfortunately, due to their extremely large parameter size, these models either (i) do not scale well or (2) consider only a single relationship at a time [10] thereby limiting their usefulness on large, real-world KGs. Despite their large model size, the aforementioned methods only use singleton triples, i.e., length-1 paths in the KG. PTransE [24] and RTransE [10] employ extended path information from 2 and 3-hop trails over the knowledge graph. These extended models achieve excellent performance due to the richness of the input data; unfortunately, their model-size grows exponentially as the path-length increases, which further exacerbates the scalability issues associated with the already high number of parameters of the underlying-models. Another curious finding is that some of the existing models are not self-contained models, i.e., they require pre-trained KG embeddings (RTransE, CVSM), pre-selected paths (PTransE, RTransE), or pre-computed content embeddings of each node (DKRL [36]) before their model training can even begin. TransR and TransH are self-contained models, but their experiments only report results using pre-trained TransE embeddings as input. With these considerations in mind, in the present work we rethink some of the basic decisions made by previous models to create a projection embedding model (ProjE) for KGC. ProjE has four parts that distinguish it from the related work: 1. Instead of measuring the distance between input triple 〈h, r, ?〉 and entity candidates on a unified or a relationship-specific plane, we choose to project the entity candidates onto a target vector representing the input data. 2. Unlike existing models that use transformation matrices, we combine the embedding vectors representing the input data into a target vector using a learnable combination operator. This avoids the addition of a large number of transformation matrices by reusing the entity-embeddings. 3. Rather than optimizing the margin-based pairwise ranking loss, we optimize a ranking loss of the list of candidate-entities (or relationships) collectively. We further use candidate sampling to handle very large data sets. 4. Unlike many of the related models that require pre-trained data from prerequisite models or explore expensive multi-hop paths through the knowledge graph, ProjE is a self-contained model over length-1 edges.

SOLUTION: ProjE: Embedding Projection for Knowledge Graph Completion

PROBLEM: Integration between biology and information science benefits both fields. Many related models have been proposed, such as computational visual cognition models, computational motor control models, integrations of both and so on. In general, the robustness and precision of recognition is one of the key problems for object recognition models. In this paper, inspired by features of human recognition process and their biological mechanisms, a new integrated and dynamic framework is proposed to mimic the semantic extraction, concept formation and feature re-selection in human visual processing. The main contributions of the proposed model are as follows: (1) Semantic feature extraction: Local semantic features are learnt from episodic features that are extracted from raw images through a deep neural network; (2) Integrated concept formation: Concepts are formed with local semantic information and structural information learnt through network. (3) Feature re-selection: When ambiguity is detected during recognition process, distinctive features according to the difference between ambiguous candidates are re-selected for recognition. Experimental results on hand-written digits and facial shape dataset show that, compared with other methods, the new proposed model exhibits higher robustness and precision for visual recognition, especially in the condition when input samples are smantic ambiguous. Meanwhile, the introduced biological mechanisms further strengthen the interaction between neuroscience and information science.

SOLUTION: A Novel Biologically Mechanism-Based Visual Cognition Model –Automatic Extraction of Semantics, Formation of Integrated Concepts and Re-selection Features for Ambiguity

PROBLEM: This paper considers a general data-fitting problem over a networked system, in which many computing nodes are connected by an undirected graph. This kind of problem can find many real-world applications and has been studied extensively in the literature. However, existing solutions either need a central controller for information sharing or requires slot synchronization among different nodes, which increases the difficulty of practical implementations, especially for a very large and heterogeneous system. As a contrast, in this paper, we treat the data-fitting problem over the network as a stochastic programming problem with many constraints. By adapting the results in a recent paper [18], we design a fully distributed and asynchronized stochastic gradient descent (SGD) algorithm. We show that our algorithm can achieve global optimality and consensus asymptotically by only local computations and communications. Additionally, we provide a sharp lower bound for the convergence speed in the regular graph case. This result fits the intuition and provides guidance to design a ‘good’ network topology to speed up the convergence. Also, the merit of our design is validated by experiments on both synthetic and real-world datasets.

SOLUTION:
Fully Distributed and Asynchronized Stochastic Gradient Descent for Networked Systems