OpenRaiser/Intern-Atlas huggingface dataset:
A Deep Relevance Matching Model for Ad-hoc Retrieval
Venue: arXiv cs.IR 2017
Year: 2017

In recent years, deep neural networks have led to exciting breakthroughs in speech recognition, computer vision, and natural language processing (NLP) tasks. However, there have been few positive results of deep models on ad-hoc retrieval tasks. This is partially due to the fact that many important characteristics of the ad-hoc retrieval task have not been well addressed in deep models yet. Typically, the ad-hoc retrieval task is formalized as a matching problem between two pieces of text in existing work using deep models, and treated equivalent to many NLP tasks such as paraphrase identification, question answering and automatic conversation. However, we argue that the ad-hoc retrieval task is mainly about relevance matching while most NLP matching tasks concern semantic matching, and there are some fundamental differences between these two matching tasks. Successful relevance matching requires proper handling of the exact matching signals, query term importance, and diverse matching requirements. In this paper, we propose a novel deep relevance matching model (DRMM) for ad-hoc retrieval. Specifically, our model employs a joint deep architecture at the query term level for relevance matching. By using matching histogram mapping, a feed forward matching network, and a term gating network, we can effectively deal with the three relevance matching factors mentioned above. Experimental results on two representative benchmark collections show that our model can significantly outperform some well-known retrieval models as well as state-of-the-art deep matching models.
Modelling Domain Relationships for Transfer Learning on Retrieval-based Question Answering Systems in E-commerce
Venue: arXiv cs.CL 2017
Year: 2017

In this paper, we study transfer learning for the PI and NLI problems, aiming to propose a general framework, which can effectively and efficiently adapt the shared knowledge learned from a resource-rich source domain to a resource- poor target domain. Specifically, since most existing transfer learning methods only focus on learning a shared feature space across domains while ignoring the relationship between the source and target domains, we propose to simultaneously learn shared representations and domain relationships in a unified framework. Furthermore, we propose an efficient and effective hybrid model by combining a sentence encoding- based method and a sentence interaction-based method as our base model. Extensive experiments on both paraphrase identification and natural language inference demonstrate that our base model is efficient and has promising performance compared to the competing models, and our transfer learning method can help to significantly boost the performance. Further analysis shows that the inter-domain and intra-domain relationship captured by our model are insightful. Last but not least, we deploy our transfer learning model for PI into our online chatbot system, which can bring in significant improvements over our existing system. Finally, we launch our new system on the chatbot platform Eva in our E-commerce site AliExpress.
A Survey on Network Embedding
Venue: arXiv cs.SI 2017
Year: 2017

Network embedding assigns nodes in a network to low-dimensional representations and effectively preserves the network structure. Recently, a significant amount of progresses have been made toward this emerging network analysis paradigm. In this survey, we focus on categorizing and then reviewing the current development on network embedding methods, and point out its future research directions. We first summarize the motivation of network embedding. We discuss the classical graph embedding algorithms and their relationship with network embedding. Afterwards and primarily, we provide a comprehensive overview of a large number of network embedding methods in a systematic manner, covering the structure- and property-preserving network embedding methods, the network embedding methods with side information and the advanced information preserving network embedding methods. Moreover, several evaluation approaches for network embedding and some useful online resources, including the network data sets and softwares, are reviewed, too. Finally, we discuss the framework of exploiting these network embedding methods to build an effective system and point out some potential future directions.
A surgical system for automatic registration, stiffness mapping and dynamic image overlay
Venue: arXiv cs.RO 2017
Year: 2017

In this paper we develop a surgical system using the da Vinci research kit (dVRK) that is capable of autonomously searching for tumors and dynamically displaying the tumor location using augmented reality. Such a system has the potential to quickly reveal the location and shape of tumors and visually overlay that information to reduce the cognitive overload of the surgeon. We believe that our approach is one of the first to incorporate state-of-the-art methods in registration, force sensing and tumor localization into a unified surgical system. First, the preoperative model is registered to the intra-operative scene using a Bingham distribution-based filtering approach. An active level set estimation is then used to find the location and the shape of the tumors. We use a recently developed miniature force sensor to perform the palpation. The estimated stiffness map is then dynamically overlaid onto the registered preoperative model of the organ. We demonstrate the efficacy of our system by performing experiments on phantom prostate models with embedded stiff inclusions.
Exploring Approximations for Floating-Point Arithmetic using UppSAT
Venue: arXiv cs.LO 2017
Year: 2017

We consider the problem of solving floating-point constraints obtained from software verification. We present UppSAT --- a new implementation of a systematic approximation refinement framework [ZWR17] as an abstract SMT solver. Provided with an approximation and a decision procedure (implemented in an off-the-shelf SMT solver), UppSAT yields an approximating SMT solver. Additionally, UppSAT includes a library of predefined approximation components which can be combined and extended to define new encodings, orderings and solving strategies. We propose that UppSAT can be used as a sandbox for easy and flexible exploration of new approximations. To substantiate this, we explore several approximations of floating-point arithmetic. Approximations can be viewed as a composition of an encoding into a target theory, a precision ordering, and a number of strategies for model reconstruction and precision (or approximation) refinement. We present encodings of floating-point arithmetic into reduced precision floating-point arithmetic, real-arithmetic, and fixed-point arithmetic (encoded in the theory of bit-vectors). In an experimental evaluation, we compare the advantages and disadvantages of approximating solvers obtained by combining various encodings and decision procedures (based on existing state-of-the-art SMT solvers for floating-point, real, and bit-vector arithmetic).
Random Words in a (Weighted) Regular Language: a Free Energy Approach
Venue: arXiv cs.FL 2017
Year: 2017

We study random words in a weighted regular language that achieve the maximal free energy using thermodynamics formalism. In particular, typical words in the language are algorithmically generated which have applications in computer security (anomaly detection) and software enegineering (test case generation).
Technical Report: A Trace-Based Performance Study of Autoscaling Workloads of Workflows in Datacenters
Venue: arXiv cs.DC 2017
Year: 2017

To improve customer experience, datacenter operators offer support for simplifying application and resource management. For example, running workloads of workflows on behalf of customers is desirable, but requires increasingly more sophisticated autoscaling policies, that is, policies that dynamically provision resources for the customer. Although selecting and tuning autoscaling policies is a challenging task for datacenter operators, so far relatively few studies investigate the performance of autoscaling for workloads of workflows. Complementing previous knowledge, in this work we propose the first comprehensive performance study in the field. Using trace-based simulation, we compare state-of-the-art autoscaling policies across multiple application domains, workload arrival patterns (e.g., burstiness), and system utilization levels. We further investigate the interplay between autoscaling and regular allocation policies, and the complexity cost of autoscaling. Our quantitative study focuses not only on traditional performance metrics and on state-of-the-art elasticity metrics, but also on time- and memory-related autoscaling-complexity metrics. Our main results give strong and quantitative evidence about previously unreported operational behavior, for example, that autoscaling policies perform differently across application domains and by how much they differ.
SENATUS: An Approach to Joint Traffic Anomaly Detection and Root Cause Analysis
Venue: arXiv cs.NI 2017
Year: 2017

In this paper, we propose a novel approach, called SENATUS, for joint traffic anomaly detection and root-cause analysis. Inspired from the concept of a senate, the key idea of the proposed approach is divided into three stages: election, voting and decision. At the election stage, a small number of \nop{traffic flow sets (termed as senator flows)}senator flows are chosen\nop{, which are used} to represent approximately the total (usually huge) set of traffic flows. In the voting stage, anomaly detection is applied on the senator flows and the detected anomalies are correlated to identify the most possible anomalous time bins. Finally in the decision stage, a machine learning technique is applied to the senator flows of each anomalous time bin to find the root cause of the anomalies. We evaluate SENATUS using traffic traces collected from the Pan European network, GEANT, and compare against another approach which detects anomalies using lossless compression of traffic histograms. We show the effectiveness of SENATUS in diagnosing anomaly types: network scans and DoS/DDoS attacks.
Interactive Robot Learning of Gestures, Language and Affordances
Venue: arXiv cs.RO 2017
Year: 2017

A growing field in robotics and Artificial Intelligence (AI) research is human-robot collaboration, whose target is to enable effective teamwork between humans and robots. However, in many situations human teams are still superior to human-robot teams, primarily because human teams can easily agree on a common goal with language, and the individual members observe each other effectively, leveraging their shared motor repertoire and sensorimotor resources. This paper shows that for cognitive robots it is possible, and indeed fruitful, to combine knowledge acquired from interacting with elements of the environment (affordance exploration) with the probabilistic observation of another agent's actions. We propose a model that unites (i) learning robot affordances and word descriptions with (ii) statistical recognition of human gestures with vision sensors. We discuss theoretical motivations, possible implementations, and we show initial results which highlight that, after having acquired knowledge of its surrounding environment, a humanoid robot can generalize this knowledge to the case when it observes another agent (human partner) performing the same motor actions previously executed during training.
Efficient and Invariant Convolutional Neural Networks for Dense Prediction
Venue: arXiv cs.CV 2017
Year: 2017

Convolutional neural networks have shown great success on feature extraction from raw input data such as images. Although convolutional neural networks are invariant to translations on the inputs, they are not invariant to other transformations, including rotation and flip. Recent attempts have been made to incorporate more invariance in image recognition applications, but they are not applicable to dense prediction tasks, such as image segmentation. In this paper, we propose a set of methods based on kernel rotation and flip to enable rotation and flip invariance in convolutional neural networks. The kernel rotation can be achieved on kernels of 3 $\times$ 3, while kernel flip can be applied on kernels of any size. By rotating in eight or four angles, the convolutional layers could produce the corresponding number of feature maps based on eight or four different kernels. By using flip, the convolution layer can produce three feature maps. By combining produced feature maps using maxout, the resource requirement could be significantly reduced while still retain the invariance properties. Experimental results demonstrate that the proposed methods can achieve various invariance at reasonable resource requirements in terms of both memory and time.
Geometric robustness of deep networks: analysis and improvement
Venue: arXiv cs.CV 2017
Year: 2017

Deep convolutional neural networks have been shown to be vulnerable to arbitrary geometric transformations. However, there is no systematic method to measure the invariance properties of deep networks to such transformations. We propose ManiFool as a simple yet scalable algorithm to measure the invariance of deep networks. In particular, our algorithm measures the robustness of deep networks to geometric transformations in a worst-case regime as they can be problematic for sensitive applications. Our extensive experimental results show that ManiFool can be used to measure the invariance of fairly complex networks on high dimensional datasets and these values can be used for analyzing the reasons for it. Furthermore, we build on Manifool to propose a new adversarial training scheme and we show its effectiveness on improving the invariance properties of deep neural networks.
Cascade Attribute Learning Network
Venue: arXiv cs.AI 2017
Year: 2017

We propose the cascade attribute learning network (CALNet), which can learn attributes in a control task separately and assemble them together. Our contribution is twofold: first we propose attribute learning in reinforcement learning (RL). Attributes used to be modeled using constraint functions or terms in the objective function, making it hard to transfer. Attribute learning, on the other hand, models these task properties as modules in the policy network. We also propose using novel cascading compensative networks in the CALNet to learn and assemble attributes. Using the CALNet, one can zero shoot an unseen task by separately learning all its attributes, and assembling the attribute modules. We have validated the capacity of our model on a wide variety of control problems with attributes in time, position, velocity and acceleration phases.
Observer-Side Parameter Estimation For Adaptive Control
Venue: arXiv cs.SY 2017
Year: 2017

In adaptive control, a controller is precisely designed for a certain model of the system, but that model's parameters are updated online by another mechanism called the adaptive update. This allows the controller to aim for the benefits of exact model knowledge while simultaneously remaining robust to model uncertainty. Like most nonlinear controllers, adaptive controllers are often designed and analyzed under the assumption of deterministic full state feedback. However, doing so inherently decouples the adaptive update mechanism from the probabilistic information provided by modern state observers. The simplest way to reconcile this is to let the observer produce both state estimates and model parameter estimates, so that all probabilistic information is shared within the framework of the observer. While this technique is becoming increasingly common, it is still not widely accepted due to a lack of general closed-loop stability proofs. In this thesis, we will investigate observer-side parameter estimation for adaptive control by precisely juxtaposing its mechanics against the current, widely accepted adaptive control designs. Additionally, we will propose a new technique that increases the robustness of observer-based adaptive control by following the same line of reasoning used for the popular concurrent learning method.
An Exploration of Word Embedding Initialization in Deep-Learning Tasks
Venue: arXiv cs.CL 2017
Year: 2017

Word embeddings are the interface between the world of discrete units of text processing and the continuous, differentiable world of neural networks. In this work, we examine various random and pretrained initialization methods for embeddings used in deep networks and their effect on the performance on four NLP tasks with both recurrent and convolutional architectures. We confirm that pretrained embeddings are a little better than random initialization, especially considering the speed of learning. On the other hand, we do not see any significant difference between various methods of random initialization, as long as the variance is kept reasonably low. High-variance initialization prevents the network to use the space of embeddings and forces it to use other free parameters to accomplish the task. We support this hypothesis by observing the performance in learning lexical relations and by the fact that the network can learn to perform reasonably in its task even with fixed random embeddings.
Disentangling Dynamics and Content for Control and Planning
Venue: arXiv cs.LG 2017
Year: 2017

In this paper, We study the problem of learning a controllable representation for high-dimensional observations of dynamical systems. Specifically, we consider a situation where there are multiple sets of observations of dynamical systems with identical underlying dynamics. Only one of these sets has information about the effect of actions on the observation and the rest are just some random observations of the system. Our goal is to utilize the information in that one set and find a representation for the other sets that can be used for planning and ling-term prediction.
4, 8, 32, 64 bit Substitution Box generation using Irreducible or Reducible Polynomials over Galois Field GF(p^q) for Smart Applications
Venue: arXiv cs.CR 2017
Year: 2017

Substitution Box or S-Box had been generated using 4-bit Boolean Functions (BFs) for Encryption and Decryption Algorithm of Lucifer and Data Encryption Standard (DES) in late sixties and late seventies respectively. The S-box of Advance Encryption Standard have also been generated using Irreducible Polynomials over Galois field GF(2^8) adding an additive constant in early twenty first century. In this paper Substitution Boxes have been generated from Irreducible or Reducible Polynomials over Galois field GF(p^q). Binary Galois fields have been used to generate Substitution Boxes. Since the Galois Field Number or the Number generated from coefficients of a polynomial over a particular Binary Galois field (2q) is similar to log 2 q+1 bit BFs. So generation of log 2 q+1 bit S-boxes is Possible. Now if p = prime or non-prime number then generation of S-Boxes is possible using Galois field GF (p^q). where, q = p-1.
Real-Time Capable Micro-Doppler Signature Decomposition of Walking Human Limbs
Venue: arXiv cs.CV 2017
Year: 2017

Unique micro-Doppler signature ($\boldsymbolμ$-D) of a human body motion can be analyzed as the superposition of different body parts $\boldsymbolμ$-D signatures. Extraction of human limbs $\boldsymbolμ$-D signatures in real-time can be used to detect, classify and track human motion especially for safety application. In this paper, two methods are combined to simulate $\boldsymbolμ$-D signatures of a walking human. Furthermore, a novel limbs $μ$-D signature time independent decomposition feasibility study is presented based on features as $μ$-D signatures and range profiles also known as micro-Range ($μ$-R). Walking human body parts can be divided into four classes (base, arms, legs, feet) and a decision tree classifier is used. Validation is done and the classifier is able to decompose $μ$-D signatures of limbs from a walking human signature on real-time basis.
Selling to a No-Regret Buyer
Venue: arXiv cs.GT 2017
Year: 2017

We consider the problem of a single seller repeatedly selling a single item to a single buyer (specifically, the buyer has a value drawn fresh from known distribution $D$ in every round). Prior work assumes that the buyer is fully rational and will perfectly reason about how their bids today affect the seller's decisions tomorrow. In this work we initiate a different direction: the buyer simply runs a no-regret learning algorithm over possible bids. We provide a fairly complete characterization of optimal auctions for the seller in this domain. Specifically: - If the buyer bids according to EXP3 (or any "mean-based" learning algorithm), then the seller can extract expected revenue arbitrarily close to the expected welfare. This auction is independent of the buyer's valuation $D$, but somewhat unnatural as it is sometimes in the buyer's interest to overbid. - There exists a learning algorithm $\mathcal{A}$ such that if the buyer bids according to $\mathcal{A}$ then the optimal strategy for the seller is simply to post the Myerson reserve for $D$ every round. - If the buyer bids according to EXP3 (or any "mean-based" learning algorithm), but the seller is restricted to "natural" auction formats where overbidding is dominated (e.g. Generalized First-Price or Generalized Second-Price), then the optimal strategy for the seller is a pay-your-bid format with decreasing reserves over time. Moreover, the seller's optimal achievable revenue is characterized by a linear program, and can be unboundedly better than the best truthful auction yet simultaneously unboundedly worse than the expected welfare.
A Formal Specification Framework for Smart Grid Components
Venue: arXiv cs.LO 2017
Year: 2017

Smart grid can be considered as the next step in the evolution of power systems. It comprises of different entities and objects ranging from smart appliances, smart meters, generators, smart storages, and more. One key problem in modeling smart grid is that while currently there is a considerable focus on the practical aspects of smart grid, there are very few modeling attempts and even lesser attempts at formalization. To the best of our knowledge, among other formal methods, formal specification has previously not been applied in the domain of smart grid. In this paper, we attempt to bridge this gap by presenting a novel approach to modeling smart grid components using a formal specification approach. We use a state-based formal specification language namely Z (pronounced as `Zed') since we believe Z is particularly suited for modeling smart grid components.We demonstrate the application of Z on key smart grid components. The presented formal specification can be considered as first steps towards modeling of smart grid using a Software Engineering formalism. It also demonstrates how complex systems, such as the smart grid, can be modeled elegantly using formal specification.
Multiple Instance Curriculum Learning for Weakly Supervised Object Detection
Venue: arXiv cs.CV 2017
Year: 2017

When supervising an object detector with weakly labeled data, most existing approaches are prone to trapping in the discriminative object parts, e.g., finding the face of a cat instead of the full body, due to lacking the supervision on the extent of full objects. To address this challenge, we incorporate object segmentation into the detector training, which guides the model to correctly localize the full objects. We propose the multiple instance curriculum learning (MICL) method, which injects curriculum learning (CL) into the multiple instance learning (MIL) framework. The MICL method starts by automatically picking the easy training examples, where the extent of the segmentation masks agree with detection bounding boxes. The training set is gradually expanded to include harder examples to train strong detectors that handle complex images. The proposed MICL method with segmentation in the loop outperforms the state-of-the-art weakly supervised object detectors by a substantial margin on the PASCAL VOC datasets.