OpenRaiser/Intern-Atlas huggingface dataset:
A Deep Generative Framework for Paraphrase Generation
Venue: arXiv cs.CL 2017
Year: 2017

Paraphrase generation is an important problem in NLP, especially in question answering, information retrieval, information extraction, conversation systems, to name a few. In this paper, we address the problem of generating paraphrases automatically. Our proposed method is based on a combination of deep generative models (VAE) with sequence-to-sequence models (LSTM) to generate paraphrases, given an input sentence. Traditional VAEs when combined with recurrent neural networks can generate free text but they are not suitable for paraphrase generation for a given sentence. We address this problem by conditioning the both, encoder and decoder sides of VAE, on the original sentence, so that it can generate the given sentence's paraphrases. Unlike most existing models, our model is simple, modular and can generate multiple paraphrases, for a given sentence. Quantitative evaluation of the proposed method on a benchmark paraphrase dataset demonstrates its efficacy, and its performance improvement over the state-of-the-art methods by a significant margin, whereas qualitative human evaluation indicate that the generated paraphrases are well-formed, grammatically correct, and are relevant to the input sentence. Furthermore, we evaluate our method on a newly released question paraphrase dataset, and establish a new baseline for future research.
A Semantic Approach to the Analysis of Rewriting-Based Systems
Venue: arXiv cs.PL 2017
Year: 2017

Properties expressed as the provability of a first-order sentence can be disproved by just finding a model of the negation of the sentence. This fact, however, is meaningful in restricted cases only, depending on the shape of the sentence and the class of systems at stake. In this paper we show that a number of interesting properties of rewriting-based systems can be investigated in this way, including infeasibility and non-joinability of critical pairs in (conditional) rewriting, non-loopingness of conditional rewrite systems, or the secure access to protected pages of a web site modeled as an order-sorted rewrite theory. Interestingly, this uniform, semantic approach succeeds when specific techniques developed to deal with the aforementioned problems fail.
A Streaming Accelerator for Deep Convolutional Neural Networks with Image and Feature Decomposition for Resource-limited System Applications
Venue: arXiv cs.AR 2017
Year: 2017

Deep convolutional neural networks (CNN) are widely used in modern artificial intelligence (AI) and smart vision systems but also limited by computation latency, throughput, and energy efficiency on a resource-limited scenario, such as mobile devices, internet of things (IoT), unmanned aerial vehicles (UAV), and so on. A hardware streaming architecture is proposed to accelerate convolution and pooling computations for state-of-the-art deep CNNs. It is optimized for energy efficiency by maximizing local data reuse to reduce off-chip DRAM data access. In addition, image and feature decomposition techniques are introduced to optimize memory access pattern for an arbitrary size of image and number of features within limited on-chip SRAM capacity. A prototype accelerator was implemented in TSMC 65 nm CMOS technology with 2.3 mm x 0.8 mm core area, which achieves 144 GOPS peak throughput and 0.8 TOPS/W peak energy efficiency.
Unsupervised state representation learning with robotic priors: a robustness benchmark
Venue: arXiv cs.AI 2017
Year: 2017

Our understanding of the world depends highly on our capacity to produce intuitive and simplified representations which can be easily used to solve problems. We reproduce this simplification process using a neural network to build a low dimensional state representation of the world from images acquired by a robot. As in Jonschkowski et al. 2015, we learn in an unsupervised way using prior knowledge about the world as loss functions called robotic priors and extend this approach to high dimension richer images to learn a 3D representation of the hand position of a robot from RGB images. We propose a quantitative evaluation of the learned representation using nearest neighbors in the state space that allows to assess its quality and show both the potential and limitations of robotic priors in realistic environments. We augment image size, add distractors and domain randomization, all crucial components to achieve transfer learning to real robots. Finally, we also contribute a new prior to improve the robustness of the representation. The applications of such low dimensional state representation range from easing reinforcement learning (RL) and knowledge transfer across tasks, to facilitating learning from raw data with more efficient and compact high level representations. The results show that the robotic prior approach is able to extract high level representation as the 3D position of an arm and organize it into a compact and coherent space of states in a challenging dataset.
Transcribing Against Time
Venue: arXiv cs.CL 2017
Year: 2017

We investigate the problem of manually correcting errors from an automatic speech transcript in a cost-sensitive fashion. This is done by specifying a fixed time budget, and then automatically choosing location and size of segments for correction such that the number of corrected errors is maximized. The core components, as suggested by previous research [1], are a utility model that estimates the number of errors in a particular segment, and a cost model that estimates annotation effort for the segment. In this work we propose a dynamic updating framework that allows for the training of cost models during the ongoing transcription process. This removes the need for transcriber enrollment prior to the actual transcription, and improves correction efficiency by allowing highly transcriber-adaptive cost modeling. We first confirm and analyze the improvements afforded by this method in a simulated study. We then conduct a realistic user study, observing efficiency improvements of 15% relative on average, and 42% for the participants who deviated most strongly from our initial, transcriber-agnostic cost model. Moreover, we find that our updating framework can capture dynamically changing factors, such as transcriber fatigue and topic familiarity, which we observe to have a large influence on the transcriber's working behavior.
Chebyshev Approximation and Higher Order Derivatives of Lyapunov Functions for Estimating the Domain of Attraction
Venue: arXiv cs.SY 2017
Year: 2017

Estimating the Domain of Attraction (DA) of non-polynomial systems is a challenging problem. Taylor expansion is widely adopted for transforming a nonlinear analytic function into a polynomial function, but the performance of Taylor expansion is not always satisfactory. This paper provides solvable ways for estimating the DA via Chebyshev approximation. Firstly, for Chebyshev approximation without the remainder, higher order derivatives of Lyapunov functions are used for estimating the DA, and the largest estimate is obtained by solving a generalized eigenvalue problem. Moreover, for Chebyshev approximation with the remainder, an uncertain polynomial system is reformulated, and a condition is proposed for ensuring the convergence to the largest estimate with a selected Lyapunov function. Numerical examples demonstrate that both accuracy and efficiency are improved compared to Taylor approximation.
An Evolutionary Game for User Access Mode Selection in Fog Radio Access Networks
Venue: arXiv cs.IT 2017
Year: 2017

The fog radio access network (F-RAN) is a promising paradigm for the fifth generation wireless communication systems to provide high spectral efficiency and energy efficiency. Characterizing users to select an appropriate communication mode among fog access point (F-AP), and device-to-device (D2D) in F-RANs is critical for performance optimization. Using evolutionary game theory, we investigate the dynamics of user access mode selection in F-RANs. Specifically, the competition among groups of potential users space is formulated as a dynamic evolutionary game, and the evolutionary equilibrium is the solution to this game. Stochastic geometry tool is used to derive the proposals' payoff expressions for both F-AP and D2D users by taking into account the different nodes locations, cache sizes as well as the delay cost. The analytical results obtained from the game model are evaluated via simulations, which show that the evolutionary game based access mode selection algorithm can reach a much higher payoff than the max rate based algorithm.
Commonsense Scene Semantics for Cognitive Robotics: Towards Grounding Embodied Visuo-Locomotive Interactions
Venue: arXiv cs.RO 2017
Year: 2017

We present a commonsense, qualitative model for the semantic grounding of embodied visuo-spatial and locomotive interactions. The key contribution is an integrative methodology combining low-level visual processing with high-level, human-centred representations of space and motion rooted in artificial intelligence. We demonstrate practical applicability with examples involving object interactions, and indoor movement.
And That's A Fact: Distinguishing Factual and Emotional Argumentation in Online Dialogue
Venue: arXiv cs.CL 2017
Year: 2017

We investigate the characteristics of factual and emotional argumentation styles observed in online debates. Using an annotated set of "factual" and "feeling" debate forum posts, we extract patterns that are highly correlated with factual and emotional arguments, and then apply a bootstrapping methodology to find new patterns in a larger pool of unannotated forum posts. This process automatically produces a large set of patterns representing linguistic expressions that are highly correlated with factual and emotional language. Finally, we analyze the most discriminating patterns to better understand the defining characteristics of factual and emotional arguments.
Harvesting Creative Templates for Generating Stylistically Varied Restaurant Reviews
Venue: arXiv cs.CL 2017
Year: 2017

Many of the creative and figurative elements that make language exciting are lost in translation in current natural language generation engines. In this paper, we explore a method to harvest templates from positive and negative reviews in the restaurant domain, with the goal of vastly expanding the types of stylistic variation available to the natural language generator. We learn hyperbolic adjective patterns that are representative of the strongly-valenced expressive language commonly used in either positive or negative reviews. We then identify and delexicalize entities, and use heuristics to extract generation templates from review sentences. We evaluate the learned templates against more traditional review templates, using subjective measures of "convincingness", "interestingness", and "naturalness". Our results show that the learned templates score highly on these measures. Finally, we analyze the linguistic categories that characterize the learned positive and negative templates. We plan to use the learned templates to improve the conversational style of dialogue systems in the restaurant domain.
Cystoid macular edema segmentation of Optical Coherence Tomography images using fully convolutional neural networks and fully connected CRFs
Venue: arXiv cs.CV 2017
Year: 2017

In this paper we present a new method for cystoid macular edema (CME) segmentation in retinal Optical Coherence Tomography (OCT) images, using a fully convolutional neural network (FCN) and a fully connected conditional random fields (dense CRFs). As a first step, the framework trains the FCN model to extract features from retinal layers in OCT images, which exhibit CME, and then segments CME regions using the trained model. Thereafter, dense CRFs are used to refine the segmentation according to the edema appearance. We have trained and tested the framework with OCT images from 10 patients with diabetic macular edema (DME). Our experimental results show that fluid and concrete macular edema areas were segmented with good adherence to boundaries. A segmentation accuracy of $0.61\pm 0.21$ (Dice coefficient) was achieved, with respect to the ground truth, which compares favourably with the previous state-of-the-art that used a kernel regression based method ($0.51\pm 0.34$). Our approach is versatile and we believe it can be easily adapted to detect other macular defects.
Anomaly Detection for a Water Treatment System Using Unsupervised Machine Learning
Venue: arXiv cs.LG 2017
Year: 2017

In this paper, we propose and evaluate the application of unsupervised machine learning to anomaly detection for a Cyber-Physical System (CPS). We compare two methods: Deep Neural Networks (DNN) adapted to time series data generated by a CPS, and one-class Support Vector Machines (SVM). These methods are evaluated against data from the Secure Water Treatment (SWaT) testbed, a scaled-down but fully operational raw water purification plant. For both methods, we first train detectors using a log generated by SWaT operating under normal conditions. Then, we evaluate the performance of both methods using a log generated by SWaT operating under 36 different attack scenarios. We find that our DNN generates fewer false positives than our one-class SVM while our SVM detects slightly more anomalies. Overall, our DNN has a slightly better F measure than our SVM. We discuss the characteristics of the DNN and one-class SVM used in this experiment, and compare the advantages and disadvantages of the two methods.
Supervised and Unsupervised Speech Enhancement Using Nonnegative Matrix Factorization
Venue: arXiv cs.SD 2017
Year: 2017

Reducing the interference noise in a monaural noisy speech signal has been a challenging task for many years. Compared to traditional unsupervised speech enhancement methods, e.g., Wiener filtering, supervised approaches, such as algorithms based on hidden Markov models (HMM), lead to higher-quality enhanced speech signals. However, the main practical difficulty of these approaches is that for each noise type a model is required to be trained a priori. In this paper, we investigate a new class of supervised speech denoising algorithms using nonnegative matrix factorization (NMF). We propose a novel speech enhancement method that is based on a Bayesian formulation of NMF (BNMF). To circumvent the mismatch problem between the training and testing stages, we propose two solutions. First, we use an HMM in combination with BNMF (BNMF-HMM) to derive a minimum mean square error (MMSE) estimator for the speech signal with no information about the underlying noise type. Second, we suggest a scheme to learn the required noise BNMF model online, which is then used to develop an unsupervised speech enhancement system. Extensive experiments are carried out to investigate the performance of the proposed methods under different conditions. Moreover, we compare the performance of the developed algorithms with state-of-the-art speech enhancement schemes using various objective measures. Our simulations show that the proposed BNMF-based methods outperform the competing algorithms substantially.
Combining Search with Structured Data to Create a More Engaging User Experience in Open Domain Dialogue
Venue: arXiv cs.CL 2017
Year: 2017

The greatest challenges in building sophisticated open-domain conversational agents arise directly from the potential for ongoing mixed-initiative multi-turn dialogues, which do not follow a particular plan or pursue a particular fixed information need. In order to make coherent conversational contributions in this context, a conversational agent must be able to track the types and attributes of the entities under discussion in the conversation and know how they are related. In some cases, the agent can rely on structured information sources to help identify the relevant semantic relations and produce a turn, but in other cases, the only content available comes from search, and it may be unclear which semantic relations hold between the search results and the discourse context. A further constraint is that the system must produce its contribution to the ongoing conversation in real-time. This paper describes our experience building SlugBot for the 2017 Alexa Prize, and discusses how we leveraged search and structured data from different sources to help SlugBot produce dialogic turns and carry on conversations whose length over the semi-finals user evaluation period averaged 8:17 minutes.
Process-oriented Iterative Multiple Alignment for Medical Process Mining
Venue: arXiv cs.DS 2017
Year: 2017

Adapted from biological sequence alignment, trace alignment is a process mining technique used to visualize and analyze workflow data. Any analysis done with this method, however, is affected by the alignment quality. The best existing trace alignment techniques use progressive guide-trees to heuristically approximate the optimal alignment in O(N2L2) time. These algorithms are heavily dependent on the selected guide-tree metric, often return sum-of-pairs-score-reducing errors that interfere with interpretation, and are computationally intensive for large datasets. To alleviate these issues, we propose process-oriented iterative multiple alignment (PIMA), which contains specialized optimizations to better handle workflow data. We demonstrate that PIMA is a flexible framework capable of achieving better sum-of-pairs score than existing trace alignment algorithms in only O(NL2) time. We applied PIMA to analyzing medical workflow data, showing how iterative alignment can better represent the data and facilitate the extraction of insights from data visualization.
A Kinodynamic Aggressive Trajectory Planner For Narrow Passages
Venue: arXiv cs.SY 2017
Year: 2017

Planning a path for a nonholonomic robot is a challenging topic in motion planning and it becomes more difficult when the desired path contains narrow passages. This kind of scenario can arise, for instance, when quadcopters search a collapsed building after a natural disaster. Choosing the quadcopter as the target platform, this paper proposes the Kinodynamic Aggressive Trajectory (KAT) motion planning algorithm, which aims to compute aggressive trajectories for narrow passages under nonholonomic constraints. This type of maneuvers is necessary because the dynamics of quadcopters entail that some narrow passages can only be traversed at high speed. To find the best path, the KAT uses RRT to determine a holonomic path first and then adjusts it to satisfy the nonholonomic constraints. The innovations in this process are: 1) The states of the robot are divided into near-holonomic set and non-holonomic set, which makes the constraints local rather than global; 2) For each of the most confined waypoints in the path, KAT plans forward and backward simultaneously around the waypoint to find a feasible local trajectory traversing the narrow passage. Our approach thus transforms a globally-constrained planning problem into a problem with local constraints, and as a result, the computation becomes tractable. We evaluate KAT by applying it to a quadcopter flying through two inclined holes that require aggressive maneuvers in a simulated environment. The average computation time to successfully find a solution for passing two 50 degree inclined holes is around 32 seconds.
Codes over Affine Algebras with a Finite Commutative Chain coefficient Ring
Venue: arXiv cs.IT 2017
Year: 2017

We consider codes defined over an affine algebra $\mathcal A=R[X_1,\dots,X_r]/\left\langle t_1(X_1),\dots,t_r(X_r)\right\rangle$, where $t_i(X_i)$ is a monic univariate polynomial over a finite commutative chain ring $R$. Namely, we study the $\mathcal A-$submodules of $\mathcal A^l$ ($l\in \mathbb{N}$). These codes generalize both the codes over finite quotients of polynomial rings and the multivariable codes over finite chain rings. {Some codes over Frobenius local rings that are not chain rings are also of this type}. A canonical generator matrix for these codes is introduced with the help of the Canonical Generating System. Duality of the codes is also considered.
sPIN: High-performance streaming Processing in the Network
Venue: arXiv cs.DC 2017
Year: 2017

Optimizing communication performance is imperative for large-scale computing because communication overheads limit the strong scalability of parallel applications. Today's network cards contain rather powerful processors optimized for data movement. However, these devices are limited to fixed functions, such as remote direct memory access. We develop sPIN, a portable programming model to offload simple packet processing functions to the network card. To demonstrate the potential of the model, we design a cycle-accurate simulation environment by combining the network simulator LogGOPSim and the CPU simulator gem5. We implement offloaded message matching, datatype processing, and collective communications and demonstrate transparent full-application speedups. Furthermore, we show how sPIN can be used to accelerate redundant in-memory filesystems and several other use cases. Our work investigates a portable packet-processing network acceleration model similar to compute acceleration with CUDA or OpenCL. We show how such network acceleration enables an eco-system that can significantly speed up applications and system services.
Speech Dereverberation Using Nonnegative Convolutive Transfer Function and Spectro temporal Modeling
Venue: arXiv cs.SD 2017
Year: 2017

This paper presents two single channel speech dereverberation methods to enhance the quality of speech signals that have been recorded in an enclosed space. For both methods, the room acoustics are modeled using a nonnegative approximation of the convolutive transfer function (NCTF), and to additionally exploit the spectral properties of the speech signal, such as the low rank nature of the speech spectrogram, the speech spectrogram is modeled using nonnegative matrix factorization (NMF). Two methods are described to combine the NCTF and NMF models. In the first method, referred to as the integrated method, a cost function is constructed by directly integrating the speech NMF model into the NCTF model, while in the second method, referred to as the weighted method, the NCTF and NMF based cost functions are weighted and summed. Efficient update rules are derived to solve both optimization problems. In addition, an extension of the integrated method is presented, which exploits the temporal dependencies of the speech signal. Several experiments are performed on reverberant speech signals with and without background noise, where the integrated method yields a considerably higher speech quality than the baseline NCTF method and a state of the art spectral enhancement method. Moreover, the experimental results indicate that the weighted method can even lead to a better performance in terms of instrumental quality measures, but that the optimal weighting parameter depends on the room acoustics and the utilized NMF model. Modeling the temporal dependencies in the integrated method was found to be useful only for highly reverberant conditions.
Generalized PMC model for the hybrid diagnosis of multiprocessor systems
Venue: arXiv cs.DC 2017
Year: 2017

Fault diagnosis is important to the design and maintenance of large multiprocessor systems. PMC model is the most famous diagnosis model in the system level diagnosis of multiprocessor systems. Under the PMC model, only node faults are allowed. But in real circumstances, link faults may occur. So based on the PMC model, we propose in this paper a diagnosis model called the generalized PMC(GPMC) model to adapt to the real circumstances. The foundation of GPMC model has been established. And to measure the fault diagnosis capability of multiprocessor systems under the GPMC model, the fault diagnosis capability measuring parameters: $h$-edge restricted diagnosability and $h$-vertex restricted edge diagnosability have been introduced. As an application, the $h$-edge restricted diagnosability and $h$-vertex restricted edge diagnosability of hypercubes are explored.