OpenRaiser/Intern-Atlas huggingface dataset:
Exploring the power of GPU's for training Polyglot language models
Venue: arXiv cs.LG 2014
Year: 2014

One of the major research trends currently is the evolution of heterogeneous parallel computing. GP-GPU computing is being widely used and several applications have been designed to exploit the massive parallelism that GP-GPU's have to offer. While GPU's have always been widely used in areas of computer vision for image processing, little has been done to investigate whether the massive parallelism provided by GP-GPU's can be utilized effectively for Natural Language Processing(NLP) tasks. In this work, we investigate and explore the power of GP-GPU's in the task of learning language models. More specifically, we investigate the performance of training Polyglot language models using deep belief neural networks. We evaluate the performance of training the model on the GPU and present optimizations that boost the performance on the GPU.One of the key optimizations, we propose increases the performance of a function involved in calculating and updating the gradient by approximately 50 times on the GPU for sufficiently large batch sizes. We show that with the above optimizations, the GP-GPU's performance on the task increases by factor of approximately 3-4. The optimizations we made are generic Theano optimizations and hence potentially boost the performance of other models which rely on these operations.We also show that these optimizations result in the GPU's performance at this task being now comparable to that on the CPU. We conclude by presenting a thorough evaluation of the applicability of GP-GPU's for this task and highlight the factors limiting the performance of training a Polyglot model on the GPU.
A high-level model of embedded flash energy consumption
Venue: arXiv cs.AR 2014
Year: 2014

The alignment of code in the flash memory of deeply embedded SoCs can have a large impact on the total energy consumption of a computation. We investigate the effect of code alignment in six SoCs and find that a large proportion of this energy (up to 15% of total SoC energy consumption) can be saved by changes to the alignment. A flexible model is created to predict the read-access energy consumption of flash memory on deeply embedded SoCs, where code is executed in place. This model uses the instruction level memory accesses performed by the processor to calculate the flash energy consumption of a sequence of instructions. We derive the model parameters for five SoCs and validate them. The error is as low as 5%, with a 11% average normalized RMS deviation overall. The scope for using this model to optimize code alignment is explored across a range of benchmarks and SoCs. Analysis shows that over 30% of loops can be better aligned. This can significantly reduce energy while increasing code size by less than 4%. We conclude that this effect has potential as an effective optimization, saving significant energy in deeply embedded SoCs.
A Denoising Autoencoder that Guides Stochastic Search
Venue: arXiv cs.NE 2014
Year: 2014

An algorithm is described that adaptively learns a non-linear mutation distribution. It works by training a denoising autoencoder (DA) online at each generation of a genetic algorithm to reconstruct a slowly decaying memory of the best genotypes so far. A compressed hidden layer forces the autoencoder to learn hidden features in the training set that can be used to accelerate search on novel problems with similar structure. Its output neurons define a probability distribution that we sample from to produce offspring solutions. The algorithm outperforms a canonical genetic algorithm on several combinatorial optimisation problems, e.g. multidimensional 0/1 knapsack problem, MAXSAT, HIFF, and on parameter optimisation problems, e.g. Rastrigin and Rosenbrock functions.
Proposal of a multiagent-based smart environment for the IoT
Venue: arXiv cs.SE 2014
Year: 2014

This work relates to context-awareness of things that belong to IoT networks. Preferences understood as a priority in selection are considered, and dynamic preference models for such systems are built. Preference models are based on formal logic, and they are built on-the-fly by software agents observing the behavior of users/inhabitants, and gathering knowledge about preferences expressed in terms of logical specifications. A 3-level structure of agents has been introduced to support IoT inference. These agents cooperate with each other basing on the graph representation of the system knowledge. An example of such a system is presented.
Channel Assignment With Access Contention Resolution for Cognitive Radio Networks
Venue: arXiv cs.IT 2014
Year: 2014

In this paper, we consider the channel assignment problem for cognitive radio networks with hardware-constrained secondary users (SUs). In particular, we assume that SUs exploit spectrum holes on a set of channels where each SU can use at most one available channel for communication. We present the optimal brute-force search algorithm to solve the corresponding nonlinear integer optimization problem and analyze its complexity. Because the optimal solution has exponential complexity with the numbers of channels and SUs, we develop two low-complexity channel assignment algorithms that can efficiently utilize the spectrum holes. In the first algorithm, SUs are assigned distinct sets of channels. We show that this algorithm achieves the maximum throughput limit if the number of channels is sufficiently large. In addition, we propose an overlapping channel assignment algorithm that can improve the throughput performance compared with its nonoverlapping channel assignment counterpart. Moreover, we design a distributed medium access control (MAC) protocol for access contention resolution and integrate it into the overlapping channel assignment algorithm. We then analyze the saturation throughput and the complexity of the proposed channel assignment algorithms. We also present several potential extensions, including the development of greedy channel assignment algorithms under the max-min fairness criterion and throughput analysis, considering sensing errors. Finally, numerical results are presented to validate the developed theoretical results and illustrate the performance gains due to the proposed channel assignment algorithms.
Exact quantum algorithms have advantage for almost all Boolean functions
Venue: arXiv cs.CC 2014
Year: 2014

It has been proved that almost all $n$-bit Boolean functions have exact classical query complexity $n$. However, the situation seemed to be very different when we deal with exact quantum query complexity. In this paper, we prove that almost all $n$-bit Boolean functions can be computed by an exact quantum algorithm with less than $n$ queries. More exactly, we prove that ${AND}_n$ is the only $n$-bit Boolean function, up to isomorphism, that requires $n$ queries.
Streaming Algorithms for Partitioning Integer Sequences
Venue: arXiv cs.DS 2014
Year: 2014

We study the problem of partitioning integer sequences in the one-pass data streaming model. Given is an input stream of integers $X \in \{0, 1, \dots, m \}^n$ of length $n$ with maximum element $m$, and a parameter $p$. The goal is to output the positions of separators splitting the input stream into $p$ contiguous blocks such that the maximal weight of a block is minimized. We show that computing an optimal solution requires linear space, and we design space efficient $(1+ε)$-approximation algorithms for this problem following the parametric search framework. We demonstrate that parametric search can be successfully applied in the streaming model, and we present more space efficient refinements of the basic method. All discussed algorithms require space $O( \frac{1}ε \mathrm{polylog} (m,n,\frac{1}ε))$, and we prove that the linear dependency on $\frac{1}ε$ is necessary for any possibly randomized one-pass streaming algorithm that computes a $(1+ε)$-approximation.
Node, Node-Link, and Node-Link-Group Diagrams: An Evaluation
Venue: arXiv cs.HC 2014
Year: 2014

Effectively showing the relationships between objects in a dataset is one of the main tasks in information visualization. Typically there is a well-defined notion of distance between pairs of objects, and traditional approaches such as principal component analysis or multi-dimensional scaling are used to place the objects as points in 2D space, so that similar objects are close to each other. In another typical setting, the dataset is visualized as a network graph, where related nodes are connected by links. More recently, datasets are also visualized as maps, where in addition to nodes and links, there is an explicit representation of groups and clusters. We consider these three Techniques, characterized by a progressive increase of the amount of encoded information: node diagrams, node-link diagrams and node-link-group diagrams. We assess these three types of diagrams with a controlled experiment that covers nine different tasks falling broadly in three categories: node-based tasks, network-based tasks and group-based tasks. Our findings indicate that adding links, or links and group representations, does not negatively impact performance (time and accuracy) of node-based tasks. Similarly, adding group representations does not negatively impact the performance of network-based tasks. Node-link-group diagrams outperform the others on group-based tasks. These conclusions contradict results in other studies, in similar but subtly different settings. Taken together, however, such results can have significant implications for the design of standard and domain specific visualizations tools.
Iterative Detection and LDPC Decoding Algorithms for MIMO Systems in Block-Fading Channels
Venue: arXiv cs.IT 2014
Year: 2014

We propose an Iterative Detection and Decoding (IDD) scheme with Low Density Parity Check (LDPC) codes for Multiple Input Multiple Output (MIMO) systems for block-fading $F = 2$ and fast fading Rayleigh channels. An IDD receiver with soft information processing that exploits the code structure and the behaviour of the log likelihood ratios (LLR)'s is developed. Minimum Mean Square Error (MMSE) with Successive Interference Cancellation (SIC) and with Parallel Interference Cancellation (PIC) schemes are considered. The soft \textit{a posteriori} output of the decoder in a block-fading channel with Root-Check LDPC codes has allowed us to create a new strategy to improve the Bit Error Rate (BER) of a MIMO IDD scheme. Our proposed strategy in some scenarios has resulted in up to 3dB of gain in terms of BER for block-fading channels and up to 1dB in fast fading channels.
Aspect-Based Opinion Extraction from Customer reviews
Venue: arXiv cs.CL 2014
Year: 2014

Text is the main method of communicating information in the digital age. Messages, blogs, news articles, reviews, and opinionated information abound on the Internet. People commonly purchase products online and post their opinions about purchased items. This feedback is displayed publicly to assist others with their purchasing decisions, creating the need for a mechanism with which to extract and summarize useful information for enhancing the decision-making process. Our contribution is to improve the accuracy of extraction by combining different techniques from three major areas, named Data Mining, Natural Language Processing techniques and Ontologies. The proposed framework sequentially mines products aspects and users opinions, groups representative aspects by similarity, and generates an output summary. This paper focuses on the task of extracting product aspects and users opinions by extracting all possible aspects and opinions from reviews using natural language, ontology, and frequent (tag) sets. The proposed framework, when compared with an existing baseline model, yielded promising results.
Towards the Model-Driven Engineering of Secure yet Safe Embedded Systems
Venue: arXiv cs.SE 2014
Year: 2014

We introduce SysML-Sec, a SysML-based Model-Driven Engineering environment aimed at fostering the collaboration between system designers and security experts at all methodological stages of the development of an embedded system. A central issue in the design of an embedded system is the definition of the hardware/software partitioning of the architecture of the system, which should take place as early as possible. SysML-Sec aims to extend the relevance of this analysis through the integration of security requirements and threats. In particular, we propose an agile methodology whose aim is to assess early on the impact of the security requirements and of the security mechanisms designed to satisfy them over the safety of the system. Security concerns are captured in a component-centric manner through existing SysML diagrams with only minimal extensions. After the requirements captured are derived into security and cryptographic mechanisms, security properties can be formally verified over this design. To perform the latter, model transformation techniques are implemented in the SysML-Sec toolchain in order to derive a ProVerif specification from the SysML models. An automotive firmware flashing procedure serves as a guiding example throughout our presentation.
Multiple-Symbol Differential Detection for Distributed Space-Time Coding
Venue: arXiv cs.IT 2014
Year: 2014

Differential distributed space-time coding (D-DSTC) technique has been considered for relay networks to provide both diversity gain and high throughput in the absence of channel state information. Conventional differential detection (CDD) or two-symbol non-coherent detection over slow -fading channels has been examined and shown to suffer 3-4 dB loss when compared to coherent detections. Moreover, it has also been shown that the performance of CDD severely degrades in fast-fading channels and an irreducible error floor exists at high signal-to-noise ratio region. To overcome the error floor experienced with fast-fading, a nearly optimal "multiple-symbol" differential detection (MSDD) is developed in this paper. The MSDD algorithm jointly processes a larger window of received signals for detection and significantly improves the performance of D-DSTC in fast-fading channels. The error performance of the MSDD algorithm is illustrated with simulation results under different fading scenarios.
General analytical framework for cooperative sensing and access trade-off optimization
Venue: arXiv cs.NI 2014
Year: 2014

In this paper, we investigate the joint cooperative spectrum sensing and access design problem for multi-channel cognitive radio networks. A general heterogeneous setting is considered where the probabilities that different channels are available, SNRs of the signals received at secondary users (SUs) due to transmissions from primary users (PUs) for different users and channels can be different. We assume a cooperative sensing strategy with a general a-out-of-b aggregation rule and design a synchronized MAC protocol so that SUs can exploit available channels. We analyze the sensing performance and the throughput achieved by the joint sensing and access design. Based on this analysis, we develop algorithms to find optimal parameters for the sensing and access protocols and to determine channel assignment for SUs to maximize the system throughput. Finally, numerical results are presented to verify the effectiveness of our design and demonstrate the relative performance of our proposed algorithms and the optimal ones.
Idealized computational models for auditory receptive fields
Venue: arXiv cs.SD 2014
Year: 2014

This paper presents a theory by which idealized models of auditory receptive fields can be derived in a principled axiomatic manner, from a set of structural properties to enable invariance of receptive field responses under natural sound transformations and ensure internal consistency between spectro-temporal receptive fields at different temporal and spectral scales. For defining a time-frequency transformation of a purely temporal sound signal, it is shown that the framework allows for a new way of deriving the Gabor and Gammatone filters as well as a novel family of generalized Gammatone filters, with additional degrees of freedom to obtain different trade-offs between the spectral selectivity and the temporal delay of time-causal temporal window functions. When applied to the definition of a second-layer of receptive fields from a spectrogram, it is shown that the framework leads to two canonical families of spectro-temporal receptive fields, in terms of spectro-temporal derivatives of either spectro-temporal Gaussian kernels for non-causal time or the combination of a time-causal generalized Gammatone filter over the temporal domain and a Gaussian filter over the logspectral domain. For each filter family, the spectro-temporal receptive fields can be either separable over the time-frequency domain or be adapted to local glissando transformations that represent variations in logarithmic frequencies over time. Within each domain of either non-causal or time-causal time, these receptive field families are derived by uniqueness from the assumptions. It is demonstrated how the presented framework allows for computation of basic auditory features for audio processing and that it leads to predictions about auditory receptive fields with good qualitative similarity to biological receptive fields measured in the inferior colliculus (ICC) and primary auditory cortex (A1) of mammals.
An efficient time domain representation for Single-Carrier Frequency Division Multiple Access
Venue: arXiv cs.IT 2014
Year: 2014

This paper presents a physical model for Single Carrier-Frequency Division Mutliple Access (SC-FDMA). We specifically show that by using mutlirate signal processing we derive a general time domain description of Localised SC-FDMA systems relying on circular convolution. This general model has the advantage of encompassing different implementations with flexible rates as well as additional frequency precoding such as spectral shaping. Based on this time-domain model, we study the Power Spectral Density (PSD) and the Signal to Interference and Noise Ratio (SINR). Different implementations of SC-FDMA are investigated and analytical expressions of both PSD and SINR compared to simulations results.
Characterization of Coded Random Access with Compressive Sensing based Multi-User Detection
Venue: arXiv cs.IT 2014
Year: 2014

The emergence of Machine-to-Machine (M2M) communication requires new Medium Access Control (MAC) schemes and physical (PHY) layer concepts to support a massive number of access requests. The concept of coded random access, introduced recently, greatly outperforms other random access methods and is inherently capable to take advantage of the capture effect from the PHY layer. Furthermore, at the PHY layer, compressive sensing based multi-user detection (CS-MUD) is a novel technique that exploits sparsity in multi-user detection to achieve a joint activity and data detection. In this paper, we combine coded random access with CS-MUD on the PHY layer and show very promising results for the resulting protocol.
Performance Analysis of Hybrid ARQ with Incremental Redundancy over Amplify-and-Forward Dual-Hop Relay Channels
Venue: arXiv cs.IT 2014
Year: 2014

In this paper, we consider a three node relay network comprising a source, a relay, and a destination. The source transmits the message to the destination using hybrid automatic repeat request (HARQ) with incremental redundancy (IR). The relay overhears the transmitted message, amplifies it using a variable gain amplifier, and then forwards the message to the destination. This latter combines both the source and the relay message and tries to decode the information. In case of decoding failure, the destination sends a negative acknowledgement. A new replica of the message containing new parity bits is then transmitted in the subsequent HARQ round. This process continues until successful decoding occurs at the destination or a maximum number $M$ of rounds is reached. We study the performance of HARQ-IR over the considered relay channel from an information theoretic perspective. We derive exact expressions and bounds for the information outage probability, the average number of transmissions, and the average transmission rate. Moreover, we evaluate the delay experienced by Poisson arriving packets over the considered relay network. We also provide analytical expressions for the expected waiting time, the sojourn time, and the energy efficiency. The derived exact expressions are validated by Monte Carlo simulations.
An Efficient Solution for Model Checking Abstract State Machine Using Bogor
Venue: arXiv cs.SE 2014
Year: 2014

Nowadays, publish subscribe and event based architecture are frequently used for developing loosely coupled distributed systems. Hence, it is desirable to find a proper solution to specify different systems through these architectures. Abstract state machine (ASM) is a useful means to visually and formally model publish subscribe and event based architectures. However, modeling per se is not enough since the designers want to be able to verify the designed models. As the model checking is a proper approach to verify software and hardware systems. In this paper, we present an approach to verify ASM models specified in terms of AsmetaL language using Bogor. In our approach, the AsmetaL specification is automatically encoded to BIR, the input language of the Bogor.
Performance Improvement of PAPR Reduction for OFDM Signal In LTE System
Venue: arXiv cs.NI 2014
Year: 2014

Orthogonal frequency division multiplexing (OFDM) is an emerging research field of wireless communication. It is one of the most proficient multi-carrier transmission techniques widely used today as broadband wired & wireless applications having several attributes such as provides greater immunity to multipath fading & impulse noise, eliminating inter symbol interference (ISI), inter carrier interference (ICI) & the need for equalizers. OFDM signals have a general problem of high peak to average power ratio (PAPR) which is defined as the ratio of the peak power to the average power of the OFDM signal. The drawback of high PAPR is that the dynamic range of the power amplifier (PA) and digital-to-analog converter (DAC). In this paper, an improved scheme of amplitude clipping & filtering method is proposed and implemented which shows the significant improvement in case of PAPR reduction while increasing slight BER compare to an existing method. Also, the comparative studies of different parameters will be covered.
Virtual Prototyping and Distributed Control for Solar Array with Distributed Multilevel Inverter
Venue: arXiv cs.DC 2014
Year: 2014

In this paper, we present the virtual prototyping of a solar array with a grid-tie implemented as a distributed inverter and controlled using distributed algorithms. Due to the distributed control and inherent redundancy in the array composed of many panels and inverter modules, the virtual prototype exhibits fault-tolerance capabilities. The distributed identifier algorithm allows the system to keep track of the number of operating panels to appropriately regulate the DC voltage output of the panels using buck-boost converters, and determine appropriate switching times for H-bridges in the grid-tie. We evaluate the distributed inverter, its control strategy, and fault-tolerance through simulation in Simulink/Stateflow. Our virtual prototyping framework allows for generating arrays and grid-ties consisting of many panels, and we evaluate arrays of five to dozens of panels. Our analysis suggests the achievable total harmonic distortion (THD) of the system may allow for operating the array in spite of failures of the power electronics, control software, and other subcomponents.