OpenRaiser/Intern-Atlas huggingface dataset:
Ristretto: Hardware-Oriented Approximation of Convolutional Neural Networks
Venue: arXiv cs.CV 2016
Year: 2016

Convolutional neural networks (CNN) have achieved major breakthroughs in recent years. Their performance in computer vision have matched and in some areas even surpassed human capabilities. Deep neural networks can capture complex non-linear features; however this ability comes at the cost of high computational and memory requirements. State-of-art networks require billions of arithmetic operations and millions of parameters. To enable embedded devices such as smartphones, Google glasses and monitoring cameras with the astonishing power of deep learning, dedicated hardware accelerators can be used to decrease both execution time and power consumption. In applications where fast connection to the cloud is not guaranteed or where privacy is important, computation needs to be done locally. Many hardware accelerators for deep neural networks have been proposed recently. A first important step of accelerator design is hardware-oriented approximation of deep networks, which enables energy-efficient inference. We present Ristretto, a fast and automated framework for CNN approximation. Ristretto simulates the hardware arithmetic of a custom hardware accelerator. The framework reduces the bit-width of network parameters and outputs of resource-intense layers, which reduces the chip area for multiplication units significantly. Alternatively, Ristretto can remove the need for multipliers altogether, resulting in an adder-only arithmetic. The tool fine-tunes trimmed networks to achieve high classification accuracy. Since training of deep neural networks can be time-consuming, Ristretto uses highly optimized routines which run on the GPU. This enables fast compression of any given network. Given a maximum tolerance of 1%, Ristretto can successfully condense CaffeNet and SqueezeNet to 8-bit. The code for Ristretto is available.
Shape Recognition by Bag of Skeleton-associated Contour Parts
Venue: arXiv cs.CV 2016
Year: 2016

Contour and skeleton are two complementary representations for shape recognition. However combining them in a principal way is nontrivial, as they are generally abstracted by different structures (closed string vs graph), respectively. This paper aims at addressing the shape recognition problem by combining contour and skeleton according to the correspondence between them. The correspondence provides a straightforward way to associate skeletal information with a shape contour. More specifically, we propose a new shape descriptor. named Skeleton-associated Shape Context (SSC), which captures the features of a contour fragment associated with skeletal information. Benefited from the association, the proposed shape descriptor provides the complementary geometric information from both contour and skeleton parts, including the spatial distribution and the thickness change along the shape part. To form a meaningful shape feature vector for an overall shape, the Bag of Features framework is applied to the SSC descriptors extracted from it. Finally, the shape feature vector is fed into a linear SVM classifier to recognize the shape. The encouraging experimental results demonstrate that the proposed way to combine contour and skeleton is effective for shape recognition, which achieves the state-of-the-art performances on several standard shape benchmarks.
X-ray image separation via coupled dictionary learning
Venue: arXiv cs.CV 2016
Year: 2016

In support of art investigation, we propose a new source sepa- ration method that unmixes a single X-ray scan acquired from double-sided paintings. Unlike prior source separation meth- ods, which are based on statistical or structural incoherence of the sources, we use visual images taken from the front- and back-side of the panel to drive the separation process. The coupling of the two imaging modalities is achieved via a new multi-scale dictionary learning method. Experimental results demonstrate that our method succeeds in the discrimination of the sources, while state-of-the-art methods fail to do so.
Deep Roots: Improving CNN Efficiency with Hierarchical Filter Groups
Venue: arXiv cs.NE 2016
Year: 2016

We propose a new method for creating computationally efficient and compact convolutional neural networks (CNNs) using a novel sparse connection structure that resembles a tree root. This allows a significant reduction in computational cost and number of parameters compared to state-of-the-art deep CNNs, without compromising accuracy, by exploiting the sparsity of inter-layer filter dependencies. We validate our approach by using it to train more efficient variants of state-of-the-art CNN architectures, evaluated on the CIFAR10 and ILSVRC datasets. Our results show similar or higher accuracy than the baseline architectures with much less computation, as measured by CPU and GPU timings. For example, for ResNet 50, our model has 40% fewer parameters, 45% fewer floating point operations, and is 31% (12%) faster on a CPU (GPU). For the deeper ResNet 200 our model has 25% fewer floating point operations and 44% fewer parameters, while maintaining state-of-the-art accuracy. For GoogLeNet, our model has 7% fewer parameters and is 21% (16%) faster on a CPU (GPU).
On Content-Based Recommendation and User Privacy in Social-Tagging Systems
Venue: arXiv cs.IR 2016
Year: 2016

Recommendation systems and content filtering approaches based on annotations and ratings, essentially rely on users expressing their preferences and interests through their actions, in order to provide personalised content. This activity, in which users engage collectively, has been named social tagging. Although it has opened a myriad of new possibilities for application interoperability on the semantic web, it is also posing new privacy threats. Social tagging consists in describing online or online resources by using free-text labels (i.e. tags), therefore exposing the user's profile and activity to privacy attacks. Tag forgery is a privacy enhancing technology consisting of generating tags for categories or resources that do not reflect the user's actual preferences. By modifying their profile, tag forgery may have a negative impact on the quality of the recommendation system, thus protecting user privacy to a certain extent but at the expenses of utility loss. The impact of tag forgery on content-based recommendation is, therefore, investigated in a real-world application scenario where different forgery strategies are evaluated, and the consequent loss in utility is measured and compared.
Vulnerable GPU Memory Management: Towards Recovering Raw Data from GPU
Venue: arXiv cs.CR 2016
Year: 2016

In this paper, we present that security threats coming with existing GPU memory management strategy are overlooked, which opens a back door for adversaries to freely break the memory isolation: they enable adversaries without any privilege in a computer to recover the raw memory data left by previous processes directly. More importantly, such attacks can work on not only normal multi-user operating systems, but also cloud computing platforms. To demonstrate the seriousness of such attacks, we recovered original data directly from GPU memory residues left by exited commodity applications, including Google Chrome, Adobe Reader, GIMP, Matlab. The results show that, because of the vulnerable memory management strategy, commodity applications in our experiments are all affected.
Bending the Curve: Improving the ROC Curve Through Error Redistribution
Venue: arXiv cs.LG 2016
Year: 2016

Classification performance is often not uniform over the data. Some areas in the input space are easier to classify than others. Features that hold information about the "difficulty" of the data may be non-discriminative and are therefore disregarded in the classification process. We propose a meta-learning approach where performance may be improved by post-processing. This improvement is done by establishing a dynamic threshold on the base-classifier results. Since the base-classifier is treated as a "black box" the method presented can be used on any state of the art classifier in order to try an improve its performance. We focus our attention on how to better control the true-positive/false-positive trade-off known as the ROC curve. We propose an algorithm for the derivation of optimal thresholds by redistributing the error depending on features that hold information about difficulty. We demonstrate the resulting benefit on both synthetic and real-life data.
Information and Communications Technologies (ICT) and Pre-Service Education Professionals: A Case Study of Motivation and Knowledge
Venue: arXiv cs.CY 2016
Year: 2016

The importance of knowing ICT training and motivation -so relevant in today's society- which currently offers the first year college students, mostly in degrees in Education, focuses the object of interest in this study. The following targets have been proposed: [1] knowing what basic skills regarding initial instrumental knowledge presents the prospective teacher (aptitudes) and [2] knowing their motivation for the educational use of ICT in the classroom (attitudes). For this purpose a non-experimental descriptive quantitative methodology has been used, with a sample of subjects (N=282) of the Autonomous Region of Extremadura (Spain). The results show that new degree college students possess a basic knowledge of ICT alongside a highly positive motivation towards the use of these. However, it is worrying that they only show an instrumental and technical knowledge of computing and telematic tools implied in social environments, but not pedagogical ones. Also they are unfamiliar with the true power of social, economic, political, ethical, influence as well as the effects and problems that their misuse can generate in their future students (addictions, manipulation, consumerism, etc.). Dimensions therefore for which they are urged to be trained at University for a proper performance as future professionals in education.
Cross Domain Adaptation by Learning Partially Shared Classifiers and Weighting Source Data Points in the Shared Subspaces
Venue: arXiv cs.LG 2016
Year: 2016

Transfer learning is a problem defined over two domains. These two domains share the same feature space and class label space, but have significantly different distributions. One domain has sufficient labels, named as source domain, and the other domain has few labels, named as target do- main. The problem is to learn a effective classifier for the target domain. In this paper, we propose a novel transfer learning method for this problem by learning a partially shared classifier for the target domain, and weighting the source domain data points. We learn some shared subspaces for both the data points of the two domains, and a shared classifier in the shared subspaces. We hope that in the shared subspaces, the distributions of two domain can match each other well, and to match the distributions, we weight the source domain data points with different weighting factors. Moreover, we adapt the shared classifier to each domain by learning different adaptation functions. To learn the subspace transformation matrices, the classifier parameters, and the adaptation parameters, we build a objective function with weighted clas- sification errors, parameter regularization, local reconstruction regularization, and distribution matching. This objective function is minimized by an itera- tive algorithm. Experiments show its effectiveness over benchmark data sets, including travel destination review data set, face expression data set, spam email data set, etc.
Tensors in Power System Computation I: Distributed Computation for Optimal Power Flow, DC OPF
Venue: arXiv cs.CE 2016
Year: 2016

Tensor decomposition plays a key role in identifying common features across a collection of matrices in many areas of science. A fundamental need in big data research is to process data tabulated as large-scale matrices using eigenvectors. A higher order generalized singular value decomposition technique successfully captures the common features of the same organ from multiple animals in genomic signal processing. A recent semidefinite programming approach to solve an AC optimal power flow was accompanied by the problem formulation in the Cartesian coordinate system. The collection of nodal Kirchhoff laws introduces a 3D tensor with a common feature of individual matrices to maintain local power balance. In this paper, the mathematical process is established and the common feature is identified. The common feature is a key element to a fully decentralized and therefore scalable algorithm to solve AC optimal power flow.
Space-Efficient Karatsuba Multiplication for Multi-Precision Integers
Venue: arXiv cs.NA 2016
Year: 2016

The traditional Karatsuba algorithm for the multiplication of polynomials and multi-precision integers has a time complexity of $O(n^{1.59})$ and a space complexity of $O(n)$. Roche proposed an improved algorithm with the same $O(n^{1.59})$ time complexity but with a much reduced $O(\log n)$ space complexity. In Roche's paper details were provided for multiplication of polynomials, but not for multi-precision integers. Multi-precision integers differ from polynomials by the presence of carries, which poses difficulties in implementing Roche's scheme in multi-precision integers. This paper provides a detailed solution to these difficulties. Finally, numerical comparisons between the schoolbook, traditional Karatsuba, and space-efficient Karatsuba algorithms are provided.
3D Face Tracking and Texture Fusion in the Wild
Venue: arXiv cs.CV 2016
Year: 2016

We present a fully automatic approach to real-time 3D face reconstruction from monocular in-the-wild videos. With the use of a cascaded-regressor based face tracking and a 3D Morphable Face Model shape fitting, we obtain a semi-dense 3D face shape. We further use the texture information from multiple frames to build a holistic 3D face representation from the video frames. Our system is able to capture facial expressions and does not require any person-specific training. We demonstrate the robustness of our approach on the challenging 300 Videos in the Wild (300-VW) dataset. Our real-time fitting framework is available as an open source library at http://4dface.org.
Automated Resolution Selection for Image Segmentation
Venue: arXiv cs.CV 2016
Year: 2016

It is well-known in image processing that computational cost increases rapidly with the number and dimensions of the images to be processed. Several fields, such as medical imaging, routinely use numerous very large images, which might also be 3D and/or captured at several frequency bands, all adding to the computational expense. Multiresolution analysis is a method of increasing the efficiency of the segmentation process. One multiresolution approach is the coarse-to-fine segmentation strategy, whereby the segmentation starts at a coarse resolution and is then fine-tuned during subsequent steps. The starting resolution for segmentation is generally selected arbitrarily with no clear selection criteria. The research reported in this paper showed that starting from different resolutions for image segmentation results in different accuracies and computational times, even for images of the same category (depicting similar scenes or objects). An automated method for resolution selection for an input image would thus be beneficial. This paper introduces a framework for the automated selection of the best resolution for image segmentation. We propose a measure for defining the best resolution based on user/system criteria, offering a trade-off between accuracy and computation time. A learning approach is then introduced for the selection of the resolution, whereby extracted image features are mapped to the previously determined best resolution. In the learning process, class (i.e., resolution) distribution is generally imbalanced, making effective learning from the data difficult. Experiments conducted with three datasets using two different segmentation algorithms show that the resolutions selected through learning enable much faster segmentation than the original ones, while retaining at least the original accuracy.
Information-Theoretic Lower Bounds on the Storage Cost of Shared Memory Emulation
Venue: arXiv cs.DC 2016
Year: 2016

The focus of this paper is to understand storage costs of emulating an atomic shared memory over an asynchronous, distributed message passing system. Previous literature has developed several shared memory emulation algorithms based on replication and erasure coding techniques. In this paper, we present information-theoretic lower bounds on the storage costs incurred by shared memory emulation algorithms. Our storage cost lower bounds are universally applicable, that is, we make no assumption on the structure of the algorithm or the method of encoding the data. We consider an arbitrary algorithm $A$ that implements an atomic multi-writer single-reader (MWSR) shared memory variable whose values come from a finite set $\mathcal{V}$ over a system of $N$ servers connected by point-to-point asynchronous links. We require that in every fair execution of algorithm $A$ where the number of server failures is smaller than a parameter $f$, every operation invoked at a non-failing client terminates. We define the storage cost of a server in algorithm $A$ as the logarithm (to base 2) of number of states it can take on; the total-storage cost of algorithm $A$ is the sum of the storage cost of all servers. Our results are as follows. (i) We show that if algorithm $A$ does not use server gossip, then the total storage cost is lower bounded by $2 \frac{N}{N-f+1}\log_2|\mathcal{V}|-o(\log_2|\mathcal{V}|)$. (ii) The total storage cost is at least $2 \frac{N}{N-f+2} \log_{2}|\mathcal{V}|-o(\log_{2}|\mathcal{V}|)$ even if the algorithm uses server gossip. (iii) We consider algorithms where the write protocol sends information about the value in at most one phase. We show that the total storage cost is at least $ν^* \frac{N}{N-f+ν^*-1} \log_2( |\mathcal{V}|)- o(\log_2(|\mathcal{V}|),$ where $ν^*$ is the minimum of $f+1$ and the number of active write operations of an execution.
Building Cyber-Physical Energy Systems
Venue: arXiv cs.SY 2016
Year: 2016

The built environment, as hallmark of modern society, has become one of the key drivers of energy demand. This makes for meaningful application of novel paradigms, such as cyber-physical systems, with large scale impact for both primary energy consumption reduction as well as (micro-) grid stability problems. In a bottom-up approach we analyze the drivers of CPS design, deployment and adoption in smart buildings. This ranges from low-level embedded and real time system challenges, instrumentation and control issues, up to ICT security layers protecting information in a world of ubiquitous connectivity. A modeling and predictive control framework is also discussed with outlook of deployment for HVAC optimization to a new facility for research from our campus.
PROJECTION Algorithm for Motif Finding on GPUs
Venue: arXiv cs.DC 2016
Year: 2016

Motif finding is one of the NP-complete problems in Computational Biology. Existing nondeterministic algorithms for motif finding do not guarantee the global optimality of results and are sensitive to initial parameters. To address this problem, the PROJECTION algorithm provides a good initial estimate that can be further refined using local optimization algorithms such as EM, MEME or Gibbs. For large enough input (600-1000 bp per sequence) or for challenging motif finding problems, the PROJECTION algorithm may run in an inordinate amount of time. In this paper we present a parallel implementation of the PROJECTION algorithm in Graphics Processing Units (GPUs) using CUDA. We also list down several major issues we have encountered including performing space optimizations because of the GPU's space limitations.
madmom: a new Python Audio and Music Signal Processing Library
Venue: arXiv cs.SD 2016
Year: 2016

In this paper, we present madmom, an open-source audio processing and music information retrieval (MIR) library written in Python. madmom features a concise, NumPy-compatible, object oriented design with simple calling conventions and sensible default values for all parameters, which facilitates fast prototyping of MIR applications. Prototypes can be seamlessly converted into callable processing pipelines through madmom's concept of Processors, callable objects that run transparently on multiple cores. Processors can also be serialised, saved, and re-run to allow results to be easily reproduced anywhere. Apart from low-level audio processing, madmom puts emphasis on musically meaningful high-level features. Many of these incorporate machine learning techniques and madmom provides a module that implements some in MIR commonly used methods such as hidden Markov models and neural networks. Additionally, madmom comes with several state-of-the-art MIR algorithms for onset detection, beat, downbeat and meter tracking, tempo estimation, and piano transcription. These can easily be incorporated into bigger MIR systems or run as stand-alone programs.
Incorporating Quotation and Evaluation into Church's Type Theory: Syntax and Semantics
Venue: arXiv cs.LO 2016
Year: 2016

${\rm \small CTT}_{\rm qe}$ is a version of Church's type theory that includes quotation and evaluation operators that are similar to quote and eval in the Lisp programming language. With quotation and evaluation it is possible to reason in ${\rm \small CTT}_{\rm qe}$ about the interplay of the syntax and semantics of expressions and, as a result, to formalize syntax-based mathematical algorithms. We present the syntax and semantics of ${\rm \small CTT}_{\rm qe}$ and give several examples that illustrate the usefulness of having quotation and evaluation in ${\rm \small CTT}_{\rm qe}$. We do not give a proof system for ${\rm \small CTT}_{\rm qe}$, but we do sketch what a proof system could look like.
Stable and accurate interface capturing advection schemes
Venue: arXiv cs.CE 2016
Year: 2016

In this paper, stable and "low-diffusive" multidimensional interface capturing (IC) schemes using slope limiters are discussed. It is known that direction-by-direction slope-limited MUSCL schemes create geometrical artifacts and thus return a poor accuracy. We here focus on this particular issue and show that the reconstruction of gradient directions are an important factor of accuracy. The use of a multidimensional limiting process (MLP) added with an adequate time integration scheme leads to an artifact-free and instability-free interface capturing (IC) approach. Numerical experiments like the reference Kothe-Rider forward-backward advection case show the accuracy of the approach. We also show that the approach can be extended to the more complex compressible multimaterial hydrodynamics case, with potentially an arbitrary number of fluids. We also believe that this approach is appropriate for multicore/manycore architecture because of its SIMD feature, which may be another asset compared to interface reconstruction approaches.