OpenRaiser/Intern-Atlas huggingface dataset:
Mining Student Responses to Infer Student Satisfaction Predictors
Venue: arXiv cs.CY 2020
Year: 2020

The identification and analysis of student satisfaction is a challenging issue. This is becoming increasingly important since a measure of student satisfaction is taken as an indication of how well a course has been taught. However, it remains a challenging problem as student satisfaction has various aspects. In this paper, we formulate the student satisfaction estimation as a prediction problem where we predict different levels of student satisfaction and infer the influential predictors related to course and instructor. We present five different aspects of student satisfaction in terms of 1) course content, 2) class participation, 3) achievement of initial expectations about the course, 4) relevancy towards professional development, and 5) if the course connects them and helps to explore the real-world situations. We employ state-of-the-art machine learning techniques to predict each of these aspects of student satisfaction levels. For our experiment, we utilize a large student evaluation dataset which includes student perception using different attributes related to courses and the instructors. Our experimental results and comprehensive analysis reveal that student satisfaction is more influenced by course attributes in comparison to instructor related attributes.
Geodesic-HOF: 3D Reconstruction Without Cutting Corners
Venue: arXiv cs.CV 2020
Year: 2020

Single-view 3D object reconstruction is a challenging fundamental problem in computer vision, largely due to the morphological diversity of objects in the natural world. In particular, high curvature regions are not always captured effectively by methods trained using only set-based loss functions, resulting in reconstructions short-circuiting the surface or cutting corners. In particular, high curvature regions are not always captured effectively by methods trained using only set-based loss functions, resulting in reconstructions short-circuiting the surface or cutting corners. To address this issue, we propose learning an image-conditioned mapping function from a canonical sampling domain to a high dimensional space where the Euclidean distance is equal to the geodesic distance on the object. The first three dimensions of a mapped sample correspond to its 3D coordinates. The additional lifted components contain information about the underlying geodesic structure. Our results show that taking advantage of these learned lifted coordinates yields better performance for estimating surface normals and generating surfaces than using point cloud reconstructions alone. Further, we find that this learned geodesic embedding space provides useful information for applications such as unsupervised object decomposition.
FinBERT: A Pretrained Language Model for Financial Communications
Venue: arXiv cs.CL 2020
Year: 2020

Contextual pretrained language models, such as BERT (Devlin et al., 2019), have made significant breakthrough in various NLP tasks by training on large scale of unlabeled text re-sources.Financial sector also accumulates large amount of financial communication text.However, there is no pretrained finance specific language models available. In this work,we address the need by pretraining a financial domain specific BERT models, FinBERT, using a large scale of financial communication corpora. Experiments on three financial sentiment classification tasks confirm the advantage of FinBERT over generic domain BERT model. The code and pretrained models are available at https://github.com/yya518/FinBERT. We hope this will be useful for practitioners and researchers working on financial NLP tasks.
Iterative Algorithm Induced Deep-Unfolding Neural Networks: Precoding Design for Multiuser MIMO Systems
Venue: arXiv cs.IT 2020
Year: 2020

Optimization theory assisted algorithms have received great attention for precoding design in multiuser multiple-input multiple-output (MU-MIMO) systems. Although the resultant optimization algorithms are able to provide excellent performance, they generally require considerable computational complexity, which gets in the way of their practical application in real-time systems. In this work, in order to address this issue, we first propose a framework for deep-unfolding, where a general form of iterative algorithm induced deep-unfolding neural network (IAIDNN) is developed in matrix form to better solve the problems in communication systems. Then, we implement the proposed deepunfolding framework to solve the sum-rate maximization problem for precoding design in MU-MIMO systems. An efficient IAIDNN based on the structure of the classic weighted minimum mean-square error (WMMSE) iterative algorithm is developed. Specifically, the iterative WMMSE algorithm is unfolded into a layer-wise structure, where a number of trainable parameters are introduced to replace the highcomplexity operations in the forward propagation. To train the network, a generalized chain rule of the IAIDNN is proposed to depict the recurrence relation of gradients between two adjacent layers in the back propagation. Moreover, we discuss the computational complexity and generalization ability of the proposed scheme. Simulation results show that the proposed IAIDNN efficiently achieves the performance of the iterative WMMSE algorithm with reduced computational complexity.
NeuroCard: One Cardinality Estimator for All Tables
Venue: arXiv cs.DB 2020
Year: 2020

Query optimizers rely on accurate cardinality estimates to produce good execution plans. Despite decades of research, existing cardinality estimators are inaccurate for complex queries, due to making lossy modeling assumptions and not capturing inter-table correlations. In this work, we show that it is possible to learn the correlations across all tables in a database without any independence assumptions. We present NeuroCard, a join cardinality estimator that builds a single neural density estimator over an entire database. Leveraging join sampling and modern deep autoregressive models, NeuroCard makes no inter-table or inter-column independence assumptions in its probabilistic modeling. NeuroCard achieves orders of magnitude higher accuracy than the best prior methods (a new state-of-the-art result of 8.5$\times$ maximum error on JOB-light), scales to dozens of tables, while being compact in space (several MBs) and efficient to construct or update (seconds to minutes).
Load-balanced Service Function Chaining in Edge Computing over FiWi Access Networks for Internet of Things
Venue: arXiv cs.NI 2020
Year: 2020

Service function chaining (SFC) is promising to implement flexible and scalable virtual network infrastructure for the Internet of Things (IoT). Edge computing is envisioned to be an effective solution to process huge amount of IoT application data. In order to uniformly provide services to IoT applications among the distributed edge computing nodes (ECNs), we present a unified SFC orchestration framework based on the coordination of SDN and NFV, which provides a synergic edge cloud platform by exploiting the connectivity of FiWi access networks. In addition, we study the VNF deployment problem under our synergic framework, and we formulate it as a mixed-integer nonlinear programming (MINLP) problem jointly considering the load balancing of networking and computing for chaining VNFs. We also propose two approximation optimal deployment algorithms named Greedy-Bisection Multi-Path (GBMP) and KSP MultiPath (KSMP) taking advantage of the multi-instance virtual network functions (VNFs) deployed in ECNs and the multipath capacity in FiWi access networks. Extensive simulations are conducted in two types of IoT application scenarios in the EC over FiWi access networks. The numerical results show that our proposed algorithms are superior to single path and ECMP based deployment algorithms in terms of load balancing, service acceptance ratio, and network utilization in both two typical scenarios.
Filter design for small target detection on infrared imagery using normalized-cross-correlation layer
Venue: arXiv cs.CV 2020
Year: 2020

In this paper, we introduce a machine learning approach to the problem of infrared small target detection filter design. For this purpose, similarly to a convolutional layer of a neural network, the normalized-cross-correlational (NCC) layer, which we utilize for designing a target detection/recognition filter bank, is proposed. By employing the NCC layer in a neural network structure, we introduce a framework, in which supervised training is used to calculate the optimal filter shape and the optimum number of filters required for a specific target detection/recognition task on infrared images. We also propose the mean-absolute-deviation NCC (MAD-NCC) layer, an efficient implementation of the proposed NCC layer, designed especially for FPGA systems, in which square root operations are avoided for real-time computation. As a case study we work on dim-target detection on mid-wave infrared imagery and obtain the filters that can discriminate a dim target from various types of background clutter, specific to our operational concept.
Markov-Lipschitz Deep Learning
Venue: arXiv cs.LG 2020
Year: 2020

We propose a novel framework, called Markov-Lipschitz deep learning (MLDL), to tackle geometric deterioration caused by collapse, twisting, or crossing in vector-based neural network transformations for manifold-based representation learning and manifold data generation. A prior constraint, called locally isometric smoothness (LIS), is imposed across-layers and encoded into a Markov random field (MRF)-Gibbs distribution. This leads to the best possible solutions for local geometry preservation and robustness as measured by locally geometric distortion and locally bi-Lipschitz continuity. Consequently, the layer-wise vector transformations are enhanced into well-behaved, LIS-constrained metric homeomorphisms. Extensive experiments, comparisons, and ablation study demonstrate significant advantages of MLDL for manifold learning and manifold data generation. MLDL is general enough to enhance any vector transformation-based networks. The code is available at https://github.com/westlake-cairi/Markov-Lipschitz-Deep-Learning.
Randomized polynomial-time equivalence between determinant and trace-IMM equivalence tests
Venue: arXiv cs.CC 2020
Year: 2020

Equivalence testing for a polynomial family {g_m} over a field F is the following problem: Given black-box access to an n-variate polynomial f(x), where n is the number of variables in g_m, check if there exists an A in GL(n,F) such that f(x) = g_m(Ax). If yes, then output such an A. The complexity of equivalence testing has been studied for a number of important polynomial families, including the determinant (Det) and the two popular variants of the iterated matrix multiplication polynomial: IMM_{w,d} (the (1,1) entry of the product of d many w $\times$ w symbolic matrices) and Tr-IMM_{w,d} (the trace of the product of d many w $\times$ w symbolic matrices). The families Det, IMM and Tr-IMM are VBP-complete, and so, in this sense, they have the same complexity. But, do they have the same equivalence testing complexity? We show that the answer is 'yes' for Det and Tr-IMM (modulo the use of randomness). The result is obtained by connecting the two problems via another well-studied problem called the full matrix algebra isomorphism problem (FMAI). In particular, we prove the following: 1. Testing equivalence of polynomials to Tr-IMM_{w,d}, for d$\geq$ 3 and w$\geq$ 2, is randomized polynomial-time Turing reducible to testing equivalence of polynomials to Det_w, the determinant of the w $\times$ w matrix of formal variables. (Here, d need not be a constant.) 2. FMAI is randomized polynomial-time Turing reducible to equivalence testing (in fact, to tensor isomorphism testing) for the family of matrix multiplication tensors {Tr-IMM_{w,3}}. These in conjunction with the randomized poly-time reduction from determinant equivalence testing to FMAI [Garg,Gupta,Kayal,Saha19], imply that FMAI, equivalence testing for Tr-IMM and for Det, and the $3$-tensor isomorphism problem for the family of matrix multiplication tensors are randomized poly-time equivalent under Turing reductions.
Detecting unusual input to neural networks
Venue: arXiv cs.LG 2020
Year: 2020

Evaluating a neural network on an input that differs markedly from the training data might cause erratic and flawed predictions. We study a method that judges the unusualness of an input by evaluating its informative content compared to the learned parameters. This technique can be used to judge whether a network is suitable for processing a certain input and to raise a red flag that unexpected behavior might lie ahead. We compare our approach to various methods for uncertainty evaluation from the literature for various datasets and scenarios. Specifically, we introduce a simple, effective method that allows to directly compare the output of such metrics for single input points even if these metrics live on different scales.
Wat zei je? Detecting Out-of-Distribution Translations with Variational Transformers
Venue: arXiv cs.CL 2020
Year: 2020

We detect out-of-training-distribution sentences in Neural Machine Translation using the Bayesian Deep Learning equivalent of Transformer models. For this we develop a new measure of uncertainty designed specifically for long sequences of discrete random variables -- i.e. words in the output sentence. Our new measure of uncertainty solves a major intractability in the naive application of existing approaches on long sentences. We use our new measure on a Transformer model trained with dropout approximate inference. On the task of German-English translation using WMT13 and Europarl, we show that with dropout uncertainty our measure is able to identify when Dutch source sentences, sentences which use the same word types as German, are given to the model instead of German.
On Lipschitz Regularization of Convolutional Layers using Toeplitz Matrix Theory
Venue: arXiv cs.LG 2020
Year: 2020

This paper tackles the problem of Lipschitz regularization of Convolutional Neural Networks. Lipschitz regularity is now established as a key property of modern deep learning with implications in training stability, generalization, robustness against adversarial examples, etc. However, computing the exact value of the Lipschitz constant of a neural network is known to be NP-hard. Recent attempts from the literature introduce upper bounds to approximate this constant that are either efficient but loose or accurate but computationally expensive. In this work, by leveraging the theory of Toeplitz matrices, we introduce a new upper bound for convolutional layers that is both tight and easy to compute. Based on this result we devise an algorithm to train Lipschitz regularized Convolutional Neural Networks.
Computing Shortest Paths and Diameter in the Hybrid Network Model
Venue: arXiv cs.DC 2020
Year: 2020

The $\mathsf{HYBRID}$ model, introduced in [Augustine et al., SODA '20], provides a theoretical foundation for networks that allow multiple communication modes. The model follows the principles of synchronous message passing, whereas nodes are allowed to use \textit{two} fundamentally different communication modes. First, a local mode where nodes may exchange arbitrary information per round over edges of a local communication graph $G$ (akin to the $\mathsf{LOCAL}$ model). Second, a global mode where every node may exchange $O(\log n)$ messages of size $O(\log n)$ bits per round with arbitrary nodes in the network. The $\mathsf{HYBRID}$ model intends to reflect the conditions of many real hybrid networks, where high-bandwidth but inherently local communication is combined with highly flexible global communication with restricted bandwidth. We continue to explore the power and limitations of the $\mathsf{HYBRID}$ model by investigating the complexity of computing shortest paths and diameter of the local communication graph $G$. We improve on the known upper bound for the exact all pairs shortest paths problem (APSP) from [Augustine et al., SODA '20] and provide algorithms to approximate solutions for the $k$ source shortest paths problem ($k$-SSSP). We demonstrate that our results for APSP and $k$-SSP are almost tight (up to poly-logarithmic factors). Furthermore, we give an improved algorithm for the exact single source shortest paths problem for graphs with large diameter. For the problem of approximating the diameter of the local communication network we give the first non-trivial upper bound. This upper bound is complemented by a lower bound for the exact diameter problem.
Machine Common Sense
Venue: arXiv cs.AI 2020
Year: 2020

Machine common sense remains a broad, potentially unbounded problem in artificial intelligence (AI). There is a wide range of strategies that can be employed to make progress on this challenge. This article deals with the aspects of modeling commonsense reasoning focusing on such domain as interpersonal interactions. The basic idea is that there are several types of commonsense reasoning: one is manifested at the logical level of physical actions, the other deals with the understanding of the essence of human-human interactions. Existing approaches, based on formal logic and artificial neural networks, allow for modeling only the first type of common sense. To model the second type, it is vital to understand the motives and rules of human behavior. This model is based on real-life heuristics, i.e., the rules of thumb, developed through knowledge and experience of different generations. Such knowledge base allows for development of an expert system with inference and explanatory mechanisms (commonsense reasoning algorithms and personal models). Algorithms provide tools for a situation analysis, while personal models make it possible to identify personality traits. The system so designed should perform the function of amplified intelligence for interactions, including human-machine.
Combinatorial Group Testing and Sparse Recovery Schemes with Near-Optimal Decoding Time
Venue: arXiv cs.IT 2020
Year: 2020

In the long-studied problem of combinatorial group testing, one is asked to detect a set of $k$ defective items out of a population of size $n$, using $m \ll n$ disjunctive measurements. In the non-adaptive setting, the most widely used combinatorial objects are disjunct and list-disjunct matrices, which define incidence matrices of test schemes. Disjunct matrices allow the identification of the exact set of defectives, whereas list disjunct matrices identify a small superset of the defectives. Apart from the combinatorial guarantees, it is often of key interest to equip measurement designs with efficient decoding algorithms. The most efficient decoders should run in sublinear time in $n$, and ideally near-linear in the number of measurements $m$. In this work, we give several constructions with an optimal number of measurements and near-optimal decoding time for the most fundamental group testing tasks, as well as for central tasks in the compressed sensing and heavy hitters literature. For many of those tasks, the previous measurement-optimal constructions needed time either quadratic in the number of measurements or linear in the universe size. Most of our results are obtained via a clean and novel approach which avoids list-recoverable codes or related complex techniques which were present in almost every state-of-the-art work on efficiently decodable constructions of such objects.
Calibration of the von Wolffersdorff model using Genetic Algorithms
Venue: arXiv cs.CE 2020
Year: 2020

This article proposes an optimization framework, based on Genetic Algorithms (GA), to calibrate the constitutive law of von Wolffersdorff. This constitutive law is known as Sand Hypoplasticity (SH), and allows for robust and accurate modeling of the soil behavior but requires a complex calibration involving eight parameters. The proposed optimization can automatically fit these parameters from the results of an oedometric and a triaxial drained compression test, by combining the GA with a numerical solver that integrates the SH in the test conditions. By repeating the same calibration several times, the stochastic nature of the optimizer enables the uncertainty quantification of the calibration parameters and allows studying their relative importance on the model prediction. After validating the numerical solver on the ExCaliber-Laboratory software from the SoilModels' website, the GA calibration is tested on a synthetic dataset to analyze the convergence and the statistics of the results. In particular, a correlation analysis reveals that two couples of the eight model parameters are strongly correlated. Finally, the calibration procedure is tested on the results from von Wolffersdorff, 1996, and Herle & Gudehus, 1999, on the Hochstetten sand. The model parameters identified by the Genetic Algorithm optimization improves the matching with the experimental data and hence lead to a better calibration.
Efficient Ab-Initio Molecular Dynamic Simulations by Offloading Fast Fourier Transformations to FPGAs
Venue: arXiv cs.DC 2020
Year: 2020

A large share of today's HPC workloads is used for Ab-Initio Molecular Dynamics (AIMD) simulations, where the interatomic forces are computed on-the-fly by means of accurate electronic structure calculations. They are computationally intensive and thus constitute an interesting application class for energy-efficient hardware accelerators such as FPGAs. In this paper, we investigate the potential of offloading 3D Fast Fourier Transformations (FFTs) as a critical routine of plane-wave-based electronic structure calculations to FPGA and in conjunction demonstrate the tolerance of these simulations to lower precision computations.
Taxonomy and Practical Evaluation of Primality Testing Algorithms
Venue: arXiv cs.CR 2020
Year: 2020

Modern cryptography algorithms are commonly used to ensure information security. Prime numbers are needed in many asymmetric cryptography algorithms. For example, RSA algorithm selects two large prime numbers and multiplies to each other to obtain a large composite number whose factorization is very difficult. Producing a prime number is not an easy task as they are not distributed regularly through integers. Primality testing algorithms are used to determine whether a particular number is prime or composite. In this paper, an intensive survey is thoroughly conducted among the several primality testing algorithms showing the pros and cons, the time complexity, and a brief summary of each algorithm. Besides, an implementation of these algorithms is accomplished using Java and Python as programming languages to evaluate the efficiency of both the algorithms and the programming languages.
A Machine Learning-Based Migration Strategy for Virtual Network Function Instances
Venue: arXiv cs.NI 2020
Year: 2020

With the growing demand for data connectivity, network service providers are faced with the task of reducing their capital and operational expenses while simultaneously improving network performance and addressing the increased demand. Although Network Function Virtualization (NFV) has been identified as a promising solution, several challenges must be addressed to ensure its feasibility. In this paper, we address the Virtual Network Function (VNF) migration problem by developing the VNF Neural Network for Instance Migration (VNNIM), a migration strategy for VNF instances. The performance of VNNIM is further improved through the optimization of the learning rate hyperparameter through particle swarm optimization. Results show that the VNNIM is very effective in predicting the post-migration server exhibiting a binary accuracy of 99.07% and a delay difference distribution that is centered around a mean of zero when compared to the optimization model. The greatest advantage of VNNIM, however, is its run-time efficiency highlighted through a run-time analysis.