OpenRaiser/Intern-Atlas huggingface dataset:
L2P: Learning to Place for Estimating Heavy-Tailed Distributed Outcomes
Venue: arXiv cs.LG 2023
Year: 2023

Many real-world prediction tasks have outcome variables that have characteristic heavy-tail distributions. Examples include copies of books sold, auction prices of art pieces, demand for commodities in warehouses, etc. By learning heavy-tailed distributions, "big and rare" instances (e.g., the best-sellers) will have accurate predictions. Most existing approaches are not dedicated to learning heavy-tailed distribution; thus, they heavily under-predict such instances. To tackle this problem, we introduce Learning to Place (L2P), which exploits the pairwise relationships between instances for learning. In its training phase, L2P learns a pairwise preference classifier: is instance A > instance B? In its placing phase, L2P obtains a prediction by placing the new instance among the known instances. Based on its placement, the new instance is then assigned a value for its outcome variable. Experiments on real data show that L2P outperforms competing approaches in terms of accuracy and ability to reproduce heavy-tailed outcome distribution. In addition, L2P provides an interpretable model by placing each predicted instance in relation to its comparable neighbors. Interpretable models are highly desirable when lives and treasure are at stake.
A Linear Exponential Comonad in s-finite Transition Kernels and Probabilistic Coherent Spaces
Venue: arXiv cs.LO 2023
Year: 2023

This paper concerns a stochastic construction of probabilistic coherent spaces by employing novel ingredients (i) linear exponential comonads arising properly in the measure-theory (ii) continuous orthogonality between measures and measurable functions. A linear exponential comonad is constructed over a symmetric monoidal category of transition kernels, relaxing Markov kernels of Panangaden's stochastic relations into s-finite kernels. The model supports an orthogonality in terms of an integral between measures and measurable functions, which can be seen as a continuous extension of Girard-Danos-Ehrhard's linear duality for probabilistic coherent spaces. The orthogonality is formulated by a Hyland-Schalk double glueing construction, into which our measure theoretic monoidal comonad structure is accommodated. As an application to countable measurable spaces, a dagger compact closed category is obtained, whose double glueing gives rise to the familiar category of probabilistic coherent spaces.
Student Engagement Detection Using Emotion Analysis, Eye Tracking and Head Movement with Machine Learning
Venue: arXiv cs.CV 2023
Year: 2023

With the increase of distance learning, in general, and e-learning, in particular, having a system capable of determining the engagement of students is of primordial importance, and one of the biggest challenges, both for teachers, researchers and policy makers. Here, we present a system to detect the engagement level of the students. It uses only information provided by the typical built-in web-camera present in a laptop computer, and was designed to work in real time. We combine information about the movements of the eyes and head, and facial emotions to produce a concentration index with three classes of engagement: "very engaged", "nominally engaged" and "not engaged at all". The system was tested in a typical e-learning scenario, and the results show that it correctly identifies each period of time where students were "very engaged", "nominally engaged" and "not engaged at all". Additionally, the results also show that the students with best scores also have higher concentration indexes.
OpenVSLAM: A Versatile Visual SLAM Framework
Venue: arXiv cs.CV 2023
Year: 2023

In this paper, we introduce OpenVSLAM, a visual SLAM framework with high usability and extensibility. Visual SLAM systems are essential for AR devices, autonomous control of robots and drones, etc. However, conventional open-source visual SLAM frameworks are not appropriately designed as libraries called from third-party programs. To overcome this situation, we have developed a novel visual SLAM framework. This software is designed to be easily used and extended. It incorporates several useful features and functions for research and development.
Differentiable Sparsification for Deep Neural Networks
Venue: arXiv cs.LG 2023
Year: 2023

Deep neural networks have significantly alleviated the burden of feature engineering, but comparable efforts are now required to determine effective architectures for these networks. Furthermore, as network sizes have become excessively large, a substantial amount of resources is invested in reducing their sizes. These challenges can be effectively addressed through the sparsification of over-complete models. In this study, we propose a fully differentiable sparsification method for deep neural networks, which can zero out unimportant parameters by directly optimizing a regularized objective function with stochastic gradient descent. Consequently, the proposed method can learn both the sparsified structure and weights of a network in an end-to-end manner. It can be directly applied to various modern deep neural networks and requires minimal modification to the training process. To the best of our knowledge, this is the first fully differentiable sparsification method.
Bayesian open games
Venue: arXiv cs.GT 2023
Year: 2023

This paper generalises the treatment of compositional game theory as introduced by Ghani et al. in 2018, where games are modelled as morphisms of a symmetric monoidal category. From an economic modelling perspective, the notion of a game in the work by Ghani et al. is not expressive enough for many applications. This includes stochastic environments, stochastic choices by players, as well as incomplete information regarding the game being played. The current paper addresses these three issues all at once.
Operator Responsibility for Outcomes: A Demonstration of the ResQu Model
Venue: arXiv cs.HC 2023
Year: 2023

In systems with advanced automation, human responsibility for outcomes becomes equivocal. We developed the Responsibility Quantification (ResQu) model to compute a measure of operator responsibility (Douer & Meyer, 2020) and compared it to observed and subjective levels of responsibility (Douer & Meyer, 2019). We used the model to calculate operators' objective responsibility in a common fault event in the control room in a dairy factory. We compared the results to the subjective assessments made by different functions in the diary. The capabilities of the automation greatly exceeded those of the human, and the operator should comply with the indications of the automation. Thus, the objective causal human responsibility is 0. Outside observers, such as managers, assigned much higher responsibility to the operator, possibly holding operators responsible for adverse outcomes in situations in which they rightly trusted the automation.
NLPExplorer: Exploring the Universe of NLP Papers
Venue: arXiv cs.IR 2023
Year: 2023

Understanding the current research trends, problems, and their innovative solutions remains a bottleneck due to the ever-increasing volume of scientific articles. In this paper, we propose NLPExplorer, a completely automatic portal for indexing, searching, and visualizing Natural Language Processing (NLP) research volume. NLPExplorer presents interesting insights from papers, authors, venues, and topics. In contrast to previous topic modelling based approaches, we manually curate five course-grained non-exclusive topical categories namely Linguistic Target (Syntax, Discourse, etc.), Tasks (Tagging, Summarization, etc.), Approaches (unsupervised, supervised, etc.), Languages (English, Chinese,etc.) and Dataset types (news, clinical notes, etc.). Some of the novel features include a list of young popular authors, popular URLs, and datasets, a list of topically diverse papers and recent popular papers. Also, it provides temporal statistics such as yearwise popularity of topics, datasets, and seminal papers. To facilitate future research and system development, we make all the processed datasets accessible through API calls. The current system is available at http://lingo.iitgn.ac.in:5001/
Marpa, A practical general parser: the recognizer
Venue: arXiv cs.CL 2023
Year: 2023

The Marpa recognizer is described. Marpa is a practical and fully implemented algorithm for the recognition, parsing and evaluation of context-free grammars. The Marpa recognizer is the first to unite the improvements to Earley's algorithm found in Joop Leo's 1991 paper to those in Aycock and Horspool's 2002 paper. Marpa tracks the full state of the parse, at it proceeds, in a form convenient for the application. This greatly improves error detection and enables event-driven parsing. One such technique is "Ruby Slippers" parsing, in which the input is altered in response to the parser's expectations.
Proposing a Model for Predicting Passenger Origin-Destination in Online Taxi-Hailing Systems
Venue: arXiv cs.LG 2023
Year: 2023

Due to the significance of transportation planning, traffic management, and dispatch optimization, predicting passenger origin-destination has emerged as a crucial requirement for intelligent transportation systems management. In this study, we present a model designed to forecast the origin and destination of travels within a specified time window. To derive meaningful travel flows, we employ K-means clustering in a four-dimensional space with a maximum cluster size constraint for origin and destination zones. Given the large number of clusters, we utilize non-negative matrix factorization to reduce the number of travel clusters. Furthermore, we implement a stacked recurrent neural network model to predict the travel count in each cluster. A comparison of our results with existing models reveals that our proposed model achieves a 5-7\% lower mean absolute percentage error (MAPE) for 1-hour time windows and a 14\% lower MAPE for 30-minute time windows.
Sub-Nyquist Sampling of Sparse and Correlated Signals in Array Processing
Venue: arXiv cs.IT 2023
Year: 2023

This paper considers efficient sampling of simultaneously sparse and correlated (S$\&$C) signals. Such signals arise in various applications in array processing. We propose an implementable sampling architecture for the acquisition of S$\&$C at a sub-Nyquist rate. We prove a sampling theorem showing exact and stable reconstruction of the acquired signals even when the sampling rate is smaller than the Nyquist rate by orders of magnitude. Quantitatively, our results state that an ensemble $M$ signals, composed of a-priori unknown latent $R$ signals, each bandlimited to $W/2$ but only $S$-sparse in the Fourier domain, can be reconstructed exactly from compressive sampling only at a rate $RS\log^α W$ samples per second. When $R \ll M$, and $S\ll W$, this amounts to a significant reduction in sampling rate compared to the Nyquist rate of $MW$ samples per second. This is the first result that presents an implementable sampling architecture, and a sampling theorem for the compressive acquisition of S$\&$C signals. The signal reconstruction from sub-Nyquist rate boils down to a sparse and low-rank (S$\&$L) matrix recovery from a few linear measurements. The conventional convex penalties for S$\&$L matrices are provably not optimal in the number of measurements. We resort to a two-step algorithm to recover S$\&$L matrix from a near-optimal number of measurements. This result then translates into a signal reconstruction algorithm from a sub-Nyquist sampling rate.
Demystifying Graph Databases: Analysis and Taxonomy of Data Organization, System Designs, and Graph Queries
Venue: arXiv cs.DB 2023
Year: 2023

Graph processing has become an important part of multiple areas of computer science, such as machine learning, computational sciences, medical applications, social network analysis, and many others. Numerous graphs such as web or social networks may contain up to trillions of edges. Often, these graphs are also dynamic (their structure changes over time) and have domain-specific rich data associated with vertices and edges. Graph database systems such as Neo4j enable storing, processing, and analyzing such large, evolving, and rich datasets. Due to the sheer size of such datasets, combined with the irregular nature of graph processing, these systems face unique design challenges. To facilitate the understanding of this emerging domain, we present the first survey and taxonomy of graph database systems. We focus on identifying and analyzing fundamental categories of these systems (e.g., triple stores, tuple stores, native graph database systems, or object-oriented systems), the associated graph models (e.g., RDF or Labeled Property Graph), data organization techniques (e.g., storing graph data in indexing structures or dividing data into records), and different aspects of data distribution and query execution (e.g., support for sharding and ACID). 51 graph database systems are presented and compared, including Neo4j, OrientDB, or Virtuoso. We outline graph database queries and relationships with associated domains (NoSQL stores, graph streaming, and dynamic graph algorithms). Finally, we describe research and engineering challenges to outline the future of graph databases.
Hydra: Resilient and Highly Available Remote Memory
Venue: arXiv cs.DC 2023
Year: 2023

We present Hydra, a low-latency, low-overhead, and highly available resilience mechanism for remote memory. Hydra can access erasure-coded remote memory within a single-digit microsecond read/write latency, significantly improving the performance-efficiency trade-off over the state-of-the-art -- it performs similar to in-memory replication with 1.6X lower memory overhead. We also propose CodingSets, a novel coding group placement algorithm for erasure-coded data, that provides load balancing while reducing the probability of data loss under correlated failures by an order of magnitude. With Hydra, even when only 50% of memory is local, unmodified memory-intensive applications achieve performance close to that of the fully in-memory case in the presence of remote failures and outperform the state-of-the-art solutions by up to 4.35X.
Divide and Scale: Formalization and Roadmap to Robust Sharding
Venue: arXiv cs.DC 2023
Year: 2023

Sharding distributed ledgers is a promising on-chain solution for scaling blockchains but lacks formal grounds, nurturing skepticism on whether such complex systems can scale blockchains securely. We fill this gap by introducing the first formal framework as well as a roadmap to robust sharding. In particular, we first define the properties sharded distributed ledgers should fulfill. We build upon and extend the Bitcoin backbone protocol by defining consistency and scalability. Consistency encompasses the need for atomic execution of cross-shard transactions to preserve safety, whereas scalability encapsulates the speedup a sharded system can gain in comparison to a non-sharded system. Using our model, we explore the limitations of sharding. We show that a sharded ledger with $n$ participants cannot scale under a fully adaptive adversary, but it can scale up to $m$ shards where $n=c'm\log m$, under an epoch-adaptive adversary; the constant $c'$ encompasses the trade-off between security and scalability. This is possible only if the sharded ledgers create succinct proofs of the valid state updates at every epoch. We leverage our results to identify the sufficient components for robust sharding, which we incorporate in a protocol abstraction termed Divide & Scale. To demonstrate the power of our framework, we analyze the most prominent sharded blockchains (Elastico, Monoxide, OmniLedger, RapidChain) and pinpoint where they fail to meet the desired properties.
A Near-Optimal Gradient Flow for Learning Neural Energy-Based Models
Venue: arXiv cs.CV 2023
Year: 2023

In this paper, we propose a novel numerical scheme to optimize the gradient flows for learning energy-based models (EBMs). From a perspective of physical simulation, we redefine the problem of approximating the gradient flow utilizing optimal transport (i.e. Wasserstein) metric. In EBMs, the learning process of stepwise sampling and estimating data distribution performs the functional gradient of minimizing the global relative entropy between the current and target real distribution, which can be treated as dynamic particles moving from disorder to target manifold. Previous learning schemes mainly minimize the entropy concerning the consecutive time KL divergence in each learning step. However, they are prone to being stuck in the local KL divergence by projecting non-smooth information within smooth manifold, which is against the optimal transport principle. To solve this problem, we derive a second-order Wasserstein gradient flow of the global relative entropy from Fokker-Planck equation. Compared with existing schemes, Wasserstein gradient flow is a smoother and near-optimal numerical scheme to approximate real data densities. We also derive this near-proximal scheme and provide its numerical computation equations. Our extensive experiments demonstrate the practical superiority and potentials of our proposed scheme on fitting complex distributions and generating high-quality, high-dimensional data with neural EBMs.
Incentive-aware Contextual Pricing with Non-parametric Market Noise
Venue: arXiv cs.LG 2023
Year: 2023

We consider a dynamic pricing problem for repeated contextual second-price auctions with multiple strategic buyers who aim to maximize their long-term time discounted utility. The seller has limited information on buyers' overall demand curves which depends on a non-parametric market-noise distribution, and buyers may potentially submit corrupted bids (relative to true valuations) to manipulate the seller's pricing policy for more favorable reserve prices in the future. We focus on designing the seller's learning policy to set contextual reserve prices where the seller's goal is to minimize regret compared to the revenue of a benchmark clairvoyant policy that has full information of buyers' demand. We propose a policy with a phased-structure that incorporates randomized "isolation" periods, during which a buyer is randomly chosen to solely participate in the auction. We show that this design allows the seller to control the number of periods in which buyers significantly corrupt their bids. We then prove that our policy enjoys a $T$-period regret of $\widetilde{\mathcal{O}}(\sqrt{T})$ facing strategic buyers. Finally, we conduct numerical simulations to compare our proposed algorithm to standard pricing policies. Our numerical results show that our algorithm outperforms these policies under various buyer bidding behavior.
A New Approximation Algorithm for the Minimum 2-Edge-Connected Spanning Subgraph Problem
Venue: arXiv cs.DS 2023
Year: 2023

We present a new approximation algorithm for the minimum 2-edge-connected spanning subgraph problem. Its approximation ratio is $\frac{4}{3}$, which matches the current best ratio. The approximation ratio of the algorithm is $\frac{6}{5}$ on subcubic graphs, which is an improvement upon the previous best ratio of $\frac{5}{4}$. The algorithm is a novel extension of the primal-dual schema, which consists of two distinct phases. Both the algorithm and the analysis are much simpler than those of the previous approaches.
Region Normalization for Image Inpainting
Venue: arXiv cs.CV 2023
Year: 2023

Feature Normalization (FN) is an important technique to help neural network training, which typically normalizes features across spatial dimensions. Most previous image inpainting methods apply FN in their networks without considering the impact of the corrupted regions of the input image on normalization, e.g. mean and variance shifts. In this work, we show that the mean and variance shifts caused by full-spatial FN limit the image inpainting network training and we propose a spatial region-wise normalization named Region Normalization (RN) to overcome the limitation. RN divides spatial pixels into different regions according to the input mask, and computes the mean and variance in each region for normalization. We develop two kinds of RN for our image inpainting network: (1) Basic RN (RN-B), which normalizes pixels from the corrupted and uncorrupted regions separately based on the original inpainting mask to solve the mean and variance shift problem; (2) Learnable RN (RN-L), which automatically detects potentially corrupted and uncorrupted regions for separate normalization, and performs global affine transformation to enhance their fusion. We apply RN-B in the early layers and RN-L in the latter layers of the network respectively. Experiments show that our method outperforms current state-of-the-art methods quantitatively and qualitatively. We further generalize RN to other inpainting networks and achieve consistent performance improvements. Our code is available at https://github.com/geekyutao/RN.
DNA Linear Block Codes: Generation, Error-detection and Error-correction of DNA Codeword
Venue: arXiv cs.IT 2023
Year: 2023

In modern age, the increasing complexity of computation and communication technology is leading us towards the necessity of new paradigm. As a result, unconventional approach like DNA coding theory is gaining considerable attention. The storage capacity, information processing and transmission properties of DNA molecules stimulate the notion of DNA coding theory as well as DNA cryptography. In this paper we generate DNA codeword using DNA linear block codes which ensures the secure transmission of information. In the proposed code design strategy DNA-based XOR operation (DNAX) is applied for effective construction of DNA codewords which are quadruples generated over the set of alphabets consisting of four DNA bases adenine, thymine, guanine, and cytosine. By worked out examples we explain the use of generator matrix and parity check matrix in encryption and decryption of coded data in the form of short single stranded DNA sequences. The newly developed technique can detect as well as correcting error in transmission of DNA codewords through biological channels from sender to the intended receiver. Through DNA coding theory we are expanding the paths towards data compression and error correction in the form of DNA strands. This leads us towards a broader domain of DNA cryptography.
Optimization of Convolutional Neural Network Using the Linearly Decreasing Weight Particle Swarm Optimization
Venue: arXiv cs.NE 2023
Year: 2023

Convolutional neural network (CNN) is one of the most frequently used deep learning techniques. Various forms of models have been proposed and im-proved for learning at CNN. When learning with CNN, it is necessary to determine the optimal hyperparameters. However, the number of hyperparameters is so large that it is difficult to do it manually, so much research has been done on automation. A method that uses metaheuristic algorithms is attracting attention in research on hyperparameter optimization. Metaheuristic algorithms are naturally inspired and include evolution strategies, genetic algorithms, antcolony optimization and particle swarm optimization. In particular, particle swarm optimization converges faster than genetic algorithms, and various models have been proposed. In this paper, we pro-pose CNN hyperparameter optimization with linearly decreasing weight particle swarm optimization (LDWPSO). In the experiment, the MNIST data set and CIFAR-10 data set, which are often used as benchmark data sets, are used. By opti-mizing CNN hyperparameters with LDWPSO, learning the MNIST and CIFAR-10 datasets, we compare the accuracy with a standard CNN based on LeNet-5. As a result, when using the MNIST dataset, the baseline CNN is 94.02% at the 5th epoch, compared to 98.95% for LDWPSO CNN, which improves accuracy. When using the CIFAR-10 dataset, the Baseline CNN is 28.07% at the 10th epoch, compared to 69.37% for the LDWPSO CNN, which greatly improves accuracy. This paper is presented at the 36th Annual Conference of the Japanese Society for Artificial In-telligence. The final version is available at the following URL: https://doi.org/10.11517/pjsai.JSAI2022.0_2S4IS2b03