OpenRaiser/Intern-Atlas huggingface dataset:
Toward Learning Human-aligned Cross-domain Robust Models by Countering Misaligned Features
Venue: arXiv cs.LG 2022
Year: 2022

Machine learning has demonstrated remarkable prediction accuracy over i.i.d data, but the accuracy often drops when tested with data from another distribution. In this paper, we aim to offer another view of this problem in a perspective assuming the reason behind this accuracy drop is the reliance of models on the features that are not aligned well with how a data annotator considers similar across these two datasets. We refer to these features as misaligned features. We extend the conventional generalization error bound to a new one for this setup with the knowledge of how the misaligned features are associated with the label. Our analysis offers a set of techniques for this problem, and these techniques are naturally linked to many previous methods in robust machine learning literature. We also compared the empirical strength of these methods demonstrated the performance when these previous techniques are combined, with an implementation available at https://github.com/OoDBag/WR
Hopcroft's Problem, Log-Star Shaving, 2D Fractional Cascading, and Decision Trees
Venue: arXiv cs.CG 2022
Year: 2022

We revisit Hopcroft's problem and related fundamental problems about geometric range searching. Given $n$ points and $n$ lines in the plane, we show how to count the number of point-line incidence pairs or the number of point-above-line pairs in $O(n^{4/3})$ time, which matches the conjectured lower bound and improves the best previous time bound of $n^{4/3}2^{O(\log^*n)}$ obtained almost 30 years ago by Matoušek. We describe two interesting and different ways to achieve the result: the first is randomized and uses a new 2D version of fractional cascading for arrangements of lines; the second is deterministic and uses decision trees in a manner inspired by the sorting technique of Fredman (1976). The second approach extends to any constant dimension. Many consequences follow from these new ideas: for example, we obtain an $O(n^{4/3})$-time algorithm for line segment intersection counting in the plane, $O(n^{4/3})$-time randomized algorithms for bichromatic closest pair and Euclidean minimum spanning tree in three or four dimensions, and a randomized data structure for halfplane range counting in the plane with $O(n^{4/3})$ preprocessing time and space and $O(n^{1/3})$ query time.
Confidence Composition for Monitors of Verification Assumptions
Venue: arXiv cs.LO 2022
Year: 2022

Closed-loop verification of cyber-physical systems with neural network controllers offers strong safety guarantees under certain assumptions. It is, however, difficult to determine whether these guarantees apply at run time because verification assumptions may be violated. To predict safety violations in a verified system, we propose a three-step confidence composition (CoCo) framework for monitoring verification assumptions. First, we represent the sufficient condition for verified safety with a propositional logical formula over assumptions. Second, we build calibrated confidence monitors that evaluate the probability that each assumption holds. Third, we obtain the confidence in the verification guarantees by composing the assumption monitors using a composition function suitable for the logical formula. Our CoCo framework provides theoretical bounds on the calibration and conservatism of compositional monitors. Two case studies show that compositional monitors are calibrated better than their constituents and successfully predict safety violations.
Flying Trapeze Act Motion Planning Algorithm for Two-Link Free-Flying Acrobatic Robot
Venue: arXiv cs.RO 2022
Year: 2022

A flying trapeze act can be a challenging task for a robotics system since some act requires the performer to catch another trapeze or catcher at the end after being airborne. The objective of this paper is to design and validate a motion planning algorithm for a two-link free-flying acrobatic robot that can accurately land on another trapeze after free-flying in the air. First, the proposed algorithm plan the robot trajectory with the non-linear constrained optimization method. Then, a feedback controller is implemented to stabilize the posture. However, since the spatial position of the center-of-mass of the robot cannot be controlled, this paper proposes a trajectory correction scheme that manipulates the robot's posture such that the robot is still able to land on the target. Lastly, the whole algorithm is validated in the simulation that mimics real-world circumstances.
Computing Simple Mechanisms: Lift-and-Round over Marginal Reduced Forms
Venue: arXiv cs.GT 2022
Year: 2022

We study revenue maximization in multi-item multi-bidder auctions under the natural item-independence assumption - a classical problem in Multi-Dimensional Bayesian Mechanism Design. One of the biggest challenges in this area is developing algorithms to compute (approximately) optimal mechanisms that are not brute-force in the size of the bidder type space, which is usually exponential in the number of items in multi-item auctions. Unfortunately, such algorithms were only known for basic settings of our problem when bidders have unit-demand [CHMS10,CMS15] or additive valuations [Yao15]. In this paper, we significantly improve the previous results and design the first algorithm that runs in time polynomial in the number of items and the number of bidders to compute mechanisms that are $O(1)$-approximations to the optimal revenue when bidders have XOS valuations, resolving an open problem raised in [CM16,CZ17]. Moreover, the computed mechanism has a simple structure: It is either a posted price mechanism or a two-part tariff mechanism. As a corollary of our result, we show how to compute an approximately optimal and simple mechanism efficiently using only sample access to the bidders' value distributions. Our algorithm builds on two innovations that allow us to search over the space of mechanisms efficiently: (i) a new type of succinct representation of mechanisms - the marginal reduced forms, and (ii) a novel Lift-and-Round procedure that concavifies the problem.
Consistency and Consensus Driven for Hesitant Fuzzy Linguistic Decision Making with Pairwise Comparisons
Venue: arXiv cs.AI 2022
Year: 2022

Hesitant fuzzy linguistic preference relation (HFLPR) is of interest because it provides an efficient way for opinion expression under uncertainty. For enhancing the theory of decision making with HFLPR, the paper introduces an algorithm for group decision making with HFLPRs based on the acceptable consistency and consensus measurements, which involves (1) defining a hesitant fuzzy linguistic geometric consistency index (HFLGCI) and proposing a procedure for consistency checking and inconsistency improving for HFLPR; (2) measuring the group consensus based on the similarity between the original individual HFLPRs and the overall perfect HFLPR, then establishing a procedure for consensus ensuring including the determination of decision-makers weights. The convergence and monotonicity of the proposed two procedures have been proved. Some experiments are furtherly performed to investigate the critical values of the defined HFLGCI, and comparative analyses are conducted to show the effectiveness of the proposed algorithm. A case concerning the performance evaluation of venture capital guiding funds is given to illustrate the availability of the proposed algorithm. As an application of our work, an online decision-making portal is finally provided for decision-makers to utilize the proposed algorithms to solve decision-making problems.
Terahertz Wireless Channels: A Holistic Survey on Measurement, Modeling, and Analysis
Venue: arXiv cs.IT 2022
Year: 2022

Terahertz (0.1-10 THz) communications are envisioned as a key technology for sixth generation (6G) wireless systems. The study of underlying THz wireless propagation channels provides the foundations for the development of reliable THz communication systems and their applications. This article provides a comprehensive overview of the study of THz wireless channels. First, the three most popular THz channel measurement methodologies, namely, frequency-domain channel measurement based on a vector network analyzer (VNA), time-domain channel measurement based on sliding correlation, and time-domain channel measurement based on THz pulses from time-domain spectroscopy (THz-TDS), are introduced and compared. Current channel measurement systems and measurement campaigns are reviewed. Then, existing channel modeling methodologies are categorized into deterministic, stochastic, and hybrid approaches. State-of-the-art THz channel models are analyzed, and the channel simulators that are based on them are introduced. Next, an in-depth review of channel characteristics in the THz band is presented. Finally, open problems and future research directions for research studies on THz wireless channels for 6G are elaborated.
High-degree compression functions on alternative models of elliptic curves and their applications
Venue: arXiv cs.CR 2022
Year: 2022

This paper presents method for obtaining high-degree compression functions using natural symmetries in a given model of an elliptic curve. Such symmetries may be found using symmetry of involution $[-1]$ and symmetry of translation morphism $τ_T=P+T$, where $T$ is the $n$-torsion point which naturally belongs to the $E(\mathbb K)$ for a given elliptic curve model. We will study alternative models of elliptic curves with points of order $2$ and $4$, and specifically Huff's curves and the Hessian family of elliptic curves (like Hessian, twisted Hessian and generalized Hessian curves) with a point of order $3$. We bring up some known compression functions on those models and present new ones as well. For (almost) every presented compression function, differential addition and point doubling formulas are shown. As in the case of high-degree compression functions manual investigation of differential addition and doubling formulas is very difficult, we came up with a Magma program which relies on the Gröbner basis. We prove that if for a model $E$ of an elliptic curve exists an isomorphism $ϕ:E \to E_M$, where $E_M$ is the Montgomery curve and for any $P \in E(\mathbb K)$ holds that $ϕ(P)=(ϕ_x(P), ϕ_y(P))$, then for a model $E$ one may find compression function of degree $2$. Moreover, one may find, defined for this compression function, differential addition and doubling formulas of the same efficiency as Montgomery's. However, it seems that for the family of elliptic curves having a natural point of order $3$, compression functions of the same efficiency do not exist.
A Dataset Perspective on Offline Reinforcement Learning
Venue: arXiv cs.LG 2022
Year: 2022

The application of Reinforcement Learning (RL) in real world environments can be expensive or risky due to sub-optimal policies during training. In Offline RL, this problem is avoided since interactions with an environment are prohibited. Policies are learned from a given dataset, which solely determines their performance. Despite this fact, how dataset characteristics influence Offline RL algorithms is still hardly investigated. The dataset characteristics are determined by the behavioral policy that samples this dataset. Therefore, we define characteristics of behavioral policies as exploratory for yielding high expected information in their interaction with the Markov Decision Process (MDP) and as exploitative for having high expected return. We implement two corresponding empirical measures for the datasets sampled by the behavioral policy in deterministic MDPs. The first empirical measure SACo is defined by the normalized unique state-action pairs and captures exploration. The second empirical measure TQ is defined by the normalized average trajectory return and captures exploitation. Empirical evaluations show the effectiveness of TQ and SACo. In large-scale experiments using our proposed measures, we show that the unconstrained off-policy Deep Q-Network family requires datasets with high SACo to find a good policy. Furthermore, experiments show that policy constraint algorithms perform well on datasets with high TQ and SACo. Finally, the experiments show, that purely dataset-constrained Behavioral Cloning performs competitively to the best Offline RL algorithms for datasets with high TQ.
A Sphere-Packing Error Exponent for Mismatched Decoding
Venue: arXiv cs.IT 2022
Year: 2022

We derive a sphere-packing error exponent for coded transmission over discrete memoryless channels with a fixed decoding metric. By studying the error probability of the code over an auxiliary channel, we find a lower bound to the probability of error of mismatched decoding. The bound is shown to decay exponentially for coding rates smaller than a new upper bound to the mismatch capacity which is established in this paper. For rates higher than the new upper bound, the error probability is shown to be bounded away from zero. The new upper bound is shown to improve over previous upper bounds to the mismatch capacity.
TAGLETS: A System for Automatic Semi-Supervised Learning with Auxiliary Data
Venue: arXiv cs.LG 2022
Year: 2022

Machine learning practitioners often have access to a spectrum of data: labeled data for the target task (which is often limited), unlabeled data, and auxiliary data, the many available labeled datasets for other tasks. We describe TAGLETS, a system built to study techniques for automatically exploiting all three types of data and creating high-quality, servable classifiers. The key components of TAGLETS are: (1) auxiliary data organized according to a knowledge graph, (2) modules encapsulating different methods for exploiting auxiliary and unlabeled data, and (3) a distillation stage in which the ensembled modules are combined into a servable model. We compare TAGLETS with state-of-the-art transfer learning and semi-supervised learning methods on four image classification tasks. Our study covers a range of settings, varying the amount of labeled data and the semantic relatedness of the auxiliary data to the target task. We find that the intelligent incorporation of auxiliary and unlabeled data into multiple learning techniques enables TAGLETS to match-and most often significantly surpass-these alternatives. TAGLETS is available as an open-source system at github.com/BatsResearch/taglets.
Automated generation of 0D and 1D reduced-order models of patient-specific blood flow
Venue: arXiv cs.CE 2022
Year: 2022

Three-dimensional (3D) cardiovascular fluid dynamics simulations typically require hours to days of computing time on a high-performance computing cluster. One-dimensional (1D) and lumped-parameter zero-dimensional (0D) models show great promise for accurately predicting blood bulk flow and pressure waveforms with only a fraction of the cost. They can also accelerate uncertainty quantification, optimization, and design parameterization studies. Despite several prior studies generating 1D and 0D models and comparing them to 3D solutions, these were typically limited to either 1D or 0D and a singular category of vascular anatomies. This work proposes a fully automated and openly available framework to generate and simulate 1D and 0D models from 3D patient-specific geometries, automatically detecting vessel junctions and stenosis segments. Our only input is the 3D geometry; we do not use any prior knowledge from 3D simulations. All computational tools presented in this work are implemented in the open-source software platform SimVascular. We demonstrate the reduced-order approximation quality against rigid-wall 3D solutions in a comprehensive comparison with N=72 publicly available models from various anatomies, vessel types, and disease conditions. Relative average approximation errors of flows and pressures typically ranged from 1% to 10% for both 1D and 0D models, measured at the outlets of terminal vessel branches. In general, 0D model errors were only slightly higher than 1D model errors despite requiring only a third of the 1D runtime. Automatically generated ROMs can significantly speed up model development and shift the computational load from high-performance machines to personal computers.
FPM: A Collection of Large-scale Foundation Pre-trained Language Models
Venue: arXiv cs.CL 2022
Year: 2022

Large-scale Transformer models have significantly promoted the recent development of natural language processing applications. However, little effort has been made to unify the effective models. In this paper, driven by providing a new set of baseline models in the future, we adopt various novel transformer architectures and launch a model set with the help of recent mainstream technologies. We focus the discussions on optimizing the depth of the networks based on the existing powerful encode-decoder structures. We show that by properly avoiding training defects such as non-convergence and degradation, scaling up off-the-shelf transformer architectures consistently delivers better performance. To stimulate future research on large-scale language model pretraining, we present extensive results and detailed discussions on network performance improvements with respect to the network depth and confirm the existence of the optimal number of layers under specific tasks. To the best of our knowledge, we provide the largest Chinese generative model and the largest Chinese encoding model. The BERT language models we trained on English datasets deliver a 14.45% higher F1 score than the Turing-NLR.
A Survey and Empirical Evaluation of Parallel Deep Learning Frameworks
Venue: arXiv cs.LG 2022
Year: 2022

The field of deep learning has witnessed a remarkable shift towards extremely compute- and memory-intensive neural networks. These newer larger models have enabled researchers to advance state-of-the-art tools across a variety of fields. This phenomenon has spurred the development of algorithms for distributed training of neural networks over a larger number of hardware accelerators. In this paper, we discuss and compare current state-of-the-art frameworks for large scale distributed deep learning. First, we survey current practices in distributed learning and identify the different types of parallelism used. Then, we present empirical results comparing their performance on large image and language training tasks. Additionally, we address their statistical efficiency and memory consumption behavior. Based on our results, we discuss algorithmic and implementation portions of each framework which hinder performance.
Optimal Rate-Distortion-Leakage Tradeoff for Single-Server Information Retrieval
Venue: arXiv cs.IT 2022
Year: 2022

Private information retrieval protocols guarantee that a user can privately and losslessly retrieve a single file from a database stored across multiple servers. In this work, we propose to simultaneously relax the conditions of perfect retrievability and privacy in order to obtain improved download rates when all files are stored uncoded on a single server. Information leakage is measured in terms of the average success probability for the server of correctly guessing the identity of the desired file. The main findings are: i) The derivation of the optimal tradeoff between download rate, distortion, and information leakage when the file size is infinite. Closed-form expressions of the optimal tradeoff for the special cases of "no-leakage" and "no-privacy" are also given. ii) A novel approach based on linear programming (LP) to construct schemes for a finite file size and an arbitrary number of files. The proposed LP approach can be leveraged to find provably optimal schemes with corresponding closed-form expressions for the rate-distortion-leakage tradeoff when the database contains at most four bits. Finally, for a database that contains 320 bits, we compare two construction methods based on the LP approach with a nonconstructive scheme downloading subsets of files using a finite-length lossy compressor based on random coding.
On Training Implicit Models
Venue: arXiv cs.LG 2022
Year: 2022

This paper focuses on training implicit models of infinite layers. Specifically, previous works employ implicit differentiation and solve the exact gradient for the backward propagation. However, is it necessary to compute such an exact but expensive gradient for training? In this work, we propose a novel gradient estimate for implicit models, named phantom gradient, that 1) forgoes the costly computation of the exact gradient; and 2) provides an update direction empirically preferable to the implicit model training. We theoretically analyze the condition under which an ascent direction of the loss landscape could be found, and provide two specific instantiations of the phantom gradient based on the damped unrolling and Neumann series. Experiments on large-scale tasks demonstrate that these lightweight phantom gradients significantly accelerate the backward passes in training implicit models by roughly 1.7 times, and even boost the performance over approaches based on the exact gradient on ImageNet.
An Open-Source RRAM Compiler
Venue: arXiv cs.ET 2022
Year: 2022

Memory compilers are necessary tools to boost the design procedure of digital circuits. However, only a few are available to academia. Resistive Random Access Memory (RRAM) is characterised by high density, high speed, non volatility and is a potential candidate of future digital memories. To the best of the authors' knowledge, this paper presents the first open source RRAM compiler for automatic memory generation including its peripheral circuits, verification and timing characterisation. The RRAM compiler is written with Cadence SKILL programming language and is integrated in Cadence environment. The layout verification procedure takes place in Siemens Mentor Calibre tool. The technology used by the compiler is TSMC 180nm. This paper analyses the novel results of a plethora of M x N RRAMs generated by the compiler, up to M = 128, N = 64 and word size B = 16 bits, for clock frequency equal to 12.5 MHz. Finally, the compiler achieves density of up to 0.024 Mb/mm2.
SGD Through the Lens of Kolmogorov Complexity
Venue: arXiv cs.LG 2022
Year: 2022

We prove that stochastic gradient descent (SGD) finds a solution that achieves $(1-ε)$ classification accuracy on the entire dataset. We do so under two main assumptions: (1. Local progress) The model accuracy improves on average over batches. (2. Models compute simple functions) The function computed by the model is simple (has low Kolmogorov complexity). It is sufficient that these assumptions hold only for a tiny fraction of the epochs. Intuitively, the above means that intermittent local progress of SGD implies global progress. Assumption 2 trivially holds for underparameterized models, hence, our work gives the first convergence guarantee for general, underparameterized models. Furthermore, this is the first result which is completely model agnostic - we do not require the model to have any specific architecture or activation function, it may not even be a neural network. Our analysis makes use of the entropy compression method, which was first introduced by Moser and Tardos in the context of the Lovász local lemma.