OpenRaiser/Intern-Atlas huggingface dataset:
A Context-Integrated Transformer-Based Neural Network for Auction Design
Venue: arXiv cs.GT 2023
Year: 2023

One of the central problems in auction design is developing an incentive-compatible mechanism that maximizes the auctioneer's expected revenue. While theoretical approaches have encountered bottlenecks in multi-item auctions, recently, there has been much progress on finding the optimal mechanism through deep learning. However, these works either focus on a fixed set of bidders and items, or restrict the auction to be symmetric. In this work, we overcome such limitations by factoring \emph{public} contextual information of bidders and items into the auction learning framework. We propose $\mathtt{CITransNet}$, a context-integrated transformer-based neural network for optimal auction design, which maintains permutation-equivariance over bids and contexts while being able to find asymmetric solutions. We show by extensive experiments that $\mathtt{CITransNet}$ can recover the known optimal solutions in single-item settings, outperform strong baselines in multi-item auctions, and generalize well to cases other than those in training.
SMGRL: Scalable Multi-resolution Graph Representation Learning
Venue: arXiv cs.LG 2023
Year: 2023

Graph convolutional networks (GCNs) allow us to learn topologically-aware node embeddings, which can be useful for classification or link prediction. However, they are unable to capture long-range dependencies between nodes without adding additional layers -- which in turn leads to over-smoothing and increased time and space complexity. Further, the complex dependencies between nodes make mini-batching challenging, limiting their applicability to large graphs. We propose a Scalable Multi-resolution Graph Representation Learning (SMGRL) framework that enables us to learn multi-resolution node embeddings efficiently. Our framework is model-agnostic and can be applied to any existing GCN model. We dramatically reduce training costs by training only on a reduced-dimension coarsening of the original graph, then exploit self-similarity to apply the resulting algorithm at multiple resolutions. The resulting multi-resolution embeddings can be aggregated to yield high-quality node embeddings that capture both long- and short-range dependencies. Our experiments show that this leads to improved classification accuracy, without incurring high computational costs.
Graph Representation Learning via Aggregation Enhancement
Venue: arXiv cs.LG 2023
Year: 2023

Graph neural networks (GNNs) have become a powerful tool for processing graph-structured data but still face challenges in effectively aggregating and propagating information between layers, which limits their performance. We tackle this problem with the kernel regression (KR) approach, using KR loss as the primary loss in self-supervised settings or as a regularization term in supervised settings. We show substantial performance improvements compared to state-of-the-art in both scenarios on multiple transductive and inductive node classification datasets, especially for deep networks. As opposed to mutual information (MI), KR loss is convex and easy to estimate in high-dimensional cases, even though it indirectly maximizes the MI between its inputs. Our work highlights the potential of KR to advance the field of graph representation learning and enhance the performance of GNNs. The code to reproduce our experiments is available at https://github.com/Anonymous1252022/KR_for_GNNs
Ergodic Sum Rate Capacity Achieving Transmit Design for Massive MIMO LEO Satellite Uplink Transmission
Venue: arXiv cs.IT 2023
Year: 2023

In this paper, we investigate the ergodic sum rate (ESR) capacity achieving uplink (UL) transmit design for massive multiple-input multiple-output (MIMO) low-earth-orbit (LEO) satellite communications with statistical channel state information at the user terminals (UTs). The UL massive MIMO LEO satellite channel model with uniform planar array configurations at the satellite and UTs is presented. We prove that the rank of each UT's optimal transmit covariance matrix does not exceed that of its channel correlation matrix at the UT side, which reveals the maximum number of independent data streams transmitted from each UT to the satellite. We then prove that the transmit covariance matrix design can be transformed into the lower-dimensional matrix design without loss of optimality. We also obtain a necessary and sufficient condition when single data stream transmission from each UT to the satellite can achieve the ESR capacity. A conditional gradient (CG) method is developed to compute the ESR capacity achieving transmit covariance matrices. Furthermore, to avoid the exhaustive sample average, we utilize an asymptotic expression of the ESR and devise a simplified CG method to compute the transmit covariance matrices, which can approximate the ESR capacity. Simulations demonstrate the effectiveness of the proposed approaches.
MDS multi-twisted Reed-Solomon codes with small dimensional hull
Venue: arXiv cs.IT 2023
Year: 2023

In this paper, we find a necessary and sufficient condition for multi-twisted Reed-Solomon codes to be MDS. In particular, we introduce a new class of MDS double-twisted Reed-Solomon codes $\mathcal{C}_{\bm α, \bm t, \bm h, \bm η}$ with twists $\bm t = (1, 2)$ and hooks $\bm h = (0, 1)$ over the finite field $\mathbb{F}_q$, providing a non-trivial example over $\mathbb{F}_{16}$ and enumeration over the finite fields of size up to 17. Moreover, we obtain necessary conditions for the existence of multi-twisted Reed-Solomon codes with small dimensional hull. Consequently, we derive conditions for the existence of MDS multi-twisted Reed-Solomon codes with small dimensional hull.
Trajectory balance: Improved credit assignment in GFlowNets
Venue: arXiv cs.LG 2023
Year: 2023

Generative flow networks (GFlowNets) are a method for learning a stochastic policy for generating compositional objects, such as graphs or strings, from a given unnormalized density by sequences of actions, where many possible action sequences may lead to the same object. We find previously proposed learning objectives for GFlowNets, flow matching and detailed balance, which are analogous to temporal difference learning, to be prone to inefficient credit propagation across long action sequences. We thus propose a new learning objective for GFlowNets, trajectory balance, as a more efficient alternative to previously used objectives. We prove that any global minimizer of the trajectory balance objective can define a policy that samples exactly from the target distribution. In experiments on four distinct domains, we empirically demonstrate the benefits of the trajectory balance objective for GFlowNet convergence, diversity of generated samples, and robustness to long action sequences and large action spaces.
SnAKe: Bayesian Optimization with Pathwise Exploration
Venue: arXiv cs.LG 2023
Year: 2023

Bayesian Optimization is a very effective tool for optimizing expensive black-box functions. Inspired by applications developing and characterizing reaction chemistry using droplet microfluidic reactors, we consider a novel setting where the expense of evaluating the function can increase significantly when making large input changes between iterations. We further assume we are working asynchronously, meaning we have to select new queries before evaluating previous experiments. This paper investigates the problem and introduces 'Sequential Bayesian Optimization via Adaptive Connecting Samples' (SnAKe), which provides a solution by considering large batches of queries and preemptively building optimization paths that minimize input costs. We investigate some convergence properties and empirically show that the algorithm is able to achieve regret similar to classical Bayesian Optimization algorithms in both synchronous and asynchronous settings, while reducing input costs significantly. We show the method is robust to the choice of its single hyper-parameter and provide a parameter-free alternative.
"How trustworthy is this research?" Designing a Tool to Help Readers Understand Evidence and Uncertainty in Science Journalism
Venue: arXiv cs.DL 2023
Year: 2023

This article reports on a Research through Design study exploring how to design a tool for helping readers of science journalism understand the strength and uncertainty of scientific evidence in news stories about health science, using both textual and visual information. A central aim has been to teach readers about criteria for assessing scientific evidence, in particular in order to help readers differentiate between science and pseudoscience. Working in a research-in-the-wild collaboration with a website for popular science, the study presents the design and evaluation of the Scientific Evidence Indicator, which uses metadata about scientific publications to present an assessment of evidence strength to the readers. Evaluations of the design demonstrate some success in helping readers recognize whether studies have undergone scientific peer review or not, but point to challenges in facilitating a more in-depth understanding. Insights from the study point to a potential for developing similar tools aimed at journalists rather than directly at audiences.
A Higher-Order Language for Markov Kernels and Linear Operators
Venue: arXiv cs.LO 2023
Year: 2023

Much work has been done to give semantics to probabilistic programming languages. In recent years, most of the semantics used to reason about probabilistic programs fall in two categories: semantics based on Markov kernels and semantics based on linear operators. Both styles of semantics have found numerous applications in reasoning about probabilistic programs, but they each have their strengths and weaknesses. Though it is believed that there is a connection between them there are no languages that can handle both styles of programming. In this work we address these questions by defining a two-level calculus and its categorical semantics which makes it possible to program with both kinds of semantics. From the logical side of things we see this language as an alternative resource interpretation of linear logic, where the resource being kept track of is sampling instead of variable use.
Entanglement-Assisted Quantum Error-Correcting Codes over Local Frobenius Rings
Venue: arXiv cs.IT 2023
Year: 2023

In this paper, we provide a framework for constructing entanglement-assisted quantum error-correcting codes (EAQECCs) from classical additive codes over a finite commutative local Frobenius ring $\mathcal{R}$. At the heart of the framework, and this is one of the main technical contributions of our paper, is a procedure to construct, for an additive code $\mathcal{C}$ over $\mathcal{R}$, a generating set for $\mathcal{C}$ that is in standard form, meaning that it consists purely of isotropic generators and hyperbolic pairs. Moreover, when $\mathcal{R}$ is a Galois ring, we give an exact expression for the minimum number of pairs of maximally entangled qudits required to construct an EAQECC from an additive code over $\mathcal{R}$, which significantly extends known results for EAQECCs over finite fields. We also demonstrate how adding extra coordinates to an additive code can give us a certain degree of flexibility in determining the parameters of the EAQECCs that result from our construction.
DoCoM: Compressed Decentralized Optimization with Near-Optimal Sample Complexity
Venue: arXiv cs.LG 2023
Year: 2023

This paper proposes the Doubly Compressed Momentum-assisted stochastic gradient tracking algorithm $\texttt{DoCoM}$ for communication-efficient decentralized optimization. The algorithm features two main ingredients to achieve a near-optimal sample complexity while allowing for communication compression. First, the algorithm tracks both the averaged iterate and stochastic gradient using compressed gossiping consensus. Second, a momentum step is incorporated for adaptive variance reduction with the local gradient estimates. We show that $\texttt{DoCoM}$ finds a near-stationary solution at all participating agents satisfying $\mathbb{E}[ \| \nabla f( θ) \|^2 ] = \mathcal{O}( 1 / T^{2/3} )$ in $T$ iterations, where $f(θ)$ is a smooth (possibly non-convex) objective function. Notice that the proof is achieved via analytically designing a new potential function that tightly tracks the one-iteration progress of $\texttt{DoCoM}$. As a corollary, our analysis also established the linear convergence of $\texttt{DoCoM}$ to a global optimal solution for objective functions with the Polyak-Łojasiewicz condition. Numerical experiments demonstrate that our algorithm outperforms several state-of-the-art algorithms in practice.
Multi-Order Networks for Action Unit Detection
Venue: arXiv cs.CV 2023
Year: 2023

Action Units (AU) are muscular activations used to describe facial expressions. Therefore accurate AU recognition unlocks unbiaised face representation which can improve face-based affective computing applications. From a learning standpoint AU detection is a multi-task problem with strong inter-task dependencies. To solve such problem, most approaches either rely on weight sharing, or add explicit dependency modelling by decomposing the joint task distribution using Bayes chain rule. If the latter strategy yields comprehensive inter-task relationships modelling, it requires imposing an arbitrary order into an unordered task set. Crucially, this ordering choice has been identified as a source of performance variations. In this paper, we present Multi-Order Network (MONET), a multi-task method with joint task order optimization. MONET uses a differentiable order selection to jointly learn task-wise modules with their optimal chaining order. Furthermore, we introduce warmup and order dropout to enhance order selection by encouraging order exploration. Experimentally, we first demonstrate MONET capacity to retrieve the optimal order in a toy environment. Second, we validate MONET architecture by showing that MONET outperforms existing multi-task baselines on multiple attribute detection problems chosen for their wide range of dependency settings. More importantly, we demonstrate that MONET significantly extends state-of-the-art performance in AU detection.
Remove, Reduce, Inform: What Actions do People Want Social Media Platforms to Take on Potentially Misleading Content?
Venue: arXiv cs.CY 2023
Year: 2023

To reduce the spread of misinformation, social media platforms may take enforcement actions against offending content, such as adding informational warning labels, reducing distribution, or removing content entirely. However, both their actions and their inactions have been controversial and plagued by allegations of partisan bias. When it comes to specific content items, surprisingly little is known about what ordinary people want the platforms to do. We provide empirical evidence about a politically balanced panel of lay raters' preferences for three potential platform actions on 368 news articles. Our results confirm that on many articles there is a lack of consensus on which actions to take. We find a clear hierarchy of perceived severity of actions with a majority of raters wanting informational labels on the most articles and removal on the fewest. There was no partisan difference in terms of how many articles deserve platform actions but conservatives did prefer somewhat more action on content from liberal sources, and vice versa. We also find that judgments about two holistic properties, misleadingness and harm, could serve as an effective proxy to determine what actions would be approved by a majority of raters.
Experimental Investigation of Trust in Anthropomorphic Agents as Task Partners
Venue: arXiv cs.HC 2023
Year: 2023

This study investigated whether human trust in a social robot with anthropomorphic physicality is similar to that in an AI agent or in a human in order to clarify how anthropomorphic physicality influences human trust in an agent. We conducted an online experiment using two types of cognitive tasks, calculation and emotion recognition tasks, where participants answered after referring to the answers of an AI agent, a human, or a social robot. During the experiment, the participants rated their trust levels in their partners. As a result, trust in the social robot was basically neither similar to that in the AI agent nor in the human and instead settled between them. The results showed a possibility that manipulating anthropomorphic features would help assist human users in appropriately calibrating trust in an agent.
Learning to reason about and to act on physical cascading events
Venue: arXiv cs.AI 2023
Year: 2023

Reasoning and interacting with dynamic environments is a fundamental problem in AI, but it becomes extremely challenging when actions can trigger cascades of cross-dependent events. We introduce a new supervised learning setup called {\em Cascade} where an agent is shown a video of a physically simulated dynamic scene, and is asked to intervene and trigger a cascade of events, such that the system reaches a "counterfactual" goal. For instance, the agent may be asked to "Make the blue ball hit the red one, by pushing the green ball". The agent intervention is drawn from a continuous space, and cascades of events makes the dynamics highly non-linear. We combine semantic tree search with an event-driven forward model and devise an algorithm that learns to search in semantic trees in continuous spaces. We demonstrate that our approach learns to effectively follow instructions to intervene in previously unseen complex scenes. It can also reason about alternative outcomes, when provided an observed cascade of events.
Orthogonal Fold & Cut
Venue: arXiv cs.CG 2023
Year: 2023

We characterize the cut patterns that can be produced by "orthogonal fold & cut": folding an axis-aligned rectangular sheet of paper along horizontal and vertical creases, and then making a single straight cut (at any angle). Along the way, we solve a handful of related problems: orthogonal fold & punch, 1D fold & cut, signed 1D fold & cut, and 1D interval fold & cut.
Multi-Output Gaussian Process-Based Data Augmentation for Multi-Building and Multi-Floor Indoor Localization
Venue: arXiv cs.NI 2023
Year: 2023

Location fingerprinting based on RSSI becomes a mainstream indoor localization technique due to its advantage of not requiring the installation of new infrastructure and the modification of existing devices, especially given the prevalence of Wi-Fi-enabled devices and the ubiquitous Wi-Fi access in modern buildings. The use of AI/ML technologies like DNNs makes location fingerprinting more accurate and reliable, especially for large-scale multi-building and multi-floor indoor localization. The application of DNNs for indoor localization, however, depends on a large amount of preprocessed and deliberately-labeled data for their training. Considering the difficulty of the data collection in an indoor environment, especially under the current epidemic situation of COVID-19, we investigate three different methods of RSSI data augmentation based on Multi-Output Gaussian Process (MOGP), i.e., by a single floor, by neighboring floors, and by a single building; unlike Single-Output Gaussian Process (SOGP), MOGP can take into account the correlation among RSSI observations from multiple Access Points (APs) deployed closely to each other (e.g., APs on the same floor of a building) by collectively handling them. The feasibility of the MOGP-based RSSI data augmentation is demonstrated through experiments based on the state-of-the-art RNN indoor localization model and the UJIIndoorLoc, i.e., the most popular publicly-available multi-building and multi-floor indoor localization database, where the RNN model trained with the UJIIndoorLoc database augmented by using the whole RSSI data of a building in fitting an MOGP model (i.e., by a single building) outperforms the other two augmentation methods as well as the RNN model trained with the original UJIIndoorLoc database, resulting in the mean three-dimensional positioning error of 8.42 m.
Learning Interpretable, High-Performing Policies for Autonomous Driving
Venue: arXiv cs.LG 2023
Year: 2023

Gradient-based approaches in reinforcement learning (RL) have achieved tremendous success in learning policies for autonomous vehicles. While the performance of these approaches warrants real-world adoption, these policies lack interpretability, limiting deployability in the safety-critical and legally-regulated domain of autonomous driving (AD). AD requires interpretable and verifiable control policies that maintain high performance. We propose Interpretable Continuous Control Trees (ICCTs), a tree-based model that can be optimized via modern, gradient-based, RL approaches to produce high-performing, interpretable policies. The key to our approach is a procedure for allowing direct optimization in a sparse decision-tree-like representation. We validate ICCTs against baselines across six domains, showing that ICCTs are capable of learning interpretable policy representations that parity or outperform baselines by up to 33% in AD scenarios while achieving a 300x-600x reduction in the number of policy parameters against deep learning baselines. Furthermore, we demonstrate the interpretability and utility of our ICCTs through a 14-car physical robot demonstration.
Verifying Inverse Model Neural Networks
Venue: arXiv cs.LG 2023
Year: 2023

Inverse problems exist in a wide variety of physical domains from aerospace engineering to medical imaging. The goal is to infer the underlying state from a set of observations. When the forward model that produced the observations is nonlinear and stochastic, solving the inverse problem is very challenging. Neural networks are an appealing solution for solving inverse problems as they can be trained from noisy data and once trained are computationally efficient to run. However, inverse model neural networks do not have guarantees of correctness built-in, which makes them unreliable for use in safety and accuracy-critical contexts. In this work we introduce a method for verifying the correctness of inverse model neural networks. Our approach is to overapproximate a nonlinear, stochastic forward model with piecewise linear constraints and encode both the overapproximate forward model and the neural network inverse model as a mixed-integer program. We demonstrate this verification procedure on a real-world airplane fuel gauge case study. The ability to verify and consequently trust inverse model neural networks allows their use in a wide variety of contexts, from aerospace to medicine.