OpenRaiser/Intern-Atlas huggingface dataset:
Reconciling Shannon and Scott with a Lattice of Computable Information
Venue: arXiv cs.PL 2023
Year: 2023

This paper proposes a reconciliation of two different theories of information. The first, originally proposed in a lesser-known work by Claude Shannon, describes how the information content of channels can be described qualitatively, but still abstractly, in terms of information elements, i.e. equivalence relations over the data source domain. Shannon showed that these elements form a complete lattice, with the order expressing when one element is more informative than another. In the context of security and information flow this structure has been independently rediscovered several times, and used as a foundation for reasoning about information flow. The second theory of information is Dana Scott's domain theory, a mathematical framework for giving meaning to programs as continuous functions over a particular topology. Scott's partial ordering also represents when one element is more informative than another, but in the sense of computational progress, i.e. when one element is a more defined or evolved version of another. To give a satisfactory account of information flow in programs it is necessary to consider both theories together, to understand what information is conveyed by a program viewed as a channel (à la Shannon) but also by the definedness of its encoding (à la Scott). We combine these theories by defining the Lattice of Computable Information (LoCI), a lattice of preorders rather than equivalence relations. LoCI retains the rich lattice structure of Shannon's theory, filters out elements that do not make computational sense, and refines the remaining information elements to reflect how Scott's ordering captures the way that information is presented. We show how the new theory facilitates the first general definition of termination-insensitive information flow properties, a weakened form of information flow property commonly targeted by static program analyses.
Promoting Social Behaviour in Reducing Peak Electricity Consumption Using Multi-Agent Systems
Venue: arXiv cs.MA 2023
Year: 2023

As we transition to renewable energy sources, addressing their inflexibility during peak demand becomes crucial. It is therefore important to reduce the peak load placed on our energy system. For households, this entails spreading high-power appliance usage like dishwashers and washing machines throughout the day. Traditional approaches to spreading out usage have relied on differential pricing set by a centralised utility company, but this has been ineffective. Our previous research investigated a decentralised mechanism where agents receive an initial allocation of time-slots to use their appliances, which they can exchange with others. This was found to be an effective approach to reducing the peak load when we introduced social capital, the tracking of favours, to incentivise agents to accept exchanges that do not immediately benefit them. This system encouraged self-interested agents to learn socially beneficial behaviour to earn social capital that they could later use to improve their own performance. In this paper we expand this work by implementing real world household appliance usage data to ensure that our mechanism could adapt to the challenging demand needs of real households. We also demonstrate how smaller and more diverse populations can optimise more effectively than larger community energy systems.
Bayesian Multi-Task Learning MPC for Robotic Mobile Manipulation
Venue: arXiv cs.RO 2023
Year: 2023

Mobile manipulation in robotics is challenging due to the need of solving many diverse tasks, such as opening a door or picking-and-placing an object. Typically, a basic first-principles system description of the robot is available, thus motivating the use of model-based controllers. However, the robot dynamics and its interaction with an object are affected by uncertainty, limiting the controller's performance. To tackle this problem, we propose a Bayesian multi-task learning model that uses trigonometric basis functions to identify the error in the dynamics. In this way, data from different but related tasks can be leveraged to provide a descriptive error model that can be efficiently updated online for new, unseen tasks. We combine this learning scheme with a model predictive controller, and extensively test the effectiveness of the proposed approach, including comparisons with available baseline controllers. We present simulation tests with a ball-balancing robot, and door-opening hardware experiments with a quadrupedal manipulator.
TSN-FlexTest: Flexible TSN Measurement Testbed (Extended Version)
Venue: arXiv cs.NI 2023
Year: 2023

Robust, reliable, and deterministic networks are essential for a variety of applications. In order to provide guaranteed communication network services, Time-Sensitive Networking (TSN) unites a set of standards for time-synchronization, flow control, enhanced reliability, and management. We design the TSN-FlexTest testbed with generic commodity hardware and open-source software components to enable flexible TSN measurements. We have conducted extensive measurements to validate the TSN-FlexTest testbed and to examine TSN characteristics. The measurements provide insights into the effects of TSN configurations, such as increasing the number of synchronization messages for the Precision Time Protocol, indicating that a measurement accuracy of 15 ns can be achieved. The TSN measurements included extensive evaluations of the Time-aware Shaper (TAS) for sets of Tactile Internet (TI) packet traffic streams. The measurements elucidate the effects of different scheduling and shaping approaches, while revealing the need for pervasive network control that synchronizes the sending nodes with the network switches. We present the first measurements of distributed TAS with synchronized senders on a commodity hardware testbed, demonstrating the same Quality-of-Service as with dedicated wires for high-priority TI streams despite a 200% over-saturation cross traffic load. The testbed is provided as an open-source project to facilitate future TSN research.
Bidirectional Generation of Structure and Properties Through a Single Molecular Foundation Model
Venue: arXiv cs.LG 2023
Year: 2023

The recent success of large foundation models in artificial intelligence has prompted the emergence of chemical pre-trained models. Despite the growing interest in large molecular pre-trained models that provide informative representations for downstream tasks, attempts for multimodal pre-training approaches on the molecule domain were limited. To address this, we present a novel multimodal molecular pre-trained model that incorporates the modalities of structure and biochemical properties, drawing inspiration from recent advances in multimodal learning techniques. Our proposed model pipeline of data handling and training objectives aligns the structure/property features in a common embedding space, which enables the model to regard bidirectional information between the molecules' structure and properties. These contributions emerge synergistic knowledge, allowing us to tackle both multimodal and unimodal downstream tasks through a single model. Through extensive experiments, we demonstrate that our model shows remarkable capabilities in solving various meaningful chemical challenges, including conditional molecule generation, property prediction, molecule classification, and reaction prediction.
Intelligent Reflecting Surfaces Assisted Millimeter Wave MIMO Full Duplex Systems
Venue: arXiv cs.IT 2023
Year: 2023

Full duplex (FD) systems suffer from very high hardware cost and high power consumption to mitigate the self-interference (SI) in the analog domain. Moreover, in millimeter wave (mmWave) they rely on hybrid beamforming (HYBF) as a signal processing tool to partially deal with the SI, which presents many drawbacks such as high insertion loss and high power consumption. This article proposes the use of near-field (NF-) IRSs for FD systems with the objective to solve the aforementioned issues cost-efficiently. Namely, we propose to truncate the analog/hybrid beamforming stage of the mmWave FD systems and compensate it with an NF-IRS, to simultaneously and smartly control the uplink (UL) and downlink (DL) channels, while assisting in shaping the SI channel: this to obtain very strong passive SI cancellation. A novel joint active and passive beamforming design for the weighted sum-rate (WSR) maximization of a NF-IRS-assisted mmWave point-to-point FD system is presented. Results show that the proposed solution fully reaps the benefits of the IRSs only when they operate in the NF, which leads to considerably higher gains compared to the conventional massive MIMO (mMIMO) mmWave FD and half duplex (HD) systems.
Reward is not Necessary: How to Create a Modular & Compositional Self-Preserving Agent for Life-Long Learning
Venue: arXiv cs.AI 2023
Year: 2023

Reinforcement Learning views the maximization of rewards and avoidance of punishments as central to explaining goal-directed behavior. However, over a life, organisms will need to learn about many different aspects of the world's structure: the states of the world and state-vector transition dynamics. The number of combinations of states grows exponentially as an agent incorporates new knowledge, and there is no obvious weighted combination of pre-existing rewards or costs defined for a given combination of states, as such a weighting would need to encode information about good and bad combinations prior to an agent's experience in the world. Therefore, we must develop more naturalistic accounts of behavior and motivation in large state-spaces. We show that it is possible to use only the intrinsic motivation metric of empowerment, which measures the agent's capacity to realize many possible futures under a transition operator. We propose to scale empowerment to hierarchical state-spaces by using Operator Bellman Equations. These equations produce state-time feasibility functions, which are compositional hierarchical state-time transition operators that map an initial state and time when an agent begins a policy to the final states and times of completing a goal. Because these functions are hierarchical operators we can define hierarchical empowerment measures on them. An agent can then optimize plans to distant states and times to maximize its hierarchical empowerment-gain, allowing it to discover goals that bring about a more favorable coupling of its internal structure (physiological states) to its external environment (world structure & spatial state). Life-long agents could therefore be primarily animated by principles of compositionality and empowerment, exhibiting self-concern for the growth & maintenance of their own structural integrity without recourse to reward-maximization.
Distinctive Self-Similar Object Detection
Venue: arXiv cs.CV 2023
Year: 2023

Deep learning-based object detection has demonstrated a significant presence in the practical applications of artificial intelligence. However, objects such as fire and smoke, pose challenges to object detection because of their non-solid and various shapes, and consequently difficult to truly meet requirements in practical fire prevention and control. In this paper, we propose that the distinctive fractal feature of self-similar in fire and smoke can relieve us from struggling with their various shapes. To our best knowledge, we are the first to discuss this problem. In order to evaluate the self-similarity of the fire and smoke and improve the precision of object detection, we design a semi-supervised method that use Hausdorff distance to describe the resemblance between instances. Besides, based on the concept of self-similar, we have devised a novel methodology for evaluating this particular task in a more equitable manner. We have meticulously designed our network architecture based on well-established and representative baseline networks such as YOLO and Faster R-CNN. Our experiments have been conducted on publicly available fire and smoke detection datasets, which we have thoroughly verified to ensure the validity of our approach. As a result, we have observed significant improvements in the detection accuracy.
Taming Reachability Analysis of DNN-Controlled Systems via Abstraction-Based Training
Venue: arXiv cs.LG 2023
Year: 2023

The intrinsic complexity of deep neural networks (DNNs) makes it challenging to verify not only the networks themselves but also the hosting DNN-controlled systems. Reachability analysis of these systems faces the same challenge. Existing approaches rely on over-approximating DNNs using simpler polynomial models. However, they suffer from low efficiency and large overestimation, and are restricted to specific types of DNNs. This paper presents a novel abstraction-based approach to bypass the crux of over-approximating DNNs in reachability analysis. Specifically, we extend conventional DNNs by inserting an additional abstraction layer, which abstracts a real number to an interval for training. The inserted abstraction layer ensures that the values represented by an interval are indistinguishable to the network for both training and decision-making. Leveraging this, we devise the first black-box reachability analysis approach for DNN-controlled systems, where trained DNNs are only queried as black-box oracles for the actions on abstract states. Our approach is sound, tight, efficient, and agnostic to any DNN type and size. The experimental results on a wide range of benchmarks show that the DNNs trained by using our approach exhibit comparable performance, while the reachability analysis of the corresponding systems becomes more amenable with significant tightness and efficiency improvement over the state-of-the-art white-box approaches.
FLNeRF: 3D Facial Landmarks Estimation in Neural Radiance Fields
Venue: arXiv cs.CV 2023
Year: 2023

This paper presents the first significant work on directly predicting 3D face landmarks on neural radiance fields (NeRFs). Our 3D coarse-to-fine Face Landmarks NeRF (FLNeRF) model efficiently samples from a given face NeRF with individual facial features for accurate landmarks detection. Expression augmentation is applied to facial features in a fine scale to simulate large emotions range including exaggerated facial expressions (e.g., cheek blowing, wide opening mouth, eye blinking) for training FLNeRF. Qualitative and quantitative comparison with related state-of-the-art 3D facial landmark estimation methods demonstrate the efficacy of FLNeRF, which contributes to downstream tasks such as high-quality face editing and swapping with direct control using our NeRF landmarks. Code and data will be available. Github link: https://github.com/ZHANG1023/FLNeRF.
A Generalized EigenGame with Extensions to Multiview Representation Learning
Venue: arXiv cs.LG 2023
Year: 2023

Generalized Eigenvalue Problems (GEPs) encompass a range of interesting dimensionality reduction methods. Development of efficient stochastic approaches to these problems would allow them to scale to larger datasets. Canonical Correlation Analysis (CCA) is one example of a GEP for dimensionality reduction which has found extensive use in problems with two or more views of the data. Deep learning extensions of CCA require large mini-batch sizes, and therefore large memory consumption, in the stochastic setting to achieve good performance and this has limited its application in practice. Inspired by the Generalized Hebbian Algorithm, we develop an approach to solving stochastic GEPs in which all constraints are softly enforced by Lagrange multipliers. Then by considering the integral of this Lagrangian function, its pseudo-utility, and inspired by recent formulations of Principal Components Analysis and GEPs as games with differentiable utilities, we develop a game-theory inspired approach to solving GEPs. We show that our approaches share much of the theoretical grounding of the previous Hebbian and game theoretic approaches for the linear case but our method permits extension to general function approximators like neural networks for certain GEPs for dimensionality reduction including CCA which means our method can be used for deep multiview representation learning. We demonstrate the effectiveness of our method for solving GEPs in the stochastic setting using canonical multiview datasets and demonstrate state-of-the-art performance for optimizing Deep CCA.
Brief Announcement: Broadcasting Time in Dynamic Rooted Trees is Linear
Venue: arXiv cs.DC 2023
Year: 2023

We study the broadcast problem on dynamic networks with $n$ processes. The processes communicate in synchronous rounds along an arbitrary rooted tree. The sequence of trees is given by an adversary whose goal is to maximize the number of rounds until at least one process reaches all other processes. Previous research has shown a $\lceil{\frac{3n-1}{2}}\rceil-2$ lower bound and an $O(n\log\log n)$ upper bound. We show the first linear upper bound for this problem, namely $\lceil{(1 + \sqrt 2) n-1}\rceil \approx 2.4n$. Our result follows from a detailed analysis of the evolution of the adjacency matrix of the network over time.
Privacy in Practice: Private COVID-19 Detection in X-Ray Images (Extended Version)
Venue: arXiv cs.LG 2023
Year: 2023

Machine learning (ML) can help fight pandemics like COVID-19 by enabling rapid screening of large volumes of images. To perform data analysis while maintaining patient privacy, we create ML models that satisfy Differential Privacy (DP). Previous works exploring private COVID-19 models are in part based on small datasets, provide weaker or unclear privacy guarantees, and do not investigate practical privacy. We suggest improvements to address these open gaps. We account for inherent class imbalances and evaluate the utility-privacy trade-off more extensively and over stricter privacy budgets. Our evaluation is supported by empirically estimating practical privacy through black-box Membership Inference Attacks (MIAs). The introduced DP should help limit leakage threats posed by MIAs, and our practical analysis is the first to test this hypothesis on the COVID-19 classification task. Our results indicate that needed privacy levels might differ based on the task-dependent practical threat from MIAs. The results further suggest that with increasing DP guarantees, empirical privacy leakage only improves marginally, and DP therefore appears to have a limited impact on practical MIA defense. Our findings identify possibilities for better utility-privacy trade-offs, and we believe that empirical attack-specific privacy estimation can play a vital role in tuning for practical privacy.
Neural Meta-Symbolic Reasoning and Learning
Venue: arXiv cs.AI 2023
Year: 2023

Deep neural learning uses an increasing amount of computation and data to solve very specific problems. By stark contrast, human minds solve a wide range of problems using a fixed amount of computation and limited experience. One ability that seems crucial to this kind of general intelligence is meta-reasoning, i.e., our ability to reason about reasoning. To make deep learning do more from less, we propose the first neural meta-symbolic system (NEMESYS) for reasoning and learning: meta programming using differentiable forward-chaining reasoning in first-order logic. Differentiable meta programming naturally allows NEMESYS to reason and learn several tasks efficiently. This is different from performing object-level deep reasoning and learning, which refers in some way to entities external to the system. In contrast, NEMESYS enables self-introspection, lifting from object- to meta-level reasoning and vice versa. In our extensive experiments, we demonstrate that NEMESYS can solve different kinds of tasks by adapting the meta-level programs without modifying the internal reasoning system. Moreover, we show that NEMESYS can learn meta-level programs given examples. This is difficult, if not impossible, for standard differentiable logic programming
Formal Abstractions for Packet Scheduling
Venue: arXiv cs.NI 2023
Year: 2023

Early programming models for software-defined networking (SDN) focused on basic features for controlling network-wide forwarding paths, but more recent work has considered richer features, such as packet scheduling and queueing, that affect performance. In particular, PIFO trees, proposed by Sivaraman et al., offer a flexible and efficient primitive for programmable packet scheduling. Prior work has shown that PIFO trees can express a wide range of practical algorithms including strict priority, weighted fair queueing, and hierarchical schemes. However, the semantic properties of PIFO trees are not well understood. This paper studies PIFO trees from a programming language perspective. We formalize the syntax and semantics of PIFO trees in an operational model that decouples the scheduling policy running on a tree from the topology of the tree. Building on this formalization, we develop compilation algorithms that allow the behavior of a PIFO tree written against one topology to be realized using a tree with a different topology. Such a compiler could be used to optimize an implementation of PIFO trees, or realize a logical PIFO tree on a target with a fixed topology baked into the hardware. To support experimentation, we develop a software simulator for PIFO trees, and we present case studies illustrating its behavior on standard and custom algorithms.
Visual Dexterity: In-Hand Reorientation of Novel and Complex Object Shapes
Venue: arXiv cs.RO 2023
Year: 2023

In-hand object reorientation is necessary for performing many dexterous manipulation tasks, such as tool use in less structured environments that remain beyond the reach of current robots. Prior works built reorientation systems assuming one or many of the following: reorienting only specific objects with simple shapes, limited range of reorientation, slow or quasistatic manipulation, simulation-only results, the need for specialized and costly sensor suites, and other constraints which make the system infeasible for real-world deployment. We present a general object reorientation controller that does not make these assumptions. It uses readings from a single commodity depth camera to dynamically reorient complex and new object shapes by any rotation in real-time, with the median reorientation time being close to seven seconds. The controller is trained using reinforcement learning in simulation and evaluated in the real world on new object shapes not used for training, including the most challenging scenario of reorienting objects held in the air by a downward-facing hand that must counteract gravity during reorientation. Our hardware platform only uses open-source components that cost less than five thousand dollars. Although we demonstrate the ability to overcome assumptions in prior work, there is ample scope for improving absolute performance. For instance, the challenging duck-shaped object not used for training was dropped in 56 percent of the trials. When it was not dropped, our controller reoriented the object within 0.4 radians (23 degrees) 75 percent of the time. Videos are available at: https://taochenshh.github.io/projects/visual-dexterity.
Continuous Functions on Final Comodels of Free Algebraic Theories
Venue: arXiv cs.LO 2023
Year: 2023

In 2009, Ghani, Hancock and Pattinson gave a tree-like representation of stream processors $A^{\mathbb{N}} \rightarrow B^{\mathbb{N}}$. In 2021, Garner showed that this representation can be established in terms of algebraic theory and comodels: the set of infinite streams $A^{\mathbb{N}}$ is the final comodel of the algebraic theory of $A$-valued input $\mathbb{T}_A$ and the set of stream processors $\mathit{Top}(A^{\mathbb{N}},B^{\mathbb{N}})$ can be seen as the final $\mathbb{T}_A$-$\mathbb{T}_B$-bimodel. In this paper, we generalize Garner's results to the case of free algebraic theories.