OpenRaiser/Intern-Atlas huggingface dataset:
How Semantic Information G Measure Relates to Distortion, Freshness, Purposiveness, and Efficiency
Venue: arXiv cs.IT 2022
Year: 2022

To improve communication efficiency and provide more useful information, we need to measure semantic information by combining inaccuracy or distortion, freshness, purposiveness, and efficiency. The author proposed the semantic information G measure before. This measure is more compatible with Shannon information theory than other semantic or generalized information measures and has been applied to machine learning. This paper focuses on semantic predictive information (including observation information) and purposive (or goal-related) information (involving semantic communication and constraint control). The GPS pointer is used as an example to discuss the change of semantic predictive information with inaccuracy and time (age of the information). An example of constraint control (controlling probability distributions) is provided for measuring purposive information and maximizing this information and the information efficiency. The information rate fidelity function (a generalization of the information rate distortion function) is introduced for the optimization. Two computing examples demonstrate how to measure predictive and goal-related information and optimizing information efficiency. The results accord with theoretical conclusions well. The G measure is related to deep learning; its application to machine learning is worth exploring. Communication efficiency also involves utilities or information values; semantic communication optimization combining utilities needs further research.
On the `Semantics' of Differential Privacy: A Bayesian Formulation
Venue: arXiv cs.CR 2023
Year: 2023

Differential privacy is a definition of "privacy'" for algorithms that analyze and publish information about statistical databases. It is often claimed that differential privacy provides guarantees against adversaries with arbitrary side information. In this paper, we provide a precise formulation of these guarantees in terms of the inferences drawn by a Bayesian adversary. We show that this formulation is satisfied by both "vanilla" differential privacy as well as a relaxation known as (epsilon,delta)-differential privacy. Our formulation follows the ideas originally due to Dwork and McSherry [Dwork 2006]. This paper is, to our knowledge, the first place such a formulation appears explicitly. The analysis of the relaxed definition is new to this paper, and provides some concrete guidance for setting parameters when using (epsilon,delta)-differential privacy.
Domain Adaptation: Learning Bounds and Algorithms
Venue: arXiv cs.LG 2023
Year: 2023

This paper addresses the general problem of domain adaptation which arises in a variety of applications where the distribution of the labeled sample available somewhat differs from that of the test data. Building on previous work by Ben-David et al. (2007), we introduce a novel distance between distributions, discrepancy distance, that is tailored to adaptation problems with arbitrary loss functions. We give Rademacher complexity bounds for estimating the discrepancy distance from finite samples for different loss functions. Using this distance, we derive novel generalization bounds for domain adaptation for a wide family of loss functions. We also present a series of novel adaptation bounds for large classes of regularization-based algorithms, including support vector machines and kernel ridge regression based on the empirical discrepancy. This motivates our analysis of the problem of minimizing the empirical discrepancy for various loss functions for which we also give novel algorithms. We report the results of preliminary experiments that demonstrate the benefits of our discrepancy minimization algorithms for domain adaptation.
Local algorithms for the maximum flow and minimum cut in bounded-degree networks
Venue: arXiv cs.DS 2023
Year: 2023

We show a deterministic constant-time local algorithm for constructing an approximately maximum flow and minimum fractional cut in multisource-multitarget networks with bounded degrees and bounded edge capacities. Locality means that the decision we make about each edge only depends on its constant radius neighborhood. We show two applications of the algorithms: one is related to the Aldous-Lyons Conjecture, and the other is about approximating the neighborhood distribution of graphs by bounded-size graphs. The scope of our results can be extended to unimodular random graphs and networks. As a corollary, we generalize the Maximum Flow Minimum Cut Theorem to unimodular random flow networks.
Distributed Algorithms for Large-Scale Graphs
Venue: arXiv cs.DC 2023
Year: 2023

Motivated by the increasing need for fast processing of large-scale graphs, we study a number of fundamental graph problems in a message-passing model for distributed computing, called $k$-machine model, where we have $k$ machines that jointly perform computations on $n$-node graphs. The graph is assumed to be partitioned in a balanced fashion among the $k$ machines, a common implementation in many real-world systems. Communication is point-to-point via bandwidth-constrained links, and the goal is to minimize the round complexity, i.e., the number of communication rounds required to finish a computation. We present a generic methodology that allows to obtain efficient algorithms in the $k$-machine model using distributed algorithms for the classical CONGEST model of distributed computing. Using this methodology, we obtain algorithms for various fundamental graph problems such as connectivity, minimum spanning trees, shortest paths, maximal independent sets, and finding subgraphs, showing that many of these problems can be solved in $\tilde{O}(n/k)$ rounds; this shows that one can achieve speedup nearly linear in $k$. To complement our upper bounds, we present lower bounds on the round complexity that quantify the fundamental limitations of solving graph problems distributively. We first show a lower bound of $Ω(n/k)$ rounds for computing a spanning tree of the input graph. This result implies the same bound for other fundamental problems such as computing a minimum spanning tree, breadth-first tree, or shortest paths tree. We also show a $\tilde Ω(n/k^2)$ lower bound for connectivity, spanning tree verification and other related problems. The latter lower bounds follow from the development and application of novel results in a random-partition variant of the classical communication complexity model.
Toward Online Mobile Facility Location on General Metrics
Venue: arXiv cs.DS 2023
Year: 2023

We introduce an online variant of mobile facility location (MFL) (introduced by Demaine et al. [SODA' 07]). We call this new problem online mobile facility location (OMFL). In the OMFL problem, initially, we are given a set of $k$ mobile facilities with their starting locations. One by one, requests are added. After each request arrives, one can make some changes to the facility locations before the subsequent request arrives. Each request is always assigned to the nearest facility. The cost of this assignment is the distance from the request to the facility. The objective is to minimize the total cost, which consists of the relocation cost of facilities and the distance cost of requests to their nearest facilities. We provide a lower bound for the OMFL problem that even holds on uniform metrics. A natural approach to solve the OMFL problem for general metric spaces is to utilize hierarchically well-separated trees (HSTs) and directly solve the OMFL problem on HSTs. In this paper, we provide the first step in this direction by solving a generalized variant of the OMFL problem on uniform metrics that we call G-OMFL. We devise a simple deterministic online algorithm and provide a tight analysis for the algorithm. The second step remains an open question. Inspired by the $k$-server problem, we introduce a new variant of the OMFL problem that focuses solely on minimizing movement cost. We refer to this variant as M-OMFL. Additionally, we provide a lower bound for M-OMFL that is applicable even on uniform metrics.
A Survey on the Local Divisor Technique
Venue: arXiv cs.FL 2023
Year: 2023

Local divisors allow a powerful induction scheme on the size of a monoid. We survey this technique by giving several examples of this proof method. These applications include linear temporal logic, rational expressions with Kleene stars restricted to prefix codes with bounded synchronization delay, Church-Rosser congruential languages, and Simon's Factorization Forest Theorem. We also introduce the notion of localizable language class as a new abstract concept which unifies some of the proofs for the results above. The current arXiv-version includes some additional material about codes of bounded synchronization delay as well as some updates concerning related literature.
Low Cost Swarm Based Diligent Cargo Transit System
Venue: arXiv cs.RO 2023
Year: 2023

The goal of this paper is to present the design and development of a low cost cargo transit system which can be adapted in developing countries like India where there is abundant and cheap human labour which makes the process of automation in any industry a challenge to innovators. The need of the hour is an automation system that can diligently transfer cargo from one place to another and minimize human intervention in the cargo transit industry. Therefore, a solution is being proposed which could effectively bring down human labour and the resources needed to implement them. The reduction in human labour and resources is achieved by the use of low cost components and very limited modification of the surroundings and the existing vehicles themselves. The operation of the cargo transit system has been verified and the relevant results are presented. An economical and robust cargo transit system is designed and implemented.
Efficient Approaches for Enclosing the United Solution Set of the Interval Generalized Sylvester Matrix Equations
Venue: arXiv cs.NA 2023
Year: 2023

In this work, we investigate the interval generalized Sylvester matrix equation ${\bf{A}}X{\bf{B}}+{\bf{C}}X{\bf{D}}={\bf{F}}$ and develop some techniques for obtaining outer estimations for the so-called united solution set of this interval system. First, we propose a modified variant of the Krawczyk operator which causes reducing computational complexity to cubic, compared to Kronecker product form. We then propose an iterative technique for enclosing the solution set. These approaches are based on spectral decompositions of the midpoints of ${\bf{A}}$, ${\bf{B}}$, ${\bf{C}}$ and ${\bf{D}}$ and in both of them we suppose that the midpoints of ${\bf{A}}$ and ${\bf{C}}$ are simultaneously diagonalizable as well as for the midpoints of the matrices ${\bf{B}}$ and ${\bf{D}}$. Some numerical experiments are given to illustrate the performance of the proposed methods.
Communication-Efficient Learning of Deep Networks from Decentralized Data
Venue: arXiv cs.LG 2023
Year: 2023

Modern mobile devices have access to a wealth of data suitable for learning models, which in turn can greatly improve the user experience on the device. For example, language models can improve speech recognition and text entry, and image models can automatically select good photos. However, this rich data is often privacy sensitive, large in quantity, or both, which may preclude logging to the data center and training there using conventional approaches. We advocate an alternative that leaves the training data distributed on the mobile devices, and learns a shared model by aggregating locally-computed updates. We term this decentralized approach Federated Learning. We present a practical method for the federated learning of deep networks based on iterative model averaging, and conduct an extensive empirical evaluation, considering five different model architectures and four datasets. These experiments demonstrate the approach is robust to the unbalanced and non-IID data distributions that are a defining characteristic of this setting. Communication costs are the principal constraint, and we show a reduction in required communication rounds by 10-100x as compared to synchronized stochastic gradient descent.
The CMA Evolution Strategy: A Tutorial
Venue: arXiv cs.LG 2023
Year: 2023

This tutorial introduces the CMA Evolution Strategy (ES), where CMA stands for Covariance Matrix Adaptation. The CMA-ES is a stochastic, or randomized, method for real-parameter (continuous domain) optimization of non-linear, non-convex functions. We try to motivate and derive the algorithm from intuitive concepts and from requirements of non-linear, non-convex search in continuous domain.
Bandit algorithms to emulate human decision making using probabilistic distortions
Venue: arXiv cs.LG 2023
Year: 2023

Motivated by models of human decision making proposed to explain commonly observed deviations from conventional expected value preferences, we formulate two stochastic multi-armed bandit problems with distorted probabilities on the reward distributions: the classic $K$-armed bandit and the linearly parameterized bandit settings. We consider the aforementioned problems in the regret minimization as well as best arm identification framework for multi-armed bandits. For the regret minimization setting in $K$-armed as well as linear bandit problems, we propose algorithms that are inspired by Upper Confidence Bound (UCB) algorithms, incorporate reward distortions, and exhibit sublinear regret. For the $K$-armed bandit setting, we derive an upper bound on the expected regret for our proposed algorithm, and then we prove a matching lower bound to establish the order-optimality of our algorithm. For the linearly parameterized setting, our algorithm achieves a regret upper bound that is of the same order as that of regular linear bandit algorithm called Optimism in the Face of Uncertainty Linear (OFUL) bandit algorithm, and unlike OFUL, our algorithm handles distortions and an arm-dependent noise model. For the best arm identification problem in the $K$-armed bandit setting, we propose algorithms, derive guarantees on their performance, and also show that these algorithms are order optimal by proving matching fundamental limits on performance. For best arm identification in linear bandits, we propose an algorithm and establish sample complexity guarantees. Finally, we present simulation experiments which demonstrate the advantages resulting from using distortion-aware learning algorithms in a vehicular traffic routing application.
Maximizing the Conditional Expected Reward for Reaching the Goal
Venue: arXiv cs.LO 2023
Year: 2023

The paper addresses the problem of computing maximal conditional expected accumulated rewards until reaching a target state (briefly called maximal conditional expectations) in finite-state Markov decision processes where the condition is given as a reachability constraint. Conditional expectations of this type can, e.g., stand for the maximal expected termination time of probabilistic programs with non-determinism, under the condition that the program eventually terminates, or for the worst-case expected penalty to be paid, assuming that at least three deadlines are missed. The main results of the paper are (i) a polynomial-time algorithm to check the finiteness of maximal conditional expectations, (ii) PSPACE-completeness for the threshold problem in acyclic Markov decision processes where the task is to check whether the maximal conditional expectation exceeds a given threshold, (iii) a pseudo-polynomial-time algorithm for the threshold problem in the general (cyclic) case, and (iv) an exponential-time algorithm for computing the maximal conditional expectation and an optimal scheduler.
Bayesian definition of random sequences with respect to conditional probabilities
Venue: arXiv cs.IT 2023
Year: 2023

We study Martin-Löf random (ML-random) points on computable probability measures on sample and parameter spaces (Bayes models). We consider variants of conditional randomness defined by ML-randomness on Bayes models and those of conditional blind randomness. We show that variants of conditional blind randomness are ill-defined from the Bayes statistical point of view. We prove that if the sets of random sequences of uniformly computable parametric models are pairwise disjoint then there is a consistent estimator for the model. Finally, we present an algorithmic solution to a classical problem in Bayes statistics, i.e., the posterior distributions converge weakly to almost all parameters if and only if the posterior distributions converge weakly to all ML-random parameters.
Synchronizability of Communicating Finite State Machines is not Decidable
Venue: arXiv cs.DC 2023
Year: 2023

A system of communicating finite state machines is synchronizable if its send trace semantics, i.e.the set of sequences of sendings it can perform, is the same when its communications are FIFO asynchronous and when they are just rendez-vous synchronizations. This property was claimed to be decidable in several conference and journal papers for either mailboxes or peer-to-peer communications, thanks to a form of small model property. In this paper, we show that this small model property does not hold neither for mailbox communications, nor for peer-to-peer communications, therefore the decidability of synchronizability becomes an open question. We close this question for peer-to-peer communications, and we show that synchronizability is actually undecidable. We show that synchronizability is decidable if the topology of communications is an oriented ring. We also show that, in this case, synchronizability implies the absence of unspecified receptions and orphan messages, and the channel-recognizability of the reachability set.
Design of the Artificial: lessons from the biological roots of general intelligence
Venue: arXiv cs.AI 2023
Year: 2023

Our fascination with intelligent machines goes back to ancient times with the mythical automaton Talos, Aristotle's mode of mechanical thought (syllogism) and Heron of Alexandria's mechanical machines. However, the quest for Artificial General Intelligence (AGI) has been troubled with repeated failures. Recently, there has been a shift towards bio-inspired software and hardware, but their singular design focus makes them inefficient in achieving AGI. Which set of requirements have to be met in the design of AGI? What are the limits in the design of the artificial? A careful examination of computation in biological systems suggests that evolutionary tinkering of contextual processing of information enabled by a hierarchical architecture is key to building AGI.
Gabor Convolutional Networks
Venue: arXiv cs.CV 2023
Year: 2023

Steerable properties dominate the design of traditional filters, e.g., Gabor filters, and endow features the capability of dealing with spatial transformations. However, such excellent properties have not been well explored in the popular deep convolutional neural networks (DCNNs). In this paper, we propose a new deep model, termed Gabor Convolutional Networks (GCNs or Gabor CNNs), which incorporates Gabor filters into DCNNs to enhance the resistance of deep learned features to the orientation and scale changes. By only manipulating the basic element of DCNNs based on Gabor filters, i.e., the convolution operator, GCNs can be easily implemented and are compatible with any popular deep learning architecture. Experimental results demonstrate the super capability of our algorithm in recognizing objects, where the scale and rotation changes occur frequently. The proposed GCNs have much fewer learnable network parameters, and thus is easier to train with an end-to-end pipeline.
Two-Level Type Theory and Applications
Venue: arXiv cs.LO 2023
Year: 2023

We define and develop two-level type theory (2LTT), a version of Martin-Löf type theory which combines two different type theories. We refer to them as the inner and the outer type theory. In our case of interest, the inner theory is homotopy type theory (HoTT) which may include univalent universes and higher inductive types. The outer theory is a traditional form of type theory validating uniqueness of identity proofs (UIP). One point of view on it is as internalised meta-theory of the inner type theory. There are two motivations for 2LTT. Firstly, there are certain results about HoTT which are of meta-theoretic nature, such as the statement that semisimplicial types up to level $n$ can be constructed in HoTT for any externally fixed natural number $n$. Such results cannot be expressed in HoTT itself, but they can be formalised and proved in 2LTT, where $n$ will be a variable in the outer theory. This point of view is inspired by observations about conservativity of presheaf models. Secondly, 2LTT is a framework which is suitable for formulating additional axioms that one might want to add to HoTT. This idea is heavily inspired by Voevodsky's Homotopy Type System (HTS), which constitutes one specific instance of a 2LTT. HTS has an axiom ensuring that the type of natural numbers behaves like the external natural numbers, which allows the construction of a universe of semisimplicial types. In 2LTT, this axiom can be stated simply be asking the inner and outer natural numbers to be isomorphic. After defining 2LTT, we set up a collection of tools with the goal of making 2LTT a convenient language for future developments. As a first such application, we develop the theory of Reedy fibrant diagrams in the style of Shulman. Continuing this line of thought, we suggest a definition of (infinity,1)-category and give some examples.
Distributed interaction between computer virus and patch: A modeling study
Venue: arXiv cs.SI 2023
Year: 2023

The decentralized patch distribution mechanism holds significant promise as an alternative to its centralized counterpart. For the purpose of accurately evaluating the performance of the decentralized patch distribution mechanism and based on the exact SIPS model that accurately captures the average dynamics of the interaction between viruses and patches, a new virus-patch interacting model, which is known as the generic SIPS model, is proposed. This model subsumes the linear SIPS model. The dynamics of the generic SIPS model is studied comprehensively. In particular, a set of criteria for the final extinction or/and long-term survival of viruses or/and patches are presented. Some conditions for the linear SIPS model to accurately capture the average dynamics of the virus-patch interaction are empirically found. As a consequence, the linear SIPS model can be adopted as a standard model for assessing the performance of the distributed patch distribution mechanism, provided the proper conditions are satisfied.
Network Capacity Bound for Personalized PageRank in Multimodal Networks
Venue: arXiv cs.SI 2023
Year: 2023

In a former paper the concept of Bipartite PageRank was introduced and a theorem on the limit of authority flowing between nodes for personalized PageRank has been generalized. In this paper we want to extend those results to multimodal networks. In particular we deal with a hypergraph type that may be used for describing multimodal network where a hyperlink connects nodes from each of the modalities. We introduce a generalisation of PageRank for such graphs and define the respective random walk model that can be used for computations. We state and prove theorems on the limit of outflow of authority for cases where individual modalities have identical and distinct damping factors.