OpenRaiser/Intern-Atlas huggingface dataset:
Learning Model Checking and the Kernel Trick for Signal Temporal Logic on Stochastic Processes
Venue: arXiv cs.LO 2022
Year: 2022

We introduce a similarity function on formulae of signal temporal logic (STL). It comes in the form of a kernel function, well known in machine learning as a conceptually and computationally efficient tool. The corresponding kernel trick allows us to circumvent the complicated process of feature extraction, i.e. the (typically manual) effort to identify the decisive properties of formulae so that learning can be applied. We demonstrate this consequence and its advantages on the task of predicting (quantitative) satisfaction of STL formulae on stochastic processes: Using our kernel and the kernel trick, we learn (i) computationally efficiently (ii) a practically precise predictor of satisfaction, (iii) avoiding the difficult task of finding a way to explicitly turn formulae into vectors of numbers in a sensible way. We back the high precision we have achieved in the experiments by a theoretically sound PAC guarantee, ensuring our procedure efficiently delivers a close-to-optimal predictor.
Just Enough, Just in Time, Just for "Me": Fundamental Principles for Engineering IoT-native Software Systems
Venue: arXiv cs.SE 2022
Year: 2022

By seamlessly integrating everyday objects and by changing the way we interact with our surroundings, Internet of Things (IoT) is drastically improving the life quality of households and enhancing the productivity of businesses. Given the unique IoT characteristics, IoT applications have emerged distinctively from the mainstream application types. Inspired by the outlook of a programmable world, we further foresee an IoT-native trend in designing, developing, deploying, and maintaining software systems. However, although the challenges of IoT software projects are frequently discussed, addressing those challenges are still in the "crossing the chasm" period. By participating in a few various IoT projects, we gradually distilled three fundamental principles for engineering IoT-native software systems, such as just enough, just in time, and just for "me". These principles target the challenges that are associated with the most typical features of IoT environments, ranging from resource limits to technology heterogeneity of IoT devices. We expect this research to trigger dedicated efforts, techniques and theories for the topic IoT-native software engineering.
Attacks and Defenses for Free-Riders in Multi-Discriminator GAN
Venue: arXiv cs.CV 2022
Year: 2022

Generative Adversarial Networks (GANs) are increasingly adopted by the industry to synthesize realistic images. Due to data not being centrally available, Multi-Discriminator (MD)-GANs training framework employs multiple discriminators that have direct access to the real data. Distributedly training a joint GAN model entails the risk of free-riders, i.e., participants that aim to benefit from the common model while only pretending to participate in the training process. In this paper, we conduct the first characterization study of the impact of free-riders on MD-GAN. Based on two production prototypes of MD-GAN, we find that free-riders drastically reduce the ability of MD-GANs to produce images that are indistinguishable from real data, i.e., they increase the FID score -- the standard measure to assess the quality of generated images. To mitigate the model degradation, we propose a defense strategy against free-riders in MD-GAN, termed DFG. DFG distinguishes free-riders and benign participants through periodic probing and clustering of discriminators' responses based on a reference response of free-riders, which then allows the generator to exclude the detected free-riders from the training. Furthermore, we extend our defense, termed DFG+, to enable discriminators to filter out free-riders at the variant of MD-GAN that allows peer exchanges of discriminators networks. Extensive evaluation on various scenarios of free-riders, MD-GAN architecture, and three datasets show that our defenses effectively detect free-riders. With 1 to 5 free-riders, DFG and DFG+ averagely decreases FID by 5.22% to 11.53% for CIFAR10 and 5.79% to 13.22% for CIFAR100 in comparison to an attack without defense. In a shell, the proposed DFG(+) can effectively defend against free-riders without affecting benign clients at a negligible computation overhead.
Monadic Monadic Second Order Logic
Venue: arXiv cs.LO 2022
Year: 2022

One of the main reasons for the correspondence of regular languages and monadic second-order logic is that the class of regular languages is closed under images of surjective letter-to-letter homomorphisms. This closure property holds for structures such as finite words, finite trees, infinite words, infinite trees, elements of the free group, etc. Such structures can be modelled using monads. In this paper, we study which structures (understood via monads in the category of sets) are such that the class of regular languages (i.e. languages recognized by finite algebras) are closed under direct images of surjective letter-to-letter homomorphisms. We provide diverse sufficient conditions for a monad to satisfy this property. We also present numerous examples of monads, including positive examples that do not satisfy our sufficient conditions, and counterexamples where the closure property fails.
Patapasco: A Python Framework for Cross-Language Information Retrieval Experiments
Venue: arXiv cs.IR 2022
Year: 2022

While there are high-quality software frameworks for information retrieval experimentation, they do not explicitly support cross-language information retrieval (CLIR). To fill this gap, we have created Patapsco, a Python CLIR framework. This framework specifically addresses the complexity that comes with running experiments in multiple languages. Patapsco is designed to be extensible to many language pairs, to be scalable to large document collections, and to support reproducible experiments driven by a configuration file. We include Patapsco results on standard CLIR collections using multiple settings.
Heat Conduction Plate Layout Optimization using Physics-driven Convolutional Neural Networks
Venue: arXiv cs.LG 2022
Year: 2022

The layout optimization of the heat conduction is essential during design in engineering, especially for thermal sensible products. When the optimization algorithm iteratively evaluates different loading cases, the traditional numerical simulation methods used usually lead to a substantial computational cost. To effectively reduce the computational effort, data-driven approaches are used to train a surrogate model as a mapping between the prescribed external loads and various geometry. However, the existing model are trained by data-driven methods which requires intensive training samples that from numerical simulations and not really effectively solve the problem. Choosing the steady heat conduction problems as examples, this paper proposes a Physics-driven Convolutional Neural Networks (PD-CNN) method to infer the physical field solutions for random varied loading cases. After that, the Particle Swarm Optimization (PSO) algorithm is used to optimize the sizes and the positions of the hole masks in the prescribed design domain, and the average temperature value of the entire heat conduction field is minimized, and the goal of minimizing heat transfer is achieved. Compared with the existing data-driven approaches, the proposed PD-CNN optimization framework not only predict field solutions that are highly consistent with conventional simulation results, but also generate the solution space with without any pre-obtained training data.
Text and Code Embeddings by Contrastive Pre-Training
Venue: arXiv cs.CL 2022
Year: 2022

Text embeddings are useful features in many applications such as semantic search and computing text similarity. Previous work typically trains models customized for different use cases, varying in dataset choice, training objective and model architecture. In this work, we show that contrastive pre-training on unsupervised data at scale leads to high quality vector representations of text and code. The same unsupervised text embeddings that achieve new state-of-the-art results in linear-probe classification also display impressive semantic search capabilities and sometimes even perform competitively with fine-tuned models. On linear-probe classification accuracy averaging over 7 tasks, our best unsupervised model achieves a relative improvement of 4% and 1.8% over previous best unsupervised and supervised text embedding models respectively. The same text embeddings when evaluated on large-scale semantic search attains a relative improvement of 23.4%, 14.7%, and 10.6% over previous best unsupervised methods on MSMARCO, Natural Questions and TriviaQA benchmarks, respectively. Similarly to text embeddings, we train code embedding models on (text, code) pairs, obtaining a 20.8% relative improvement over prior best work on code search.
Diversity Enhancement via Magnitude
Venue: arXiv cs.NE 2022
Year: 2022

Promoting and maintaining diversity of candidate solutions is a key requirement of evolutionary algorithms in general and multi-objective evolutionary algorithms in particular. In this paper, we use the recently developed theory of magnitude to construct a gradient flow and similar notions that systematically manipulate finite subsets of Euclidean space to enhance their diversity, and apply the ideas in service of multi-objective evolutionary algorithms. We demonstrate diversity enhancement on benchmark problems using leading algorithms, and discuss extensions of the framework.
Towards Ubiquitous Sensing and Localization With Reconfigurable Intelligent Surfaces
Venue: arXiv cs.IT 2022
Year: 2022

In future cellular systems, wireless localization and sensing functions will be built-in for specific applications, e.g., navigation, transportation, and healthcare, and to support flexible and seamless connectivity. Driven by this trend, the need rises for fine-resolution sensing solutions and cm-level localization accuracy, while the accuracy of current wireless systems is limited by the quality of the propagation environment. Recently, with the development of new materials, reconfigurable intelligent surfaces (RISs) provide an opportunity to reshape and control the electromagnetic characteristics of the environment, which can be utilized to improve the performance of wireless sensing and localization. In this tutorial, we will first review the background and motivation to utilize wireless signals for sensing and localization. Next, we introduce how to incorporate RIS into applications of sensing and localization, including key challenges and enabling techniques, and then some case studies will be presented. Finally, future research directions will also be discussed.
A Classical Approach to Handcrafted Feature Extraction Techniques for Bangla Handwritten Digit Recognition
Venue: arXiv cs.CV 2022
Year: 2022

Bangla Handwritten Digit recognition is a significant step forward in the development of Bangla OCR. However, intricate shape, structural likeness and distinctive composition style of Bangla digits makes it relatively challenging to distinguish. Thus, in this paper, we benchmarked four rigorous classifiers to recognize Bangla Handwritten Digit: K-Nearest Neighbor (KNN), Support Vector Machine (SVM), Random Forest (RF), and Gradient-Boosted Decision Trees (GBDT) based on three handcrafted feature extraction techniques: Histogram of Oriented Gradients (HOG), Local Binary Pattern (LBP), and Gabor filter on four publicly available Bangla handwriting digits datasets: NumtaDB, CMARTdb, Ekush and BDRW. Here, handcrafted feature extraction methods are used to extract features from the dataset image, which are then utilized to train machine learning classifiers to identify Bangla handwritten digits. We further fine-tuned the hyperparameters of the classification algorithms in order to acquire the finest Bangla handwritten digits recognition performance from these algorithms, and among all the models we employed, the HOG features combined with SVM model (HOG+SVM) attained the best performance metrics across all datasets. The recognition accuracy of the HOG+SVM method on the NumtaDB, CMARTdb, Ekush and BDRW datasets reached 93.32%, 98.08%, 95.68% and 89.68%, respectively as well as we compared the model performance with recent state-of-art methods.
Flexible Design on Deterministic IP Networking for Mixed Traffic Transmission
Venue: arXiv cs.NI 2022
Year: 2022

Deterministic IP (DIP) networking is a promising technique that can provide delay-bounded transmission in large-scale networks. Nevertheless, DIP faces several challenges in the mixed traffic scenarios, including (i) the capability of ultra-low latency communications, (ii) the simultaneous satisfaction of diverse QoS requirements, and (iii) the network efficiency. The problems are more formidable in the dynamic surroundings without prior knowledge of traffic demands. To address the above-mentioned issues, this paper designs a flexible DIP (FDIP) network. In the proposed network, we classify the queues at the output port into multiple groups. Each group operates with different cycle lengths. FDIP can assign the time-sensitive flows with different groups, hence delivering diverse QoS requirements, simultaneously. The ultra-low latency communication can be achieved by specific groups with short cycle lengths. Moreover, the flexible scheduling with diverse cycle lengths improves resource utilization, hence increasing the throughput (i.e., the number of acceptable time-sensitive flows). We formulate a throughput maximization problem that jointly considers the admission control, transmission path selection, and cycle length assignment. A branch and bound (BnB)-based heuristic is developed. Simulation results show that the proposed FDIP significantly outperforms the standard DIP in terms of both the throughput and the latency guarantees.
Large-scale Deterministic Transmission among IEEE 802.1Qbv Time-Sensitive Networks
Venue: arXiv cs.NI 2022
Year: 2022

IEEE 802.1Qbv (TAS) is the most widely used technique in Time-Sensitive Networking (TSN) which aims to provide bounded transmission delays and ultra-low jitters in industrial local area networks. With the development of emerging technologies (e.g., cloud computing), many wide-range time-sensitive network services emerge, such as factory automation, connected vehicles, and smart grids. Nevertheless, TAS is a Layer 2 technique for local networks, and cannot provide large-scale deterministic transmission. To tackle this problem, this paper proposes a hierarchical network containing access networks and a core network. Access networks perform TAS to aggregate time-sensitive traffic. In the core network, we exploit DIP (a well-known deterministic networking mechanism for backbone networks) to achieve long-distance deterministic transmission. Due to the differences between TAS and DIP, we design cross-domain transmission mechanisms at the edge of access networks and the core network to achieve seamless deterministic transmission. We also formulate the end-to-end scheduling to maximize the amount of accepted time-sensitive traffic. Experimental simulations show that the proposed network can achieve end-to-end deterministic transmission even in high-loaded scenarios.
Toward a Minecraft Mod for Early Detection of Alzheimer's Disease in Young Adults
Venue: arXiv cs.HC 2022
Year: 2022

This paper proposes a Minecraft-based system for early detection of Alzheimer's disease in young adults. Early detection, where spatial navigation is a crucial key, is regarded as an important way to prevent the disease. The proposed system is compared with a recent existing and thoroughly studied system using a game called Sea Hero Quest (SHQ), by analyzing spatial navigational patterns of players. Our preliminary results show that spatial navigational patterns in both systems are highly correlated, indicating that the proposed system is likely as effective as the SHQ system for the detection task.
Multi-UAV Coverage Planning with Limited Endurance in Disaster Environment
Venue: arXiv cs.RO 2022
Year: 2022

For scenes such as floods and earthquakes, the disaster area is large, and rescue time is tight. Multi-UAV exploration is more efficient than a single UAV. Existing UAV exploration work is modeled as a Coverage Path Planning (CPP) task to achieve full coverage of the area in the presence of obstacles. However, the endurance capability of UAV is limited, and the rescue time is urgent. Thus, even using multiple UAVs cannot achieve complete disaster area coverage in time. Therefore, in this paper we propose a multi-Agent Endurance-limited CPP (MAEl-CPP) problem based on a priori heatmap of the disaster area, which requires the exploration of more valuable areas under limited energy. Furthermore, we propose a path planning algorithm for the MAEl-CPP problem, by ranking the possible disaster areas according to their importance through satellite or remote aerial images and completing path planning according to the importance level. Experimental results show that our proposed algorithm is at least twice as effective as the existing method in terms of search efficiency.
Explanatory Learning: Beyond Empiricism in Neural Networks
Venue: arXiv cs.LG 2022
Year: 2022

We introduce Explanatory Learning (EL), a framework to let machines use existing knowledge buried in symbolic sequences -- e.g. explanations written in hieroglyphic -- by autonomously learning to interpret them. In EL, the burden of interpreting symbols is not left to humans or rigid human-coded compilers, as done in Program Synthesis. Rather, EL calls for a learned interpreter, built upon a limited collection of symbolic sequences paired with observations of several phenomena. This interpreter can be used to make predictions on a novel phenomenon given its explanation, and even to find that explanation using only a handful of observations, like human scientists do. We formulate the EL problem as a simple binary classification task, so that common end-to-end approaches aligned with the dominant empiricist view of machine learning could, in principle, solve it. To these models, we oppose Critical Rationalist Networks (CRNs), which instead embrace a rationalist view on the acquisition of knowledge. CRNs express several desired properties by construction, they are truly explainable, can adjust their processing at test-time for harder inferences, and can offer strong confidence guarantees on their predictions. As a final contribution, we introduce Odeen, a basic EL environment that simulates a small flatland-style universe full of phenomena to explain. Using Odeen as a testbed, we show how CRNs outperform empiricist end-to-end approaches of similar size and architecture (Transformers) in discovering explanations for novel phenomena.
DocEnTr: An End-to-End Document Image Enhancement Transformer
Venue: arXiv cs.CV 2022
Year: 2022

Document images can be affected by many degradation scenarios, which cause recognition and processing difficulties. In this age of digitization, it is important to denoise them for proper usage. To address this challenge, we present a new encoder-decoder architecture based on vision transformers to enhance both machine-printed and handwritten document images, in an end-to-end fashion. The encoder operates directly on the pixel patches with their positional information without the use of any convolutional layers, while the decoder reconstructs a clean image from the encoded patches. Conducted experiments show a superiority of the proposed model compared to the state-of the-art methods on several DIBCO benchmarks. Code and models will be publicly available at: \url{https://github.com/dali92002/DocEnTR}.
Timely Status Update: Should ARQ be Used in Two-Hop Networks?
Venue: arXiv cs.NI 2022
Year: 2022

This paper investigates the information freshness of two-hop networks. Age of information (AoI) is used as the metric to characterize the information freshness, defined as the time elapsed since the latest received status update was generated. In error-prone wireless networks, prior studies indicated that Automatic Repeat-reQuest (ARQ) does not help improve the average AoI performance of single-hop networks, because sending a new packet always carries the most up-to-date information (i.e., discarding the old packet). We believe that this observation does not apply to two-hop networks. For example, when a packet transmission fails in the second hop, although a new packet has more recent information, it may require more time to be delivered (i.e., the communication has to restart from the first hop), thus leading to a high AoI. This paper analyzes the theoretical average AoI of two-hop networks with and without ARQ. Specifically, we model the two schemes using Markov chains, from which we derive the average AoI. Our theoretical and simulation results confirm that, unlike single-hop networks, ARQ should be used in two-hop networks to achieve lower average AoI. In particular, when ARQ is used, the successful decoding probability of the second hop has a greater impact on the average AoI than that of the first hop. Overall, our findings provide insight into the ARQ design for two-hop timely status update systems.
t-Deletion-s-Insertion-Burst Correcting Codes
Venue: arXiv cs.IT 2022
Year: 2022

Motivated by applications in DNA-based storage and communication systems, we study deletion and insertion errors simultaneously in a burst. In particular, we study a type of error named $t$-deletion-$s$-insertion-burst ($(t,s)$-burst for short) which is a generalization of the $(2,1)$-burst error proposed by Schoeny {\it et. al}. Such an error deletes $t$ consecutive symbols and inserts an arbitrary sequence of length $s$ at the same coordinate. We provide a sphere-packing upper bound on the size of binary codes that can correct a $(t,s)$-burst error, showing that the redundancy of such codes is at least $\log n+t-1$. For $t\geq 2s$, an explicit construction of binary $(t,s)$-burst correcting codes with redundancy $\log n+(t-s-1)\log\log n+O(1)$ is given. In particular, we construct a binary $(3,1)$-burst correcting code with redundancy at most $\log n+9$, which is optimal up to a constant.
Probability estimation and structured output prediction for learning preferences in last mile delivery
Venue: arXiv cs.AI 2022
Year: 2022

We study the problem of learning the preferences of drivers and planners in the context of last mile delivery. Given a data set containing historical decisions and delivery locations, the goal is to capture the implicit preferences of the decision-makers. We consider two ways to use the historical data: one is through a probability estimation method that learns transition probabilities between stops (or zones). This is a fast and accurate method, recently studied in a VRP setting. Furthermore, we explore the use of machine learning to infer how to best balance multiple objectives such as distance, probability and penalties. Specifically, we cast the learning problem as a structured output prediction problem, where training is done by repeatedly calling the TSP solver. Another important aspect we consider is that for last-mile delivery, every address is a potential client and hence the data is very sparse. Hence, we propose a two-stage approach that first learns preferences at the zone level in order to compute a zone routing; after which a penalty-based TSP computes the stop routing. Results show that the zone transition probability estimation performs well, and that the structured output prediction learning can improve the results further. We hence showcase a successful combination of both probability estimation and machine learning, all the while using standard TSP solvers, both during learning and to compute the final solution; this means the methodology is applicable to other, real-life, TSP variants, or proprietary solvers.