OpenRaiser/Intern-Atlas huggingface dataset:
Transcoding Billions of Unicode Characters per Second with SIMD Instructions
Venue: arXiv cs.DC 2022
Year: 2022

In software, text is often represented using Unicode formats (UTF-8 and UTF-16). We frequently have to convert text from one format to the other, a process called transcoding. Popular transcoding functions are slower than state-of-the-art disks and networks. These transcoding functions make little use of the single-instruction-multiple-data (SIMD) instructions available on commodity processors. By designing transcoding algorithms for SIMD instructions, we multiply the speed of transcoding on current systems (x64 and ARM). To ensure reproducibility, we make our software freely available as an open source library.
Geometric Fabrics: Generalizing Classical Mechanics to Capture the Physics of Behavior
Venue: arXiv cs.RO 2022
Year: 2022

Classical mechanical systems are central to controller design in energy shaping methods of geometric control. However, their expressivity is limited by position-only metrics and the intimate link between metric and geometry. Recent work on Riemannian Motion Policies (RMPs) has shown that shedding these restrictions results in powerful design tools, but at the expense of theoretical stability guarantees. In this work, we generalize classical mechanics to what we call geometric fabrics, whose expressivity and theory enable the design of systems that outperform RMPs in practice. Geometric fabrics strictly generalize classical mechanics forming a new physics of behavior by first generalizing them to Finsler geometries and then explicitly bending them to shape their behavior while maintaining stability. We develop the theory of fabrics and present both a collection of controlled experiments examining their theoretical properties and a set of robot system experiments showing improved performance over a well-engineered and hardened implementation of RMPs, our current state-of-the-art in controller design.
Robust Visual Teach and Repeat for UGVs Using 3D Semantic Maps
Venue: arXiv cs.RO 2022
Year: 2022

We propose a Visual Teach and Repeat (VTR) algorithm using semantic landmarks extracted from environmental objects for ground robots with fixed mount monocular cameras. The proposed algorithm is robust to changes in the starting pose of the camera/robot, where a pose is defined as the planar position plus the orientation around the vertical axis. VTR consists of a teach phase in which a robot moves in a prescribed path, and a repeat phase in which the robot tries to repeat the same path starting from the same or a different pose. Most available VTR algorithms are pose dependent and cannot perform well in the repeat phase when starting from an initial pose far from that of the teach phase. To achieve more robust pose independency, the key is to generate a 3D semantic map of the environment containing the camera trajectory and the positions of surrounding objects during the teach phase. For specific implementation, we use ORB-SLAM to collect the camera poses and the 3D point clouds of the environment, and YOLOv3 to detect objects in the environment. We then combine the two outputs to build the semantic map. In the repeat phase, we relocalize the robot based on the detected objects and the stored semantic map. The robot is then able to move toward the teach path, and repeat it in both forward and backward directions. We have tested the proposed algorithm in different scenarios and compared it with two most relevant recent studies. Also, we compared our algorithm with two image-based relocalization methods. One is purely based on ORB-SLAM and the other combines Superglue and RANSAC. The results show that our algorithm is much more robust with respect to pose variations as well as environmental alterations. Our code and data are available at the following Github page: https://github.com/mmahdavian/semantic_visual_teach_repeat.
Differentiable Scaffolding Tree for Molecular Optimization
Venue: arXiv cs.LG 2022
Year: 2022

The structural design of functional molecules, also called molecular optimization, is an essential chemical science and engineering task with important applications, such as drug discovery. Deep generative models and combinatorial optimization methods achieve initial success but still struggle with directly modeling discrete chemical structures and often heavily rely on brute-force enumeration. The challenge comes from the discrete and non-differentiable nature of molecule structures. To address this, we propose differentiable scaffolding tree (DST) that utilizes a learned knowledge network to convert discrete chemical structures to locally differentiable ones. DST enables a gradient-based optimization on a chemical graph structure by back-propagating the derivatives from the target properties through a graph neural network (GNN). Our empirical studies show the gradient-based molecular optimizations are both effective and sample efficient. Furthermore, the learned graph parameters can also provide an explanation that helps domain experts understand the model output.
KD-VLP: Improving End-to-End Vision-and-Language Pretraining with Object Knowledge Distillation
Venue: arXiv cs.CV 2022
Year: 2022

Self-supervised vision-and-language pretraining (VLP) aims to learn transferable multi-modal representations from large-scale image-text data and to achieve strong performances on a broad scope of vision-language tasks after finetuning. Previous mainstream VLP approaches typically adopt a two-step strategy relying on external object detectors to encode images in a multi-modal Transformer framework, which suffer from restrictive object concept space, limited image context and inefficient computation. In this paper, we propose an object-aware end-to-end VLP framework, which directly feeds image grid features from CNNs into the Transformer and learns the multi-modal representations jointly. More importantly, we propose to perform object knowledge distillation to facilitate learning cross-modal alignment at different semantic levels. To achieve that, we design two novel pretext tasks by taking object features and their semantic labels from external detectors as supervision: 1.) Object-guided masked vision modeling task focuses on enforcing object-aware representation learning in the multi-modal Transformer; 2.) Phrase-region alignment task aims to improve cross-modal alignment by utilizing the similarities between noun phrases and object labels in the linguistic space. Extensive experiments on a wide range of vision-language tasks demonstrate the efficacy of our proposed framework, and we achieve competitive or superior performances over the existing pretraining strategies.
Investigating and Modeling the Dynamics of Long Ties
Venue: arXiv cs.SI 2022
Year: 2022

Long ties, the social ties that bridge different communities, are widely believed to play crucial roles in spreading novel information in social networks. However, some existing network theories and prediction models indicate that long ties might dissolve quickly or eventually become redundant, thus putting into question the long-term value of long ties. Our empirical analysis of real-world dynamic networks shows that contrary to such reasoning, long ties are more likely to persist than other social ties, and that many of them constantly function as social bridges without being embedded in local networks. Using a novel cost-benefit analysis model combined with machine learning, we show that long ties are highly beneficial, which instinctively motivates people to expend extra effort to maintain them. This partly explains why long ties are more persistent than what has been suggested by many existing theories and models. Overall, our study suggests the need for social interventions that can promote the formation of long ties, such as mixing people with diverse backgrounds.
Beamforming Design for IRS-Aided Decode-and-Forward Relay Wireless Network
Venue: arXiv cs.IT 2022
Year: 2022

As a low-cost and low-power-consumption passive reflector, intelligent reflecting surface (IRS) can make a significant rate improvement by building a programmable wireless environment. To improve the rate performance and coverage range of wireless networks, an IRS-aided decode-and-forward (DF) relay network is proposed with multiple antennas at relay station (RS). To achieve a high rate, an alternately iterative structure (AIS) of maximizing receive power (Max-RP) at RS is proposed to jointly optimize the beamforming vectors at RS and phase shifts at IRS. Considering its high-complexity, two low-complexity Max-RP schemes of null-space projection (NSP) plus maximum ratio combining (MRC) and IRS element selection (IRSES) plus MRC are presented to reduce this complexity, respectively. For the former, NSP is used to separate the reflected signal from IRS and the direct transmitted signal from source and MRC is adopted to combine the two signals at RS. For the latter, the basic concept of IRSES is as follows: IRS is partitioned into M subsets of elements and adjusting the phases of all elements per subset make all reflected signals and the direct signal from source phase alignment (PA) at the corresponding antenna of relay. Simulation results show that the proposed three methods perform much better than the existing network with single-antenna relay in terms of rate performance. In particular, a 85% rate gain over existing scheme is achieved in the high signal-to-noise ratio region. Moreover, it is verified that the positions of RS and IRS have a substantial impact on rate performance, and there exists an optimal positions of RS and IRS.
On Crossing-Families in Planar Point Sets
Venue: arXiv cs.CG 2022
Year: 2022

A $k$-crossing family in a point set $S$ in general position is a set of $k$ segments spanned by points of $S$ such that all $k$ segments mutually cross. In this short note we present two statements on crossing families which are based on sets of small cardinality: (1) Any set of at least 15 points contains a crossing family of size 4. (2) There are sets of $n$ points which do not contain a crossing family of size larger than $8\lceil \frac{n}{41} \rceil$. Both results improve the previously best known bounds.
Label Cleaning Multiple Instance Learning: Refining Coarse Annotations on Single Whole-Slide Images
Venue: arXiv cs.CV 2022
Year: 2022

Annotating cancerous regions in whole-slide images (WSIs) of pathology samples plays a critical role in clinical diagnosis, biomedical research, and machine learning algorithms development. However, generating exhaustive and accurate annotations is labor-intensive, challenging, and costly. Drawing only coarse and approximate annotations is a much easier task, less costly, and it alleviates pathologists' workload. In this paper, we study the problem of refining these approximate annotations in digital pathology to obtain more accurate ones. Some previous works have explored obtaining machine learning models from these inaccurate annotations, but few of them tackle the refinement problem where the mislabeled regions should be explicitly identified and corrected, and all of them require a -- often very large -- number of training samples. We present a method, named Label Cleaning Multiple Instance Learning (LC-MIL), to refine coarse annotations on a single WSI without the need of external training data. Patches cropped from a WSI with inaccurate labels are processed jointly within a multiple instance learning framework, mitigating their impact on the predictive model and refining the segmentation. Our experiments on a heterogeneous WSI set with breast cancer lymph node metastasis, liver cancer, and colorectal cancer samples show that LC-MIL significantly refines the coarse annotations, outperforming state-of-the-art alternatives, even while learning from a single slide. Moreover, we demonstrate how real annotations drawn by pathologists can be efficiently refined and improved by the proposed approach. All these results demonstrate that LC-MIL is a promising, light-weight tool to provide fine-grained annotations from coarsely annotated pathology sets.
Population-scale dietary interests during the COVID-19 pandemic
Venue: arXiv cs.SI 2022
Year: 2022

The SARS-CoV-2 virus has altered people's lives around the world. Here we document population-wide shifts in dietary interests in 18 countries in 2020, as revealed through time series of Google search volumes. We find that during the first wave of the COVID-19 pandemic there was an overall surge in food interest, larger and longer-lasting than the surge during typical end-of-year holidays in Western countries. The shock of decreased mobility manifested as a drastic increase in interest in consuming food at home and a corresponding decrease in consuming food outside of home. The largest (up to threefold) increases occurred for calorie-dense carbohydrate-based foods such as pastries, bakery products, bread, and pies. The observed shifts in dietary interests have the potential to globally affect food consumption and health outcomes. These findings can inform governmental and organizational decisions regarding measures to mitigate the effects of the COVID-19 pandemic on diet and nutrition.
In-network Computation for Large-scale Federated Learning over Wireless Edge Networks
Venue: arXiv cs.IT 2022
Year: 2022

Most conventional Federated Learning (FL) models are using a star network topology where all users aggregate their local models at a single server (e.g., a cloud server). That causes significant overhead in terms of both communications and computing at the server, delaying the training process, especially for large scale FL systems with straggling nodes. This paper proposes a novel edge network architecture that enables decentralizing the model aggregation process at the server, thereby significantly reducing the training delay for the whole FL network. Specifically, we design a highly-effective in-network computation protocol (INC) consisting of a user scheduling mechanism, an in-network aggregation process (INA) which is designed for both primal- and primal-dual methods in distributed machine learning problems, and a network routing algorithm. Under the proposed INA, we then formulate a joint routing and resource optimization problem, aiming to minimize the aggregation latency. The problem is NP-hard, and thus we propose a polynomial time routing algorithm which can achieve near optimal performance with a theoretical bound. Simulation results showed that the proposed INC framework can not only help reduce the FL training latency, up to 5.6 times, but also significantly decrease cloud's traffic and computing overhead. This can enable large-scale FL.
Dependency Structure for News Document Summarization
Venue: arXiv cs.CL 2022
Year: 2022

In this work, we develop a neural network based model which leverages dependency parsing to capture cross-positional dependencies and grammatical structures. With the help of linguistic signals, sentence-level relations can be correctly captured, thus improving news documents summarization performance. Empirical studies demonstrate that this simple but effective method outperforms existing works on the benchmark dataset. Extensive analyses examine different settings and configurations of the proposed model which provide a good reference to the community.
Federated Feature Selection for Cyber-Physical Systems of Systems
Venue: arXiv cs.LG 2022
Year: 2022

Autonomous vehicles (AVs) generate a massive amount of multi-modal data that once collected and processed through Machine Learning algorithms, enable AI-based services at the Edge. In fact, not all these data contain valuable, and informative content but only a subset of the relative attributes should be exploited at the Edge. Therefore, enabling AVs to locally extract such a subset is of utmost importance to limit computation and communication workloads. Achieving a consistent subset of data in a distributed manner imposes the AVs to cooperate in finding an agreement on what attributes should be sent to the Edge. In this work, we address such a problem by proposing a federated feature selection algorithm where all the AVs collaborate to filter out, iteratively, the redundant or irrelevant attributes in a distributed manner, without any exchange of raw data. This solution builds on two components: a Mutual-Information-based feature selection algorithm run by the AVs and a novel aggregation function based on the Bayes theorem executed on the Edge. Our federated feature selection algorithm provably converges to a solution in a finite number of steps. Such an algorithm has been tested on two reference datasets: MAV with images and inertial measurements of a monitored vehicle, WESAD with a collection of samples from biophysical sensors to monitor a relative passenger. The numerical results show that the fleet finds a consensus with both the datasets on the minimum achievable subset of features, i.e., 24 out of 2166 (99\%) in MAV and 4 out of 8 (50\%) in WESAD, preserving the informative content of data.
Cross Attention-guided Dense Network for Images Fusion
Venue: arXiv cs.CV 2022
Year: 2022

In recent years, various applications in computer vision have achieved substantial progress based on deep learning, which has been widely used for image fusion and shown to achieve adequate performance. However, suffering from limited ability in modeling the spatial correspondence of different source images, it still remains a great challenge for existing unsupervised image fusion models to extract appropriate feature and achieves adaptive and balanced fusion. In this paper, we propose a novel cross-attention-guided image fusion network, which is a unified and unsupervised framework for multi-modal image fusion, multi-exposure image fusion, and multi-focus image fusion. Different from the existing self-attention module, our cross-attention module focus on modeling the cross-correlation between different source images. Using the proposed cross attention module as a core block, a densely connected cross attention-guided network is built to dynamically learn the spatial correspondence to derive better alignment of important details from different input images. Meanwhile, an auxiliary branch is also designed to model the long-range information, and a merging network is attached to finally reconstruct the fusion image. Extensive experiments have been carried out on publicly available datasets, and the results demonstrate that the proposed model outperforms the state-of-the-art quantitatively and qualitatively.
Learning to Robustly Aggregate Labeling Functions for Semi-supervised Data Programming
Venue: arXiv cs.LG 2022
Year: 2022

A critical bottleneck in supervised machine learning is the need for large amounts of labeled data which is expensive and time consuming to obtain. However, it has been shown that a small amount of labeled data, while insufficient to re-train a model, can be effectively used to generate human-interpretable labeling functions (LFs). These LFs, in turn, have been used to generate a large amount of additional noisy labeled data, in a paradigm that is now commonly referred to as data programming. However, previous approaches to automatically generate LFs make no attempt to further use the given labeled data for model training, thus giving up opportunities for improved performance. Moreover, since the LFs are generated from a relatively small labeled dataset, they are prone to being noisy, and naively aggregating these LFs can lead to very poor performance in practice. In this work, we propose an LF based reweighting framework \ouralgo{} to solve these two critical limitations. Our algorithm learns a joint model on the (same) labeled dataset used for LF induction along with any unlabeled data in a semi-supervised manner, and more critically, reweighs each LF according to its goodness, influencing its contribution to the semi-supervised loss using a robust bi-level optimization algorithm. We show that our algorithm significantly outperforms prior approaches on several text classification datasets.
How much human-like visual experience do current self-supervised learning algorithms need in order to achieve human-level object recognition?
Venue: arXiv cs.CV 2022
Year: 2022

This paper addresses a fundamental question: how good are our current self-supervised visual representation learning algorithms relative to humans? More concretely, how much "human-like" natural visual experience would these algorithms need in order to reach human-level performance in a complex, realistic visual object recognition task such as ImageNet? Using a scaling experiment, here we estimate that the answer is several orders of magnitude longer than a human lifetime: typically on the order of a million to a billion years of natural visual experience (depending on the algorithm used). We obtain even larger estimates for achieving human-level performance in ImageNet-derived robustness benchmarks. The exact values of these estimates are sensitive to some underlying assumptions, however even in the most optimistic scenarios they remain orders of magnitude larger than a human lifetime. We discuss the main caveats surrounding our estimates and the implications of these surprising results.
Recurrent Neural Networks for Partially Observed Dynamical Systems
Venue: arXiv cs.LG 2022
Year: 2022

Complex nonlinear dynamics are ubiquitous in many fields. Moreover, we rarely have access to all of the relevant state variables governing the dynamics. Delay embedding allows us, in principle, to account for unobserved state variables. Here we provide an algebraic approach to delay embedding that permits explicit approximation of error. We also provide the asymptotic dependence of the first order approximation error on the system size. More importantly, this formulation of delay embedding can be directly implemented using a Recurrent Neural Network (RNN). This observation expands the interpretability of both delay embedding and RNN and facilitates principled incorporation of structure and other constraints into these approaches.
On the Robustness of "Robust reversible data hiding scheme based on two-layer embedding strategy"
Venue: arXiv cs.MM 2022
Year: 2022

In the paper "Robust reversible data hiding scheme based on two-layer embedding strategy" published in INS recently, Kumar et al. proposed a robust reversible data hiding (RRDH) scheme based on two-layer embedding. Secret data was embedded into the most significant bit (MSB) planes to increase robustness, and a sorting strategy based on local complexity was adopted to reduce distortion. However, Kumar et al.'s reversible data hiding (RDH) scheme is not as robust against joint photographic experts group (JPEG) compression as stated and can not be called RRDH. This comment first gives a brief description of their RDH scheme, then analyses their scheme's robustness from the perspective of JPEG compression principles. JPEG compression will change pixel values, thereby destroying auxiliary information and pixel value ordering required to extract secret data correctly, making their scheme not robust. Next, the changes in both bit plane and pixel value ordering after JPEG compression are shown and analysed by different robustness-testing experiments. Finally, some suggestions are given to improve the robustness.