OpenRaiser/Intern-Atlas huggingface dataset:
Full-privacy secured search engine empowered by efficient genome-mapping algorithms
Venue: arXiv cs.IR 2022
Year: 2022

Since the 90s, keyword-based search engines have been helping people locate relevant web content via a simple query, so have the recent full-text-based search engines mainly used for plagiarism detection following an article upload. However, these "free" or paid services operate by storing users' search queries and preferences for personal profiling and targeted ads delivery, while user-uploaded articles can further profit the service providers as part of their expanding databases. In short, search engine privacy has not been an option for web exploration in the past decades. Here we demonstrate that a database or internet search, provided with the entire article as a query, can be correctly carried out without revealing users' sensitive queries by an irreversible encoding scheme and an efficient FM-index search routine that is generally used in the NGS of genomes. In our solution, Sapiens Aperio Veritas Engine (S.A.V.E.), every word in the query is encoded into one of 12 "amino acids" (a.a.) comprising a pseudo-biological sequence (PBS) at users' local machines. The PBS-mediated plagiarism detection is done by users' submission of locally encoded PBS through our cloud service to locate identical duplicates in the collected web contents which had been encoded in the same way as the query. It is found that PBSs with a length longer than 12 a.a., can return correct results with a false positive rate <0.8%. S.A.V.E. runs at a similar speed as Bowtie and is 4 orders faster than BLAST. S.A.V.E., functioning in both regular and in-private search modes, provides a new option for efficient internet search and plagiarism detection in a compressed search space without a chance of storing and revealing users' confidential contents. We expect that future privacy-aware search engines can reference the ideas proposed herein. S.A.V.E. is made available at https://dyn.life.nthu.edu.tw/SAVE/
A Mixed-Integer Programming Approach to Training Dense Neural Networks
Venue: arXiv cs.LG 2022
Year: 2022

Artificial Neural Networks (ANNs) are prevalent machine learning models that are applied across various real-world classification tasks. However, training ANNs is time-consuming and the resulting models take a lot of memory to deploy. In order to train more parsimonious ANNs, we propose a novel mixed-integer programming (MIP) formulation for training fully-connected ANNs. Our formulations can account for both binary and rectified linear unit (ReLU) activations, and for the use of a log-likelihood loss. We present numerical experiments comparing our MIP-based methods against existing approaches and show that we are able to achieve competitive out-of-sample performance with more parsimonious models.
Application of Machine Learning Methods in Inferring Surface Water Groundwater Exchanges using High Temporal Resolution Temperature Measurements
Venue: arXiv cs.LG 2022
Year: 2022

We examine the ability of machine learning (ML) and deep learning (DL) algorithms to infer surface/ground exchange flux based on subsurface temperature observations. The observations and fluxes are produced from a high-resolution numerical model representing conditions in the Columbia River near the Department of Energy Hanford site located in southeastern Washington State. Random measurement error, of varying magnitude, is added to the synthetic temperature observations. The results indicate that both ML and DL methods can be used to infer the surface/ground exchange flux. DL methods, especially convolutional neural networks, outperform the ML methods when used to interpret noisy temperature data with a smoothing filter applied. However, the ML methods also performed well and they are can better identify a reduced number of important observations, which could be useful for measurement network optimization. Surprisingly, the ML and DL methods better inferred upward flux than downward flux. This is in direct contrast to previous findings using numerical models to infer flux from temperature observations and it may suggest that combined use of ML or DL inference with numerical inference could improve flux estimation beneath river systems.
Robust Natural Language Processing: Recent Advances, Challenges, and Future Directions
Venue: arXiv cs.CL 2022
Year: 2022

Recent natural language processing (NLP) techniques have accomplished high performance on benchmark datasets, primarily due to the significant improvement in the performance of deep learning. The advances in the research community have led to great enhancements in state-of-the-art production systems for NLP tasks, such as virtual assistants, speech recognition, and sentiment analysis. However, such NLP systems still often fail when tested with adversarial attacks. The initial lack of robustness exposed troubling gaps in current models' language understanding capabilities, creating problems when NLP systems are deployed in real life. In this paper, we present a structured overview of NLP robustness research by summarizing the literature in a systemic way across various dimensions. We then take a deep-dive into the various dimensions of robustness, across techniques, metrics, embeddings, and benchmarks. Finally, we argue that robustness should be multi-dimensional, provide insights into current research, identify gaps in the literature to suggest directions worth pursuing to address these gaps.
DFA-NeRF: Personalized Talking Head Generation via Disentangled Face Attributes Neural Rendering
Venue: arXiv cs.CV 2022
Year: 2022

While recent advances in deep neural networks have made it possible to render high-quality images, generating photo-realistic and personalized talking head remains challenging. With given audio, the key to tackling this task is synchronizing lip movement and simultaneously generating personalized attributes like head movement and eye blink. In this work, we observe that the input audio is highly correlated to lip motion while less correlated to other personalized attributes (e.g., head movements). Inspired by this, we propose a novel framework based on neural radiance field to pursue high-fidelity and personalized talking head generation. Specifically, neural radiance field takes lip movements features and personalized attributes as two disentangled conditions, where lip movements are directly predicted from the audio inputs to achieve lip-synchronized generation. In the meanwhile, personalized attributes are sampled from a probabilistic model, where we design a Transformer-based variational autoencoder sampled from Gaussian Process to learn plausible and natural-looking head pose and eye blink. Experiments on several benchmarks demonstrate that our method achieves significantly better results than state-of-the-art methods.
The exact complexity of a Robinson tiling
Venue: arXiv cs.DM 2022
Year: 2022

We find the exact formula for the number of distinct $n \times n$ square patterns which appear in a Robinson tiling made of one infinite order supertile.
Improved bounds for the many-user MAC
Venue: arXiv cs.IT 2022
Year: 2022

Many-user MAC is an important model for understanding energy efficiency of massive random access in 5G and beyond. Introduced in Polyanskiy'2017 for the AWGN channel, subsequent works have provided improved bounds on the asymptotic minimum energy-per-bit required to achieve a target per-user error at a given user density and payload, going beyond the AWGN setting. The best known rigorous bounds use spatially coupled codes along with the optimal AMP algorithm. But these bounds are infeasible to compute beyond a few (around 10) bits of payload. In this paper, we provide new achievability bounds for the many-user AWGN and quasi-static Rayleigh fading MACs using the spatially coupled codebook design along with a scalar AMP algorithm. The obtained bounds are computable even up to 100 bits and outperform the previous ones at this payload.
Technical Report: Bundling Linked Data Structures for Linearizable Range Queries
Venue: arXiv cs.DS 2022
Year: 2022

We present bundled references, a new building block to provide linearizable range query operations for highly concurrent lock-based linked data structures. Bundled references allow range queries to traverse a path through the data structure that is consistent with the target atomic snapshot. We demonstrate our technique with three data structures: a linked list, skip list, and a binary search tree. Our evaluation reveals that in mixed workloads, our design can improve upon the state-of-the-art techniques by 1.2x-1.8x for a skip list and 1.3x-3.7x for a binary search tree. We also integrate our bundled data structure into the DBx1000 in-memory database, yielding up to 40% gain over the same competitors.
Balanced Nonadaptive Redundancy Scheduling
Venue: arXiv cs.IT 2022
Year: 2022

Distributed computing systems implement redundancy to reduce the job completion time and variability. Despite a large body of work about computing redundancy, the analytical performance evaluation of redundancy techniques in queuing systems is still an open problem. In this work, we take one step forward to analyze the performance of scheduling policies in systems with redundancy. In particular, we study the pattern of shared servers among replicas of different jobs. To this end, we employ combinatorics and graph theory and define and derive performance indicators using the statistics of the overlaps. We consider two classical nonadaptive scheduling policies: random and round-robin. We then propose a scheduling policy based on combinatorial block designs. Compared with conventional scheduling, the proposed scheduling improves the performance indicators. We study the expansion property of the graphs associated with round-robin and block design-based policies. It turns out the superior performance of the block design-based policy results from better expansion properties of its associated graph. As indicated by the performance indicators, the simulation results show that the block design-based policy outperforms random and round-robin scheduling in different scenarios. Specifically, it reduces the average waiting time in the queue to up to 25% compared to the random policy and up to 100% compared to the round-robin policy.
Learning Safe, Generalizable Perception-based Hybrid Control with Certificates
Venue: arXiv cs.RO 2022
Year: 2022

Many robotic tasks require high-dimensional sensors such as cameras and Lidar to navigate complex environments, but developing certifiably safe feedback controllers around these sensors remains a challenging open problem, particularly when learning is involved. Previous works have proved the safety of perception-feedback controllers by separating the perception and control subsystems and making strong assumptions on the abilities of the perception subsystem. In this work, we introduce a novel learning-enabled perception-feedback hybrid controller, where we use Control Barrier Functions (CBFs) and Control Lyapunov Functions (CLFs) to show the safety and liveness of a full-stack perception-feedback controller. We use neural networks to learn a CBF and CLF for the full-stack system directly in the observation space of the robot, without the need to assume a separate perception-based state estimator. Our hybrid controller, called LOCUS (Learning-enabled Observation-feedback Control Using Switching), can safely navigate unknown environments, consistently reach its goal, and generalizes safely to environments outside of the training dataset. We demonstrate LOCUS in experiments both in simulation and in hardware, where it successfully navigates a changing environment using feedback from a Lidar sensor.
Nipping in the Bud: Detection, Diffusion and Mitigation of Hate Speech on Social Media
Venue: arXiv cs.SI 2022
Year: 2022

Since the proliferation of social media usage, hate speech has become a major crisis. Hateful content can spread quickly and create an environment of distress and hostility. Further, what can be considered hateful is contextual and varies with time. While online hate speech reduces the ability of already marginalised groups to participate in discussion freely, offline hate speech leads to hate crimes and violence against individuals and communities. The multifaceted nature of hate speech and its real-world impact have already piqued the interest of the data mining and machine learning communities. Despite our best efforts, hate speech remains an evasive issue for researchers and practitioners alike. This article presents methodological challenges that hinder building automated hate mitigation systems. These challenges inspired our work in the broader area of combating hateful content on the web. We discuss a series of our proposed solutions to limit the spread of hate speech on social media.
Submix: Practical Private Prediction for Large-Scale Language Models
Venue: arXiv cs.LG 2022
Year: 2022

Recent data-extraction attacks have exposed that language models can memorize some training samples verbatim. This is a vulnerability that can compromise the privacy of the model's training data. In this work, we introduce SubMix: a practical protocol for private next-token prediction designed to prevent privacy violations by language models that were fine-tuned on a private corpus after pre-training on a public corpus. We show that SubMix limits the leakage of information that is unique to any individual user in the private corpus via a relaxation of group differentially private prediction. Importantly, SubMix admits a tight, data-dependent privacy accounting mechanism, which allows it to thwart existing data-extraction attacks while maintaining the utility of the language model. SubMix is the first protocol that maintains privacy even when publicly releasing tens of thousands of next-token predictions made by large transformer-based models such as GPT-2.
On the Persistence of Higher-Order Interactions in Real-World Hypergraphs
Venue: arXiv cs.SI 2022
Year: 2022

A hypergraph is a generalization of an ordinary graph, and it naturally represents group interactions as hyperedges (i.e., arbitrary-sized subsets of nodes). Such group interactions are ubiquitous in many domains: the sender and receivers of an email, the co-authors of a publication, and the items co-purchased by a customer, to name a few. A higher-order interaction (HOI) in a hypergraph is defined as the co-appearance of a set of nodes in any hyperedge. Our focus is the persistence of HOIs repeated over time, which is naturally interpreted as the strength of group relationships, aiming at answering three questions: (a) How do HOIs in real-world hypergraphs persist over time? (b) What are the key factors governing the persistence? (c) How accurately can we predict the persistence? In order to answer the questions above, we investigate the persistence of HOIs in 13 real-world hypergraphs from 6 domains. First, we define how to measure the persistence of HOIs. Then, we examine global patterns and anomalies in the persistence, revealing a power-law relationship. After that, we study the relations between the persistence and 16 structural features of HOIs, some of which are closely related to the persistence. Lastly, based on the 16 structural features, we assess the predictability of the persistence under various settings and find strong predictors of the persistence. Note that predicting the persistence of HOIs has many potential applications, such as recommending items to be purchased together and predicting missing recipients of emails.
Underwater Object Classification and Detection: first results and open challenges
Venue: arXiv cs.CV 2022
Year: 2022

This work reviews the problem of object detection in underwater environments. We analyse and quantify the shortcomings of conventional state-of-the-art (SOTA) algorithms in the computer vision community when applied to this challenging environment, as well as providing insights and general guidelines for future research efforts. First, we assessed if pretraining with the conventional ImageNet is beneficial when the object detector needs to be applied to environments that may be characterised by a different feature distribution. We then investigate whether two-stage detectors yields to better performance with respect to single-stage detectors, in terms of accuracy, intersection of union (IoU), floating operation per second (FLOPS), and inference time. Finally, we assessed the generalisation capability of each model to a lower quality dataset to simulate performance on a real scenario, in which harsher conditions ought to be expected. Our experimental results provide evidence that underwater object detection requires searching for "ad-hoc" architectures than merely training SOTA architectures on new data, and that pretraining is not beneficial.
Analysis and Optimization of Cache-Enabled mmWave HetNets with Integrated Access and Backhaul
Venue: arXiv cs.IT 2022
Year: 2022

In milimeter wave heterogenous networks with integrated access and backhaul (mABHetNets), a considerable part of spectrum resources are occupied by the backhaul link, which limits the performance of the access link. In order to overcome such backhaul "spectrum occupancy", we introduce cache in mABHetNets. Caching popular files at small base stations (SBSs) can offload the backhaul traffic and transfer spectrum from the backhaul link to the access link. To achieve the optimal performance of the cache-enabled mABHetNets, we first analyze the signal-to-interference-plus-noise ratio (SINR) distribution and derive the average potential throughput (APT) expression by stochastic geometric tools. Then, based on our analytical work, we formulate a joint optimization problem of cache decision and spectrum partition to maximize the APT. Inspired by the block coordinate descent (BCD) method, we propose a joint cache decision, spectrum partition and power allocation (JCSPA) algorithm to find the optimal solution. Simulation results show the convergence and enhancement of the proposed algorithm. Besides, we verify the APT under different parameters and find that the introduction of cache facilitates the transfer of backhaul spectrum to access link. Jointly deploying appropriate caching capacity at SBSs and performing specified spectrum partition can bring up about 90% APT gain in mABHetNets.
A new simple dynamic muscle fatigue model and its validation
Venue: arXiv cs.RO 2022
Year: 2022

Musculoskeletal disorder (MSD) is one of the major health problems in physical work especially in manual handling jobs. In several literatures, muscle fatigue is considered to be closely related to MSD, especially for muscle related disorders. In addition to many existing analysis techniques for muscle fatigue assessment and MSD risk analysis, in this paper, a new muscle fatigue model was proposed. The new proposed model reflects the influence of external load, workload history, and individual differences. This model is simple in mathematics and can be easily applied in realtime calculation, such as the application in realtime virtual work simulation and evaluation. The new model was mathematically validated with 24 existing static models by comparing the calculated METs, and qualitatively or quantitatively validated with 3 existing dynamic models. The proposed model shows high or moderate similarities in predicting the METs with all the 24 static models. Validation results with the three dynamic models were also promising. The main limitation of the model is that it still lacks experimental validation for more dynamic situations. Relevance to industry Muscle fatigue is one of the main reasons causing MSDs in industry, especially for physical work. Correct evaluation of muscle fatigue is necessary to determine work-rest regimens and reduce the risks of MSD.
Unified Field Multiplier for ECC: Inherent Resistance against Horizontal SCA Attacks
Venue: arXiv cs.CR 2022
Year: 2022

In this paper we introduce a unified field multiplier for the EC kP operation in two different types of Galois fields. The most important contributions of this paper are that the multiplier is based on the 4-segment Karatsuba multiplication method and that it is inherent resistant against selected horizontal attacks.
On the Influence of the FPGA Compiler Optimization Options on the Success of the Horizontal Attack
Venue: arXiv cs.CR 2022
Year: 2022

This paper reports about the impact of compiler options on the resistance of cryptographic implementations against side channel analysis attacks. We evaluated four compiler option for six different FPGAs from Intel and Xilinx. In order to ensure fair assessment we synthesized always the same VHDL code, kept the measurement setup and statistical analysis method etc. constant. Our analysis clearly shows that the compiler options have an impact on the success of attacks but also that the impact is unpredictable not only between different FPGAs but also for an individual FPGA.
Graph Neural Networks: a bibliometrics overview
Venue: arXiv cs.LG 2022
Year: 2022

Recently, graph neural networks have become a hot topic in machine learning community. This paper presents a Scopus based bibliometric overview of the GNNs research since 2004, when GNN papers were first published. The study aims to evaluate GNN research trend, both quantitatively and qualitatively. We provide the trend of research, distribution of subjects, active and influential authors and institutions, sources of publications, most cited documents, and hot topics. Our investigations reveal that the most frequent subject categories in this field are computer science, engineering, telecommunications, linguistics, operations research and management science, information science and library science, business and economics, automation and control systems, robotics, and social sciences. In addition, the most active source of GNN publications is Lecture Notes in Computer Science. The most prolific or impactful institutions are found in the United States, China, and Canada. We also provide must read papers and future directions. Finally, the application of graph convolutional networks and attention mechanism are now among hot topics of GNN research.
Rxn Hypergraph: a Hypergraph Attention Model for Chemical Reaction Representation
Venue: arXiv cs.LG 2022
Year: 2022

It is fundamental for science and technology to be able to predict chemical reactions and their properties. To achieve such skills, it is important to develop good representations of chemical reactions, or good deep learning architectures that can learn such representations automatically from the data. There is currently no universal and widely adopted method for robustly representing chemical reactions. Most existing methods suffer from one or more drawbacks, such as: (1) lacking universality; (2) lacking robustness; (3) lacking interpretability; or (4) requiring excessive manual pre-processing. Here we exploit graph-based representations of molecular structures to develop and test a hypergraph attention neural network approach to solve at once the reaction representation and property-prediction problems, alleviating the aforementioned drawbacks. We evaluate this hypergraph representation in three experiments using three independent data sets of chemical reactions. In all experiments, the hypergraph-based approach matches or outperforms other representations and their corresponding models of chemical reactions while yielding interpretable multi-level representations.