OpenRaiser/Intern-Atlas huggingface dataset:
Transmit Optimization for Multi-functional MIMO Systems Integrating Sensing, Communication, and Powering
Venue: arXiv cs.IT 2022
Year: 2022

This paper unifies integrated sensing and communication (ISAC) and simultaneous wireless information and power transfer (SWIPT), by investigating a new multi-functional multiple-input multiple-output (MIMO) system integrating wireless sensing, communication, and powering. In this system, one multi-antenna hybrid access point (H-AP) transmits wireless signals to communicate with one multi-antenna information decoding (ID) receiver, wirelessly charge one multi-antenna energy harvesting (EH) receiver, and perform radar sensing for a point target based on the echo signal at the same time. Under this setup, we aim to reveal the fundamental performance tradeoff limits of sensing, communication, and powering, in terms of the estimation Cram{é}r-Rao bound (CRB), achievable communication rate, and harvested energy level, respectively. Towards this end, we define the achievable CRB-rate-energy (C-R-E) region and characterize its Pareto boundary by maximizing the achievable rate at the ID receiver, subject to the estimation CRB requirement for target sensing, the harvested energy requirement at the EH receiver, and the maximum transmit power constraint at the H-AP. We obtain the semi-closed-form optimal transmit covariance solution to the formulated problem by applying advanced convex optimization techniques. Numerical results show the optimal C-R-E region boundary achieved by our proposed design, as compared to the benchmark scheme based on time switching.
Versatile Semantic Coded Transmission over MIMO Fading Channels
Venue: arXiv cs.IT 2022
Year: 2022

Semantic communications have shown great potential to boost the end-to-end transmission performance. To further improve the system efficiency, in this paper, we propose a class of novel semantic coded transmission (SCT) schemes over multiple-input multiple-output (MIMO) fading channels. In particular, we propose a high-efficiency SCT system supporting concurrent transmission of multiple streams, which can maximize the multiplexing gain of end-to-end semantic communication system. By jointly considering the entropy distribution on the source semantic features and the wireless MIMO channel states, we design a spatial multiplexing mechanism to realize adaptive coding rate allocation and stream mapping. As a result, source content and channel environment will be seamlessly coupled, which maximizes the coding gain of SCT system. Moreover, our SCT system is versatile: a single model can support various transmission rates. The whole model is optimized under the constraint of transmission rate-distortion (RD) tradeoff. Experimental results verify that our scheme substantially increases the throughput of semantic communication system. It also outperforms traditional MIMO communication systems under realistic fading channels.
gMeta: Template-based Regular Expression Generation over Noisy Examples
Venue: arXiv cs.SE 2022
Year: 2022

Regular expressions (regexes) are widely used in different fields of computer science, such as programming languages, string processing, and databases. However, existing tools for synthesizing or repairing regexes always assume that the input examples are faultless. In real industrial scenarios, this assumption does not entirely hold. Thus, this paper presents a simple but effective templated-based approach to generate regular expressions over noisy examples. Specifically, we present a data model (i.e., MetaParam) to extract features of strings for clustering all examples. Then, we propose a practical dynamic thresholding scheme to filter out anomalous examples via detecting knee points on CDF graphs. Finally, we design a template-based algorithm to translate a finite of positve examples to regular expression, which is efficient, interpretable, and extensible. We performed an experimental evaluation on four different extraction tasks applied to real-world datasets and obtained promising results in terms of F-measure. Moreover, gMeta achieves excellent results in real industrial scenarios.
Saliency Can Be All You Need In Contrastive Self-Supervised Learning
Venue: arXiv cs.CV 2022
Year: 2022

We propose an augmentation policy for Contrastive Self-Supervised Learning (SSL) in the form of an already established Salient Image Segmentation technique entitled Global Contrast based Salient Region Detection. This detection technique, which had been devised for unrelated Computer Vision tasks, was empirically observed to play the role of an augmentation facilitator within the SSL protocol. This observation is rooted in our practical attempts to learn, by SSL-fashion, aerial imagery of solar panels, which exhibit challenging boundary patterns. Upon the successful integration of this technique on our problem domain, we formulated a generalized procedure and conducted a comprehensive, systematic performance assessment with various Contrastive SSL algorithms subject to standard augmentation techniques. This evaluation, which was conducted across multiple datasets, indicated that the proposed technique indeed contributes to SSL. We hypothesize whether salient image segmentation may suffice as the only augmentation policy in Contrastive SSL when treating downstream segmentation tasks.
Symmetric Saliency-based Adversarial Attack To Speaker Identification
Venue: arXiv cs.SD 2022
Year: 2022

Adversarial attack approaches to speaker identification either need high computational cost or are not very effective, to our knowledge. To address this issue, in this paper, we propose a novel generation-network-based approach, called symmetric saliency-based encoder-decoder (SSED), to generate adversarial voice examples to speaker identification. It contains two novel components. First, it uses a novel saliency map decoder to learn the importance of speech samples to the decision of a targeted speaker identification system, so as to make the attacker focus on generating artificial noise to the important samples. It also proposes an angular loss function to push the speaker embedding far away from the source speaker. Our experimental results demonstrate that the proposed SSED yields the state-of-the-art performance, i.e. over 97% targeted attack success rate and a signal-to-noise level of over 39 dB on both the open-set and close-set speaker identification tasks, with a low computational cost.
Explainable Predictive Decision Mining for Operational Support
Venue: arXiv cs.AI 2022
Year: 2022

Several decision points exist in business processes (e.g., whether a purchase order needs a manager's approval or not), and different decisions are made for different process instances based on their characteristics (e.g., a purchase order higher than $500 needs a manager approval). Decision mining in process mining aims to describe/predict the routing of a process instance at a decision point of the process. By predicting the decision, one can take proactive actions to improve the process. For instance, when a bottleneck is developing in one of the possible decisions, one can predict the decision and bypass the bottleneck. However, despite its huge potential for such operational support, existing techniques for decision mining have focused largely on describing decisions but not on predicting them, deploying decision trees to produce logical expressions to explain the decision. In this work, we aim to enhance the predictive capability of decision mining to enable proactive operational support by deploying more advanced machine learning algorithms. Our proposed approach provides explanations of the predicted decisions using SHAP values to support the elicitation of proactive actions. We have implemented a Web application to support the proposed approach and evaluated the approach using the implementation.
Adaptive and Fair Deployment Approach to Balance Offload Traffic in Multi-UAV Cellular Networks
Venue: arXiv cs.NI 2022
Year: 2022

Unmanned aerial vehicle-aided communication (UAB-BS) is a promising solution to establish rapid wireless connectivity in sudden/temporary crowded events because of its more flexibility and mobility features than conventional ground base station (GBS). Because of these benefits, UAV-BSs can easily be deployed at high altitudes to provide more line of sight (LoS) links than GBS. Therefore, users on the ground can obtain more reliable wireless channels. In practice, the mobile nature of the ground user can create uneven user density at different times and spaces. This phenomenon leads to unbalanced user associations among UAV-BSs and may cause frequent UAV-BS overload. We propose a three-dimensional adaptive and fair deployment approach to solve this problem. The proposed approach can jointly optimize the altitude and transmission power of UAV-BS to offload the traffic from overloaded UAV-BSs. The simulation results show that the network performance improves by 37.71% in total capacity, 37.48% in total energy efficiency and 16.12% in the Jain fairness index compared to the straightforward greedy approach.
A Solvable Model of Neural Scaling Laws
Venue: arXiv cs.LG 2022
Year: 2022

Large language models with a huge number of parameters, when trained on near internet-sized number of tokens, have been empirically shown to obey neural scaling laws: specifically, their performance behaves predictably as a power law in either parameters or dataset size until bottlenecked by the other resource. To understand this better, we first identify the necessary properties allowing such scaling laws to arise and then propose a statistical model -- a joint generative data model and random feature model -- that captures this neural scaling phenomenology. By solving this model in the dual limit of large training set size and large number of parameters, we gain insight into (i) the statistical structure of datasets and tasks that lead to scaling laws, (ii) the way nonlinear feature maps, such as those provided by neural networks, enable scaling laws when trained on these datasets, (iii) the optimality of the equiparameterization scaling of training sets and parameters, and (iv) whether such scaling laws can break down and how they behave when they do. Key findings are the manner in which the power laws that occur in the statistics of natural datasets are extended by nonlinear random feature maps and then translated into power-law scalings of the test loss and how the finite extent of the data's spectral power law causes the model's performance to plateau.
Evaluation and comparison of federated learning algorithms for Human Activity Recognition on smartphones
Venue: arXiv cs.LG 2022
Year: 2022

Pervasive computing promotes the integration of smart devices in our living spaces to develop services providing assistance to people. Such smart devices are increasingly relying on cloud-based Machine Learning, which raises questions in terms of security (data privacy), reliance (latency), and communication costs. In this context, Federated Learning (FL) has been introduced as a new machine learning paradigm enhancing the use of local devices. At the server level, FL aggregates models learned locally on distributed clients to obtain a more general model. In this way, no private data is sent over the network, and the communication cost is reduced. Unfortunately, however, the most popular federated learning algorithms have been shown not to be adapted to some highly heterogeneous pervasive computing environments. In this paper, we propose a new FL algorithm, termed FedDist, which can modify models (here, deep neural network) during training by identifying dissimilarities between neurons among the clients. This permits to account for clients' specificity without impairing generalization. FedDist evaluated with three state-of-the-art federated learning algorithms on three large heterogeneous mobile Human Activity Recognition datasets. Results have shown the ability of FedDist to adapt to heterogeneous data and the capability of FL to deal with asynchronous situations.
Reward Shaping Using Convolutional Neural Network
Venue: arXiv cs.AI 2022
Year: 2022

In this paper, we propose Value Iteration Network for Reward Shaping (VIN-RS), a potential-based reward shaping mechanism using Convolutional Neural Network (CNN). The proposed VIN-RS embeds a CNN trained on computed labels using the message passing mechanism of the Hidden Markov Model. The CNN processes images or graphs of the environment to predict the shaping values. Recent work on reward shaping still has limitations towards training on a representation of the Markov Decision Process (MDP) and building an estimate of the transition matrix. The advantage of VIN-RS is to construct an effective potential function from an estimated MDP while automatically inferring the environment transition matrix. The proposed VIN-RS estimates the transition matrix through a self-learned convolution filter while extracting environment details from the input frames or sampled graphs. Due to (1) the previous success of using message passing for reward shaping; and (2) the CNN planning behavior, we use these messages to train the CNN of VIN-RS. Experiments are performed on tabular games, Atari 2600 and MuJoCo, for discrete and continuous action space. Our results illustrate promising improvements in the learning speed and maximum cumulative reward compared to the state-of-the-art.
Learning Heuristics for the Maximum Clique Enumeration Problem Using Low Dimensional Representations
Venue: arXiv cs.SI 2022
Year: 2022

Approximate solutions to various NP-hard combinatorial optimization problems have been found by learned heuristics using complex learning models. In particular, vertex (node) classification in graphs has been a helpful method towards finding the decision boundary to distinguish vertices in an optimal set from the rest. By following this approach, we use a learning framework for a pruning process of the input graph towards reducing the runtime of the maximum clique enumeration problem. We extensively study the role of using different vertex representations on the performance of this heuristic method, using graph embedding algorithms, such as Node2vec and DeepWalk, and representations using higher-order graph features comprising local subgraph counts. Our results show that Node2Vec and DeepWalk are promising embedding methods in representing nodes towards classification purposes. We observe that using local graph features in the classification process produce more accurate results when combined with a feature elimination process. Finally, we provide tests on random graphs to show the robustness and scalability of our method.
FusionFormer: Fusing Operations in Transformer for Efficient Streaming Speech Recognition
Venue: arXiv cs.SD 2022
Year: 2022

The recently proposed Conformer architecture which combines convolution with attention to capture both local and global dependencies has become the \textit{de facto} backbone model for Automatic Speech Recognition~(ASR). Inherited from the Natural Language Processing (NLP) tasks, the architecture takes Layer Normalization~(LN) as a default normalization technique. However, through a series of systematic studies, we find that LN might take 10\% of the inference time despite that it only contributes to 0.1\% of the FLOPs. This motivates us to replace LN with other normalization techniques, e.g., Batch Normalization~(BN), to speed up inference with the help of operator fusion methods and the avoidance of calculating the mean and variance statistics during inference. After examining several plain attempts which directly remove all LN layers or replace them with BN in the same place, we find that the divergence issue is mainly caused by the unstable layer output. We therefore propose to append a BN layer to each linear or convolution layer where stabilized training results are observed. We also propose to simplify the activations in Conformer, such as Swish and GLU, by replacing them with ReLU. All these exchanged modules can be fused into the weights of the adjacent linear/convolution layers and hence have zero inference cost. Therefore, we name it FusionFormer. Our experiments indicate that FusionFormer is as effective as the LN-based Conformer and is about 10\% faster.
Secure Outage Analysis of RIS-Assisted Communications with Discrete Phase Control
Venue: arXiv cs.IT 2022
Year: 2022

This correspondence investigates a reconfigurable intelligent surface (RIS)-assisted wireless communication system with security threats. The RIS is deployed to enhance the secrecy outage probability (SOP) of the data sent to a legitimate user. By deriving the distributions of the received signal-to-noise-ratios (SNRs) at the legitimate user and the eavesdropper, we formulate, in a closed-form expression, a tight bound for the SOP under the constraint of discrete phase control at the RIS. The SOP is characterized as a function of the number of antenna elements, $N$, and the number of discrete phase choices, $2^b$. It is revealed that the performance loss in terms of SOP due to the discrete phase control is ignorable for large $N$ when $b\!\geq\!3$. In addition, we explicitly quantify this SOP loss when binary phase shifts with $b\!=\!1$ is utilized. It is identified that increasing the RIS antenna elements by $1.6$ times can achieve the same SOP with binary phase shifts as that by the RIS with ideally continuous phase shifts. Numerical simulations are conducted to verify the accuracy of these theoretical observations.
EEMARQ: Efficient Lock-Free Range Queries with Memory Reclamation
Venue: arXiv cs.DB 2022
Year: 2022

Multi-Version Concurrency Control (MVCC) is a common mechanism for achieving linearizable range queries in database systems and concurrent data-structures. The core idea is to keep previous versions of nodes to serve range queries, while still providing atomic reads and updates. Existing concurrent data-structure implementations, that support linearizable range queries, are either slow, use locks, or rely on blocking reclamation schemes. We present EEMARQ, the first scheme that uses MVCC with lock-free memory reclamation to obtain a fully lock-free data-structure supporting linearizable inserts, deletes, contains, and range queries. Evaluation shows that EEMARQ outperforms existing solutions across most workloads, with lower space overhead and while providing full lock freedom.
VocabulARy replicated: comparing teenagers to young adults
Venue: arXiv cs.HC 2022
Year: 2022

A critical component of user studies is gaining access to a representative sample of the population researches intend to investigate. Nevertheless, the vast majority of human-computer interaction (HCI)studies, including augmented reality (AR) studies, rely on convenience sampling. The outcomes of these studies are often based on results obtained from university students aged between 19 and 26 years. In order to investigate how the results from one of our studies are affected by convenience sampling, we replicated the AR-supported language learning study called VocabulARy with 24 teenagers, aged between 14 and 19 years. The results verified most of the outcomes from the original study. In addition, it also revealed that teenagers found learning significantly less mentally demanding compared to young adults, and completed the study in a significantly shorter time. All this at no cost to learning outcomes.
Modelling M/M/R-JSQ-PS sojourn time distribution for Ultra-Reliable Low Latency Communication services
Venue: arXiv cs.NI 2022
Year: 2022

The future Internet promises to support time-sensitive services that require ultra low latencies and reliabilities of 99.99%. Recent advances in cellular and WiFi connections enhance the network to meet high reliability and ultra low latencies. However, the aforementioned services require that the server processing time ensures low latencies with high reliability, otherwise the end-to-end performance is not met. To that end, in this paper we use queuing theory to model the sojourn time distribution for Ultra-Reliable Low Latency Communication services of M/M/R-JSQ-PS systems: Markovian queues with R CPU servers following a join shortest queue processor-sharing discipline (for example Linux systems). We develop open-source simulation software, and develop and compare six analytical approximations for the sojourn time distribution. The proposed approximations yield Wasserstein distances below 2 time units, and upon medium loads incur into errors of less than 1.78 time units (e.g., milliseconds) for the 99.99th percentile sojourn time. Moreover, the proposed sojourn time approximations are stable regardless the number of CPUs and stay close to the simulations.
50 Ways to Bake a Cookie: Mapping the Landscape of Procedural Texts
Venue: arXiv cs.CL 2022
Year: 2022

The web is full of guidance on a wide variety of tasks, from changing the oil in your car to baking an apple pie. However, as content is created independently, a single task could have thousands of corresponding procedural texts. This makes it difficult for users to view the bigger picture and understand the multiple ways the task could be accomplished. In this work we propose an unsupervised learning approach for summarizing multiple procedural texts into an intuitive graph representation, allowing users to easily explore commonalities and differences. We demonstrate our approach on recipes, a prominent example of procedural texts. User studies show that our representation is intuitive and coherent and that it has the potential to help users with several sensemaking tasks, including adapting recipes for a novice cook and finding creative ways to spice up a dish.
When Language Model Meets Private Library
Venue: arXiv cs.PL 2022
Year: 2022

With the rapid development of pre-training techniques, a number of language models have been pre-trained on large-scale code corpora and perform well in code generation. In this paper, we investigate how to equip pre-trained language models with the ability of code generation for private libraries. In practice, it is common for programmers to write code using private libraries. However, this is a challenge for language models since they have never seen private APIs during training. Motivated by the fact that private libraries usually come with elaborate API documentation, we propose a novel framework with two modules: the APIRetriever finds useful APIs, and then the APICoder generates code using these APIs. For APIRetriever, we present a dense retrieval system and also design a friendly interaction to involve uses. For APICoder, we can directly use off-the-shelf language models, or continually pre-train the base model on a code corpus containing API information. Both modules are trained with data from public libraries and can be generalized to private ones. Furthermore, we craft three benchmarks for private libraries, named TorchDataEval, MonkeyEval, and BeatNumEval. Experimental results demonstrate the impressive performance of our framework.