OpenRaiser/Intern-Atlas huggingface dataset:
DeepEmotex: Classifying Emotion in Text Messages using Deep Transfer Learning
Venue: arXiv cs.IR 2022
Year: 2022

Transfer learning has been widely used in natural language processing through deep pretrained language models, such as Bidirectional Encoder Representations from Transformers and Universal Sentence Encoder. Despite the great success, language models get overfitted when applied to small datasets and are prone to forgetting when fine-tuned with a classifier. To remedy this problem of forgetting in transferring deep pretrained language models from one domain to another domain, existing efforts explore fine-tuning methods to forget less. We propose DeepEmotex an effective sequential transfer learning method to detect emotion in text. To avoid forgetting problem, the fine-tuning step is instrumented by a large amount of emotion-labeled data collected from Twitter. We conduct an experimental study using both curated Twitter data sets and benchmark data sets. DeepEmotex models achieve over 91% accuracy for multi-class emotion classification on test dataset. We evaluate the performance of the fine-tuned DeepEmotex models in classifying emotion in EmoInt and Stimulus benchmark datasets. The models correctly classify emotion in 73% of the instances in the benchmark datasets. The proposed DeepEmotex-BERT model outperforms Bi-LSTM result on the benchmark datasets by 23%. We also study the effect of the size of the fine-tuning dataset on the accuracy of our models. Our evaluation results show that fine-tuning with a large set of emotion-labeled data improves both the robustness and effectiveness of the resulting target task model.
Open-Ended Learning Strategies for Learning Complex Locomotion Skills
Venue: arXiv cs.RO 2022
Year: 2022

Teaching robots to learn diverse locomotion skills under complex three-dimensional environmental settings via Reinforcement Learning (RL) is still challenging. It has been shown that training agents in simple settings before moving them on to complex settings improves the training process, but so far only in the context of relatively simple locomotion skills. In this work, we adapt the Enhanced Paired Open-Ended Trailblazer (ePOET) approach to train more complex agents to walk efficiently on complex three-dimensional terrains. First, to generate more rugged and diverse three-dimensional training terrains with increasing complexity, we extend the Compositional Pattern Producing Networks - Neuroevolution of Augmenting Topologies (CPPN-NEAT) approach and include randomized shapes. Second, we combine ePOET with Soft Actor-Critic off-policy optimization, yielding ePOET-SAC, to ensure that the agent could learn more diverse skills to solve more challenging tasks. Our experimental results show that the newly generated three-dimensional terrains have sufficient diversity and complexity to guide learning, that ePOET successfully learns complex locomotion skills on these terrains, and that our proposed ePOET-SAC approach slightly improves upon ePOET.
Generating Exact Optimal Designs via Particle Swarm Optimization: Assessing Efficacy and Efficiency via Case Study
Venue: arXiv cs.NE 2022
Year: 2022

In this study we address existing deficiencies in the literature on applications of Particle Swarm Optimization to generate optimal designs. We present the results of a large computer study in which we bench-mark both efficiency and efficacy of PSO to generate high quality candidate designs for small-exact response surface scenarios commonly encountered by industrial practitioners. A preferred version of PSO is demonstrated and recommended. Further, in contrast to popular local optimizers such as the coordinate exchange, PSO is demonstrated to, even in a single run, generate highly efficient designs with large probability at small computing cost. Therefore, it appears beneficial for more practitioners to adopt and use PSO as tool for generating candidate experimental designs.
Monitoring Urban Forests from Auto-Generated Segmentation Maps
Venue: arXiv cs.CV 2022
Year: 2022

We present and evaluate a weakly-supervised methodology to quantify the spatio-temporal distribution of urban forests based on remotely sensed data with close-to-zero human interaction. Successfully training machine learning models for semantic segmentation typically depends on the availability of high-quality labels. We evaluate the benefit of high-resolution, three-dimensional point cloud data (LiDAR) as source of noisy labels in order to train models for the localization of trees in orthophotos. As proof of concept we sense Hurricane Sandy's impact on urban forests in Coney Island, New York City (NYC) and reference it to less impacted urban space in Brooklyn, NYC.
Two-terminal source coding with common sum reconstruction
Venue: arXiv cs.IT 2022
Year: 2022

We present the problem of two-terminal source coding with Common Sum Reconstruction (CSR). Consider two terminals, each with access to one of two correlated sources. Both terminals want to reconstruct the sum of the two sources under some average distortion constraint, and the reconstructions at two terminals must be identical with high probability. In this paper, we develop inner and outer bounds to the achievable rate distortion region of the CSR problem for a doubly symmetric binary source. We employ existing achievability results for Steinberg's common reconstruction and Wyner-Ziv's source coding with side information problems, and an achievability result for the lossy version of Korner-Marton's modulo-two sum computation problem.
Resource Allocation for Compression-aided Federated Learning with High Distortion Rate
Venue: arXiv cs.IT 2022
Year: 2022

Recently, a considerable amount of works have been made to tackle the communication burden in federated learning (FL) (e.g., model quantization, data sparsification, and model compression). However, the existing methods, that boost the communication efficiency in FL, result in a considerable trade-off between communication efficiency and global convergence rate. We formulate an optimization problem for compression-aided FL, which captures the relationship between the distortion rate, number of participating IoT devices, and convergence rate. Following that, the objective function is to minimize the total transmission time for FL convergence. Because the problem is non-convex, we propose to decompose it into sub-problems. Based on the property of a FL model, we first determine the number of IoT devices participating in the FL process. Then, the communication between IoT devices and the server is optimized by efficiently allocating wireless resources based on a coalition game. Our theoretical analysis shows that, by actively controlling the number of participating IoT devices, we can avoid the training divergence of compression-aided FL while maintaining the communication efficiency.
Constellation Design for Deep Joint Source-Channel Coding
Venue: arXiv cs.IT 2022
Year: 2022

Deep learning-based joint source-channel coding (JSCC) has shown excellent performance in image and feature transmission. However, the output values of the JSCC encoder are continuous, which makes the constellation of modulation complex and dense. It is hard and expensive to design radio frequency chains for transmitting such full-resolution constellation points. In this paper, two methods of mapping the full-resolution constellation to finite constellation are proposed for real system implementation. The constellation mapping results of the proposed methods correspond to regular constellation and irregular constellation, respectively. We apply the methods to existing deep JSCC models and evaluate them on AWGN channels with different signal-to-noise ratios (SNRs). Experimental results show that the proposed methods outperform the traditional uniform quadrature amplitude modulation (QAM) constellation mapping method by only adding a few additional parameters.
An Attack Resilient PUF-based Authentication Mechanism for Distributed Systems
Venue: arXiv cs.CR 2022
Year: 2022

In most PUF-based authentication schemes, a central server is usually engaged to verify the response of the device's PUF to challenge bit-streams. However, the server availability may be intermittent in practice. To tackle such an issue, this paper proposes a new protocol for supporting distributed authentication while avoiding vulnerability to information leakage where CRPs could be retrieved from hacked devices and collectively used to model the PUF. The main idea is to provision for scrambling the challenge bit-stream in a way that is dependent on the verifier. The scrambling pattern varies per authentication round for each device and independently across devices. In essence, the scrambling function becomes node- and packet-specific and the response received by two verifiers of one device for the same challenge bit-stream could vary. Thus, neither the scrambling function can be reverted, nor the PUF can be modeled even by a collusive set of malicious nodes. The validation results using data of an FPGA-based implementation demonstrate the effectiveness of our approach in thwarting PUF modeling attacks by collusive actors. We also discuss the approach resiliency against impersonation, Sybil, and reverse engineering attacks.
SBERT studies Meaning Representations: Decomposing Sentence Embeddings into Explainable Semantic Features
Venue: arXiv cs.CL 2022
Year: 2022

Models based on large-pretrained language models, such as S(entence)BERT, provide effective and efficient sentence embeddings that show high correlation to human similarity ratings, but lack interpretability. On the other hand, graph metrics for graph-based meaning representations (e.g., Abstract Meaning Representation, AMR) can make explicit the semantic aspects in which two sentences are similar. However, such metrics tend to be slow, rely on parsers, and do not reach state-of-the-art performance when rating sentence similarity. In this work, we aim at the best of both worlds, by learning to induce $S$emantically $S$tructured $S$entence BERT embeddings (S$^3$BERT). Our S$^3$BERT embeddings are composed of explainable sub-embeddings that emphasize various semantic sentence features (e.g., semantic roles, negation, or quantification). We show how to i) learn a decomposition of the sentence embeddings into semantic features, through approximation of a suite of interpretable AMR graph metrics, and how to ii) preserve the overall power of the neural embeddings by controlling the decomposition learning process with a second objective that enforces consistency with the similarity ratings of an SBERT teacher model. In our experimental studies, we show that our approach offers interpretability -- while fully preserving the effectiveness and efficiency of the neural sentence embeddings.
Optimization Heuristics for Cost-Efficient Long-Term Cloud Portfolio Allocations Under Uncertainty
Venue: arXiv cs.DC 2022
Year: 2022

Today's cloud infrastructure landscape offers a broad range of services to build and operate software applications. The myriad of options, however, has also brought along a new layer of complexity. When it comes to procuring cloud computing resources, consumers can purchase their virtual machines from different providers on different marketspaces to form so called cloud portfolios: a bundle of virtual machines whereby the virtual machines have different technical characteristics and pricing mechanisms. Thus, selecting the right server instances for a given set of applications such that the allocations are cost efficient is a non-trivial task. In this paper we propose a formal specification of the cloud portfolio management problem that takes an application-driven approach and incorporates the nuances of the commonly encountered reserved, on-demand and spot market types. We present two distinct cost optimization heuristics for this stochastic temporal bin packing problem, one taking a naive first fit strategy, while the other is built on the concepts of genetic algorithms. The results of the evaluation show that the former optimization approach significantly outperforms the latter, both in terms of execution speeds and solution quality.
Helm -- What It Can Do and Where Is It Going?
Venue: arXiv cs.DC 2022
Year: 2022

Deploying an application into a Kubernetes cluster requires sending a manifest file to the cluster's control plane interface. This action is typically performed through a kubectl client which is configured and authorized to communicate with the control plane's Uniform Resource Locator (URL). An application typically requires many Kubernetes resources such as pods, deployments, secrets, service and volumes. Configuring each of these through manifest files requires complex scripting, especially when there are numerous resources needed. A solution to the complex management tasks is Helm. Helm provides both a tool and underlying framework that packages the necessary manifest files. These packages are deployed through a single step install command which abstracts all the underlying control plane interaction from the user. Similar to application installs through Debian's package manager dpkg, packages are shared through local and remote repositories and allow the user to easily install, update, delete or handle concurrent versions.
Resource-Mediated Consensus Formation
Venue: arXiv cs.MA 2022
Year: 2022

In social sciences, simulating opinion dynamics to study the interplay between homophily and influence, and the subsequent formation of echo chambers, is of great importance. As such, in this paper we investigate echo chambers by implementing a unique social game in which we spawn in a large number of agents, each assigned one of the two opinions on an issue and a finite amount of influence in the form of a game currency. Agents attempt to have an opinion that is a majority at the end of the game, to obtain a reward also paid in the game currency. At the beginning of each round, a randomly selected agent is selected, referred to as a speaker. The second agent is selected in the radius of speaker influence (which is a set subset of the speaker's neighbors) to interact with the speaker as a listener. In this interaction, the speaker proposes a payoff in the game currency from their personal influence budget to persuade the listener to hold the speaker's opinion in future rounds until chosen listener again. The listener can either choose to accept or reject this payoff to hold the speaker's opinion for future rounds. The listener's choice is informed only by their estimate of global majority opinion through a limited view of the opinions of their neighboring agents. We show that the influence game leads to the formation of "echo chambers," or homogeneous clusters of opinions. We also investigate various scenarios to disrupt the creation of such echo chambers, including the introduction of resource disparity between agents with different opinions, initially preferentially assigning opinions to agents, and the introduction of committed agents, who never change their initial opinion.
Aeneas: Rust Verification by Functional Translation
Venue: arXiv cs.PL 2022
Year: 2022

We present Aeneas, a new verification toolchain for Rust programs based on a lightweight functional translation. We leverage Rust's rich region-based type system to eliminate memory reasoning for many Rust programs, as long as they do not rely on interior mutability or unsafe code. Doing so, we relieve the proof engineer of the burden of memory-based reasoning, allowing them to instead focus on functional properties of their code. Our first contribution is a new approach to borrows and controlled aliasing. We propose a pure, functional semantics for LLBC, a Low-Level Borrow Calculus that captures a large subset of Rust programs. Our semantics is value-based, meaning there is no notion of memory, addresses or pointer arithmetic. Our semantics is also ownership-centric, meaning that we enforce soundness of borrows via a semantic criterion based on loans rather than through a syntactic type-based lifetime discipline. We claim that our semantics captures the essence of the borrow mechanism rather than its current implementation in the Rust compiler. Our second contribution is a translation from LLBC to a pure lambda-calculus. This allows the user to reason about the original Rust program through the theorem prover of their choice. To deal with the well-known technical difficulty of terminating a borrow, we rely on a novel approach, in which we approximate the borrow graph in the presence of function calls. This in turn allows us to perform the translation using a new technical device called backward functions. We implement our toolchain in a mixture of Rust and OCaml. Our evaluation shows significant gains of verification productivity for the programmer. Rust goes to great lengths to enforce static control of aliasing; the proof engineer should not waste any time on memory reasoning when so much already comes "for free"!
Location-based Twitter Filtering for the Creation of Low-Resource Language Datasets in Indonesian Local Languages
Venue: arXiv cs.CL 2022
Year: 2022

Twitter contains an abundance of linguistic data from the real world. We examine Twitter for user-generated content in low-resource languages such as local Indonesian. For NLP to work in Indonesian, it must consider local dialects, geographic context, and regional culture influence Indonesian languages. This paper identifies the problems we faced when constructing a Local Indonesian NLP dataset. Furthermore, we are developing a framework for creating, collecting, and classifying Local Indonesian datasets for NLP. Using twitter's geolocation tool for automatic annotating.
TeKo: Text-Rich Graph Neural Networks with External Knowledge
Venue: arXiv cs.SI 2022
Year: 2022

Graph Neural Networks (GNNs) have gained great popularity in tackling various analytical tasks on graph-structured data (i.e., networks). Typical GNNs and their variants follow a message-passing manner that obtains network representations by the feature propagation process along network topology, which however ignore the rich textual semantics (e.g., local word-sequence) that exist in many real-world networks. Existing methods for text-rich networks integrate textual semantics by mainly utilizing internal information such as topics or phrases/words, which often suffer from an inability to comprehensively mine the text semantics, limiting the reciprocal guidance between network structure and text semantics. To address these problems, we propose a novel text-rich graph neural network with external knowledge (TeKo), in order to take full advantage of both structural and textual information within text-rich networks. Specifically, we first present a flexible heterogeneous semantic network that incorporates high-quality entities and interactions among documents and entities. We then introduce two types of external knowledge, that is, structured triplets and unstructured entity description, to gain a deeper insight into textual semantics. We further design a reciprocal convolutional mechanism for the constructed heterogeneous semantic network, enabling network structure and textual semantics to collaboratively enhance each other and learn high-level network representations. Extensive experimental results on four public text-rich networks as well as a large-scale e-commerce searching dataset illustrate the superior performance of TeKo over state-of-the-art baselines.
Two-Timescale Optimization for Intelligent Reflecting Surface-Assisted MIMO Transmission in Fast-Changing Channels
Venue: arXiv cs.IT 2022
Year: 2022

The application of intelligent reflecting surface (IRS) depends on the knowledge of channel state information (CSI), and has been hindered by the heavy overhead of channel training, estimation, and feedback in fast-changing channels. This paper presents a new two-timescale beamforming approach to maximizing the average achievable rate (AAR) of IRS-assisted MIMO systems, where the IRS is configured relatively infrequently based on statistical CSI (S-CSI) and the base station precoder and power allocation are updated frequently based on quickly outdated instantaneous CSI (I-CSI). The key idea is that we first reveal the optimal small-timescale power allocation based on outdated I-CSI yields a water-filling structure. Given the optimal power allocation, a new mini-batch sampling (mbs)- based particle swarm optimization (PSO) algorithm is developed to optimize the large-timescale IRS configuration with reduced channel samples. Another important aspect is that we develop a model-driven PSO algorithm to optimize the IRS configuration, which maximizes a lower bound of the AAR by only using the S-CSI and eliminates the need of channel samples. The modeldriven PSO serves as a dependable lower bound for the mbs-PSO. Simulations corroborate the superiority of the new two-timescale beamforming strategy to its alternatives in terms of the AAR and efficiency, with the benefits of the IRS demonstrated.
Nebula Graph: An open source distributed graph database
Venue: arXiv cs.DB 2022
Year: 2022

This paper introduces the recent work of Nebula Graph, an open-source, distributed, scalable, and native graph database. We present a system design trade-off and a comprehensive overview of Nebula Graph internals, including graph data models, partitioning strategies, secondary indexes, optimizer rules, storage-side transactions, graph query languages, observability, graph processing frameworks, and visualization tool-kits. In addition, three sets of large-scale graph b
Text-Aware End-to-end Mispronunciation Detection and Diagnosis
Venue: arXiv cs.SD 2022
Year: 2022

Mispronunciation detection and diagnosis (MDD) technology is a key component of computer-assisted pronunciation training system (CAPT). In the field of assessing the pronunciation quality of constrained speech, the given transcriptions can play the role of a teacher. Conventional methods have fully utilized the prior texts for the model construction or improving the system performance, e.g. forced-alignment and extended recognition networks. Recently, some end-to-end based methods attempt to incorporate the prior texts into model training and preliminarily show the effectiveness. However, previous studies mostly consider applying raw attention mechanism to fuse audio representations with text representations, without taking possible text-pronunciation mismatch into account. In this paper, we present a gating strategy that assigns more importance to the relevant audio features while suppressing irrelevant text information. Moreover, given the transcriptions, we design an extra contrastive loss to reduce the gap between the learning objective of phoneme recognition and MDD. We conducted experiments using two publicly available datasets (TIMIT and L2-Arctic) and our best model improved the F1 score from $57.51\%$ to $61.75\%$ compared to the baselines. Besides, we provide a detailed analysis to shed light on the effectiveness of gating mechanism and contrastive learning on MDD.