OpenRaiser/Intern-Atlas huggingface dataset:
Widely Interpretable Semantic Representation: Frameless Meaning Representation for Broader Applicability
Venue: arXiv cs.CL 2023
Year: 2023

This paper presents a novel semantic representation, WISeR, that overcomes challenges for Abstract Meaning Representation (AMR). Despite its strengths, AMR is not easily applied to languages or domains without predefined semantic frames, and its use of numbered arguments results in semantic role labels, which are not directly interpretable and are semantically overloaded for parsers. We examine the numbered arguments of predicates in AMR and convert them to thematic roles that do not require reference to semantic frames. We create a new corpus of 1K English dialogue sentences annotated in both WISeR and AMR. WISeR shows stronger inter-annotator agreement for beginner and experienced annotators, with beginners becoming proficient in WISeR annotation more quickly. Finally, we train a state-of-the-art parser on the AMR 3.0 corpus and a WISeR corpus converted from AMR 3.0. The parser is evaluated on these corpora and our dialogue corpus. The WISeR model exhibits higher accuracy than its AMR counterpart across the board, demonstrating that WISeR is easier for parsers to learn.
Level Up: Private Non-Interactive Decision Tree Evaluation using Levelled Homomorphic Encryption
Venue: arXiv cs.CR 2023
Year: 2023

As machine learning as a service continues gaining popularity, concerns about privacy and intellectual property arise. Users often hesitate to disclose their private information to obtain a service, while service providers aim to protect their proprietary models. Decision trees, a widely used machine learning model, are favoured for their simplicity, interpretability, and ease of training. In this context, Private Decision Tree Evaluation (PDTE) enables a server holding a private decision tree to provide predictions based on a client's private attributes. The protocol is such that the server learns nothing about the client's private attributes. Similarly, the client learns nothing about the server's model besides the prediction and some hyperparameters. In this paper, we propose two novel non-interactive PDTE protocols, XXCMP-PDTE and RCC-PDTE, based on two new non-interactive comparison protocols, XXCMP and RCC. Our evaluation of these comparison operators demonstrates that our proposed constructions can efficiently evaluate high-precision numbers. Specifically, RCC can compare 32-bit numbers in under 10 milliseconds. We assess our proposed PDTE protocols on decision trees trained over UCI datasets and compare our results with existing work in the field. Moreover, we evaluate synthetic decision trees to showcase scalability, revealing that RCC-PDTE can evaluate a decision tree with over 1000 nodes and 16 bits of precision in under 2 seconds. In contrast, the current state-of-the-art requires over 10 seconds to evaluate such a tree with only 11 bits of precision.
Leveraging Large Language Models and Weak Supervision for Social Media data annotation: an evaluation using COVID-19 self-reported vaccination tweets
Venue: arXiv cs.CL 2023
Year: 2023

The COVID-19 pandemic has presented significant challenges to the healthcare industry and society as a whole. With the rapid development of COVID-19 vaccines, social media platforms have become a popular medium for discussions on vaccine-related topics. Identifying vaccine-related tweets and analyzing them can provide valuable insights for public health research-ers and policymakers. However, manual annotation of a large number of tweets is time-consuming and expensive. In this study, we evaluate the usage of Large Language Models, in this case GPT-4 (March 23 version), and weak supervision, to identify COVID-19 vaccine-related tweets, with the purpose of comparing performance against human annotators. We leveraged a manu-ally curated gold-standard dataset and used GPT-4 to provide labels without any additional fine-tuning or instructing, in a single-shot mode (no additional prompting).
Exploring the Benefits of Differentially Private Pre-training and Parameter-Efficient Fine-tuning for Table Transformers
Venue: arXiv cs.LG 2023
Year: 2023

For machine learning with tabular data, Table Transformer (TabTransformer) is a state-of-the-art neural network model, while Differential Privacy (DP) is an essential component to ensure data privacy. In this paper, we explore the benefits of combining these two aspects together in the scenario of transfer learning -- differentially private pre-training and fine-tuning of TabTransformers with a variety of parameter-efficient fine-tuning (PEFT) methods, including Adapter, LoRA, and Prompt Tuning. Our extensive experiments on the ACSIncome dataset show that these PEFT methods outperform traditional approaches in terms of the accuracy of the downstream task and the number of trainable parameters, thus achieving an improved trade-off among parameter efficiency, privacy, and accuracy. Our code is available at github.com/IBM/DP-TabTransformer.
Strong-Weak Integrated Semi-supervision for Unsupervised Single and Multi Target Domain Adaptation
Venue: arXiv cs.CV 2023
Year: 2023

Unsupervised domain adaptation (UDA) focuses on transferring knowledge learned in the labeled source domain to the unlabeled target domain. Despite significant progress that has been achieved in single-target domain adaptation for image classification in recent years, the extension from single-target to multi-target domain adaptation is still a largely unexplored problem area. In general, unsupervised domain adaptation faces a major challenge when attempting to learn reliable information from a single unlabeled target domain. Increasing the number of unlabeled target domains further exacerbate the problem rather significantly. In this paper, we propose a novel strong-weak integrated semi-supervision (SWISS) learning strategy for image classification using unsupervised domain adaptation that works well for both single-target and multi-target scenarios. Under the proposed SWISS-UDA framework, a strong representative set with high confidence but low diversity target domain samples and a weak representative set with low confidence but high diversity target domain samples are updated constantly during the training process. Both sets are fused to generate an augmented strong-weak training batch with pseudo-labels to train the network during every iteration. The extension from single-target to multi-target domain adaptation is accomplished by exploring the class-wise distance relationship between domains and replacing the strong representative set with much stronger samples from peer domains via peer scaffolding. Moreover, a novel adversarial logit loss is proposed to reduce the intra-class divergence between source and target domains, which is back-propagated adversarially with a gradient reverse layer between the classifier and the rest of the network. Experimental results based on three benchmarks, Office-31, Office-Home, and DomainNet, show the effectiveness of the proposed SWISS framework.
An overview of VANET vehicular networks
Venue: arXiv cs.NI 2023
Year: 2023

Today, with the development of intercity and metropolitan roadways and with various cars moving in various directions, there is a greater need than ever for a network to coordinate commutes. Nowadays, people spend a lot of time in their vehicles. Smart automobiles have developed to make that time safer, more effective, more fun, pollution-free, and affordable. However, maintaining the optimum use of resources and addressing rising needs continues to be a challenge given the popularity of vehicle users and the growing diversity of requests for various services. As a result, VANET will require modernized working practices in the future. Modern intelligent transportation management and driver assistance systems are created using cutting-edge communication technology. Vehicular Ad-hoc networks promise to increase transportation effectiveness, accident prevention, and pedestrian comfort by allowing automobiles and road infrastructure to communicate entertainment and traffic information. By constructing thorough frameworks, workflow patterns, and update procedures, including block-chain, artificial intelligence, and SDN (Software Defined Networking), this paper addresses VANET-related technologies, future advances, and related challenges. An overview of the VANET upgrade solution is given in this document in order to handle potential future problems.
Reasoning with Latent Diffusion in Offline Reinforcement Learning
Venue: arXiv cs.LG 2023
Year: 2023

Offline reinforcement learning (RL) holds promise as a means to learn high-reward policies from a static dataset, without the need for further environment interactions. However, a key challenge in offline RL lies in effectively stitching portions of suboptimal trajectories from the static dataset while avoiding extrapolation errors arising due to a lack of support in the dataset. Existing approaches use conservative methods that are tricky to tune and struggle with multi-modal data (as we show) or rely on noisy Monte Carlo return-to-go samples for reward conditioning. In this work, we propose a novel approach that leverages the expressiveness of latent diffusion to model in-support trajectory sequences as compressed latent skills. This facilitates learning a Q-function while avoiding extrapolation error via batch-constraining. The latent space is also expressive and gracefully copes with multi-modal data. We show that the learned temporally-abstract latent space encodes richer task-specific information for offline RL tasks as compared to raw state-actions. This improves credit assignment and facilitates faster reward propagation during Q-learning. Our method demonstrates state-of-the-art performance on the D4RL benchmarks, particularly excelling in long-horizon, sparse-reward tasks.
Narrative as a Dynamical System
Venue: arXiv cs.CL 2023
Year: 2023

There is increasing evidence that human activity in general, and narrative in particular, can be treated as a dynamical system in the physics sense; a system whose evolution is described by an action integral, such that the average of all possible paths from point A to point B is given by the extremum of the action. We create by construction three such paths by averaging about 500 different narratives, and we show that the average path is consistent with an action principle.
Pump, Dump, and then What? The Long-Term Impact of Cryptocurrency Pump-and-Dump Schemes
Venue: arXiv cs.CR 2023
Year: 2023

The pump and dump scheme is a form of market manipulation attack in which coordinated actors drive up the price of an asset in order to sell at a higher price. Due in part to a lack of enforcement, these schemes are widespread within the cryptocurrency marketplace, but the negative impact of these events on the coins they target is not yet fully understood. Drawing upon a novel dataset of pump events extracted from Telegram channels, an order of magnitude larger than the nearest comparable dataset in the literature, we explore the differing tactics of pumping channels and the long-term impact of pump and dump schemes across 765 coins. We find that, despite a short-term positive impact in some cases, the long-term impact of pump and dump schemes on the targeted assets is negative, amounting to an average 30% relative drop in price a year after the pump event.
A Reinforcement Learning Approach for Robotic Unloading from Visual Observations
Venue: arXiv cs.RO 2023
Year: 2023

In this work, we focus on a robotic unloading problem from visual observations, where robots are required to autonomously unload stacks of parcels using RGB-D images as their primary input source. While supervised and imitation learning have accomplished good results in these types of tasks, they heavily rely on labeled data, which are challenging to obtain in realistic scenarios. Our study aims to develop a sample efficient controller framework that can learn unloading tasks without the need for labeled data during the learning process. To tackle this challenge, we propose a hierarchical controller structure that combines a high-level decision-making module with classical motion control. The high-level module is trained using Deep Reinforcement Learning (DRL), wherein we incorporate a safety bias mechanism and design a reward function tailored to this task. Our experiments demonstrate that both these elements play a crucial role in achieving improved learning performance. Furthermore, to ensure reproducibility and establish a benchmark for future research, we provide free access to our code and simulation.
Epistemic Modeling Uncertainty of Rapid Neural Network Ensembles for Adaptive Learning
Venue: arXiv cs.LG 2023
Year: 2023

Emulator embedded neural networks, which are a type of physics informed neural network, leverage multi-fidelity data sources for efficient design exploration of aerospace engineering systems. Multiple realizations of the neural network models are trained with different random initializations. The ensemble of model realizations is used to assess epistemic modeling uncertainty caused due to lack of training samples. This uncertainty estimation is crucial information for successful goal-oriented adaptive learning in an aerospace system design exploration. However, the costs of training the ensemble models often become prohibitive and pose a computational challenge, especially when the models are not trained in parallel during adaptive learning. In this work, a new type of emulator embedded neural network is presented using the rapid neural network paradigm. Unlike the conventional neural network training that optimizes the weights and biases of all the network layers by using gradient-based backpropagation, rapid neural network training adjusts only the last layer connection weights by applying a linear regression technique. It is found that the proposed emulator embedded neural network trains near-instantaneously, typically without loss of prediction accuracy. The proposed method is demonstrated on multiple analytical examples, as well as an aerospace flight parameter study of a generic hypersonic vehicle.
Semi-supervised Classification of Malware Families Under Extreme Class Imbalance via Hierarchical Non-Negative Matrix Factorization with Automatic Model Selection
Venue: arXiv cs.CR 2023
Year: 2023

Identification of the family to which a malware specimen belongs is essential in understanding the behavior of the malware and developing mitigation strategies. Solutions proposed by prior work, however, are often not practicable due to the lack of realistic evaluation factors. These factors include learning under class imbalance, the ability to identify new malware, and the cost of production-quality labeled data. In practice, deployed models face prominent, rare, and new malware families. At the same time, obtaining a large quantity of up-to-date labeled malware for training a model can be expensive. In this paper, we address these problems and propose a novel hierarchical semi-supervised algorithm, which we call the HNMFk Classifier, that can be used in the early stages of the malware family labeling process. Our method is based on non-negative matrix factorization with automatic model selection, that is, with an estimation of the number of clusters. With HNMFk Classifier, we exploit the hierarchical structure of the malware data together with a semi-supervised setup, which enables us to classify malware families under conditions of extreme class imbalance. Our solution can perform abstaining predictions, or rejection option, which yields promising results in the identification of novel malware families and helps with maintaining the performance of the model when a low quantity of labeled data is used. We perform bulk classification of nearly 2,900 both rare and prominent malware families, through static analysis, using nearly 388,000 samples from the EMBER-2018 corpus. In our experiments, we surpass both supervised and semi-supervised baseline models with an F1 score of 0.80.
Fault-Tolerant Spanners against Bounded-Degree Edge Failures: Linearly More Faults, Almost For Free
Venue: arXiv cs.DS 2023
Year: 2023

We study a new and stronger notion of fault-tolerant graph structures whose size bounds depend on the degree of the failing edge set, rather than the total number of faults. For a subset of faulty edges $F \subseteq G$, the faulty-degree $°(F)$ is the largest number of faults in $F$ incident to any given vertex. We design new fault-tolerant structures with size comparable to previous constructions, but which tolerate every fault set of small faulty-degree $°(F)$, rather than only fault sets of small size $|F|$. Our main results are: - New FT-Certificates: For every $n$-vertex graph $G$ and degree threshold $f$, one can compute a connectivity certificate $H \subseteq G$ with $|E(H)| = \widetilde{O}(fn)$ edges that has the following guarantee: for any edge set $F$ with faulty-degree $°(F)\leq f$ and every vertex pair $u,v$, it holds that $u$ and $v$ are connected in $H \setminus F$ iff they are connected in $G \setminus F$. This bound on $|E(H)|$ is nearly tight. Since our certificates handle some fault sets of size up to $|F|=O(fn)$, prior work did not imply any nontrivial upper bound for this problem, even when $f=1$. - New FT-Spanners: We show that every $n$-vertex graph $G$ admits a $(2k-1)$-spanner $H$ with $|E(H)| = O_k(f^{1-1/k} n^{1+1/k})$ edges, which tolerates any fault set $F$ of faulty-degree at most $f$. This bound on $|E(H)|$ optimal up to its hidden dependence on $k$, and it is close to the bound of $O_k(|F|^{1/2} n^{1+1/k} + |F|n)$ that is known for the case where the total number of faults is $|F|$ [Bodwin, Dinitz, Robelle SODA '22]. Our proof of this theorem is non-constructive, but by following a proof strategy of Dinitz and Robelle [PODC '20], we show that the runtime can be made polynomial by paying an additional $\text{polylog } n$ factor in spanner size.
Distributed Learning Dynamics for Coalitional Games
Venue: arXiv cs.GT 2023
Year: 2023

In the framework of transferable utility coalitional games, a scoring (characteristic) function determines the value of any subset/coalition of agents. Agents decide on both which coalitions to form and the allocations of the values of the formed coalitions among their members. An important concept in coalitional games is that of a core solution, which is a partitioning of agents into coalitions and an associated allocation to each agent under which no group of agents can get a higher allocation by forming an alternative coalition. We present distributed learning dynamics for coalitional games that converge to a core solution whenever one exists. In these dynamics, an agent maintains a state consisting of (i) an aspiration level for its allocation and (ii) the coalition, if any, to which it belongs. In each stage, a randomly activated agent proposes to form a new coalition and changes its aspiration based on the success or failure of its proposal. The coalition membership structure is changed, accordingly, whenever the proposal succeeds. Required communications are that: (i) agents in the proposed new coalition need to reveal their current aspirations to the proposing agent, and (ii) agents are informed if they are joining the proposed coalition or if their existing coalition is broken. The proposing agent computes the feasibility of forming the coalition. We show that the dynamics hit an absorbing state whenever a core solution is reached. We further illustrate the distributed learning dynamics on a multi-agent task allocation setting.
Predicting Fatigue Crack Growth via Path Slicing and Re-Weighting
Venue: arXiv cs.LG 2023
Year: 2023

Predicting potential risks associated with the fatigue of key structural components is crucial in engineering design. However, fatigue often involves entangled complexities of material microstructures and service conditions, making diagnosis and prognosis of fatigue damage challenging. We report a statistical learning framework to predict the growth of fatigue cracks and the life-to-failure of the components under loading conditions with uncertainties. Digital libraries of fatigue crack patterns and the remaining life are constructed by high-fidelity physical simulations. Dimensionality reduction and neural network architectures are then used to learn the history dependence and nonlinearity of fatigue crack growth. Path-slicing and re-weighting techniques are introduced to handle the statistical noises and rare events. The predicted fatigue crack patterns are self-updated and self-corrected by the evolving crack patterns. The end-to-end approach is validated by representative examples with fatigue cracks in plates, which showcase the digital-twin scenario in real-time structural health monitoring and fatigue life prediction for maintenance management decision-making.
On correlation distribution of Niho-type decimation $d=3(p^m-1)+1$
Venue: arXiv cs.IT 2023
Year: 2023

The cross-correlation problem is a classic problem in sequence design. In this paper we compute the cross-correlation distribution of the Niho-type decimation $d=3(p^m-1)+1$ over $\mathrm{GF}(p^{2m})$ for any prime $p \ge 5$. Previously this problem was solved by Xia et al. only for $p=2$ and $p=3$ in a series of papers. The main difficulty of this problem for $p \ge 5$, as pointed out by Xia et al., is to count the number of codewords of "pure weight" 5 in $p$-ary Zetterberg codes. It turns out this counting problem can be transformed by the MacWilliams identity into counting codewords of weight at most 5 in $p$-ary Melas codes, the most difficult of which is related to a K3 surface well studied in the literature and can be computed. When $p \ge 7$, the theory of elliptic curves over finite fields also plays an important role in the resolution of this problem.
Enhancing Keyphrase Generation by BART Finetuning with Splitting and Shuffling
Venue: arXiv cs.CL 2023
Year: 2023

Keyphrase generation is a task of identifying a set of phrases that best repre-sent the main topics or themes of a given text. Keyphrases are dividend int pre-sent and absent keyphrases. Recent approaches utilizing sequence-to-sequence models show effectiveness on absent keyphrase generation. However, the per-formance is still limited due to the hardness of finding absent keyphrases. In this paper, we propose Keyphrase-Focused BART, which exploits the differ-ences between present and absent keyphrase generations, and performs fine-tuning of two separate BART models for present and absent keyphrases. We further show effective approaches of shuffling keyphrases and candidate keyphrase ranking. For absent keyphrases, our Keyphrase-Focused BART achieved new state-of-the-art score on F1@5 in two out of five keyphrase gen-eration benchmark datasets.