OpenRaiser/Intern-Atlas huggingface dataset:
An Approach to Remove Key Escrow Problem in ID-Based Encryption From Pairing
Venue: arXiv cs.CR 2023
Year: 2023

Key escrow refers to storing a copy of a cryptographic key with a trusted third party, typically a government agency or some other organization. Key escrow aims to ensure that law enforcement agencies can access encrypted data when necessary, for example, in criminal investigations or national security matters. However, key escrow also raises concerns about privacy and security. If the trusted third party is compromised, the stored keys could be exposed, and unauthorized parties could access sensitive information. This could result in a significant breach of privacy and potentially harm national security. In identity-based cryptography, the key escrow problem arises because a trusted third party, the Private Key Generator (PKG), generates the private keys for all users. This means that the PKG has complete control over the private keys, which raises concerns about the security and privacy of the users. To balance security and privacy concerns, some approaches have been proposed to address the key escrow problem. We propose an efficient democratic identity-based encryption model that balances the government's and users' rights while ensuring security and privacy. The key objective of the proposed scheme is to provide the government with authority to monitor unlawful messages while ensuring the user's privacy for their lawful messages. To achieve this, the scheme involves two entities: PKG and PKPO. The user's partial key is escrowed at PKG, while the partial key is stored at PKPO. The latter provides a privacy service to the user by confusing their signature, which the user with their personal information can only unlock.
LongNet: Scaling Transformers to 1,000,000,000 Tokens
Venue: arXiv cs.CL 2023
Year: 2023

Scaling sequence length has become a critical demand in the era of large language models. However, existing methods struggle with either computational complexity or model expressivity, rendering the maximum sequence length restricted. To address this issue, we introduce LongNet, a Transformer variant that can scale sequence length to more than 1 billion tokens, without sacrificing the performance on shorter sequences. Specifically, we propose dilated attention, which expands the attentive field exponentially as the distance grows. LongNet has significant advantages: 1) it has a linear computation complexity and a logarithm dependency between any two tokens in a sequence; 2) it can be served as a distributed trainer for extremely long sequences; 3) its dilated attention is a drop-in replacement for standard attention, which can be seamlessly integrated with the existing Transformer-based optimization. Experiments results demonstrate that LongNet yields strong performance on both long-sequence modeling and general language tasks. Our work opens up new possibilities for modeling very long sequences, e.g., treating a whole corpus or even the entire Internet as a sequence.
TablEye: Seeing small Tables through the Lens of Images
Venue: arXiv cs.LG 2023
Year: 2023

The exploration of few-shot tabular learning becomes imperative. Tabular data is a versatile representation that captures diverse information, yet it is not exempt from limitations, property of data and model size. Labeling extensive tabular data can be challenging, and it may not be feasible to capture every important feature. Few-shot tabular learning, however, remains relatively unexplored, primarily due to scarcity of shared information among independent datasets and the inherent ambiguity in defining boundaries within tabular data. To the best of our knowledge, no meaningful and unrestricted few-shot tabular learning techniques have been developed without imposing constraints on the dataset. In this paper, we propose an innovative framework called TablEye, which aims to overcome the limit of forming prior knowledge for tabular data by adopting domain transformation. It facilitates domain transformation by generating tabular images, which effectively conserve the intrinsic semantics of the original tabular data. This approach harnesses rigorously tested few-shot learning algorithms and embedding functions to acquire and apply prior knowledge. Leveraging shared data domains allows us to utilize this prior knowledge, originally learned from the image domain. Specifically, TablEye demonstrated a superior performance by outstripping the TabLLM in a 4-shot task with a maximum 0.11 AUC and a STUNT in a 1- shot setting, where it led on average by 3.17% accuracy.
FREEDOM: Target Label & Source Data & Domain Information-Free Multi-Source Domain Adaptation for Unsupervised Personalization
Venue: arXiv cs.LG 2023
Year: 2023

From a service perspective, Multi-Source Domain Adaptation (MSDA) is a promising scenario to adapt a deployed model to a client's dataset. It can provide adaptation without a target label and support the case where a source dataset is constructed from multiple domains. However, it is impractical, wherein its training heavily relies on prior domain information of the multi-source dataset -- how many domains exist and the domain label of each data sample. Moreover, MSDA requires both source and target datasets simultaneously (physically), causing storage limitations on the client device or data privacy issues by transferring client data to a server. For a more practical scenario of model adaptation from a service provider's point of view, we relax these constraints and present a novel problem scenario of Three-Free Domain Adaptation, namely TFDA, where 1) target labels, 2) source dataset, and mostly 3) source domain information (domain labels + the number of domains) are unavailable. Under the problem scenario, we propose a practical adaptation framework called FREEDOM. It leverages the power of the generative model, disentangling data into class and style aspects, where the style is defined as the class-independent information from the source data and designed with a nonparametric Bayesian approach. In the adaptation stage, FREEDOM aims to match the source class distribution with the target's under the philosophy that class distribution is consistent even if the style is different; after then, only part of the classification model is deployed as a personalized network. As a result, FREEDOM achieves state-of-the-art or comparable performance even without domain information, with reduced final model size on the target side, independent of the number of source domains.
Comparison of Neural FEM and Neural Operator Methods for applications in Solid Mechanics
Venue: arXiv cs.CE 2023
Year: 2023

Machine Learning methods belong to the group of most up-to-date approaches for solving partial differential equations. The current work investigates two classes, Neural FEM and Neural Operator Methods, for the use in elastostatics by means of numerical experiments. The Neural Operator methods require expensive training but then allow for solving multiple boundary value problems with the same Machine Learning model. Main differences between the two classes are the computational effort and accuracy. Especially the accuracy requires more research for practical applications.
Interpretable Computer Vision Models through Adversarial Training: Unveiling the Robustness-Interpretability Connection
Venue: arXiv cs.CV 2023
Year: 2023

With the perpetual increase of complexity of the state-of-the-art deep neural networks, it becomes a more and more challenging task to maintain their interpretability. Our work aims to evaluate the effects of adversarial training utilized to produce robust models - less vulnerable to adversarial attacks. It has been shown to make computer vision models more interpretable. Interpretability is as essential as robustness when we deploy the models to the real world. To prove the correlation between these two problems, we extensively examine the models using local feature-importance methods (SHAP, Integrated Gradients) and feature visualization techniques (Representation Inversion, Class Specific Image Generation). Standard models, compared to robust are more susceptible to adversarial attacks, and their learned representations are less meaningful to humans. Conversely, these models focus on distinctive regions of the images which support their predictions. Moreover, the features learned by the robust model are closer to the real ones.
Mainline Automatic Train Horn and Brake Performance Metric
Venue: arXiv cs.RO 2023
Year: 2023

This paper argues for the introduction of a mainline rail-oriented performance metric for driver-replacing on-board perception systems. Perception at the head of a train is divided into several subfunctions. This article presents a preliminary submetric for the obstacle detection subfunction. To the best of the author's knowledge, no other such proposal for obstacle detection exists. A set of submetrics for the subfunctions should facilitate the comparison of perception systems among each other and guide the measurement of human driver performance. It should also be useful for a standardized prediction of the number of accidents for a given perception system in a given operational design domain. In particular, for the proposal of the obstacle detection submetric, the professional readership is invited to provide their feedback and quantitative information to the author. The analysis results of the feedback will be published separately later.
Homo-Loggatus. The anthropological condition of historians in the digital world
Venue: arXiv cs.HC 2023
Year: 2023

Computerization has created a digital ecological niche where humans live in a state of interconnection that modifies their Epigenetics. Within this hyper-datafied virtual space, the logged-in agent enhances their intellectual and rational abilities, giving rise to a new cognitive entity. Humans are evolving towards a new anthropological status that shifts the terms of the Digital History debate from History to the historian, compelling the latter to reflect on the positions of Fichte and Schelling regarding the mind-body-world relationship (ecological niche). This reflection leads to the possibility of overcoming the crisis of History imposed by presentism and the necessity of redefining the research methodology based on the new vision of the interconnection between the mind and the digital niche as an investigative tool.
MRecGen: Multimodal Appropriate Reaction Generator
Venue: arXiv cs.CV 2023
Year: 2023

Verbal and non-verbal human reaction generation is a challenging task, as different reactions could be appropriate for responding to the same behaviour. This paper proposes the first multiple and multimodal (verbal and nonverbal) appropriate human reaction generation framework that can generate appropriate and realistic human-style reactions (displayed in the form of synchronised text, audio and video streams) in response to an input user behaviour. This novel technique can be applied to various human-computer interaction scenarios by generating appropriate virtual agent/robot behaviours. Our demo is available at \url{https://github.com/SSYSteve/MRecGen}.
The Importance of Knowing the Arrival Order in Combinatorial Bayesian Settings
Venue: arXiv cs.GT 2023
Year: 2023

We study the measure of order-competitive ratio introduced by Ezra et al. [2023] for online algorithms in Bayesian combinatorial settings. In our setting, a decision-maker observes a sequence of elements that are associated with stochastic rewards that are drawn from known priors, but revealed one by one in an online fashion. The decision-maker needs to decide upon the arrival of each element whether to select it or discard it (according to some feasibility constraint), and receives the associated rewards of the selected elements. The order-competitive ratio is defined as the worst-case ratio (over all distribution sequences) between the performance of the best order-unaware and order-aware algorithms, and quantifies the loss incurred due to the lack of knowledge of the arrival order. Ezra et al. [2023] showed how to design algorithms that achieve better approximations with respect to the new benchmark (order-competitive ratio) in the single-choice setting, which raises the natural question of whether the same can be achieved in combinatorial settings. In particular, whether it is possible to achieve a constant approximation with respect to the best online algorithm for downward-closed feasibility constraints, whether $ω(1/n)$-approximation is achievable for general (non-downward-closed) feasibility constraints, or whether a convergence rate to $1$ of $o(1/\sqrt{k})$ is achievable for the multi-unit setting. We show, by devising novel constructions that may be of independent interest, that for all three scenarios, the asymptotic lower bounds with respect to the old benchmark, also hold with respect to the new benchmark.
Successful Combination of Database Search and Snowballing for Identification of Primary Studies in Systematic Literature Studies
Venue: arXiv cs.SE 2023
Year: 2023

Background: A good search strategy is essential for a successful systematic literature study. Historically, database searches have been the norm, which has later been complemented with snowball searches. Our conjecture is that we can perform even better searches if combining the two search approaches, referred to as a hybrid search strategy. Objective: Our main objective was to compare and evaluate a hybrid search strategy. Furthermore, we compared some alternative hybrid search strategies to assess whether it was possible to identify more cost-efficient ways of searching for relevant primary studies. Method: To compare and evaluate the hybrid search strategy, we replicated an SLR on industry-academia collaboration in software engineering. The SLR used a more traditional approach to searching for relevant articles for an SLR, while the replication was conducted using a hybrid search strategy. Results: In our evaluation, the hybrid search strategy was superior in identifying relevant primary studies. It identified 30 percent more primary studies and even more when focusing only on peer-reviewed articles. To embrace individual viewpoints when assessing research articles and minimise the risk of missing primary studies, we introduced two new concepts, wild cards and borderline articles, when conducting systematic literature studies. Conclusions: The hybrid search strategy is a strong contender for being used when conducting systematic literature studies. Furthermore, alternative hybrid search strategies may be viable if selected wisely in relation to the start set for snowballing. Finally, the two new concepts were judged as essential to cater for different individual judgements and to minimise the risk of excluding primary studies that ought to be included.
Real-time Workload Pattern Analysis for Large-scale Cloud Databases
Venue: arXiv cs.DB 2023
Year: 2023

Hosting database services on cloud systems has become a common practice. This has led to the increasing volume of database workloads, which provides the opportunity for pattern analysis. Discovering workload patterns from a business logic perspective is conducive to better understanding the trends and characteristics of the database system. However, existing workload pattern discovery systems are not suitable for large-scale cloud databases which are commonly employed by the industry. This is because the workload patterns of large-scale cloud databases are generally far more complicated than those of ordinary databases. In this paper, we propose Alibaba Workload Miner (AWM), a real-time system for discovering workload patterns in complicated large-scale workloads. AWM encodes and discovers the SQL query patterns logged from user requests and optimizes the querying processing based on the discovered patterns. First, Data Collection & Preprocessing Module collects streaming query logs and encodes them into high-dimensional feature embeddings with rich semantic contexts and execution features. Next, Online Workload Mining Module separates encoded queries by business groups and discovers the workload patterns for each group. Meanwhile, Offline Training Module collects labels and trains the classification model using the labels. Finally, Pattern-based Optimizing Module optimizes query processing in cloud databases by exploiting discovered patterns. Extensive experimental results on one synthetic dataset and two real-life datasets (extracted from Alibaba Cloud databases) show that AWM enhances the accuracy of pattern discovery by 66% and reduce the latency of online inference by 22%, compared with the state-of-the-arts.
Dense Retrieval Adaptation using Target Domain Description
Venue: arXiv cs.IR 2023
Year: 2023

In information retrieval (IR), domain adaptation is the process of adapting a retrieval model to a new domain whose data distribution is different from the source domain. Existing methods in this area focus on unsupervised domain adaptation where they have access to the target document collection or supervised (often few-shot) domain adaptation where they additionally have access to (limited) labeled data in the target domain. There also exists research on improving zero-shot performance of retrieval models with no adaptation. This paper introduces a new category of domain adaptation in IR that is as-yet unexplored. Here, similar to the zero-shot setting, we assume the retrieval model does not have access to the target document collection. In contrast, it does have access to a brief textual description that explains the target domain. We define a taxonomy of domain attributes in retrieval tasks to understand different properties of a source domain that can be adapted to a target domain. We introduce a novel automatic data construction pipeline that produces a synthetic document collection, query set, and pseudo relevance labels, given a textual domain description. Extensive experiments on five diverse target domains show that adapting dense retrieval models using the constructed synthetic data leads to effective retrieval performance on the target domain.
Active Learning with Contrastive Pre-training for Facial Expression Recognition
Venue: arXiv cs.CV 2023
Year: 2023

Deep learning has played a significant role in the success of facial expression recognition (FER), thanks to large models and vast amounts of labelled data. However, obtaining labelled data requires a tremendous amount of human effort, time, and financial resources. Even though some prior works have focused on reducing the need for large amounts of labelled data using different unsupervised methods, another promising approach called active learning is barely explored in the context of FER. This approach involves selecting and labelling the most representative samples from an unlabelled set to make the best use of a limited 'labelling budget'. In this paper, we implement and study 8 recent active learning methods on three public FER datasets, FER13, RAF-DB, and KDEF. Our findings show that existing active learning methods do not perform well in the context of FER, likely suffering from a phenomenon called 'Cold Start', which occurs when the initial set of labelled samples is not well representative of the entire dataset. To address this issue, we propose contrastive self-supervised pre-training, which first learns the underlying representations based on the entire unlabelled dataset. We then follow this with the active learning methods and observe that our 2-step approach shows up to 9.2% improvement over random sampling and up to 6.7% improvement over the best existing active learning baseline without the pre-training. We will make the code for this study public upon publication at: github.com/ShuvenduRoy/ActiveFER.
Intent-driven Intelligent Control and Orchestration in O-RAN Via Hierarchical Reinforcement Learning
Venue: arXiv cs.NI 2023
Year: 2023

rApps and xApps need to be controlled and orchestrated well in the open radio access network (O-RAN) so that they can deliver a guaranteed network performance in a complex multi-vendor environment. This paper proposes a novel intent-driven intelligent control and orchestration scheme based on hierarchical reinforcement learning (HRL). The proposed scheme can orchestrate multiple rApps or xApps according to the operator's intent of optimizing certain key performance indicators (KPIs), such as throughput, energy efficiency, and latency. Specifically, we propose a bi-level architecture with a meta-controller and a controller. The meta-controller provides the target performance in terms of KPIs, while the controller performs xApp orchestration at the lower level. Our simulation results show that the proposed HRL-based intent-driven xApp orchestration mechanism achieves 7.5% and 21.4% increase in average system throughput with respect to two baselines, i.e., a single xApp baseline and a non-machine learning-based algorithm, respectively. Similarly, 17.3% and 37.9% increase in energy efficiency are observed in comparison to the same baselines.
Approximation Algorithms for Directed Weighted Spanners
Venue: arXiv cs.DS 2023
Year: 2023

In the pairwise weighted spanner problem, the input consists of an $n$-vertex-directed graph, where each edge is assigned a cost and a length. Given $k$ vertex pairs and a distance constraint for each pair, the goal is to find a minimum-cost subgraph in which the distance constraints are satisfied. This formulation captures many well-studied connectivity problems, including spanners, distance preservers, and Steiner forests. In the offline setting, we show: 1. An $\tilde{O}(n^{4/5 + ε})$-approximation algorithm for pairwise weighted spanners. When the edges have unit costs and lengths, the best previous algorithm gives an $\tilde{O}(n^{3/5 + ε})$-approximation, due to Chlamtáč, Dinitz, Kortsarz, and Laekhanukit (TALG, 2020). 2. An $\tilde{O}(n^{1/2+ε})$-approximation algorithm for all-pair weighted distance preservers. When the edges have unit costs and arbitrary lengths, the best previous algorithm gives an $\tilde{O}(n^{1/2})$-approximation for all-pair spanners, due to Berman, Bhattacharyya, Makarychev, Raskhodnikova, and Yaroslavtsev (Information and Computation, 2013). In the online setting, we show: 1. An $\tilde{O}(k^{1/2 + ε})$-competitive algorithm for pairwise weighted spanners. The state-of-the-art results are $\tilde{O}(n^{4/5})$-competitive when edges have unit costs and arbitrary lengths, and $\min\{\tilde{O}(k^{1/2 + ε}), \tilde{O}(n^{2/3 + ε})\}$-competitive when edges have unit costs and lengths, due to Grigorescu, Lin, and Quanrud (APPROX, 2021). 2. An $\tilde{O}(k^ε)$-competitive algorithm for single-source weighted spanners. Without distance constraints, this problem is equivalent to the directed Steiner tree problem. The best previous algorithm for online directed Steiner trees is $\tilde{O}(k^ε)$-competitive, due to Chakrabarty, Ene, Krishnaswamy, and Panigrahi (SICOMP, 2018).