OpenRaiser/Intern-Atlas huggingface dataset:
Accuracy and Fidelity Comparison of Luna and DALL-E 2 Diffusion-Based Image Generation Systems
Venue: arXiv cs.CV 2023
Year: 2023

We qualitatively examine the accuracy and fidelity between two diffusion-based image generation systems, namely DALL-E 2 and Luna, which have massive differences in training datasets, algorithmic approaches, prompt resolvement, and output upscaling. The methodology used is a qualitative benchmark created by Saharia et al. and in our research we conclude that DALL-E 2 significantly edges Luna in both alignment and fidelity comparisons.
Corrupted by Algorithms? How AI-generated and Human-written Advice Shape (Dis)honesty
Venue: arXiv cs.HC 2023
Year: 2023

Artificial Intelligence (AI) increasingly becomes an indispensable advisor. New ethical concerns arise if AI persuades people to behave dishonestly. In an experiment, we study how AI advice (generated by a Natural-Language-Processing algorithm) affects (dis)honesty, compare it to equivalent human advice, and test whether transparency about advice source matters. We find that dishonesty-promoting advice increases dishonesty, whereas honesty-promoting advice does not increase honesty. This is the case for both AI- and human advice. Algorithmic transparency, a commonly proposed policy to mitigate AI risks, does not affect behaviour. The findings mark the first steps towards managing AI advice responsibly.
A Survey of Code-switching: Linguistic and Social Perspectives for Language Technologies
Venue: arXiv cs.CL 2023
Year: 2023

The analysis of data in which multiple languages are represented has gained popularity among computational linguists in recent years. So far, much of this research focuses mainly on the improvement of computational methods and largely ignores linguistic and social aspects of C-S discussed across a wide range of languages within the long-established literature in linguistics. To fill this gap, we offer a survey of code-switching (C-S) covering the literature in linguistics with a reflection on the key issues in language technologies. From the linguistic perspective, we provide an overview of structural and functional patterns of C-S focusing on the literature from European and Indian contexts as highly multilingual areas. From the language technologies perspective, we discuss how massive language models fail to represent diverse C-S types due to lack of appropriate training data, lack of robust evaluation benchmarks for C-S (across multilingual situations and types of C-S) and lack of end-to-end systems that cover sociolinguistic aspects of C-S as well. Our survey will be a step towards an outcome of mutual benefit for computational scientists and linguists with a shared interest in multilingualism and C-S.
Emotion-Cause Pair Extraction as Question Answering
Venue: arXiv cs.CL 2023
Year: 2023

The task of Emotion-Cause Pair Extraction (ECPE) aims to extract all potential emotion-cause pairs of a document without any annotation of emotion or cause clauses. Previous approaches on ECPE have tried to improve conventional two-step processing schemes by using complex architectures for modeling emotion-cause interaction. In this paper, we cast the ECPE task to the question answering (QA) problem and propose simple yet effective BERT-based solutions to tackle it. Given a document, our Guided-QA model first predicts the best emotion clause using a fixed question. Then the predicted emotion is used as a question to predict the most potential cause for the emotion. We evaluate our model on a standard ECPE corpus. The experimental results show that despite its simplicity, our Guided-QA achieves promising results and is easy to reproduce. The code of Guided-QA is also provided.
The Evolutionary Computation Methods No One Should Use
Venue: arXiv cs.NE 2023
Year: 2023

The center-bias (or zero-bias) operator has recently been identified as one of the problems plaguing the benchmarking of evolutionary computation methods. This operator lets the methods that utilize it easily optimize functions that have their respective optima in the center of the feasible set. In this paper, we describe a simple procedure that can be used to identify methods that incorporate a center-bias operator and use it to investigate 90 evolutionary computation methods that were published between 1987 and 2022. We show that more than half (47 out of the 90) of the considered methods have the center-bias problem. We also show that the center-bias is a relatively new phenomenon (with the first identified method being from 2012), but its inclusion has become extremely prevalent in the last few years. Lastly, we briefly discuss the possible root causes of this issue.
New Results for the Pointing Errors Model in Two Asymptotic Cases
Venue: arXiv cs.IT 2023
Year: 2023

Several precise and computationally efficient results for pointing errors models in two asymptotic cases are derived in this paper. The normalized mean-squared error (NMSE) performance metric is employed to quantify the accuracy of different models. For the case that the beam width is relatively larger than the detection aperture, we propose the three kinds of models that have the form of $c_1\exp(-c_2r^2) $.It is shown that the modified intensity uniform model not only achieves a comparable accuracy with the best linearized model, but also is expressed in an elegant mathematical way when compared to the traditional Farid model. This indicates that the modified intensity uniform model is preferable in the performance analysis of free space optical (FSO) systems considering the effects of the pointing errors. By analogizing the beam spot with a point in the case that beam width is smaller than the detection aperture, the solution of the pointing errors model is transformed to a smooth function approximation problem, and we find that a more accurate approximation can be achieved by the proposed point approximation model when compared to the model that is induced from the Vasylyev model in some scenarios.
Self-Motivated Multi-Agent Exploration
Venue: arXiv cs.LG 2023
Year: 2023

In cooperative multi-agent reinforcement learning (CMARL), it is critical for agents to achieve a balance between self-exploration and team collaboration. However, agents can hardly accomplish the team task without coordination and they would be trapped in a local optimum where easy cooperation is accessed without enough individual exploration. Recent works mainly concentrate on agents' coordinated exploration, which brings about the exponentially grown exploration of the state space. To address this issue, we propose Self-Motivated Multi-Agent Exploration (SMMAE), which aims to achieve success in team tasks by adaptively finding a trade-off between self-exploration and team cooperation. In SMMAE, we train an independent exploration policy for each agent to maximize their own visited state space. Each agent learns an adjustable exploration probability based on the stability of the joint team policy. The experiments on highly cooperative tasks in StarCraft II micromanagement benchmark (SMAC) demonstrate that SMMAE can explore task-related states more efficiently, accomplish coordinated behaviours and boost the learning performance.
A Direct and New Construction of Near-Optimal Multiple ZCZ Sequence Sets
Venue: arXiv cs.IT 2023
Year: 2023

In this paper, for the first time, we present a direct and new construction of multiple zero-correlation zone (ZCZ) sequence sets with inter-set zero-cross correlation zone (ZCCZ) from generalised Boolean function. Tang \emph{et al.} in their 2010 paper, proposed an open problem to construct $N$ binary ZCZ sequence sets such that each of these ZCZ sequence sets is optimal and if the union of these $N$ sets is taken then that union is again an optimal ZCZ sequence set. The proposed construction partially settles this open problem by presenting a construction of optimal ZCZ sequence sets such that their union is a near-optimal ZCZ sequence set. Further, the performance parameter of each binary ZCZ sequence set in the proposed construction is $1$ and tends to $1$ for their union. The proposed construction is presented by a two-layer graphical representation and compared with the existing state-of-the-art. Finally, novel multi-cluster quasi synchronous-code division multiple access (QS-CDMA) system model is provided by using the proposed multiple ZCZ sequence sets.
L-HYDRA: Multi-Head Physics-Informed Neural Networks
Venue: arXiv cs.LG 2023
Year: 2023

We introduce multi-head neural networks (MH-NNs) to physics-informed machine learning, which is a type of neural networks (NNs) with all nonlinear hidden layers as the body and multiple linear output layers as multi-head. Hence, we construct multi-head physics-informed neural networks (MH-PINNs) as a potent tool for multi-task learning (MTL), generative modeling, and few-shot learning for diverse problems in scientific machine learning (SciML). MH-PINNs connect multiple functions/tasks via a shared body as the basis functions as well as a shared distribution for the head. The former is accomplished by solving multiple tasks with MH-PINNs with each head independently corresponding to each task, while the latter by employing normalizing flows (NFs) for density estimate and generative modeling. To this end, our method is a two-stage method, and both stages can be tackled with standard deep learning tools of NNs, enabling easy implementation in practice. MH-PINNs can be used for various purposes, such as approximating stochastic processes, solving multiple tasks synergistically, providing informative prior knowledge for downstream few-shot learning tasks such as meta-learning and transfer learning, learning representative basis functions, and uncertainty quantification. We demonstrate the effectiveness of MH-PINNs in five benchmarks, investigating also the possibility of synergistic learning in regression analysis. We name the open-source code "Lernaean Hydra" (L-HYDRA), since this mythical creature possessed many heads for performing important multiple tasks, as in the proposed method.
A Shannon-Theoretic Approach to the Storage-Retrieval Tradeoff in PIR Systems
Venue: arXiv cs.IT 2023
Year: 2023

We consider the storage-retrieval rate tradeoff in private information retrieval (PIR) systems using a Shannon-theoretic approach. Our focus is mostly on the canonical two-message two-database case, for which a coding scheme based on random codebook generation and the binning technique is proposed. This coding scheme reveals a hidden connection between PIR and the classic multiple description source coding problem. We first show that when the retrieval rate is kept optimal, the proposed non-linear scheme can achieve better performance over any linear scheme. Moreover, a non-trivial storage-retrieval rate tradeoff can be achieved beyond space-sharing between this extreme point and the other optimal extreme point, achieved by the retrieve-everything strategy. We further show that with a method akin to the expurgation technique, one can extract a zero-error PIR code from the random code. Outer bounds are also studied and compared to establish the superiority of the non-linear codes over linear codes.
Machine Fault Classification using Hamiltonian Neural Networks
Venue: arXiv cs.LG 2023
Year: 2023

A new approach is introduced to classify faults in rotating machinery based on the total energy signature estimated from sensor measurements. The overall goal is to go beyond using black-box models and incorporate additional physical constraints that govern the behavior of mechanical systems. Observational data is used to train Hamiltonian neural networks that describe the conserved energy of the system for normal and various abnormal regimes. The estimated total energy function, in the form of the weights of the Hamiltonian neural network, serves as the new feature vector to discriminate between the faults using off-the-shelf classification models. The experimental results are obtained using the MaFaulDa database, where the proposed model yields a promising area under the curve (AUC) of $0.78$ for the binary classification (normal vs abnormal) and $0.84$ for the multi-class problem (normal, and $5$ different abnormal regimes).
A Direct Construction of Optimal 2D-ZCACS with Flexible Array Size and Large Set Size
Venue: arXiv cs.IT 2023
Year: 2023

In this paper, we propose a direct construction of optimal two-dimensional Z-complementary array code sets (2D-ZCACS) using multivariable functions (MVFs). In contrast to earlier works, the proposed construction allows for a flexible array size and a large set size. Additionally, the proposed design can be transformed into a one-dimensional Z-complementary code set (1D-ZCCS). Many of the 1D-ZCCS described in the literature appeared to be special cases of this proposed construction. At last, we compare our work with the current state of the art and then draw our conclusions.
You Truly Understand What I Need: Intellectual and Friendly Dialogue Agents grounding Knowledge and Persona
Venue: arXiv cs.CL 2023
Year: 2023

To build a conversational agent that interacts fluently with humans, previous studies blend knowledge or personal profile into the pre-trained language model. However, the model that considers knowledge and persona at the same time is still limited, leading to hallucination and a passive way of using personas. We propose an effective dialogue agent that grounds external knowledge and persona simultaneously. The agent selects the proper knowledge and persona to use for generating the answers with our candidate scoring implemented with a poly-encoder. Then, our model generates the utterance with lesser hallucination and more engagingness utilizing retrieval augmented generation with knowledge-persona enhanced query. We conduct experiments on the persona-knowledge chat and achieve state-of-the-art performance in grounding and generation tasks on the automatic metrics. Moreover, we validate the answers from the models regarding hallucination and engagingness through human evaluation and qualitative results. We show our retriever's effectiveness in extracting relevant documents compared to the other previous retrievers, along with the comparison of multiple candidate scoring methods. Code is available at https://github.com/dlawjddn803/INFO
CyberLoc: Towards Accurate Long-term Visual Localization
Venue: arXiv cs.CV 2023
Year: 2023

This technical report introduces CyberLoc, an image-based visual localization pipeline for robust and accurate long-term pose estimation under challenging conditions. The proposed method comprises four modules connected in a sequence. First, a mapping module is applied to build accurate 3D maps of the scene, one map for each reference sequence if there exist multiple reference sequences under different conditions. Second, a single-image-based localization pipeline (retrieval--matching--PnP) is performed to estimate 6-DoF camera poses for each query image, one for each 3D map. Third, a consensus set maximization module is proposed to filter out outlier 6-DoF camera poses, and outputs one 6-DoF camera pose for a query. Finally, a robust pose refinement module is proposed to optimize 6-DoF query poses, taking candidate global 6-DoF camera poses and their corresponding global 2D-3D matches, sparse 2D-2D feature matches between consecutive query images and SLAM poses of the query sequence as input. Experiments on the 4seasons dataset show that our method achieves high accuracy and robustness. In particular, our approach wins the localization challenge of ECCV 2022 workshop on Map-based Localization for Autonomous Driving (MLAD-ECCV2022).
High-Performance Deterministic Concurrency using Lingua Franca
Venue: arXiv cs.PL 2023
Year: 2023

Actor frameworks and similar reactive programming techniques are widely used for building concurrent systems. They promise to be efficient and scale well to a large number of cores or nodes in a distributed system. However, they also expose programmers to nondeterminism, which often makes implementations hard to understand, debug, and test. The recently proposed reactor model is a promising alternative that enables efficient deterministic concurrency. In this paper, we show that determinacy does neither imply a loss in expressivity nor in performance. To show this, we evaluate Lingua Franca (LF), a reactor-oriented coordination language that equips mainstream programming languages with a concurrency model that automatically takes advantage of opportunities to exploit parallelism that do not introduce nondeterminism. Our implementation of the Savina benchmark suite demonstrates that, in terms of execution time, the runtime performance of LF programs even exceeds popular and highly optimized actor frameworks. We compare against Akka and CAF, which LF outperforms by 1.86x and 1.42x, respectively.
Delay-Doppler Domain Tomlinson-Harashima Precoding for OTFS-based Downlink MU-MIMO Transmissions: Linear Complexity Implementation and Scaling Law Analysis
Venue: arXiv cs.IT 2023
Year: 2023

Orthogonal time frequency space (OTFS) modulation is a recently proposed delay-Doppler (DD) domain communication scheme, which has shown promising performance in general wireless communications, especially over high-mobility channels. In this paper, we investigate DD domain Tomlinson-Harashima precoding (THP) for downlink multiuser multiple-input and multiple-output OTFS (MU-MIMO-OTFS) transmissions. Instead of directly applying THP based on the huge equivalent channel matrix, we propose a simple implementation of THP that does not require any matrix decomposition or inversion. Such a simple implementation is enabled by the DD domain channel property, i.e., different resolvable paths do not share the same delay and Doppler shifts, which makes it possible to pre-cancel all the DD domain interference in a symbol-by-symbol manner. We also study the achievable rate performance for the proposed scheme by leveraging the information-theoretical equivalent models. In particular, we show that the proposed scheme can achieve a near optimal performance in the high signal-to-noise ratio (SNR) regime. More importantly, scaling laws for achievable rates with respect to number of antennas and users are derived, which indicate that the achievable rate increases logarithmically with the number of antennas and linearly with the number of users. Our numerical results align well with our findings and also demonstrate a significant improvement compared to existing MU-MIMO schemes on OTFS and orthogonal frequency-division multiplexing (OFDM).
OPD@NL4Opt: An ensemble approach for the NER task of the optimization problem
Venue: arXiv cs.CL 2023
Year: 2023

In this paper, we present an ensemble approach for the NL4Opt competition subtask 1(NER task). For this task, we first fine tune the pretrained language models based on the competition dataset. Then we adopt differential learning rates and adversarial training strategies to enhance the model generalization and robustness. Additionally, we use a model ensemble method for the final prediction, which achieves a micro-averaged F1 score of 93.3% and attains the second prize in the NER task.
Architect, Regularize and Replay (ARR): a Flexible Hybrid Approach for Continual Learning
Venue: arXiv cs.LG 2023
Year: 2023

In recent years we have witnessed a renewed interest in machine learning methodologies, especially for deep representation learning, that could overcome basic i.i.d. assumptions and tackle non-stationary environments subject to various distributional shifts or sample selection biases. Within this context, several computational approaches based on architectural priors, regularizers and replay policies have been proposed with different degrees of success depending on the specific scenario in which they were developed and assessed. However, designing comprehensive hybrid solutions that can flexibly and generally be applied with tunable efficiency-effectiveness trade-offs still seems a distant goal. In this paper, we propose "Architect, Regularize and Replay" (ARR), an hybrid generalization of the renowned AR1 algorithm and its variants, that can achieve state-of-the-art results in classic scenarios (e.g. class-incremental learning) but also generalize to arbitrary data streams generated from real-world datasets such as CIFAR-100, CORe50 and ImageNet-1000.
Graph-Collaborated Auto-Encoder Hashing for Multi-view Binary Clustering
Venue: arXiv cs.CV 2023
Year: 2023

Unsupervised hashing methods have attracted widespread attention with the explosive growth of large-scale data, which can greatly reduce storage and computation by learning compact binary codes. Existing unsupervised hashing methods attempt to exploit the valuable information from samples, which fails to take the local geometric structure of unlabeled samples into consideration. Moreover, hashing based on auto-encoders aims to minimize the reconstruction loss between the input data and binary codes, which ignores the potential consistency and complementarity of multiple sources data. To address the above issues, we propose a hashing algorithm based on auto-encoders for multi-view binary clustering, which dynamically learns affinity graphs with low-rank constraints and adopts collaboratively learning between auto-encoders and affinity graphs to learn a unified binary code, called Graph-Collaborated Auto-Encoder Hashing for Multi-view Binary Clustering (GCAE). Specifically, we propose a multi-view affinity graphs learning model with low-rank constraint, which can mine the underlying geometric information from multi-view data. Then, we design an encoder-decoder paradigm to collaborate the multiple affinity graphs, which can learn a unified binary code effectively. Notably, we impose the decorrelation and code balance constraints on binary codes to reduce the quantization errors. Finally, we utilize an alternating iterative optimization scheme to obtain the multi-view clustering results. Extensive experimental results on $5$ public datasets are provided to reveal the effectiveness of the algorithm and its superior performance over other state-of-the-art alternatives.
Fuzzers for stateful systems: Survey and Research Directions
Venue: arXiv cs.CR 2023
Year: 2023

Fuzzing is a security testing methodology effective in finding bugs. In a nutshell, a fuzzer sends multiple slightly malformed messages to the software under test, hoping for crashes or weird system behaviour. The methodology is relatively simple, although applications that keep internal states are challenging to fuzz. The research community has responded to this challenge by developing fuzzers tailored to stateful systems, but a clear understanding of the variety of strategies is still missing. In this paper, we present the first taxonomy of fuzzers for stateful systems and provide a systematic comparison and classification of these fuzzers.