OpenRaiser/Intern-Atlas huggingface dataset:
CodeMark: Imperceptible Watermarking for Code Datasets against Neural Code Completion Models
Venue: arXiv cs.SE 2023
Year: 2023

Code datasets are of immense value for training neural-network-based code completion models, where companies or organizations have made substantial investments to establish and process these datasets. Unluckily, these datasets, either built for proprietary or public usage, face the high risk of unauthorized exploits, resulting from data leakages, license violations, etc. Even worse, the ``black-box'' nature of neural models sets a high barrier for externals to audit their training datasets, which further connives these unauthorized usages. Currently, watermarking methods have been proposed to prohibit inappropriate usage of image and natural language datasets. However, due to domain specificity, they are not directly applicable to code datasets, leaving the copyright protection of this emerging and important field of code data still exposed to threats. To fill this gap, we propose a method, named CodeMark, to embed user-defined imperceptible watermarks into code datasets to trace their usage in training neural code completion models. CodeMark is based on adaptive semantic-preserving transformations, which preserve the exact functionality of the code data and keep the changes covert against rule-breakers. We implement CodeMark in a toolkit and conduct an extensive evaluation of code completion models. CodeMark is validated to fulfill all desired properties of practical watermarks, including harmlessness to model accuracy, verifiability, robustness, and imperceptibility.
The Effect of an Exergame on the Shadow Play Skill Based on Muscle Memory for Young Female Participants: The Case of Forehand Drive in Table Tennis
Venue: arXiv cs.HC 2023
Year: 2023

Learning and practicing table tennis with traditional methods is a long, tedious process and may even lead to the internalization of incorrect techniques if not supervised by a coach. To overcome these issues, the presented study proposes an exergame with the aim of enhancing young female novice players' performance by boosting muscle memory, making practice more interesting, and decreasing the probability of faulty training. Specifically, we propose an exergame based on skeleton tracking and a virtual avatar to support correct shadow practice to learn forehand drive technique without the presence of a coach. We recruited 44 schoolgirls aged between 8 and 12 years without a background in playing table tennis and divided them into control and experimental groups. We examined their stroke skills (via the Mott-Lockhart test) and the error coefficient of their forehand drives (using a ball machine) in the pretest, post-test, and follow-up tests (10 days after the post-test). Our results showed that the experimental group had progress in the short and long term, while the control group had an improvement only in the short term. Further, the scale of improvement in the experimental group was significantly higher than in the control group. Given that the early stages of learning, particularly in girls children, are important in the internalization of individual skills in would-be athletes, this method could support promoting correct training for young females.
Infomathic
Venue: arXiv cs.CY 2023
Year: 2023

Since its existence, the computer tool has often supported mathematicians, whether it is to implement an approximation method (numerical calculation of a root, of an integral, ...) or to simulate a phenomenon (geometric in nature, probabilistic, ...) to verify or establish a conjecture. But, and this is another point on which we will concentrate our attention here, computing will also have served the cause of mathematics by inspiring certain reasoning or by taking charge of whole sections of a demonstration. We will illustrate this fruitful partnership with two example saccessible from high school.
Introduction to ethics in the age of digital communication
Venue: arXiv cs.CY 2023
Year: 2023

Ethics, or moral philosophy, have existed throughout civil human history. Ethics can be described simplistically as the study of what is good and bad or good and evil. More relevant for contemporary societal discourses, are behavioural understandings of ethics, and ethical practice. It is integral for individuals operating in the fields of communications, design and technological development to grasp what ethics are, how they relate and apply to specific domains, where basic principles or similarities lie from context to context, and where there may be differences. This article serves as an introduction to ethics in the field of digital communication. It gives a brief overview of applied ethics as a practical sub-field of ethics and observes ethics in contemporary professional practice from practical, research, and theoretical perspectives.The article also discusses the ways in which the nature of ethics in the field of communication has been changing, and the impact of emerging technology on these changes.
Using ChatGPT as a Static Application Security Testing Tool
Venue: arXiv cs.CR 2023
Year: 2023

In recent years, artificial intelligence has had a conspicuous growth in almost every aspect of life. One of the most applicable areas is security code review, in which a lot of AI-based tools and approaches have been proposed. Recently, ChatGPT has caught a huge amount of attention with its remarkable performance in following instructions and providing a detailed response. Regarding the similarities between natural language and code, in this paper, we study the feasibility of using ChatGPT for vulnerability detection in Python source code. Toward this goal, we feed an appropriate prompt along with vulnerable data to ChatGPT and compare its results on two datasets with the results of three widely used Static Application Security Testing tools (Bandit, Semgrep and SonarQube). We implement different kinds of experiments with ChatGPT and the results indicate that ChatGPT reduces the false positive and false negative rates and has the potential to be used for Python source code vulnerability detection.
An Empirical Study of Consistency Regularization for End-to-End Speech-to-Text Translation
Venue: arXiv cs.CL 2023
Year: 2023

Consistency regularization methods, such as R-Drop (Liang et al., 2021) and CrossConST (Gao et al., 2023), have achieved impressive supervised and zero-shot performance in the neural machine translation (NMT) field. Can we also boost end-to-end (E2E) speech-to-text translation (ST) by leveraging consistency regularization? In this paper, we conduct empirical studies on intra-modal and cross-modal consistency and propose two training strategies, SimRegCR and SimZeroCR, for E2E ST in regular and zero-shot scenarios. Experiments on the MuST-C benchmark show that our approaches achieve state-of-the-art (SOTA) performance in most translation directions. The analyses prove that regularization brought by the intra-modal consistency, instead of modality gap, is crucial for the regular E2E ST, and the cross-modal consistency could close the modality gap and boost the zero-shot E2E ST performance.
Open-VERSO: a vision of 5G experimentation infrastructures, hurdles and challenges
Venue: arXiv cs.NI 2023
Year: 2023

5G led to a digital revolution for networks by leveraging virtualisation techniques to manage software-based network functions through provided standard interfaces, which have matured recently for cloud infrastructure that is widely employed across domains and sectors. This undiscovered potential to adequately respond to concurrent and specialised traffic demands is promising for a wide spectrum of industries. Moreover, it exposes the networking ecosystem to prospects beyond the traditional value chain. However, the configuration, deployment and operation of a 5G network are challenging. Thus, different scientific and research entities have built their own open, evolvable and updateable testbed infrastructure that can be used for experimentation purposes. Such testbeds enable different stakeholders to integrate new systems or features exploiting new technologies, assess the performance of innovative services, and customise operation policies to find optimal setups from a cost-effective perspective. Furthermore, federations of infrastructure allow for wider and more complex experiments to be performed in distributed domains. However, numerous technical and procedural obstacles exist during the building of 5G network testbeds. In addition, some technical barriers persist despite the testing of alternatives and ongoing efforts within open-source systems and commercial equipment portfolios. All these limitations and challenges are relevant for experimenters and stakeholders as they attempt to determine the scope of 5G set expectations.
Time-Frequency Transformer: A Novel Time Frequency Joint Learning Method for Speech Emotion Recognition
Venue: arXiv cs.SD 2023
Year: 2023

In this paper, we propose a novel time-frequency joint learning method for speech emotion recognition, called Time-Frequency Transformer. Its advantage is that the Time-Frequency Transformer can excavate global emotion patterns in the time-frequency domain of speech signal while modeling the local emotional correlations in the time domain and frequency domain respectively. For the purpose, we first design a Time Transformer and Frequency Transformer to capture the local emotion patterns between frames and inside frequency bands respectively, so as to ensure the integrity of the emotion information modeling in both time and frequency domains. Then, a Time-Frequency Transformer is proposed to mine the time-frequency emotional correlations through the local time-domain and frequency-domain emotion features for learning more discriminative global speech emotion representation. The whole process is a time-frequency joint learning process implemented by a series of Transformer models. Experiments on IEMOCAP and CASIA databases indicate that our proposed method outdoes the state-of-the-art methods.
Neural Network Training Strategy to Enhance Anomaly Detection Performance: A Perspective on Reconstruction Loss Amplification
Venue: arXiv cs.CV 2023
Year: 2023

Unsupervised anomaly detection (UAD) is a widely adopted approach in industry due to rare anomaly occurrences and data imbalance. A desirable characteristic of an UAD model is contained generalization ability which excels in the reconstruction of seen normal patterns but struggles with unseen anomalies. Recent studies have pursued to contain the generalization capability of their UAD models in reconstruction from different perspectives, such as design of neural network (NN) structure and training strategy. In contrast, we note that containing of generalization ability in reconstruction can also be obtained simply from steep-shaped loss landscape. Motivated by this, we propose a loss landscape sharpening method by amplifying the reconstruction loss, dubbed Loss AMPlification (LAMP). LAMP deforms the loss landscape into a steep shape so the reconstruction error on unseen anomalies becomes greater. Accordingly, the anomaly detection performance is improved without any change of the NN architecture. Our findings suggest that LAMP can be easily applied to any reconstruction error metrics in UAD settings where the reconstruction model is trained with anomaly-free samples only.
LatentDR: Improving Model Generalization Through Sample-Aware Latent Degradation and Restoration
Venue: arXiv cs.CV 2023
Year: 2023

Despite significant advances in deep learning, models often struggle to generalize well to new, unseen domains, especially when training data is limited. To address this challenge, we propose a novel approach for distribution-aware latent augmentation that leverages the relationships across samples to guide the augmentation procedure. Our approach first degrades the samples stochastically in the latent space, mapping them to augmented labels, and then restores the samples from their corrupted versions during training. This process confuses the classifier in the degradation step and restores the overall class distribution of the original samples, promoting diverse intra-class/cross-domain variability. We extensively evaluate our approach on a diverse set of datasets and tasks, including domain generalization benchmarks and medical imaging datasets with strong domain shift, where we show our approach achieves significant improvements over existing methods for latent space augmentation. We further show that our method can be flexibly adapted to long-tail recognition tasks, demonstrating its versatility in building more generalizable models. Code is available at https://github.com/nerdslab/LatentDR.
Fairness Through Domain Awareness: Mitigating Popularity Bias For Music Discovery
Venue: arXiv cs.CY 2023
Year: 2023

As online music platforms grow, music recommender systems play a vital role in helping users navigate and discover content within their vast musical databases. At odds with this larger goal, is the presence of popularity bias, which causes algorithmic systems to favor mainstream content over, potentially more relevant, but niche items. In this work we explore the intrinsic relationship between music discovery and popularity bias. To mitigate this issue we propose a domain-aware, individual fairness-based approach which addresses popularity bias in graph neural network (GNNs) based recommender systems. Our approach uses individual fairness to reflect a ground truth listening experience, i.e., if two songs sound similar, this similarity should be reflected in their representations. In doing so, we facilitate meaningful music discovery that is robust to popularity bias and grounded in the music domain. We apply our BOOST methodology to two discovery based tasks, performing recommendations at both the playlist level and user level. Then, we ground our evaluation in the cold start setting, showing that our approach outperforms existing fairness benchmarks in both performance and recommendation of lesser-known content. Finally, our analysis explains why our proposed methodology is a novel and promising approach to mitigating popularity bias and improving the discovery of new and niche content in music recommender systems.
MS-Net: A Multi-modal Self-supervised Network for Fine-Grained Classification of Aircraft in SAR Images
Venue: arXiv cs.CV 2023
Year: 2023

Synthetic aperture radar (SAR) imaging technology is commonly used to provide 24-hour all-weather earth observation. However, it still has some drawbacks in SAR target classification, especially in fine-grained classification of aircraft: aircrafts in SAR images have large intra-class diversity and inter-class similarity; the number of effective samples is insufficient and it's hard to annotate. To address these issues, this article proposes a novel multi-modal self-supervised network (MS-Net) for fine-grained classification of aircraft. Firstly, in order to entirely exploit the potential of multi-modal information, a two-sided path feature extraction network (TSFE-N) is constructed to enhance the image feature of the target and obtain the domain knowledge feature of text mode. Secondly, a contrastive self-supervised learning (CSSL) framework is employed to effectively learn useful label-independent feature from unbalanced data, a similarity per-ception loss (SPloss) is proposed to avoid network overfitting. Finally, TSFE-N is used as the encoder of CSSL to obtain the classification results. Through a large number of experiments, our MS-Net can effectively reduce the difficulty of classifying similar types of aircrafts. In the case of no label, the proposed algorithm achieves an accuracy of 88.46% for 17 types of air-craft classification task, which has pioneering significance in the field of fine-grained classification of aircraft in SAR images.
TRIVEA: Transparent Ranking Interpretation using Visual Explanation of Black-Box Algorithmic Rankers
Venue: arXiv cs.IR 2023
Year: 2023

Ranking schemes drive many real-world decisions, like, where to study, whom to hire, what to buy, etc. Many of these decisions often come with high consequences. For example, a university can be deemed less prestigious if not featured in a top-k list, and consumers might not even explore products that do not get recommended to buyers. At the heart of most of these decisions are opaque ranking schemes, which dictate the ordering of data entities, but their internal logic is inaccessible or proprietary. Drawing inferences about the ranking differences is like a guessing game to the stakeholders, like, the rankees (i.e., the entities who are ranked, like product companies) and the decision-makers (i.e., who use the rankings, like buyers). In this paper, we aim to enable transparency in ranking interpretation by using algorithmic rankers that learn from available data and by enabling human reasoning about the learned ranking differences using explainable AI (XAI) methods. To realize this aim, we leverage the exploration-explanation paradigm of human-data interaction to let human stakeholders explore subsets and groupings of complex multi-attribute ranking data using visual explanations of model fit and attribute influence on rankings. We realize this explanation paradigm for transparent ranking interpretation in TRIVEA, a visual analytic system that is fueled by: i) visualizations of model fit derived from algorithmic rankers that learn the associations between attributes and rankings from available data and ii) visual explanations derived from XAI methods that help abstract important patterns, like, the relative influence of attributes in different ranking ranges. Using TRIVEA, end users not trained in data science have the agency to transparently reason about the global and local behavior of the rankings without the need to open black-box ranking models and develop confidence in the resulting attribute-based inferences. We demonstrate the efficacy of TRIVEA using multiple usage scenarios and subjective feedback from researchers with diverse domain expertise. Keywords: Visual Analytics, Learning-to-Rank, Explainable ML, Ranking
Accelerating package expansion in Rust through development of a semantic versioning tool
Venue: arXiv cs.PL 2023
Year: 2023

In many programming languages there exist countless nuances, making developers accidentally release new versions of their packages that are not backwards-compatible. Such releases can directly impact projects which are using their packages, causing bugs or even compilation errors when using the latest version. One of the affected languages is Rust, which also lacks (itself) a built-in mechanism for enforcing semantic versioning. The aim of this thesis is to describe the development of a tool for Rust programmers to reduce the chances of publishing a new version of the code that violates semantic versioning. There are already on-going plans to bundle this tool into the language's standard development toolchain. It would make it commonly used and therefore help users to safely get bug fixes, security patches and new functionality, without worrying about their app being broken by a dependency change.
Zip to Zip-it: Compression to Achieve Local Differential Privacy
Venue: arXiv cs.CR 2023
Year: 2023

Local differential privacy techniques for numerical data typically transform a dataset to ensure a bound on the likelihood that, given a query, a malicious user could infer information on the original samples. Queries are often solely based on users and their requirements, limiting the design of the perturbation to processes that, while privatizing the results, do not jeopardize their usefulness. In this paper, we propose a privatization technique called Zeal, where perturbator and aggregator are designed as a unit, resulting in a locally differentially private mechanism that, by-design, improves the compressibility of the perturbed dataset compared to the original, saves on transmitted bits for data collection and protects against a privacy vulnerabilities due to floating point arithmetic that affect other state-of-the-art schemes. We prove that the utility error on querying the average is invariant to the bias introduced by Zeal in a wide range of conditions, and that under the same circumstances, Zeal also guarantee protection against the aforementioned vulnerability. Our numerical results show up to 94% improvements in compression and up to 95% more efficient data transmissions, while keeping utility errors within 2%.
Joint Active User Detection, Channel Estimation, and Data Detection for Massive Grant-Free Transmission in Cell-Free Systems
Venue: arXiv cs.IT 2023
Year: 2023

Cell-free communication has the potential to significantly improve grant-free transmission in massive machine-type communication, wherein multiple access points jointly serve a large number of user equipments to improve coverage and spectral efficiency. In this paper, we propose a novel framework for joint active user detection (AUD), channel estimation (CE), and data detection (DD) for massive grant-free transmission in cell-free systems. We formulate an optimization problem for joint AUD, CE, and DD by considering both the sparsity of the data matrix, which arises from intermittent user activity, and the sparsity of the effective channel matrix, which arises from intermittent user activity and large-scale fading. We approximately solve this optimization problem with a box-constrained forward-backward splitting algorithm, which significantly improves AUD, CE, and DD performance. We demonstrate the effectiveness of the proposed framework through simulation experiments.
Joint Multiple Intent Detection and Slot Filling with Supervised Contrastive Learning and Self-Distillation
Venue: arXiv cs.CL 2023
Year: 2023

Multiple intent detection and slot filling are two fundamental and crucial tasks in spoken language understanding. Motivated by the fact that the two tasks are closely related, joint models that can detect intents and extract slots simultaneously are preferred to individual models that perform each task independently. The accuracy of a joint model depends heavily on the ability of the model to transfer information between the two tasks so that the result of one task can correct the result of the other. In addition, since a joint model has multiple outputs, how to train the model effectively is also challenging. In this paper, we present a method for multiple intent detection and slot filling by addressing these challenges. First, we propose a bidirectional joint model that explicitly employs intent information to recognize slots and slot features to detect intents. Second, we introduce a novel method for training the proposed joint model using supervised contrastive learning and self-distillation. Experimental results on two benchmark datasets MixATIS and MixSNIPS show that our method outperforms state-of-the-art models in both tasks. The results also demonstrate the contributions of both bidirectional design and the training method to the accuracy improvement. Our source code is available at https://github.com/anhtunguyen98/BiSLU