OpenRaiser/Intern-Atlas huggingface dataset:
An Extension of Fisher's Criterion: Theoretical Results with a Neural Network Realization
Venue: arXiv cs.LG 2022
Year: 2022

Fisher's criterion is a widely used tool in machine learning for feature selection. For large search spaces, Fisher's criterion can provide a scalable solution to select features. A challenging limitation of Fisher's criterion, however, is that it performs poorly when mean values of class-conditional distributions are close to each other. Motivated by this challenge, we propose an extension of Fisher's criterion to overcome this limitation. The proposed extension utilizes the available heteroscedasticity of class-conditional distributions to distinguish one class from another. Additionally, we describe how our theoretical results can be casted into a neural network framework, and conduct a proof-of-concept experiment to demonstrate the viability of our approach to solve classification problems.
Superimposed Channel Estimation in OTFS Modulation Using Compressive Sensing
Venue: arXiv cs.IT 2022
Year: 2022

Orthogonal time frequency space (OTFS) technique is a two-dimensional modulation method that multiplexes information symbols in the delay-Doppler (DD) domain. OTFS combats high Doppler shift existing in high speed wireless communication. However, conventional channel estimation in OTFS suffers from high pilot overhead because guard symbols occupy a significant part of the DD domain grids. In this paper, a superimposed channel estimation is proposed which can completely estimate channel parameters without considering pilot overhead and performance degradation. As the channel state information (CSI) in the DD domain is sparse, a sparse recovery algorithm orthogonal matching pursuit (OMP) is used. Besides, our proposed method does not suffer from high peak to average power ratio (PAPR). To detect information symbols, a message passing (MP) detector, which exploits the sparsity of DD channel representation, is employed.
Localizability of the approximation method
Venue: arXiv cs.CC 2022
Year: 2022

We use the approximation method of Razborov to analyze the locality barrier which arose from the investigation of the hardness magnification approach to complexity lower bounds. Adapting a limitation of the approximation method obtained by Razborov, we show that in many cases it is not possible to combine the approximation method with typical (localizable) hardness magnification theorems to derive strong circuit lower bounds. In particular, one cannot use the approximation method to derive an extremely strong constant-depth circuit lower bound and then magnify it to an $NC^1$ lower bound for an explicit function. To prove this we show that lower bounds obtained by the approximation method are in many cases localizable in the sense that they imply lower bounds for circuits which are allowed to use arbitrarily powerful oracles with small fan-in.
ChatGPT: The End of Online Exam Integrity?
Venue: arXiv cs.AI 2022
Year: 2022

This study evaluated the ability of ChatGPT, a recently developed artificial intelligence (AI) agent, to perform high-level cognitive tasks and produce text that is indistinguishable from human-generated text. This capacity raises concerns about the potential use of ChatGPT as a tool for academic misconduct in online exams. The study found that ChatGPT is capable of exhibiting critical thinking skills and generating highly realistic text with minimal input, making it a potential threat to the integrity of online exams, particularly in tertiary education settings where such exams are becoming more prevalent. Returning to invigilated and oral exams could form part of the solution, while using advanced proctoring techniques and AI-text output detectors may be effective in addressing this issue, they are not likely to be foolproof solutions. Further research is needed to fully understand the implications of large language models like ChatGPT and to devise strategies for combating the risk of cheating using these tools. It is crucial for educators and institutions to be aware of the possibility of ChatGPT being used for cheating and to investigate measures to address it in order to maintain the fairness and validity of online exams for all students.
E-NER -- An Annotated Named Entity Recognition Corpus of Legal Text
Venue: arXiv cs.CL 2022
Year: 2022

Identifying named entities such as a person, location or organization, in documents can highlight key information to readers. Training Named Entity Recognition (NER) models requires an annotated data set, which can be a time-consuming labour-intensive task. Nevertheless, there are publicly available NER data sets for general English. Recently there has been interest in developing NER for legal text. However, prior work and experimental results reported here indicate that there is a significant degradation in performance when NER methods trained on a general English data set are applied to legal text. We describe a publicly available legal NER data set, called E-NER, based on legal company filings available from the US Securities and Exchange Commission's EDGAR data set. Training a number of different NER algorithms on the general English CoNLL-2003 corpus but testing on our test collection confirmed significant degradations in accuracy, as measured by the F1-score, of between 29.4\% and 60.4\%, compared to training and testing on the E-NER collection.
Diffusing Surrogate Dreams of Video Scenes to Predict Video Memorability
Venue: arXiv cs.CV 2022
Year: 2022

As part of the MediaEval 2022 Predicting Video Memorability task we explore the relationship between visual memorability, the visual representation that characterises it, and the underlying concept portrayed by that visual representation. We achieve state-of-the-art memorability prediction performance with a model trained and tested exclusively on surrogate dream images, elevating concepts to the status of a cornerstone memorability feature, and finding strong evidence to suggest that the intrinsic memorability of visual content can be distilled to its underlying concept or meaning irrespective of its specific visual representational.
Bounds on Mixed Codes with Finite Alphabets
Venue: arXiv cs.IT 2022
Year: 2022

Mixed codes, which are error-correcting codes in the Cartesian product of different-sized spaces, model degrading storage systems well. While such codes have previously been studied for their algebraic properties (e.g., existence of perfect codes) or in the case of unbounded alphabet sizes, we focus on the case of finite alphabets, and generalize the Gilbert-Varshamov, sphere-packing, Elias-Bassalygo, and first linear programming bounds to that setting. In the latter case, our proof is also the first for the non-symmetric mono-alphabetic $q$-ary case using Navon and Samorodnitsky's Fourier-analytic approach.
Dealing with observability in interaction-based Offline Runtime Verification of Distributed Systems
Venue: arXiv cs.LO 2022
Year: 2022

Interactions are formal models describing asynchronous communications within a Distributed System (DS). They can be drawn in the fashion of sequence diagrams and executed thanks to an operational semantics akin to that of process algebras. Executions of DS can be characterized by tuples of local traces (one per subsystem) called multi-traces. For a given execution, those local traces can be collected via monitoring and the resulting multi-trace can be analysed using offline Runtime Verification (RV). To that end, interactions may serve as formal references. In practice, however, not all subsystems may be observed and, without synchronising the end of monitoring on different subsystems, some events may not be observed, e.g. the reception of a message may be observed but not the corresponding emission. So as to be able to consider all such cases of partial observation, we propose an offline RV algorithm which uses removal operations to restrict the reference interaction on-the-fly, disregarding the parts concerning no longer observed subsystems. We prove the correctness of the algorithm and assess the performance of an implementation.
Distilling Vision-Language Pre-training to Collaborate with Weakly-Supervised Temporal Action Localization
Venue: arXiv cs.CV 2022
Year: 2022

Weakly-supervised temporal action localization (WTAL) learns to detect and classify action instances with only category labels. Most methods widely adopt the off-the-shelf Classification-Based Pre-training (CBP) to generate video features for action localization. However, the different optimization objectives between classification and localization, make temporally localized results suffer from the serious incomplete issue. To tackle this issue without additional annotations, this paper considers to distill free action knowledge from Vision-Language Pre-training (VLP), since we surprisingly observe that the localization results of vanilla VLP have an over-complete issue, which is just complementary to the CBP results. To fuse such complementarity, we propose a novel distillation-collaboration framework with two branches acting as CBP and VLP respectively. The framework is optimized through a dual-branch alternate training strategy. Specifically, during the B step, we distill the confident background pseudo-labels from the CBP branch; while during the F step, the confident foreground pseudo-labels are distilled from the VLP branch. And as a result, the dual-branch complementarity is effectively fused to promote a strong alliance. Extensive experiments and ablation studies on THUMOS14 and ActivityNet1.2 reveal that our method significantly outperforms state-of-the-art methods.
Review of security techniques for memristor computing systems
Venue: arXiv cs.CR 2022
Year: 2022

Neural network (NN) algorithms have become the dominant tool in visual object recognition, natural language processing, and robotics. To enhance the computational efficiency of these algorithms, in comparison to the traditional von Neuman computing architectures, researchers have been focusing on memristor computing systems. A major drawback when using memristor computing systems today is that, in the artificial intelligence (AI) era, well-trained NN models are intellectual property and, when loaded in the memristor computing systems, face theft threats, especially when running in edge devices. An adversary may steal the well-trained NN models through advanced attacks such as learning attacks and side-channel analysis. In this paper, we review different security techniques for protecting memristor computing systems. Two threat models are described based on their assumptions regarding the adversary's capabilities: a black-box (BB) model and a white-box (WB) model. We categorize the existing security techniques into five classes in the context of these threat models: thwarting learning attacks (BB), thwarting side-channel attacks (BB), NN model encryption (WB), NN weight transformation (WB), and fingerprint embedding (WB). We also present a cross-comparison of the limitations of the security techniques. This paper could serve as an aid when designing secure memristor computing systems.
Energy-Aware Packet Schedulers for Battery-Less LoRaWAN Nodes
Venue: arXiv cs.NI 2022
Year: 2022

The Internet of Things (IoT) enables a wide variety of applications where large sensor networks are deployed in remote areas without power grid access. Thus, the sensor nodes often run on batteries, whose replacement and disposal represent important economical and environmental costs. To realize more sustainable IoT solutions, it is therefore desirable to adopt battery-less energy-neutral devices that can harvest energy from renewable sources and store it in super-capacitors, whose environmental impact is much lower than that of batteries. To achieve the energetic self-sustainability of such nodes, however, communication and computational processes must be optimized to make the best use of the scarce and volatile energy available. In this work, we propose different energy-aware packet scheduling algorithms for battery-less LoRaWAN nodes, and compare them in various simulated scenarios, using actual energy-harvesting measurements taken from a testbed. We show that an energy-aware design can significantly increase the number of transmitted packets, also lowering the mean time between packet transmissions, though (as predictable) the gain strongly depends on the harvesting capabilities of the nodes.
Positive-incentive Noise
Venue: arXiv cs.LG 2022
Year: 2022

Noise is conventionally viewed as a severe problem in diverse fields, e.g., engineering, learning systems. However, this paper aims to investigate whether the conventional proposition always holds. It begins with the definition of task entropy, which extends from the information entropy and measures the complexity of the task. After introducing the task entropy, the noise can be classified into two kinds, Positive-incentive noise (Pi-noise or $π$-noise) and pure noise, according to whether the noise can reduce the complexity of the task. Interestingly, as shown theoretically and empirically, even the simple random noise can be the $π$-noise that simplifies the task. $π$-noise offers new explanations for some models and provides a new principle for some fields, such as multi-task learning, adversarial training, etc. Moreover, it reminds us to rethink the investigation of noises.
Privacy Adhering Machine Un-learning in NLP
Venue: arXiv cs.CL 2022
Year: 2022

Regulations introduced by General Data Protection Regulation (GDPR) in the EU or California Consumer Privacy Act (CCPA) in the US have included provisions on the \textit{right to be forgotten} that mandates industry applications to remove data related to an individual from their systems. In several real world industry applications that use Machine Learning to build models on user data, such mandates require significant effort both in terms of data cleansing as well as model retraining while ensuring the models do not deteriorate in prediction quality due to removal of data. As a result, continuous removal of data and model retraining steps do not scale if these applications receive such requests at a very high frequency. Recently, a few researchers proposed the idea of \textit{Machine Unlearning} to tackle this challenge. Despite the significant importance of this task, the area of Machine Unlearning is under-explored in Natural Language Processing (NLP) tasks. In this paper, we explore the Unlearning framework on various GLUE tasks \cite{Wang:18}, such as, QQP, SST and MNLI. We propose computationally efficient approaches (SISA-FC and SISA-A) to perform \textit{guaranteed} Unlearning that provides significant reduction in terms of both memory (90-95\%), time (100x) and space consumption (99\%) in comparison to the baselines while keeping model performance constant.
Visconde: Multi-document QA with GPT-3 and Neural Reranking
Venue: arXiv cs.CL 2022
Year: 2022

This paper proposes a question-answering system that can answer questions whose supporting evidence is spread over multiple (potentially long) documents. The system, called Visconde, uses a three-step pipeline to perform the task: decompose, retrieve, and aggregate. The first step decomposes the question into simpler questions using a few-shot large language model (LLM). Then, a state-of-the-art search engine is used to retrieve candidate passages from a large collection for each decomposed question. In the final step, we use the LLM in a few-shot setting to aggregate the contents of the passages into the final answer. The system is evaluated on three datasets: IIRC, Qasper, and StrategyQA. Results suggest that current retrievers are the main bottleneck and that readers are already performing at the human level as long as relevant passages are provided. The system is also shown to be more effective when the model is induced to give explanations before answering a question. Code is available at \url{https://github.com/neuralmind-ai/visconde}.
MultiCoder: Multi-Programming-Lingual Pre-Training for Low-Resource Code Completion
Venue: arXiv cs.CL 2022
Year: 2022

Code completion is a valuable topic in both academia and industry. Recently, large-scale mono-programming-lingual (MonoPL) pre-training models have been proposed to boost the performance of code completion. However, the code completion on low-resource programming languages (PL) is difficult for the data-driven paradigm, while there are plenty of developers using low-resource PLs. On the other hand, there are few studies exploring the effects of multi-programming-lingual (MultiPL) pre-training for the code completion, especially the impact on low-resource programming languages. To this end, we propose the MultiCoder to enhance the low-resource code completion via MultiPL pre-training and MultiPL Mixture-of-Experts (MoE) layers. We further propose a novel PL-level MoE routing strategy (PL-MoE) for improving the code completion on all PLs. Experimental results on CodeXGLUE and MultiCC demonstrate that 1) the proposed MultiCoder significantly outperforms the MonoPL baselines on low-resource programming languages, and 2) the PL-MoE module further boosts the performance on six programming languages. In addition, we analyze the effects of the proposed method in details and explore the effectiveness of our method in a variety of scenarios.
Rate-Privacy-Storage Tradeoff in Federated Learning with Top $r$ Sparsification
Venue: arXiv cs.IT 2022
Year: 2022

We investigate the trade-off between rate, privacy and storage in federated learning (FL) with top $r$ sparsification, where the users and the servers in the FL system only share the most significant $r$ and $r'$ fractions, respectively, of updates and parameters in the FL process, to reduce the communication cost. We present schemes that guarantee information theoretic privacy of the values and indices of the sparse updates sent by the users at the expense of a larger storage cost. To this end, we generalize the scheme to reduce the storage cost by allowing a certain amount of information leakage. Thus, we provide the general trade-off between the communication cost, storage cost, and information leakage in private FL with top $r$ sparsification, along the lines of two proposed schemes.
Panoptic Lifting for 3D Scene Understanding with Neural Fields
Venue: arXiv cs.CV 2022
Year: 2022

We propose Panoptic Lifting, a novel approach for learning panoptic 3D volumetric representations from images of in-the-wild scenes. Once trained, our model can render color images together with 3D-consistent panoptic segmentation from novel viewpoints. Unlike existing approaches which use 3D input directly or indirectly, our method requires only machine-generated 2D panoptic segmentation masks inferred from a pre-trained network. Our core contribution is a panoptic lifting scheme based on a neural field representation that generates a unified and multi-view consistent, 3D panoptic representation of the scene. To account for inconsistencies of 2D instance identifiers across views, we solve a linear assignment with a cost based on the model's current predictions and the machine-generated segmentation masks, thus enabling us to lift 2D instances to 3D in a consistent way. We further propose and ablate contributions that make our method more robust to noisy, machine-generated labels, including test-time augmentations for confidence estimates, segment consistency loss, bounded segmentation fields, and gradient stopping. Experimental results validate our approach on the challenging Hypersim, Replica, and ScanNet datasets, improving by 8.4, 13.8, and 10.6% in scene-level PQ over state of the art.
Analysis of Sparse Recovery Algorithms via the Replica Method
Venue: arXiv cs.IT 2022
Year: 2022

This manuscript goes through the fundamental connections between statistical mechanics and estimation theory by focusing on the particular problem of compressive sensing. We first show that the asymptotic analysis of a sparse recovery algorithm is mathematically equivalent to the problem of calculating the free energy of a spin glass in the thermodynamic limit. We then use the replica method from statistical mechanics to evaluate the performance in the asymptotic regime. The asymptotic results have several applications in communications and signal processing. We briefly go through two instances of these applications: Characterization of joint sparse recovery algorithms used in distributed compressive sensing, and tuning of receivers employed for detection of spatially modulated signals.
Improved Long-Form Spoken Language Translation with Large Language Models
Venue: arXiv cs.CL 2022
Year: 2022

A challenge in spoken language translation is that plenty of spoken content is long-form, but short units are necessary for obtaining high-quality translations. To address this mismatch, we fine-tune a general-purpose, large language model to split long ASR transcripts into segments that can be independently translated so as to maximize the overall translation quality. We compare to several segmentation strategies and find that our approach improves BLEU score on three languages by an average of 2.7 BLEU overall compared to an automatic punctuation baseline. Further, we demonstrate the effectiveness of two constrained decoding strategies to improve well-formedness of the model output from above 99% to 100%.
Flareon: Stealthy any2any Backdoor Injection via Poisoned Augmentation
Venue: arXiv cs.CR 2022
Year: 2022

Open software supply chain attacks, once successful, can exact heavy costs in mission-critical applications. As open-source ecosystems for deep learning flourish and become increasingly universal, they present attackers previously unexplored avenues to code-inject malicious backdoors in deep neural network models. This paper proposes Flareon, a small, stealthy, seemingly harmless code modification that specifically targets the data augmentation pipeline with motion-based triggers. Flareon neither alters ground-truth labels, nor modifies the training loss objective, nor does it assume prior knowledge of the victim model architecture, training data, and training hyperparameters. Yet, it has a surprisingly large ramification on training -- models trained under Flareon learn powerful target-conditional (or "any2any") backdoors. The resulting models can exhibit high attack success rates for any target choices and better clean accuracies than backdoor attacks that not only seize greater control, but also assume more restrictive attack capabilities. We also demonstrate the effectiveness of Flareon against recent defenses. Flareon is fully open-source and available online to the deep learning community: https://github.com/lafeat/flareon.
Benchmarking person re-identification datasets and approaches for practical real-world implementations
Venue: arXiv cs.CV 2022
Year: 2022

Recently, Person Re-Identification (Re-ID) has received a lot of attention. Large datasets containing labeled images of various individuals have been released, allowing researchers to develop and test many successful approaches. However, when such Re-ID models are deployed in new cities or environments, the task of searching for people within a network of security cameras is likely to face an important domain shift, thus resulting in decreased performance. Indeed, while most public datasets were collected in a limited geographic area, images from a new city present different features (e.g., people's ethnicity and clothing style, weather, architecture, etc.). In addition, the whole frames of the video streams must be converted into cropped images of people using pedestrian detection models, which behave differently from the human annotators who created the dataset used for training. To better understand the extent of this issue, this paper introduces a complete methodology to evaluate Re-ID approaches and training datasets with respect to their suitability for unsupervised deployment for live operations. This method is used to benchmark four Re-ID approaches on three datasets, providing insight and guidelines that can help to design better Re-ID pipelines in the future.