OpenRaiser/Intern-Atlas huggingface dataset:
Distributed Statistical Min-Max Learning in the Presence of Byzantine Agents
Venue: arXiv cs.LG 2022
Year: 2022

Recent years have witnessed a growing interest in the topic of min-max optimization, owing to its relevance in the context of generative adversarial networks (GANs), robust control and optimization, and reinforcement learning. Motivated by this line of work, we consider a multi-agent min-max learning problem, and focus on the emerging challenge of contending with worst-case Byzantine adversarial agents in such a setup. By drawing on recent results from robust statistics, we design a robust distributed variant of the extra-gradient algorithm - a popular algorithmic approach for min-max optimization. Our main contribution is to provide a crisp analysis of the proposed robust extra-gradient algorithm for smooth convex-concave and smooth strongly convex-strongly concave functions. Specifically, we establish statistical rates of convergence to approximate saddle points. Our rates are near-optimal, and reveal both the effect of adversarial corruption and the benefit of collaboration among the non-faulty agents. Notably, this is the first paper to provide formal theoretical guarantees for large-scale distributed min-max learning in the presence of adversarial agents.
Separation of concerning things: a simpler basis for defining and programming with the C/C++ memory model (extended version)
Venue: arXiv cs.PL 2022
Year: 2022

The C/C++ memory model provides an interface and execution model for programmers of concurrent (shared-variable) code. It provides a range of mechanisms that abstract from underlying hardware memory models -- that govern how multicore architectures handle concurrent accesses to main memory -- as well as abstracting from compiler transformations. The C standard describes the memory model in terms of cross-thread relationships between events, and has been influenced by several research works that are similarly based. In this paper we provide a thread-local definition of the fundamental principles of the C memory model, which, for concise concurrent code, serves as a basis for relatively straightforward reasoning about the effects of the C ordering mechanisms. We argue that this definition is more practical from a programming perspective and is amenable to analysis by already established techniques for concurrent code. The key aspect is that the memory model definition is separate to other considerations of a rich programming language such as C, in particular, expression evaluation and optimisations, though we show how to reason about those considerations in the presence of C concurrency. A major simplification of our framework compared to the description in the C standard and related work in the literature is separating out considerations around the "lack of multicopy atomicity", a concept that is in any case irrelevant to developers of code for x86, Arm, RISC-V or SPARC architectures. We show how the framework is convenient for reasoning about well-structured code, and for formally addressing unintuitive behaviours such as "out-of-thin-air" writes.
mulEEG: A Multi-View Representation Learning on EEG Signals
Venue: arXiv cs.LG 2022
Year: 2022

Modeling effective representations using multiple views that positively influence each other is challenging, and the existing methods perform poorly on Electroencephalogram (EEG) signals for sleep-staging tasks. In this paper, we propose a novel multi-view self-supervised method (mulEEG) for unsupervised EEG representation learning. Our method attempts to effectively utilize the complementary information available in multiple views to learn better representations. We introduce diverse loss that further encourages complementary information across multiple views. Our method with no access to labels beats the supervised training while outperforming multi-view baseline methods on transfer learning experiments carried out on sleep-staging tasks. We posit that our method was able to learn better representations by using complementary multi-views.
Persistent Memory Objects: Fast and Easy Crash Consistency for Persistent Memory
Venue: arXiv cs.OS 2022
Year: 2022

DIMM-compatible persistent memory unites memory and storage. Prior works utilize persistent memory either by combining the filesystem with direct access on memory mapped files or by managing it as a collection of objects while abolishing the POSIX abstraction. In contrast, we propose retaining the POSIX abstraction and extending it to provide support for persistent memory, using Persistent Memory Objects (PMOs). In this work, we design and implement PMOs, a crash-consistent abstraction for managing persistent memory. We introduce psync, a single system call, that a programmer can use to specify crash consistency points in their code, without needing to orchestrate durability explicitly. When rendering data crash consistent, our design incurs a overhead of $\approx 25\%$ and $\approx 21\%$ for parallel workloads and FileBench, respectively, compared to a system without crash consistency. Compared to NOVA-Fortis, our design provides a speedup of $\approx 1.67\times$ and $\approx 3\times$ for the two set of benchmarks, respectively.
Situation Awareness for Autonomous Vehicles Using Blockchain-based Service Cooperation
Venue: arXiv cs.NI 2022
Year: 2022

Efficient Vehicle-to-Everything enabling cooperation and enhanced decision-making for autonomous vehicles is essential for optimized and safe traffic. Real-time decision-making based on vehicle sensor data, other traffic data, and environmental and contextual data becomes imperative. As a part of such Intelligent Traffic Systems, cooperation between different stakeholders needs to be facilitated rapidly, reliably, and securely. The Internet of Things provides the fabric to connect these stakeholders who share their data, refined information, and provided services with each other. However, these cloud-based systems struggle to meet the real-time requirements for smart traffic due to long distances across networks. Here, edge computing systems bring the data and services into the close proximity of fast-moving vehicles, reducing information delivery latencies and improving privacy as sensitive data is processed locally. To solve the issues of trust and latency in data sharing between these stakeholders, we propose a decentralized framework that enables smart contracts between traffic data producers and consumers based on blockchain. Autonomous vehicles connect to a local edge server, share their data, or use services based on agreements, for which the cooperating edge servers across the system provide a platform. We set up proof-of-concept experiments with Hyperledger Fabric and virtual cars to analyze the system throughput with secure unicast and multicast data transmissions. Our results show that multicast transmissions in such a scenario boost the throughput up to 2.5 times where the data packets of different sizes can be transmitted in less than one second.
Three-Module Modeling For End-to-End Spoken Language Understanding Using Pre-trained DNN-HMM-Based Acoustic-Phonetic Model
Venue: arXiv cs.CL 2022
Year: 2022

In spoken language understanding (SLU), what the user says is converted to his/her intent. Recent work on end-to-end SLU has shown that accuracy can be improved via pre-training approaches. We revisit ideas presented by Lugosch et al. using speech pre-training and three-module modeling; however, to ease construction of the end-to-end SLU model, we use as our phoneme module an open-source acoustic-phonetic model from a DNN-HMM hybrid automatic speech recognition (ASR) system instead of training one from scratch. Hence we fine-tune on speech only for the word module, and we apply multi-target learning (MTL) on the word and intent modules to jointly optimize SLU performance. MTL yields a relative reduction of 40% in intent-classification error rates (from 1.0% to 0.6%). Note that our three-module model is a streaming method. The final outcome of the proposed three-module modeling approach yields an intent accuracy of 99.4% on FluentSpeech, an intent error rate reduction of 50% compared to that of Lugosch et al. Although we focus on real-time streaming methods, we also list non-streaming methods for comparison.
A Comprehensive Review of Sign Language Recognition: Different Types, Modalities, and Datasets
Venue: arXiv cs.CV 2022
Year: 2022

A machine can understand human activities, and the meaning of signs can help overcome the communication barriers between the inaudible and ordinary people. Sign Language Recognition (SLR) is a fascinating research area and a crucial task concerning computer vision and pattern recognition. Recently, SLR usage has increased in many applications, but the environment, background image resolution, modalities, and datasets affect the performance a lot. Many researchers have been striving to carry out generic real-time SLR models. This review paper facilitates a comprehensive overview of SLR and discusses the needs, challenges, and problems associated with SLR. We study related works about manual and non-manual, various modalities, and datasets. Research progress and existing state-of-the-art SLR models over the past decade have been reviewed. Finally, we find the research gap and limitations in this domain and suggest future directions. This review paper will be helpful for readers and researchers to get complete guidance about SLR and the progressive design of the state-of-the-art SLR model
Sparse Optical Flow-Based Line Feature Tracking
Venue: arXiv cs.CV 2022
Year: 2022

In this paper we propose a novel sparse optical flow (SOF)-based line feature tracking method for the camera pose estimation problem. This method is inspired by the point-based SOF algorithm and developed based on an observation that two adjacent images in time-varying image sequences satisfy brightness invariant. Based on this observation, we re-define the goal of line feature tracking: track two endpoints of a line feature instead of the entire line based on gray value matching instead of descriptor matching. To achieve this goal, an efficient two endpoint tracking (TET) method is presented: first, describe a given line feature with its two endpoints; next, track the two endpoints based on SOF to obtain two new tracked endpoints by minimizing a pixel-level grayscale residual function; finally, connect the two tracked endpoints to generate a new line feature. The correspondence is established between the given and the new line feature. Compared with current descriptor-based methods, our TET method needs not to compute descriptors and detect line features repeatedly. Naturally, it has an obvious advantage over computation. Experiments in several public benchmark datasets show our method yields highly competitive accuracy with an obvious advantage over speed.
Robust and Explainable Autoencoders for Unsupervised Time Series Outlier Detection---Extended Version
Venue: arXiv cs.LG 2022
Year: 2022

Time series data occurs widely, and outlier detection is a fundamental problem in data mining, which has numerous applications. Existing autoencoder-based approaches deliver state-of-the-art performance on challenging real-world data but are vulnerable to outliers and exhibit low explainability. To address these two limitations, we propose robust and explainable unsupervised autoencoder frameworks that decompose an input time series into a clean time series and an outlier time series using autoencoders. Improved explainability is achieved because clean time series are better explained with easy-to-understand patterns such as trends and periodicities. We provide insight into this by means of a post-hoc explainability analysis and empirical studies. In addition, since outliers are separated from clean time series iteratively, our approach offers improved robustness to outliers, which in turn improves accuracy. We evaluate our approach on five real-world datasets and report improvements over the state-of-the-art approaches in terms of robustness and explainability. This is an extended version of "Robust and Explainable Autoencoders for Unsupervised Time Series Outlier Detection", to appear in IEEE ICDE 2022.
Towards Comparing Performance of Algorithms in Hardware and Software
Venue: arXiv cs.PF 2022
Year: 2022

In this paper, we report on a preliminary investigation of the potential performance gain of programs implemented in field-programmable gate arrays (FPGAs) using a high-level language Chisel compared to ordinary high-level software implementations executed on general-purpose computers and small and cheap computers. FPGAs inherently support parallel evaluations, while sequential computers do not. For this preliminary investigation, we have chosen a highly parallelizable program as a case study to show an upper bound of performance gain. The purpose is to demonstrate whether or not programming FPGAs has the potential for performance optimizations of ordinary programs. We have developed and evaluated Conway's Game of Life for an FPGA, a small and cheap computer Raspberry Pi 4, and a MacBook Pro Laptop. We have compared the performance of programs over different input sizes to decide the relative increase in runtime.
Delta Keyword Transformer: Bringing Transformers to the Edge through Dynamically Pruned Multi-Head Self-Attention
Venue: arXiv cs.CL 2022
Year: 2022

Multi-head self-attention forms the core of Transformer networks. However, their quadratically growing complexity with respect to the input sequence length impedes their deployment on resource-constrained edge devices. We address this challenge by proposing a dynamic pruning method, which exploits the temporal stability of data across tokens to reduce inference cost. The threshold-based method only retains significant differences between the subsequent tokens, effectively reducing the number of multiply-accumulates, as well as the internal tensor data sizes. The approach is evaluated on the Google Speech Commands Dataset for keyword spotting, and the performance is compared against the baseline Keyword Transformer. Our experiments show that we can reduce ~80% of operations while maintaining the original 98.4% accuracy. Moreover, a reduction of ~87-94% operations can be achieved when only degrading the accuracy by 1-4%, speeding up the multi-head self-attention inference by a factor of ~7.5-16.
Commitment games with conditional information disclosure
Venue: arXiv cs.GT 2022
Year: 2022

The conditional commitment abilities of mutually transparent computer agents have been studied in previous work on commitment games and program equilibrium. This literature has shown how these abilities can help resolve Prisoner's Dilemmas and other failures of cooperation in complete information settings. But inefficiencies due to private information have been neglected thus far in this literature, despite the fact that these problems are pervasive and might also be addressed by greater mutual transparency. In this work, we introduce a framework for commitment games with a new kind of conditional commitment device, which agents can use to conditionally disclose private information. We prove a folk theorem for this setting that provides sufficient conditions for ex post efficiency, and thus represents a model of ideal cooperation between agents without a third-party mediator. Connecting our framework with the literature on strategic information revelation, we explore cases where conditional disclosure can be used to achieve full cooperation while unconditional disclosure cannot. Finally, extending previous work on program equilibrium, we develop an implementation of conditional information disclosure. We show that this implementation forms program $ε$-Bayesian Nash equilibria corresponding to the Bayesian Nash equilibria of these commitment games.
QCRI's COVID-19 Disinformation Detector: A System to Fight the COVID-19 Infodemic in Social Media
Venue: arXiv cs.CL 2022
Year: 2022

Fighting the ongoing COVID-19 infodemic has been declared as one of the most important focus areas by the World Health Organization since the onset of the COVID-19 pandemic. While the information that is consumed and disseminated consists of promoting fake cures, rumors, and conspiracy theories to spreading xenophobia and panic, at the same time there is information (e.g., containing advice, promoting cure) that can help different stakeholders such as policy-makers. Social media platforms enable the infodemic and there has been an effort to curate the content on such platforms, analyze and debunk them. While a majority of the research efforts consider one or two aspects (e.g., detecting factuality) of such information, in this study we focus on a multifaceted approach, including an API,\url{https://app.swaggerhub.com/apis/yifan2019/Tanbih/0.8.0/} and a demo system,\url{https://covid19.tanbih.org}, which we made freely and publicly available. We believe that this will facilitate researchers and different stakeholders. A screencast of the API services and demo is available.\url{https://youtu.be/zhbcSvxEKMk}
Practical Issues and Challenges in CSI-based Integrated Sensing and Communication
Venue: arXiv cs.NI 2022
Year: 2022

Next-generation mobile communication network (i.e., 6G) has been envisioned to go beyond classical communication functionality and provide integrated sensing and communication (ISAC) capability to enable more emerging applications, such as smart cities, connected vehicles, AIoT and health care/elder care. Among all the ISAC proposals, the most practical and promising approach is to empower existing wireless network (e.g., WiFi, 4G/5G) with the augmented ability to sense the surrounding human and environment, and evolve wireless communication networks into intelligent communication and sensing network (e.g., 6G). In this paper, based on our experience on CSI-based wireless sensing with WiFi/4G/5G signals, we intend to identify ten major practical and theoretical problems that hinder real deployment of ISAC applications, and provide possible solutions to those critical challenges. Hopefully, this work will inspire further research to evolve existing WiFi/4G/5G networks into next-generation intelligent wireless network (i.e., 6G).
On the Correctness of Speculative Consensus
Venue: arXiv cs.DB 2022
Year: 2022

The introduction of Bitcoin fueled the development of blockchain-based resilient data management systems that are resilient against failures, enable federated data management, and can support data provenance. The key factor determining the performance of such resilient data management systems is the consensus protocol used by the system to replicate client transactions among all participants. Unfortunately, existing high-throughput consensus protocols are costly and impose significant latencies on transaction processing, which rules out their usage in responsive high-performance data management systems. In this work, we improve on this situation by introducing the Proof-of-Execution consensus protocol (PoE), a consensus protocol designed for high-performance low-latency resilient data management. PoE introduces speculative execution, which minimizes latencies by starting execution before consensus is reached, and PoE introduces proof-of-executions to guarantee successful execution to clients. Furthermore, PoE introduces a single-round check-commit protocol to reduce the overall communication costs of consensus. Hence, we believe that PoE is a promising step towards flexible general-purpose low-latency resilient data management systems.
Transfinite Modal Logic: a Semi-quantitative Explanation for Bayesian Reasoning
Venue: arXiv cs.AI 2022
Year: 2022

Bayesian reasoning plays a significant role both in human rationality and in machine learning. In this paper, we introduce transfinite modal logic, which combines modal logic with ordinal arithmetic, in order to formalize Bayesian reasoning semi-quantitatively. Technically, we first investigate some nontrivial properties of ordinal arithmetic, which then enable us to expand normal modal logic's semantics naturally and elegantly onto the novel transfinite modal logic, while still keeping the ordinary definition of Kripke models totally intact. Despite all the transfinite mathematical definition, we argue that in practice, this logic can actually fit into a completely finite interpretation as well. We suggest that transfinite modal logic captures the essence of Bayesian reasoning in a rather clear and simple form, in particular, it provides a perfect explanation for Sherlock Holmes' famous saying, "When you have eliminated the impossible, whatever remains, however improbable, must be the truth." We also prove a counterpart of finite model property theorem for our logic.
tmVar 3.0: an improved variant concept recognition and normalization tool
Venue: arXiv cs.CL 2022
Year: 2022

Previous studies have shown that automated text-mining tools are becoming increasingly important for successfully unlocking variant information in scientific literature at large scale. Despite multiple attempts in the past, existing tools are still of limited recognition scope and precision. We propose tmVar 3.0: an improved variant recognition and normalization tool. Compared to its predecessors, tmVar 3.0 is able to recognize a wide spectrum of variant related entities (e.g., allele and copy number variants), and to group different variant mentions belonging to the same concept in an article for improved accuracy. Moreover, tmVar3 provides additional variant normalization options such as allele-specific identifiers from the ClinGen Allele Registry. tmVar3 exhibits a state-of-the-art performance with over 90% accuracy in F-measure in variant recognition and normalization, when evaluated on three independent benchmarking datasets. tmVar3 is freely available for download. We have also processed the entire PubMed and PMC with tmVar3 and released its annotations on our FTP. Availability: ftp://ftp.ncbi.nlm.nih.gov/pub/lu/tmVar3
Unsupervised Prompt Learning for Vision-Language Models
Venue: arXiv cs.CV 2022
Year: 2022

Contrastive vision-language models like CLIP have shown great progress in transfer learning. In the inference stage, the proper text description, also known as prompt, needs to be carefully designed to correctly classify the given images. In order to avoid laborious prompt engineering, recent works such as CoOp, CLIP-Adapter and Tip-Adapter propose to adapt vision-language models for downstream image recognition tasks on a small set of labeled data. Though promising improvements are achieved, requiring labeled data from the target datasets may restrict the scalability. In this paper, we explore a different scenario, in which the labels of the target datasets are unprovided, and we present an unsupervised prompt learning (UPL) approach to avoid prompt engineering while simultaneously improving transfer performance of CLIP-like vision-language models. As far as we know, UPL is the first work to introduce unsupervised learning into prompt learning. Experimentally, our UPL outperforms original CLIP with prompt engineering on ImageNet as well as other 10 datasets. An enhanced version of UPL is even competitive with the 8-shot CoOp and the 8-shot TIP-Adapter on most datasets. Code and models are available at https://github.com/tonyhuang2022/UPL.