OpenRaiser/Intern-Atlas huggingface dataset:
Relational Representation Learning in Visually-Rich Documents
Venue: arXiv cs.CL 2022
Year: 2022

Relational understanding is critical for a number of visually-rich documents (VRDs) understanding tasks. Through multi-modal pre-training, recent studies provide comprehensive contextual representations and exploit them as prior knowledge for downstream tasks. In spite of their impressive results, we observe that the widespread relational hints (e.g., relation of key/value fields on receipts) built upon contextual knowledge are not excavated yet. To mitigate this gap, we propose DocReL, a Document Relational Representation Learning framework. The major challenge of DocReL roots in the variety of relations. From the simplest pairwise relation to the complex global structure, it is infeasible to conduct supervised training due to the definition of relation varies and even conflicts in different tasks. To deal with the unpredictable definition of relations, we propose a novel contrastive learning task named Relational Consistency Modeling (RCM), which harnesses the fact that existing relations should be consistent in differently augmented positive views. RCM provides relational representations which are more compatible to the urgent need of downstream tasks, even without any knowledge about the exact definition of relation. DocReL achieves better performance on a wide variety of VRD relational understanding tasks, including table structure recognition, key information extraction and reading order detection.
Fast Left Kan Extensions Using The Chase
Venue: arXiv cs.DB 2022
Year: 2022

We show how computation of left Kan extensions can be reduced to computation of free models of cartesian (finite-limit) theories. We discuss how the standard and parallel chase compute weakly free models of regular theories and free models of cartesian theories, and compare the concept of "free model" with a similar concept from database theory known as "universal model". We prove that, as algorithms for computing finite free models of cartesian theories, the standard and parallel chase are complete under fairness assumptions. Finally, we describe an optimized implementation of the parallel chase specialized to left Kan extensions that achieves an order of magnitude improvement in our performance benchmarks compared to the next fastest left Kan extension algorithm we are aware of.
Uncertainty Minimization for Personalized Federated Semi-Supervised Learning
Venue: arXiv cs.LG 2022
Year: 2022

Since federated learning (FL) has been introduced as a decentralized learning technique with privacy preservation, statistical heterogeneity of distributed data stays the main obstacle to achieve robust performance and stable convergence in FL applications. Model personalization methods have been studied to overcome this problem. However, existing approaches are mainly under the prerequisite of fully labeled data, which is unrealistic in practice due to the requirement of expertise. The primary issue caused by partial-labeled condition is that, clients with deficient labeled data can suffer from unfair performance gain because they lack adequate insights of local distribution to customize the global model. To tackle this problem, 1) we propose a novel personalized semi-supervised learning paradigm which allows partial-labeled or unlabeled clients to seek labeling assistance from data-related clients (helper agents), thus to enhance their perception of local data; 2) based on this paradigm, we design an uncertainty-based data-relation metric to ensure that selected helpers can provide trustworthy pseudo labels instead of misleading the local training; 3) to mitigate the network overload introduced by helper searching, we further develop a helper selection protocol to achieve efficient communication with acceptable performance sacrifice. Experiments show that our proposed method can obtain superior performance and more stable convergence than other related works with partially labeled data, especially in highly heterogeneous setting.
COGMEN: COntextualized GNN based Multimodal Emotion recognitioN
Venue: arXiv cs.CL 2022
Year: 2022

Emotions are an inherent part of human interactions, and consequently, it is imperative to develop AI systems that understand and recognize human emotions. During a conversation involving various people, a person's emotions are influenced by the other speaker's utterances and their own emotional state over the utterances. In this paper, we propose COntextualized Graph Neural Network based Multimodal Emotion recognitioN (COGMEN) system that leverages local information (i.e., inter/intra dependency between speakers) and global information (context). The proposed model uses Graph Neural Network (GNN) based architecture to model the complex dependencies (local and global information) in a conversation. Our model gives state-of-the-art (SOTA) results on IEMOCAP and MOSEI datasets, and detailed ablation experiments show the importance of modeling information at both levels.
dPRO: A Generic Profiling and Optimization System for Expediting Distributed DNN Training
Venue: arXiv cs.DC 2022
Year: 2022

Distributed training using multiple devices (e.g., GPUs) has been widely adopted for learning DNN models over large datasets. However, the performance of large-scale distributed training tends to be far from linear speed-up in practice. Given the complexity of distributed systems, it is challenging to identify the root cause(s) of inefficiency and exercise effective performance optimizations when unexpected low training speed occurs. To date, there exists no software tool which diagnoses performance issues and helps expedite distributed DNN training, while the training can be run using different deep learning frameworks. This paper proposes dPRO, a toolkit that includes: (1) an efficient profiler that collects runtime traces of distributed DNN training across multiple frameworks, especially fine-grained communication traces, and constructs global data flow graphs including detailed communication operations for accurate replay; (2) an optimizer that effectively identifies performance bottlenecks and explores optimization strategies (from computation, communication, and memory aspects) for training acceleration. We implement dPRO on multiple deep learning frameworks (TensorFlow, MXNet) and representative communication schemes (AllReduce and Parameter Server). Extensive experiments show that dPRO predicts the performance of distributed training in various settings with < 5% errors in most cases and finds optimization strategies with up to 3.48x speed-up over the baselines.
Scheduling Coflows with Precedence Constraints for Minimizing the Total Weighted Completion Time in Identical Parallel Networks
Venue: arXiv cs.DS 2022
Year: 2022

Coflow is a recently proposed network abstraction for data-parallel computing applications. This paper considers scheduling coflows with precedence constraints in identical parallel networks, such as to minimize the total weighted completion time of coflows. The identical parallel network is an architecture based on multiple network cores running in parallel. In the divisible coflow scheduling problem, the proposed algorithm achieves $(6-\frac{2}{m})μ$ and $(5-\frac{2}{m})μ$ approximate ratios for arbitrary release time and zero release time, respectively, where $m$ is the number of network cores and $μ$ is the coflow number of the longest path in the precedence graph. In the indivisible coflow scheduling problem, the proposed algorithm achieves $(4m+1)μ$ and $4mμ$ approximate ratios for arbitrary release time and zero release time, respectively. In the single network core scheduling problem, we propose a $5μ$-approximation algorithm with arbitrary release times, and a $4μ$-approximation without release time. Moreover, the proposed algorithm can be modified to solve the coflows of multi-stage jobs scheduling problem. In multi-stage jobs, coflow is transferred between servers to enable starting of next stage. This means that there are precedence constraints between coflows of job. Our result represents an improvement upon the previous best approximation ratio of $O(\tildeμ \log(N)/ \log(\log(N)))$ where $\tildeμ$ is the maximum number of coflows in a job and $N$ is the number of servers.
Speaker Recognition in the Wild
Venue: arXiv cs.SD 2022
Year: 2022

In this paper, we propose a pipeline to find the number of speakers, as well as audios belonging to each of these now identified speakers in a source of audio data where number of speakers or speaker labels are not known a priori. We used this approach as a part of our Data Preparation pipeline for Speech Recognition in Indic Languages (https://github.com/Open-Speech-EkStep/vakyansh-wav2vec2-experimentation). To understand and evaluate the accuracy of our proposed pipeline, we introduce two metrics: Cluster Purity, and Cluster Uniqueness. Cluster Purity quantifies how "pure" a cluster is. Cluster Uniqueness, on the other hand, quantifies what percentage of clusters belong only to a single dominant speaker. We discuss more on these metrics in section \ref{sec:metrics}. Since we develop this utility to aid us in identifying data based on speaker IDs before training an Automatic Speech Recognition (ASR) model, and since most of this data takes considerable effort to scrape, we also conclude that 98\% of data gets mapped to the top 80\% of clusters (computed by removing any clusters with less than a fixed number of utterances -- we do this to get rid of some very small clusters and use this threshold as 30), in the test set chosen.
Exploiting Correspondences with All-pairs Correlations for Multi-view Depth Estimation
Venue: arXiv cs.CV 2022
Year: 2022

Multi-view depth estimation plays a critical role in reconstructing and understanding the 3D world. Recent learning-based methods have made significant progress in it. However, multi-view depth estimation is fundamentally a correspondence-based optimization problem, but previous learning-based methods mainly rely on predefined depth hypotheses to build correspondence as the cost volume and implicitly regularize it to fit depth prediction, deviating from the essence of iterative optimization based on stereo correspondence. Thus, they suffer unsatisfactory precision and generalization capability. In this paper, we are the first to explore more general image correlations to establish correspondences dynamically for depth estimation. We design a novel iterative multi-view depth estimation framework mimicking the optimization process, which consists of 1) a correlation volume construction module that models the pixel similarity between a reference image and source images as all-to-all correlations; 2) a flow-based depth initialization module that estimates the depth from the 2D optical flow; 3) a novel correlation-guided depth refinement module that reprojects points in different views to effectively fetch relevant correlations for further fusion and integrate the fused correlation for iterative depth update. Without predefined depth hypotheses, the fused correlations establish multi-view correspondence in an efficient way and guide the depth refinement heuristically. We conduct sufficient experiments on ScanNet, DeMoN, ETH3D, and 7Scenes to demonstrate the superiority of our method on multi-view depth estimation and its best generalization ability.
ChASE -- A Distributed Hybrid CPU-GPU Eigensolver for Large-scale Hermitian Eigenvalue Problems
Venue: arXiv cs.DC 2022
Year: 2022

As modern massively parallel clusters are getting larger with beefier compute nodes, traditional parallel eigensolvers, such as direct solvers, struggle keeping the pace with the hardware evolution and being able to scale efficiently due to additional layers of communication and synchronization. This difficulty is especially important when porting traditional libraries to heterogeneous computing architectures equipped with accelerators, such as Graphics Processing Unit (GPU). Recently, there have been significant scientific contributions to the development of filter-based subspace eigensolver to compute partial eigenspectrum. The simpler structure of these type of algorithms makes for them easier to avoid the communication and synchronization bottlenecks typical of direct solvers. The Chebyshev Accelerated Subspace Eigensolver (ChASE) is a modern subspace eigensolver to compute partial extremal eigenpairs of large-scale Hermitian eigenproblems with the acceleration of a filter based on Chebyshev polynomials. In this work, we extend our previous work on ChASE by adding support for distributed hybrid CPU-multi-GPU computing architectures. Our tests show that ChASE achieves very good scaling performance up to 144 nodes with 526 NVIDIA A100 GPUs in total on dense eigenproblems of size up to $360$k.
Parallel Parking: Optimal Entry and Minimum Slot Dimensions
Venue: arXiv cs.RO 2022
Year: 2022

The problem of path planning for automated parking is usually presented as finding a collision-free path from initial to goal positions, where three out of four parking slot edges represent obstacles. We rethink the path planning problem for parallel parking by decomposing it into two independent parts. The topic of this paper is finding optimal parking slot entry positions. Path planning from initial to entry position is out of scope here. We show the relation between entry positions, parking slot dimensions, and the number of backward-forward direction changes. This information can be used as an input to optimize other parts of the automated parking process.
Rate-Splitting Multiple Access for 6G -- Part I: Principles, Applications and Future Works
Venue: arXiv cs.IT 2022
Year: 2022

This letter is the first part of a three-part tutorial focusing on rate-splitting multiple access (RSMA) for 6G. As Part I of the tutorial, the letter presents the basics of RSMA and its applications in light of 6G. To begin with, we first delineate the design principle and basic transmission frameworks of downlink and uplink RSMA. We then illustrate the applications of RSMA for addressing the challenges of various potential enabling technologies and use cases, consequently making it a promising next generation multiple access (NGMA) scheme for future networks such as 6G and beyond. We briefly discuss the challenges of RSMA and conclude the letter. In continuation of Part I, we will focus on the interplay of RSMA with integrated sensing and communication, and reconfigurable intelligent surfaces, respectively in Part II and Part III of this tutorial.
Monitoring AI systems: A Problem Analysis, Framework and Outlook
Venue: arXiv cs.AI 2022
Year: 2022

Knowledge-based systems have been used to monitor machines and processes in the real world. In this paper we propose the use of knowledge-based systems to monitor other AI systems in operation. We motivate and provide a problem analysis of this novel setting and subsequently propose a framework that allows for structuring future research related to this setting. Several directions for further research are also discussed.
DropTrack -- automatic droplet tracking using deep learning for microfluidic applications
Venue: arXiv cs.CV 2022
Year: 2022

Deep neural networks are rapidly emerging as data analysis tools, often outperforming the conventional techniques used in complex microfluidic systems. One fundamental analysis frequently desired in microfluidic experiments is counting and tracking the droplets. Specifically, droplet tracking in dense emulsions is challenging as droplets move in tightly packed configurations. Sometimes the individual droplets in these dense clusters are hard to resolve, even for a human observer. Here, two deep learning-based cutting-edge algorithms for object detection (YOLO) and object tracking (DeepSORT) are combined into a single image analysis tool, DropTrack, to track droplets in microfluidic experiments. DropTrack analyzes input videos, extracts droplets' trajectories, and infers other observables of interest, such as droplet numbers. Training an object detector network for droplet recognition with manually annotated images is a labor-intensive task and a persistent bottleneck. This work partly resolves this problem by training object detector networks (YOLOv5) with hybrid datasets containing real and synthetic images. We present an analysis of a double emulsion experiment as a case study to measure DropTrack's performance. For our test case, the YOLO networks trained with 60% synthetic images show similar performance in droplet counting as with the one trained using 100% real images, meanwhile saving the image annotation work by 60%. DropTrack's performance is measured in terms of mean average precision (mAP), mean square error in counting the droplets, and inference speed. The fastest configuration of DropTrack runs inference at about 30 frames per second, well within the standards for real-time image analysis.
Byzantine Fault Tolerance in Distributed Machine Learning : a Survey
Venue: arXiv cs.DC 2022
Year: 2022

Byzantine Fault Tolerance (BFT) is one of the most challenging problems in Distributed Machine Learning (DML), defined as the resilience of a fault-tolerant system in the presence of malicious components. Byzantine failures are still difficult to deal with due to their unrestricted nature, which results in the possibility of generating arbitrary data. Significant research efforts are constantly being made to implement BFT in DML. Some recent studies have considered various BFT approaches in DML. However, some aspects are limited, such as the few approaches analyzed, and there is no classification of the techniques used in the studied approaches. In this paper, we present a survey of recent work surrounding BFT in DML, mainly in first-order optimization methods, especially Stochastic Gradient Descent(SGD). We highlight key techniques as well as fundamental approaches. We provide an illustrative description of the techniques used in BFT in DML, with a proposed classification of BFT approaches in the context of their fundamental techniques. This classification is established on specific criteria such as communication process, optimization method, and topology setting, which characterize future work methods addressing open challenge
METGEN: A Module-Based Entailment Tree Generation Framework for Answer Explanation
Venue: arXiv cs.CL 2022
Year: 2022

Knowing the reasoning chains from knowledge to the predicted answers can help construct an explainable question answering (QA) system. Advances on QA explanation propose to explain the answers with entailment trees composed of multiple entailment steps. While current work proposes to generate entailment trees with end-to-end generative models, the steps in the generated trees are not constrained and could be unreliable. In this paper, we propose METGEN, a Module-based Entailment Tree GENeration framework that has multiple modules and a reasoning controller. Given a question and several supporting knowledge, METGEN can iteratively generate the entailment tree by conducting single-step entailment with separate modules and selecting the reasoning flow with the controller. As each module is guided to perform a specific type of entailment reasoning, the steps generated by METGEN are more reliable and valid. Experiment results on the standard benchmark show that METGEN can outperform previous state-of-the-art models with only 9% of the parameters.
Multi-Agent Deep Reinforcement Learning in Vehicular OCC
Venue: arXiv cs.LG 2022
Year: 2022

Optical camera communications (OCC) has emerged as a key enabling technology for the seamless operation of future autonomous vehicles. In this paper, we introduce a spectral efficiency optimization approach in vehicular OCC. Specifically, we aim at optimally adapting the modulation order and the relative speed while respecting bit error rate and latency constraints. As the optimization problem is NP-hard problem, we model the optimization problem as a Markov decision process (MDP) to enable the use of solutions that can be applied online. We then relaxed the constrained problem by employing Lagrange relaxation approach before solving it by multi-agent deep reinforcement learning (DRL). We verify the performance of our proposed scheme through extensive simulations and compare it with various variants of our approach and a random method. The evaluation shows that our system achieves significantly higher sum spectral efficiency compared to schemes under comparison.
Sparse Large-Scale Fading Decoding in Cell-Free Massive MIMO Systems
Venue: arXiv cs.IT 2022
Year: 2022

Cell-free massive multiple-input multiple-output (CF mMIMO) systems are characterized by having many more access points (APs) than user equipments (UEs). A key challenge is to determine which APs should serve which UEs. Previous work has tackled this combinatorial problem heuristically. This paper proposes a sparse large-scale fading decoding (LSFD) design for CF mMIMO to jointly optimize the association and LSFD. We formulate a group sparsity problem and then solve it using a proximal algorithm with block-coordinate descent. Numerical results show that sparse LSFD achieves almost the same spectral efficiency as optimal LSFD, thus achieving a higher energy efficiency since the processing and signaling are reduced.
CE-based white-box adversarial attacks will not work using super-fitting
Venue: arXiv cs.LG 2022
Year: 2022

Deep neural networks are widely used in various fields because of their powerful performance. However, recent studies have shown that deep learning models are vulnerable to adversarial attacks, i.e., adding a slight perturbation to the input will make the model obtain wrong results. This is especially dangerous for some systems with high-security requirements, so this paper proposes a new defense method by using the model super-fitting state to improve the model's adversarial robustness (i.e., the accuracy under adversarial attacks). This paper mathematically proves the effectiveness of super-fitting and enables the model to reach this state quickly by minimizing unrelated category scores (MUCS). Theoretically, super-fitting can resist any existing (even future) CE-based white-box adversarial attacks. In addition, this paper uses a variety of powerful attack algorithms to evaluate the adversarial robustness of super-fitting, and the proposed method is compared with nearly 50 defense models from recent conferences. The experimental results show that the super-fitting method in this paper can make the trained model obtain the highest adversarial robustness.
Rethinking Classifier and Adversarial Attack
Venue: arXiv cs.LG 2022
Year: 2022

Various defense models have been proposed to resist adversarial attack algorithms, but existing adversarial robustness evaluation methods always overestimate the adversarial robustness of these models (i.e., not approaching the lower bound of robustness). To solve this problem, this paper uses the proposed decouple space method to divide the classifier into two parts: non-linear and linear. Then, this paper defines the representation vector of the original example (and its space, i.e., the representation space) and uses the iterative optimization of Absolute Classification Boundaries Initialization (ACBI) to obtain a better attack starting point. Particularly, this paper applies ACBI to nearly 50 widely-used defense models (including 8 architectures). Experimental results show that ACBI achieves lower robust accuracy in all cases.