OpenRaiser/Intern-Atlas huggingface dataset:
Active User Detection and Channel Estimation for Spatial-based Random Access in Crowded Massive MIMO Systems via Blind Super-resolution
Venue: arXiv cs.IT 2022
Year: 2022

This work presents a novel framework for random access in crowded scenarios of multiple-input multiple-output(MIMO) systems. A multi-antenna base station (BS) and multiple single-antenna users are considered in these systems. A huge portion of the system resources is dedicated as orthogonal pilots for accurate channel estimation which imposes a huge training overhead. This overhead can be highly mitigated by exploiting intrinsic angular domain sparsity of massive MIMO channels and the sporadic traffic of users, i.e., few number of users are active to sent or receive data in each coherence interval. In fact, the angles of arrivals (AoAs) coming from active users are continuous parameters and can take any arbitrary values. Besides, the AoAs corresponding to each active user are alongside each other forming a specific cluster. This work revolves around exploiting these features. Specifically, a blind clustering-based algorithm is proposed that not only recovers the transmitted data by users in grant free random access and primary pilots in random access blocks of coherent transmission, but also provides accurate channel estimation. Our approach is based on transforming the unknown variables into a higher dimensional space with matrix variables. An off-grid atomic norm minimization is then proposed to obtain the unknown matrix from only a few observed arrays at the BS. Then, a clustering-based approach is employed to identify which AoAs correspond to which active users. After identifying active users and their AoAs, an alternating-based approach is performed to obtain the channels and data or primary pilots of active users. Simulation results demonstrate the effectiveness of our approach in AoA detection as well as data recovery.
A Stochastic Variance-Reduced Coordinate Descent Algorithm for Learning Sparse Bayesian Network from Discrete High-Dimensional Data
Venue: arXiv cs.LG 2022
Year: 2022

This paper addresses the problem of learning a sparse structure Bayesian network from high-dimensional discrete data. Compared to continuous Bayesian networks, learning a discrete Bayesian network is a challenging problem due to the large parameter space. Although many approaches have been developed for learning continuous Bayesian networks, few approaches have been proposed for the discrete ones. In this paper, we address learning Bayesian networks as an optimization problem and propose a score function which guarantees the learnt structure to be a sparse directed acyclic graph. Besides, we implement a block-wised stochastic coordinate descent algorithm to optimize the score function. Specifically, we use a variance reducing method in our optimization algorithm to make the algorithm work efficiently for high-dimensional data. The proposed approach is applied to synthetic data from well-known benchmark networks. The quality, scalability, and robustness of the constructed network are measured. Compared to some competitive approaches, the results reveal that our algorithm outperforms some of the well-known proposed methods.
Construction material classification on imbalanced datasets using Vision Transformer (ViT) architecture
Venue: arXiv cs.CV 2022
Year: 2022

This research proposes a reliable model for identifying different construction materials with the highest accuracy, which is exploited as an advantageous tool for a wide range of construction applications such as automated progress monitoring. In this study, a novel deep learning architecture called Vision Transformer (ViT) is used for detecting and classifying construction materials. The robustness of the employed method is assessed by utilizing different image datasets. For this purpose, the model is trained and tested on two large imbalanced datasets, namely Construction Material Library (CML) and Building Material Dataset (BMD). A third dataset is also generated by combining CML and BMD to create a more imbalanced dataset and assess the capabilities of the utilized method. The achieved results reveal an accuracy of 100 percent in evaluation metrics such as accuracy, precision, recall rate, and f1-score for each material category of three different datasets. It is believed that the suggested model accomplishes a robust tool for detecting and classifying different material types. To date, a number of studies have attempted to automatically classify a variety of building materials, which still have some errors. This research will address the mentioned shortcoming and proposes a model to detect the material type with higher accuracy. The employed model is also capable of being generalized to different datasets.
Rotation Equivariant Operators for Machine Learning on Scalar and Vector Fields
Venue: arXiv cs.LG 2022
Year: 2022

We develop theory and software for rotation equivariant operators on scalar and vector fields, with diverse applications in simulation, optimization and machine learning. Rotation equivariance (covariance) means all fields in the system rotate together, implying spatially invariant dynamics that preserve symmetry. Extending the convolution theorems of linear time invariant systems, we theorize that linear equivariant operators are characterized by tensor field convolutions using an appropriate product between the input field and a radially symmetric kernel field. Most Green's functions and differential operators are in fact equivariant operators, which can also fit unknown symmetry preserving dynamics by parameterizing the radial function. We implement the Julia package EquivariantOperators.jl for fully differentiable finite difference equivariant operators on scalar, vector and higher order tensor fields in 2d/3d. It can run forwards for simulation or image processing, or be back propagated for computer vision, inverse problems and optimal control. Code at https://aced-differentiate.github.io/EquivariantOperators.jl/
EKTVQA: Generalized use of External Knowledge to empower Scene Text in Text-VQA
Venue: arXiv cs.CV 2022
Year: 2022

The open-ended question answering task of Text-VQA often requires reading and reasoning about rarely seen or completely unseen scene-text content of an image. We address this zero-shot nature of the problem by proposing the generalized use of external knowledge to augment our understanding of the scene text. We design a framework to extract, validate, and reason with knowledge using a standard multimodal transformer for vision language understanding tasks. Through empirical evidence and qualitative results, we demonstrate how external knowledge can highlight instance-only cues and thus help deal with training data bias, improve answer entity type correctness, and detect multiword named entities. We generate results comparable to the state-of-the-art on three publicly available datasets, under the constraints of similar upstream OCR systems and training data.
An Exploration of Factors Influencing the Adoption of ICT Enabled Entrepreneurship Applications in Namibian Rural Communities
Venue: arXiv cs.CY 2022
Year: 2022

Digital services have the potential to improve rural entrepreneurs' access to wider markets and increase their competitiveness among other benefits. Moreover, during the ongoing Covid-19 pandemic in which movement and physical contacts have been limited, businesses relied much on digital services. However, many Namibian rural entrepreneurs have not been able to use digital services to maintain their livelihood. Therefore, this study investigated the factors affecting the adoption of ICT enabled services by rural entrepreneurs. The study applied a cross-sectional survey of 77 respondents comprising 14 rural entrepreneurs and 63 rural community members from four sites. It was found that the five main factors affecting the adoption of digital services by rural entrepreneurs are a lack of awareness of digital services, electricity, skills to navigate smart devices, high cost of both devices and mobile internet and cybercrime. We recommend a tailor-made training program for rural entrepreneurs which includes raising awareness of digital services and associated benefits, capacity building on digital skills and best practice for cybersecurity. In addition, we propose that the Namibian Government should enhance digital inclusion through a policy initiative to reduce the cost to make both data and smart devices affordable for the poor and rural communities.
A Framework for Computing Greedy Clique Cover
Venue: arXiv cs.DS 2022
Year: 2022

Structural parameters of graph (such as degeneracy and arboricity) had rarely been considered when designing algorithms for $\textit{(edge) clique cover}$ problems. Taking degeneracy of graph into account, we present a greedy framework and two fixed-parameter tractable algorithms for $\textit{clique cover}$ problems. We introduce a set theoretic concept and demonstrate its use in the computations of different objectives of $\textit{clique cover}$. Furthermore, we show efficacy of our algorithms in practice.
Graph Attention MLP with Reliable Label Utilization
Venue: arXiv cs.LG 2022
Year: 2022

Graph neural networks (GNNs) have recently achieved state-of-the-art performance in many graph-based applications. Despite the high expressive power, they typically need to perform an expensive recursive neighborhood expansion in multiple training epochs and face a scalability issue. Moreover, most of them are inflexible since they are restricted to fixed-hop neighborhoods and insensitive to actual receptive field demands for different nodes. We circumvent these limitations by introducing a scalable and flexible Graph Attention Multilayer Perceptron (GAMLP). With the separation of the non-linear transformation and feature propagation, GAMLP significantly improves the scalability and efficiency by performing the propagation procedure in a pre-compute manner. With three principled receptive field attention, each node in GAMLP is flexible and adaptive in leveraging the propagated features over the different sizes of reception field. We conduct extensive evaluations on the three large open graph benchmarks (e.g., ogbn-papers100M, ogbn-products and ogbn-mag), demonstrating that GAMLP not only achieves the state-of-art performance, but also additionally provide high scalability and efficiency.
Interpreting Face Inference Models using Hierarchical Network Dissection
Venue: arXiv cs.CV 2022
Year: 2022

This paper presents Hierarchical Network Dissection, a general pipeline to interpret the internal representation of face-centric inference models. Using a probabilistic formulation, our pipeline pairs units of the model with concepts in our "Face Dictionary", a collection of facial concepts with corresponding sample images. Our pipeline is inspired by Network Dissection, a popular interpretability model for object-centric and scene-centric models. However, our formulation allows to deal with two important challenges of face-centric models that Network Dissection cannot address: (1) spacial overlap of concepts: there are different facial concepts that simultaneously occur in the same region of the image, like "nose" (facial part) and "pointy nose" (facial attribute); and (2) global concepts: there are units with affinity to concepts that do not refer to specific locations of the face (e.g. apparent age). We use Hierarchical Network Dissection to dissect different face-centric inference models trained on widely-used facial datasets. The results show models trained for different tasks learned different internal representations. Furthermore, the interpretability results can reveal some biases in the training data and some interesting characteristics of the face-centric inference tasks. Finally, we conduct controlled experiments on biased data to showcase the potential of Hierarchical Network Dissection for bias discovery. The results illustrate how Hierarchical Network Dissection can be used to discover and quantify bias in the training data that is also encoded in the model.
A generalized stacked reinforcement learning method for sampled systems
Venue: arXiv cs.RO 2022
Year: 2022

A common setting of reinforcement learning (RL) is a Markov decision process (MDP) in which the environment is a stochastic discrete-time dynamical system. Whereas MDPs are suitable in such applications as video-games or puzzles, physical systems are time-continuous. A general variant of RL is of digital format, where updates of the value (or cost) and policy are performed at discrete moments in time. The agent-environment loop then amounts to a sampled system, whereby sample-and-hold is a specific case. In this paper, we propose and benchmark two RL methods suitable for sampled systems. Specifically, we hybridize model-predictive control (MPC) with critics learning the optimal Q- and value (or cost-to-go) function. Optimality is analyzed and performance comparison is done in an experimental case study with a mobile robot.
Bag of Tricks for Training Deeper Graph Neural Networks: A Comprehensive Benchmark Study
Venue: arXiv cs.LG 2022
Year: 2022

Training deep graph neural networks (GNNs) is notoriously hard. Besides the standard plights in training deep architectures such as vanishing gradients and overfitting, it also uniquely suffers from over-smoothing, information squashing, and so on, which limits their potential power for encoding the high-order neighbor structure in large-scale graphs. Although numerous efforts are proposed to address these limitations, such as various forms of skip connections, graph normalization, and random dropping, it is difficult to disentangle the advantages brought by a deep GNN architecture from those "tricks" necessary to train such an architecture. Moreover, the lack of a standardized benchmark with fair and consistent experimental settings poses an almost insurmountable obstacle to gauge the effectiveness of new mechanisms. In view of those, we present the first fair and reproducible benchmark dedicated to assessing the "tricks" of training deep GNNs. We categorize existing approaches, investigate their hyperparameter sensitivity, and unify the basic configuration. Comprehensive evaluations are then conducted on tens of representative graph datasets including the recent large-scale Open Graph Benchmark, with diverse deep GNN backbones. We demonstrate that an organic combo of initial connection, identity mapping, group and batch normalization attains the new state-of-the-art results for deep GNNs on large datasets. Codes are available: https://github.com/VITA-Group/Deep_GCN_Benchmarking.
An Efficient ADER-DG Local Time Stepping Scheme for 3D HPC Simulation of Seismic Waves in Poroelastic Media
Venue: arXiv cs.DC 2022
Year: 2022

Many applications from geosciences require simulations of seismic waves in porous media. Biot's theory of poroelasticity describes the coupling between solid and fluid phases and introduces a stiff source term, thereby increasing computational cost and motivating efficient methods utilising High-Performance Computing. We present a novel realisation of the discontinuous Galerkin scheme with Arbitrary DERivative time stepping (ADER-DG) that copes with stiff source terms. To integrate this source term with a reasonable time step size, we use an element-local space-time predictor, which needs to solve medium-sized linear systems - with 1000 to 10000 unknowns - in each element update (i.e., billions of times). We present a novel block-wise back-substitution algorithm for solving these systems efficiently. In comparison to LU decomposition, we reduce the number of floating-point operations by a factor of up to 25. The block-wise back-substitution is mapped to a sequence of small matrix-matrix multiplications, for which code generators are available to generate highly optimised code. We verify the new solver thoroughly in problems of increasing complexity. We demonstrate high-order convergence for 3D problems. We verify the correct treatment of point sources, material interfaces and traction-free boundary conditions. In addition, we compare against a finite difference code for a newly defined layer over half-space problem. We find that extremely high accuracy is required to resolve the slow P-wave at a free surface, while solid particle velocities are not affected by coarser resolutions. By using a clustered local time stepping scheme, we reduce time to solution by a factor of 6 to 10 compared to global time stepping. We conclude our study with a scaling and performance analysis, demonstrating our implementation's efficiency and its potential for extreme-scale simulations.
Misleading the Covid-19 vaccination discourse on Twitter: An exploratory study of infodemic around the pandemic
Venue: arXiv cs.CL 2022
Year: 2022

In this work, we collect a moderate-sized representative corpus of tweets (200,000 approx.) pertaining Covid-19 vaccination spanning over a period of seven months (September 2020 - March 2021). Following a Transfer Learning approach, we utilize the pre-trained Transformer-based XLNet model to classify tweets as Misleading or Non-Misleading and validate against a random subset of results manually. We build on this to study and contrast the characteristics of tweets in the corpus that are misleading in nature against non-misleading ones. This exploratory analysis enables us to design features (such as sentiments, hashtags, nouns, pronouns, etc) that can, in turn, be exploited for classifying tweets as (Non-)Misleading using various ML models in an explainable manner. Specifically, several ML models are employed for prediction, with up to 90% accuracy, and the importance of each feature is explained using SHAP Explainable AI (XAI) tool. While the thrust of this work is principally exploratory analysis in order to obtain insights on the online discourse on Covid-19 vaccination, we conclude the paper by outlining how these insights provide the foundations for a more actionable approach to mitigate misinformation. The curated dataset and code is made available (Github repository) so that the research community at large can reproduce, compare against, or build upon this work.
Are socially-aware trajectory prediction models really socially-aware?
Venue: arXiv cs.CV 2022
Year: 2022

Our field has recently witnessed an arms race of neural network-based trajectory predictors. While these predictors are at the core of many applications such as autonomous navigation or pedestrian flow simulations, their adversarial robustness has not been carefully studied. In this paper, we introduce a socially-attended attack to assess the social understanding of prediction models in terms of collision avoidance. An attack is a small yet carefully-crafted perturbations to fail predictors. Technically, we define collision as a failure mode of the output, and propose hard- and soft-attention mechanisms to guide our attack. Thanks to our attack, we shed light on the limitations of the current models in terms of their social understanding. We demonstrate the strengths of our method on the recent trajectory prediction models. Finally, we show that our attack can be employed to increase the social understanding of state-of-the-art models. The code is available online: https://s-attack.github.io/
Cob: a Leaderless Protocol for Parallel Byzantine Agreement in Incomplete Networks
Venue: arXiv cs.DC 2022
Year: 2022

In this paper we extend the \emph{Multidimensional Byzantine Agreement (MBA) Protocol}, a {leaderless} Byzantine agreement for lists of arbitrary values, into a protocol suitable for wide gossiping networks: \emph{Cob}. This generalization allows the consensus process to be run by an incomplete network of nodes provided with (non-synchronized) same-speed clocks. Not all nodes are active in every step, so the network size does not hamper the efficiency, as long as the gossiping broadcast delivers the messages to every node in reasonable time. These network assumptions model more closely real-life communication channels, so the Cob protocol may be applicable to a variety of practical problems, such as blockchain platforms implementing sharding. Cob has the same Bernoulli-like distribution that upper-bounds the number of steps as the MBA protocol. We prove its correctness and security assuming a supermajority of honest nodes in the network, and compare its performance with Algorand.
A Case for Sampling Based Learning Techniques in Coflow Scheduling
Venue: arXiv cs.DC 2022
Year: 2022

Coflow scheduling improves data-intensive application performance by improving their networking performance. State-of-the-art online coflow schedulers in essence approximate the classic Shortest-Job-First (SJF) scheduling by learning the coflow size online. In particular, they use multiple priority queues to simultaneously accomplish two goals: to sieve long coflows from short coflows, and to schedule short coflows with high priorities. Such a mechanism pays high overhead in learning the coflow size: moving a large coflow across the queues delays small and other large coflows, and moving similar-sized coflows across the queues results in inadvertent round-robin scheduling. We propose Philae, a new online coflow scheduler that exploits the spatial dimension of coflows, i.e., a coflow has many flows, to drastically reduce the overhead of coflow size learning. Philae pre-schedules sampled flows of each coflow and uses their sizes to estimate the average flow size of the coflow. It then resorts to Shortest Coflow First, where the notion of shortest is determined using the learned coflow sizes and coflow contention. We show that the sampling-based learning is robust to flow size skew and has the added benefit of much improved scalability from reduced coordinator-local agent interactions. Our evaluation using an Azure testbed, a publicly available production cluster trace from Facebook shows that compared to the prior art Aalo, Philae reduces the coflow completion time (CCT) in average (P90) cases by 1.50x (8.00x) on a 150-node testbed and 2.72x (9.78x) on a 900-node testbed. Evaluation using additional traces further demonstrates Philae's robustness to flow size skew.
ProoFVer: Natural Logic Theorem Proving for Fact Verification
Venue: arXiv cs.CL 2022
Year: 2022

Fact verification systems typically rely on neural network classifiers for veracity prediction which lack explainability. This paper proposes ProoFVer, which uses a seq2seq model to generate natural logic-based inferences as proofs. These proofs consist of lexical mutations between spans in the claim and the evidence retrieved, each marked with a natural logic operator. Claim veracity is determined solely based on the sequence of these operators. Hence, these proofs are faithful explanations, and this makes ProoFVer faithful by construction. Currently, ProoFVer has the highest label accuracy and the second-best Score in the FEVER leaderboard. Furthermore, it improves by 13.21% points over the next best model on a dataset with counterfactual instances, demonstrating its robustness. As explanations, the proofs show better overlap with human rationales than attention-based highlights and the proofs help humans predict model decisions correctly more often than using the evidence directly.
Heavy-tailed Streaming Statistical Estimation
Venue: arXiv cs.LG 2022
Year: 2022

We consider the task of heavy-tailed statistical estimation given streaming $p$-dimensional samples. This could also be viewed as stochastic optimization under heavy-tailed distributions, with an additional $O(p)$ space complexity constraint. We design a clipped stochastic gradient descent algorithm and provide an improved analysis, under a more nuanced condition on the noise of the stochastic gradients, which we show is critical when analyzing stochastic optimization problems arising from general statistical estimation problems. Our results guarantee convergence not just in expectation but with exponential concentration, and moreover does so using $O(1)$ batch size. We provide consequences of our results for mean estimation and linear regression. Finally, we provide empirical corroboration of our results and algorithms via synthetic experiments for mean estimation and linear regression.