OpenRaiser/Intern-Atlas huggingface dataset:
Social Diffusion Sources Can Escape Detection
Venue: arXiv cs.SI 2021
Year: 2021

Influencing (and being influenced by) others through social networks is fundamental to all human societies. Whether this happens through the diffusion of rumors, opinions, or viruses, identifying the diffusion source (i.e., the person that initiated it) is a problem that has attracted much research interest. Nevertheless, existing literature has ignored the possibility that the source might strategically modify the network structure (by rewiring links or introducing fake nodes) to escape detection. Here, without restricting our analysis to any particular diffusion scenario, we close this gap by evaluating two mechanisms that hide the source-one stemming from the source's actions, the other from the network structure itself. This reveals that sources can easily escape detection, and that removing links is far more effective than introducing fake nodes. Thus, efforts should focus on exposing concealed ties rather than planted entities; such exposure would drastically improve our chances of detecting the diffusion source.
Mastering Terra Mystica: Applying Self-Play to Multi-agent Cooperative Board Games
Venue: arXiv cs.MA 2021
Year: 2021

In this paper, we explore and compare multiple algorithms for solving the complex strategy game of Terra Mystica, hereafter abbreviated as TM. Previous work in the area of super-human game-play using AI has proven effective, with recent break-through for generic algorithms in games such as Go, Chess, and Shogi \cite{AlphaZero}. We directly apply these breakthroughs to a novel state-representation of TM with the goal of creating an AI that will rival human players. Specifically, we present the initial results of applying AlphaZero to this state-representation and analyze the strategies developed. A brief analysis is presented. We call this modified algorithm with our novel state-representation AlphaTM. In the end, we discuss the success and shortcomings of this method by comparing against multiple baselines and typical human scores. All code used for this paper is available at on \href{https://github.com/kandluis/terrazero}{GitHub}.
Rethinking Content and Style: Exploring Bias for Unsupervised Disentanglement
Venue: arXiv cs.CV 2021
Year: 2021

Content and style (C-S) disentanglement intends to decompose the underlying explanatory factors of objects into two independent subspaces. From the unsupervised disentanglement perspective, we rethink content and style and propose a formulation for unsupervised C-S disentanglement based on our assumption that different factors are of different importance and popularity for image reconstruction, which serves as a data bias. The corresponding model inductive bias is introduced by our proposed C-S disentanglement Module (C-S DisMo), which assigns different and independent roles to content and style when approximating the real data distributions. Specifically, each content embedding from the dataset, which encodes the most dominant factors for image reconstruction, is assumed to be sampled from a shared distribution across the dataset. The style embedding for a particular image, encoding the remaining factors, is used to customize the shared distribution through an affine transformation. The experiments on several popular datasets demonstrate that our method achieves the state-of-the-art unsupervised C-S disentanglement, which is comparable or even better than supervised methods. We verify the effectiveness of our method by downstream tasks: domain translation and single-view 3D reconstruction. Project page at https://github.com/xrenaa/CS-DisMo.
A Comprehensive Review of Computer-aided Whole-slide Image Analysis: from Datasets to Feature Extraction, Segmentation, Classification, and Detection Approaches
Venue: arXiv cs.CV 2021
Year: 2021

With the development of computer-aided diagnosis (CAD) and image scanning technology, Whole-slide Image (WSI) scanners are widely used in the field of pathological diagnosis. Therefore, WSI analysis has become the key to modern digital pathology. Since 2004, WSI has been used more and more in CAD. Since machine vision methods are usually based on semi-automatic or fully automatic computers, they are highly efficient and labor-saving. The combination of WSI and CAD technologies for segmentation, classification, and detection helps histopathologists obtain more stable and quantitative analysis results, save labor costs and improve diagnosis objectivity. This paper reviews the methods of WSI analysis based on machine learning. Firstly, the development status of WSI and CAD methods are introduced. Secondly, we discuss publicly available WSI datasets and evaluation metrics for segmentation, classification, and detection tasks. Then, the latest development of machine learning in WSI segmentation, classification, and detection are reviewed continuously. Finally, the existing methods are studied, the applicabilities of the analysis methods are analyzed, and the application prospects of the analysis methods in this field are forecasted.
Efficient Two-Stream Network for Violence Detection Using Separable Convolutional LSTM
Venue: arXiv cs.CV 2021
Year: 2021

Automatically detecting violence from surveillance footage is a subset of activity recognition that deserves special attention because of its wide applicability in unmanned security monitoring systems, internet video filtration, etc. In this work, we propose an efficient two-stream deep learning architecture leveraging Separable Convolutional LSTM (SepConvLSTM) and pre-trained MobileNet where one stream takes in background suppressed frames as inputs and other stream processes difference of adjacent frames. We employed simple and fast input pre-processing techniques that highlight the moving objects in the frames by suppressing non-moving backgrounds and capture the motion in-between frames. As violent actions are mostly characterized by body movements these inputs help produce discriminative features. SepConvLSTM is constructed by replacing convolution operation at each gate of ConvLSTM with a depthwise separable convolution that enables producing robust long-range Spatio-temporal features while using substantially fewer parameters. We experimented with three fusion methods to combine the output feature maps of the two streams. Evaluation of the proposed methods was done on three standard public datasets. Our model outperforms the accuracy on the larger and more challenging RWF-2000 dataset by more than a 2% margin while matching state-of-the-art results on the smaller datasets. Our experiments lead us to conclude, the proposed models are superior in terms of both computational efficiency and detection accuracy.
Medical Transformer: Gated Axial-Attention for Medical Image Segmentation
Venue: arXiv cs.CV 2021
Year: 2021

Over the past decade, Deep Convolutional Neural Networks have been widely adopted for medical image segmentation and shown to achieve adequate performance. However, due to the inherent inductive biases present in the convolutional architectures, they lack understanding of long-range dependencies in the image. Recently proposed Transformer-based architectures that leverage self-attention mechanism encode long-range dependencies and learn representations that are highly expressive. This motivates us to explore Transformer-based solutions and study the feasibility of using Transformer-based network architectures for medical image segmentation tasks. Majority of existing Transformer-based network architectures proposed for vision applications require large-scale datasets to train properly. However, compared to the datasets for vision applications, for medical imaging the number of data samples is relatively low, making it difficult to efficiently train transformers for medical applications. To this end, we propose a Gated Axial-Attention model which extends the existing architectures by introducing an additional control mechanism in the self-attention module. Furthermore, to train the model effectively on medical images, we propose a Local-Global training strategy (LoGo) which further improves the performance. Specifically, we operate on the whole image and patches to learn global and local features, respectively. The proposed Medical Transformer (MedT) is evaluated on three different medical image segmentation datasets and it is shown that it achieves better performance than the convolutional and other related transformer-based architectures. Code: https://github.com/jeya-maria-jose/Medical-Transformer
Information Bottleneck for a Rayleigh Fading MIMO Channel with an Oblivious Relay
Venue: arXiv cs.IT 2021
Year: 2021

This paper considers the information bottleneck (IB) problem of a Rayleigh fading multiple-input multiple-out (MIMO) channel with an oblivious relay. The relay is constrained to operate without knowledge of the codebooks, i.e., it performs oblivious processing. Moreover, due to the bottleneck constraint, it is impossible for the relay to inform the destination node of the perfect channel state information (CSI) in each channel realization. To evaluate the bottleneck rate, we first provide an upper bound by assuming that the destination node can get the perfect CSI at no cost. Then, we provide four achievable schemes where each scheme satisfies the bottleneck constraint and gives a lower bound to the bottleneck rate. In the first and second schemes, the relay splits the capacity of the relay-destination link into two parts, and conveys both the CSI and its observation to the destination node. Due to CSI transmission, the performance of these two schemes is sensitive to the MIMO channel dimension, especially the channel input dimension. To ensure that it still performs well when the channel dimension grows large, in the third and fourth achievable schemes, the relay only transmits compressed observation to the destination node. Numerical results show that with simple symbol-by-symbol oblivious relay processing and compression, the proposed achievable schemes work well and can demonstrate lower bounds coming quite close to the upper bound on a wide range of relevant system parameters.
Transferable Visual Words: Exploiting the Semantics of Anatomical Patterns for Self-supervised Learning
Venue: arXiv cs.CV 2021
Year: 2021

This paper introduces a new concept called "transferable visual words" (TransVW), aiming to achieve annotation efficiency for deep learning in medical image analysis. Medical imaging--focusing on particular parts of the body for defined clinical purposes--generates images of great similarity in anatomy across patients and yields sophisticated anatomical patterns across images, which are associated with rich semantics about human anatomy and which are natural visual words. We show that these visual words can be automatically harvested according to anatomical consistency via self-discovery, and that the self-discovered visual words can serve as strong yet free supervision signals for deep models to learn semantics-enriched generic image representation via self-supervision (self-classification and self-restoration). Our extensive experiments demonstrate the annotation efficiency of TransVW by offering higher performance and faster convergence with reduced annotation cost in several applications. Our TransVW has several important advantages, including (1) TransVW is a fully autodidactic scheme, which exploits the semantics of visual words for self-supervised learning, requiring no expert annotation; (2) visual word learning is an add-on strategy, which complements existing self-supervised methods, boosting their performance; and (3) the learned image representation is semantics-enriched models, which have proven to be more robust and generalizable, saving annotation efforts for a variety of applications through transfer learning. Our code, pre-trained models, and curated visual words are available at https://github.com/JLiangLab/TransVW.
Generalized Gapped-kmer Filters for Robust Frequency Estimation
Venue: arXiv cs.DM 2021
Year: 2021

In this paper, we study the generalized gapped k-mer filters and derive a closed form solution for their coefficients. We consider nonnegative integers $\ell$ and $k$, with $k\leq \ell$, and an $\ell$-tuple $B=(b_1,\ldots,b_{\ell})$ of integers $b_i\geq 2$, $i=1,\ldots,\ell$. We introduce and study an incidence matrix $A=A_{\ell,k;B}$. We develop a Möbius-like function $ν_B$ which helps us to obtain closed forms for a complete set of mutually orthogonal eigenvectors of $A^{\top} A$ as well as a complete set of mutually orthogonal eigenvectors of $AA^{\top}$ corresponding to nonzero eigenvalues. The reduced singular value decomposition of $A$ and combinatorial interpretations for the nullity and rank of $A$, are among the consequences of this approach. We then combine the obtained formulas, some results from linear algebra, and combinatorial identities of elementary symmetric functions and $ν_B$, to provide the entries of the Moore-Penrose pseudo-inverse matrix $A^{+}$ and the Gapped k-mer filter matrix $A^{+} A$.
Pre-Training BERT on Arabic Tweets: Practical Considerations
Venue: arXiv cs.CL 2021
Year: 2021

Pretraining Bidirectional Encoder Representations from Transformers (BERT) for downstream NLP tasks is a non-trival task. We pretrained 5 BERT models that differ in the size of their training sets, mixture of formal and informal Arabic, and linguistic preprocessing. All are intended to support Arabic dialects and social media. The experiments highlight the centrality of data diversity and the efficacy of linguistically aware segmentation. They also highlight that more data or more training step do not necessitate better models. Our new models achieve new state-of-the-art results on several downstream tasks. The resulting models are released to the community under the name QARiB.
Design, Integration and Sea Trials of 3D Printed Unmanned Aerial Vehicle and Unmanned Surface Vehicle for Cooperative Missions
Venue: arXiv cs.RO 2021
Year: 2021

In recent years, Unmanned Surface Vehicles (USV) have been extensively deployed for maritime applications. However, USV has a limited detection range with sensor installed at the same elevation with the targets. In this research, we propose a cooperative Unmanned Aerial Vehicle - Unmanned Surface Vehicle (UAV-USV) platform to improve the detection range of USV. A floatable and waterproof UAV is designed and 3D printed, which allows it to land on the sea. A catamaran USV and landing platform are also developed. To land UAV on the USV precisely in various lighting conditions, IR beacon detector and IR beacon are implemented on the UAV and USV, respectively. Finally, a two-phase UAV precise landing method, USV control algorithm and USV path following algorithm are proposed and tested.
Three dimensional unique identifier based automated georeferencing and coregistration of point clouds in underground environment
Venue: arXiv cs.CV 2021
Year: 2021

Spatially and geometrically accurate laser scans are essential in modelling infrastructure for applications in civil, mining and transportation. Monitoring of underground or indoor environments such as mines or tunnels is challenging due to unavailability of a sensor positioning framework, complicated structurally symmetric layouts, repetitive features and occlusions. Current practices largely include a manual selection of discernable reference points for georeferencing and coregistration purpose. This study aims at overcoming these practical challenges in underground or indoor laser scanning. The developed approach involves automatically and uniquely identifiable three dimensional unique identifiers (3DUIDs) in laser scans, and a 3D registration (3DReG) workflow. Field testing of the method in an underground tunnel has been found accurate, effective and efficient. Additionally, a method for automatically extracting roadway tunnel profile has been exhibited. The developed 3DUID can be used in roadway profile extraction, guided automation, sensor calibration, reference targets for routine survey and deformation monitoring.
Provable Super-Convergence with a Large Cyclical Learning Rate
Venue: arXiv cs.LG 2021
Year: 2021

Conventional wisdom dictates that learning rate should be in the stable regime so that gradient-based algorithms don't blow up. This letter introduces a simple scenario where an unstably large learning rate scheme leads to a super fast convergence, with the convergence rate depending only logarithmically on the condition number of the problem. Our scheme uses a Cyclical Learning Rate (CLR) where we periodically take one large unstable step and several small stable steps to compensate for the instability. These findings also help explain the empirical observations of [Smith and Topin, 2019] where they show that CLR with a large maximum learning rate can dramatically accelerate learning and lead to so-called "super-convergence". We prove that our scheme excels in the problems where Hessian exhibits a bimodal spectrum and the eigenvalues can be grouped into two clusters (small and large). The unstably large step is the key to enabling fast convergence over the small eigen-spectrum.
Joint Estimation of Multipath Angles and Delays for Millimeter-Wave Cylindrical Arrays with Hybrid Front-ends
Venue: arXiv cs.IT 2021
Year: 2021

Accurate channel parameter estimation is challenging for wideband millimeter-wave (mmWave) large-scale hybrid arrays, due to beam squint and much fewer radio frequency (RF) chains than antennas. This paper presents a novel joint delay and angle estimation approach for wideband mmWave fully-connected hybrid uniform cylindrical arrays. We first design a new hybrid beamformer to reduce the dimension of received signals on the horizontal plane by exploiting the convergence of the Bessel function, and to reduce the active beams in the vertical direction through preselection. The important recurrence relationship of the received signals needed for subspace-based angle and delay estimation is preserved, even with substantially fewer RF chains than antennas. Then, linear interpolation is generalized to reconstruct the received signals of the hybrid beamformer, so that the signals can be coherently combined across the whole band to suppress the beam squint. As a result, efficient subspace-based algorithm algorithms can be developed to estimate the angles and delays of multipath components. The estimated delays and angles are further matched and correctly associated with different paths in the presence of non-negligible noises, by putting forth perturbation operations. Simulations show that the proposed approach can approach the Cramér-Rao lower bound (CRLB) of the estimation with a significantly lower computational complexity than existing techniques.
Provably Improved Context-Based Offline Meta-RL with Attention and Contrastive Learning
Venue: arXiv cs.LG 2021
Year: 2021

Meta-learning for offline reinforcement learning (OMRL) is an understudied problem with tremendous potential impact by enabling RL algorithms in many real-world applications. A popular solution to the problem is to infer task identity as augmented state using a context-based encoder, for which efficient learning of robust task representations remains an open challenge. In this work, we provably improve upon one of the SOTA OMRL algorithms, FOCAL, by incorporating intra-task attention mechanism and inter-task contrastive learning objectives, to robustify task representation learning against sparse reward and distribution shift. Theoretical analysis and experiments are presented to demonstrate the superior performance and robustness of our end-to-end and model-free framework compared to prior algorithms across multiple meta-RL benchmarks.
PCB-Fire: Automated Classification and Fault Detection in PCB
Venue: arXiv cs.CV 2021
Year: 2021

Printed Circuit Boards are the foundation for the functioning of any electronic device, and therefore are an essential component for various industries such as automobile, communication, computation, etc. However, one of the challenges faced by the PCB manufacturers in the process of manufacturing of the PCBs is the faulty placement of its components including missing components. In the present scenario the infrastructure required to ensure adequate quality of the PCB requires a lot of time and effort. The authors present a novel solution for detecting missing components and classifying them in a resourceful manner. The presented algorithm focuses on pixel theory and object detection, which has been used in combination to optimize the results from the given dataset.
ReINTEL Challenge 2020: Exploiting Transfer Learning Models for Reliable Intelligence Identification on Vietnamese Social Network Sites
Venue: arXiv cs.CL 2021
Year: 2021

This paper presents the system that we propose for the Reliable Intelligence Indentification on Vietnamese Social Network Sites (ReINTEL) task of the Vietnamese Language and Speech Processing 2020 (VLSP 2020) Shared Task. In this task, the VLSP 2020 provides a dataset with approximately 6,000 trainning news/posts annotated with reliable or unreliable labels, and a test set consists of 2,000 examples without labels. In this paper, we conduct experiments on different transfer learning models, which are bert4news and PhoBERT fine-tuned to predict whether the news is reliable or not. In our experiments, we achieve the AUC score of 94.52% on the private test set from ReINTEL's organizers.
Clustering Algorithm to Detect Adversaries in Federated Learning
Venue: arXiv cs.LG 2021
Year: 2021

In recent times, federated machine learning has been very useful in building intelligent intrusion detection systems for IoT devices. As IoT devices are equipped with a security architecture vulnerable to various attacks, these security loopholes may bring a risk during federated training of decentralized IoT devices. Adversaries can take control over these IoT devices and inject false gradients to degrade the global model performance. In this paper, we have proposed an approach that detects the adversaries with the help of a clustering algorithm. After clustering, it further rewards the clients for detecting honest and malicious clients. Our proposed gradient filtration approach does not require any processing power from the client-side and does not use excessive bandwidth, making it very much feasible for IoT devices. Further, our approach has been very successful in boosting the global model accuracy, up to 99% even in the presence of 40% adversaries.
ikd-Tree: An Incremental K-D Tree for Robotic Applications
Venue: arXiv cs.RO 2021
Year: 2021

This paper proposes an efficient data structure, ikd-Tree, for dynamic space partition. The ikd-Tree incrementally updates a k-d tree with new coming points only, leading to much lower computation time than existing static k-d trees. Besides point-wise operations, the ikd-Tree supports several features such as box-wise operations and down-sampling that are practically useful in robotic applications. In parallel to the incremental operations (i.e., insert, re-insert, and delete), ikd-Tree actively monitors the tree structure and partially re-balances the tree, which enables efficient nearest point search in later stages. The ikd-Tree is carefully engineered and supports multi-thread parallel computing to maximize the overall efficiency. We validate the ikd-Tree in both theory and practical experiments. On theory level, a complete time complexity analysis is presented to prove the high efficiency. On experiment level, the ikd-Tree is tested on both randomized datasets and real-world LiDAR point data in LiDAR-inertial odometry and mapping application. In all tests, ikd-Tree consumes only 4% of the running time in a static k-d tree.