OpenRaiser/Intern-Atlas huggingface dataset:
Multimodal Fake News Detection
Venue: arXiv cs.CL 2021
Year: 2021

Over the last years, there has been an unprecedented proliferation of fake news. As a consequence, we are more susceptible to the pernicious impact that misinformation and disinformation spreading can have in different segments of our society. Thus, the development of tools for automatic detection of fake news plays and important role in the prevention of its negative effects. Most attempts to detect and classify false content focus only on using textual information. Multimodal approaches are less frequent and they typically classify news either as true or fake. In this work, we perform a fine-grained classification of fake news on the Fakeddit dataset, using both unimodal and multimodal approaches. Our experiments show that the multimodal approach based on a Convolutional Neural Network (CNN) architecture combining text and image data achieves the best results, with an accuracy of 87%. Some fake news categories such as Manipulated content, Satire or False connection strongly benefit from the use of images. Using images also improves the results of the other categories, but with less impact. Regarding the unimodal approaches using only text, Bidirectional Encoder Representations from Transformers (BERT) is the best model with an accuracy of 78%. Therefore, exploiting both text and image data significantly improves the performance of fake news detection.
Knowledge Distillation for Object Detection via Rank Mimicking and Prediction-guided Feature Imitation
Venue: arXiv cs.CV 2021
Year: 2021

Knowledge Distillation (KD) is a widely-used technology to inherit information from cumbersome teacher models to compact student models, consequently realizing model compression and acceleration. Compared with image classification, object detection is a more complex task, and designing specific KD methods for object detection is non-trivial. In this work, we elaborately study the behaviour difference between the teacher and student detection models, and obtain two intriguing observations: First, the teacher and student rank their detected candidate boxes quite differently, which results in their precision discrepancy. Second, there is a considerable gap between the feature response differences and prediction differences between teacher and student, indicating that equally imitating all the feature maps of the teacher is the sub-optimal choice for improving the student's accuracy. Based on the two observations, we propose Rank Mimicking (RM) and Prediction-guided Feature Imitation (PFI) for distilling one-stage detectors, respectively. RM takes the rank of candidate boxes from teachers as a new form of knowledge to distill, which consistently outperforms the traditional soft label distillation. PFI attempts to correlate feature differences with prediction differences, making feature imitation directly help to improve the student's accuracy. On MS COCO and PASCAL VOC benchmarks, extensive experiments are conducted on various detectors with different backbones to validate the effectiveness of our method. Specifically, RetinaNet with ResNet50 achieves 40.4% mAP in MS COCO, which is 3.5% higher than its baseline, and also outperforms previous KD methods.
A Bilingual, OpenWorld Video Text Dataset and End-to-end Video Text Spotter with Transformer
Venue: arXiv cs.CV 2021
Year: 2021

Most existing video text spotting benchmarks focus on evaluating a single language and scenario with limited data. In this work, we introduce a large-scale, Bilingual, Open World Video text benchmark dataset(BOVText). There are four features for BOVText. Firstly, we provide 2,000+ videos with more than 1,750,000+ frames, 25 times larger than the existing largest dataset with incidental text in videos. Secondly, our dataset covers 30+ open categories with a wide selection of various scenarios, e.g., Life Vlog, Driving, Movie, etc. Thirdly, abundant text types annotation (i.e., title, caption or scene text) are provided for the different representational meanings in video. Fourthly, the BOVText provides bilingual text annotation to promote multiple cultures live and communication. Besides, we propose an end-to-end video text spotting framework with Transformer, termed TransVTSpotter, which solves the multi-orient text spotting in video with a simple, but efficient attention-based query-key mechanism. It applies object features from the previous frame as a tracking query for the current frame and introduces a rotation angle prediction to fit the multiorient text instance. On ICDAR2015(video), TransVTSpotter achieves the state-of-the-art performance with 44.1% MOTA, 9 fps. The dataset and code of TransVTSpotter can be found at github:com=weijiawu=BOVText and github:com=weijiawu=TransVTSpotter, respectively.
Latent Space Explanation by Intervention
Venue: arXiv cs.LG 2021
Year: 2021

The success of deep neural nets heavily relies on their ability to encode complex relations between their input and their output. While this property serves to fit the training data well, it also obscures the mechanism that drives prediction. This study aims to reveal hidden concepts by employing an intervention mechanism that shifts the predicted class based on discrete variational autoencoders. An explanatory model then visualizes the encoded information from any hidden layer and its corresponding intervened representation. By the assessment of differences between the original representation and the intervened representation, one can determine the concepts that can alter the class, hence providing interpretability. We demonstrate the effectiveness of our approach on CelebA, where we show various visualizations for bias in the data and suggest different interventions to reveal and change bias.
Few-Shot Keypoint Detection as Task Adaptation via Latent Embeddings
Venue: arXiv cs.RO 2021
Year: 2021

Dense object tracking, the ability to localize specific object points with pixel-level accuracy, is an important computer vision task with numerous downstream applications in robotics. Existing approaches either compute dense keypoint embeddings in a single forward pass, meaning the model is trained to track everything at once, or allocate their full capacity to a sparse predefined set of points, trading generality for accuracy. In this paper we explore a middle ground based on the observation that the number of relevant points at a given time are typically relatively few, e.g. grasp points on a target object. Our main contribution is a novel architecture, inspired by few-shot task adaptation, which allows a sparse-style network to condition on a keypoint embedding that indicates which point to track. Our central finding is that this approach provides the generality of dense-embedding models, while offering accuracy significantly closer to sparse-keypoint approaches. We present results illustrating this capacity vs. accuracy trade-off, and demonstrate the ability to zero-shot transfer to new object instances (within-class) using a real-robot pick-and-place task.
DVHN: A Deep Hashing Framework for Large-scale Vehicle Re-identification
Venue: arXiv cs.CV 2021
Year: 2021

In this paper, we make the very first attempt to investigate the integration of deep hash learning with vehicle re-identification. We propose a deep hash-based vehicle re-identification framework, dubbed DVHN, which substantially reduces memory usage and promotes retrieval efficiency while reserving nearest neighbor search accuracy. Concretely,~DVHN directly learns discrete compact binary hash codes for each image by jointly optimizing the feature learning network and the hash code generating module. Specifically, we directly constrain the output from the convolutional neural network to be discrete binary codes and ensure the learned binary codes are optimal for classification. To optimize the deep discrete hashing framework, we further propose an alternating minimization method for learning binary similarity-preserved hashing codes. Extensive experiments on two widely-studied vehicle re-identification datasets- \textbf{VehicleID} and \textbf{VeRi}-~have demonstrated the superiority of our method against the state-of-the-art deep hash methods. \textbf{DVHN} of $2048$ bits can achieve 13.94\% and 10.21\% accuracy improvement in terms of \textbf{mAP} and \textbf{Rank@1} for \textbf{VehicleID (800)} dataset. For \textbf{VeRi}, we achieve 35.45\% and 32.72\% performance gains for \textbf{Rank@1} and \textbf{mAP}, respectively.
Testing Probabilistic Circuits
Venue: arXiv cs.DS 2021
Year: 2021

Probabilistic circuits (PCs) are a powerful modeling framework for representing tractable probability distributions over combinatorial spaces. In machine learning and probabilistic programming, one is often interested in understanding whether the distributions learned using PCs are close to the desired distribution. Thus, given two probabilistic circuits, a fundamental problem of interest is to determine whether their distributions are close to each other. The primary contribution of this paper is a closeness test for PCs with respect to the total variation distance metric. Our algorithm utilizes two common PC queries, counting and sampling. In particular, we provide a poly-time probabilistic algorithm to check the closeness of two PCs when the PCs support tractable approximate counting and sampling. We demonstrate the practical efficiency of our algorithmic framework via a detailed experimental evaluation of a prototype implementation against a set of 475 PC benchmarks. We find that our test correctly decides the closeness of all 475 PCs within 3600 seconds.
Bringing Atomistic Deep Learning to Prime Time
Venue: arXiv cs.LG 2021
Year: 2021

Artificial intelligence has not yet revolutionized the design of materials and molecules. In this perspective, we identify four barriers preventing the integration of atomistic deep learning, molecular science, and high-performance computing. We outline focused research efforts to address the opportunities presented by these challenges.
The geometry of one-weight codes in the sum-rank metric
Venue: arXiv cs.IT 2021
Year: 2021

We provide a geometric characterization of $k$-dimensional $\mathbb{F}_{q^m}$-linear sum-rank metric codes as tuples of $\mathbb{F}_q$-subspaces of $\mathbb{F}_{q^m}^k$. We then use this characterization to study one-weight codes in the sum-rank metric. This leads us to extend the family of linearized Reed-Solomon codes in order to obtain a doubly-extended version of them. We prove that these codes are still maximum sum-rank distance (MSRD) codes and, when $k=2$, they are one-weight, as in the Hamming-metric case. We then focus on constant rank-profile codes in the sum-rank metric, which are a special family of one weight-codes, and derive constraints on their parameters with the aid of an associated Hamming-metric code. Furthermore, we introduce the $n$-simplex codes in the sum-rank metric, which are obtained as the orbit of a Singer subgroup of $\mathrm{GL}(k,q^m)$. They turn out to be constant rank-profile - and hence one-weight - and generalize the simplex codes in both the Hamming and the rank metric. Finally, we focus on $2$-dimensional one-weight codes, deriving constraints on the parameters of those which are also MSRD, and we find a new construction of one-weight MSRD codes when $q=2$.
End-to-End Learning of Joint Geometric and Probabilistic Constellation Shaping
Venue: arXiv cs.IT 2021
Year: 2021

We present a novel autoencoder-based learning of joint geometric and probabilistic constellation shaping for coded-modulation systems. It can maximize either the mutual information (for symbol-metric decoding) or the generalized mutual information (for bit-metric decoding).
Deep Learning based Differential Distinguisher for Lightweight Block Ciphers
Venue: arXiv cs.CR 2021
Year: 2021

Recent years have seen an increasing involvement of Deep Learning in the cryptanalysis of various ciphers. The present study is inspired by past works on differential distinguishers, to develop a Deep Neural Network-based differential distinguisher for round reduced lightweight block ciphers PRESENT and Simeck. We make improvements in the state-of-the-art approach and extend its use to the two structurally different block ciphers, PRESENT-80 and Simeck64/128. The obtained results suggest the universality of our cryptanalysis method. The proposed method can distinguish random data from the cipher data obtained until 6 rounds of PRESENT and 7 rounds of Simeck encryption with high accuracy. In addition to this, we explore a new approach to select good input differentials, which to the best of our knowledge has not been explored in the past. We also provide a minimum-security requirement for the discussed ciphers against our differential attack.
Unique Assembly Verification in Two-Handed Self-Assembly
Venue: arXiv cs.CG 2021
Year: 2021

One of the most fundamental and well-studied problems in Tile Self-Assembly is the Unique Assembly Verification (UAV) problem. This algorithmic problem asks whether a given tile system uniquely assembles a specific assembly. The complexity of this problem in the 2-Handed Assembly Model (2HAM) at a constant temperature is a long-standing open problem since the model was introduced. Previously, only membership in the class coNP was known and that the problem is in P if the temperature is one ($τ=1$). The problem is known to be hard for many generalizations of the model, such as allowing one step into the third dimension or allowing the temperature of the system to be a variable, but the most fundamental version has remained open. In this paper, we prove the UAV problem in the 2HAM is hard even with a small constant temperature ($τ= 2$), and finally answer the complexity of this problem (open since 2013). Further, this result proves that UAV in the staged self-assembly model is coNP-complete with a single bin and stage (open since 2007), and that UAV in the q-tile model is also coNP-complete (open since 2004). We reduce from Monotone Planar 3-SAT with Neighboring Variable Pairs, a special case of 3SAT recently proven to be NP-hard. We accompany this reduction with a positive result showing that UAV is solvable in polynomial time with the promise that the given target assembly will have a tree-shaped bond graph, i.e., contains no cycles. We provide a $\mathcal{O}(n^5)$ algorithm for UAV on tree-bonded assemblies when the temperature is fixed to $2$, and a $\mathcal{O}(n^5\log τ)$ time algorithm when the temperature is part of the input.
A Survey on Echo Chambers on Social Media: Description, Detection and Mitigation
Venue: arXiv cs.SI 2021
Year: 2021

Echo chambers on social media are a significant problem that can elicit a number of negative consequences, most recently affecting the response to COVID-19. Echo chambers promote conspiracy theories about the virus and are found to be linked to vaccine hesitancy, less compliance with mask mandates, and the practice of social distancing. Moreover, the problem of echo chambers is connected to other pertinent issues like political polarization and the spread of misinformation. An echo chamber is defined as a network of users in which users only interact with opinions that support their pre-existing beliefs and opinions, and they exclude and discredit other viewpoints. This survey aims to examine the echo chamber phenomenon on social media from a social computing perspective and provide a blueprint for possible solutions. We survey the related literature to understand the attributes of echo chambers and how they affect the individual and society at large. Additionally, we show the mechanisms, both algorithmic and psychological, that lead to the formation of echo chambers. These mechanisms could be manifested in two forms: (1) the bias of social media's recommender systems and (2) internal biases such as confirmation bias and homophily. While it is immensely challenging to mitigate internal biases, there has been great efforts seeking to mitigate the bias of recommender systems. These recommender systems take advantage of our own biases to personalize content recommendations to keep us engaged in order to watch more ads. Therefore, we further investigate different computational approaches for echo chamber detection and prevention, mainly based around recommender systems.
Web Platform for Visualisation of Kinematic Data captured from a Motor Tele-rehabilitation System
Venue: arXiv cs.HC 2021
Year: 2021

Stroke can have a severe impact on an individual's quality of life, leading to consequences such as motor loss and communication problems, especially among the elderly. Studies have shown that early and easy access to stroke rehabilitation can improve an elderly individual's quality of life, and that telerehabilitation is a solution that facilitates this improvement. In this work, we visualize movement to music during rehabilitation exercises captured by the Kinect motion sensor, using a dedicated Serious Game called `Move to the Music'(MoMu). Our system provides a quantitative view of progress made by patients during a motor rehabilitation regime for healthcare professionals to track remotely (tele-rehab).
Classification of Anuran Frog Species Using Machine Learning
Venue: arXiv cs.LG 2021
Year: 2021

Acoustic classification of frogs has gotten a lot of attention recently due to its potential applicability in ecological investigations. Numerous studies have been presented for identifying frog species, although the majority of recorded species are thought to be monotypic. The purpose of this study is to demonstrate a method for classifying various frog species using an audio recording. To be more exact, continuous frog recordings are cut into audio snippets first (10 seconds). Then, for each ten-second recording, several time-frequency representations are constructed. Following that, rather than using manually created features, Machine Learning methods are employed to classify the frog species. Data reduction techniques; Principal Component Analysis (PCA) and Independent Component Analysis (ICA) are used to extract the most important features before classification. Finally, to validate our classification accuracy, cross validation and prediction accuracy are used. Experimental results show that PCA extracted features that achieved better classification accuracy both with cross validation and prediction accuracy.
CLIP2StyleGAN: Unsupervised Extraction of StyleGAN Edit Directions
Venue: arXiv cs.CV 2021
Year: 2021

The success of StyleGAN has enabled unprecedented semantic editing capabilities, on both synthesized and real images. However, such editing operations are either trained with semantic supervision or described using human guidance. In another development, the CLIP architecture has been trained with internet-scale image and text pairings and has been shown to be useful in several zero-shot learning settings. In this work, we investigate how to effectively link the pretrained latent spaces of StyleGAN and CLIP, which in turn allows us to automatically extract semantically labeled edit directions from StyleGAN, finding and naming meaningful edit operations without any additional human guidance. Technically, we propose two novel building blocks; one for finding interesting CLIP directions and one for labeling arbitrary directions in CLIP latent space. The setup does not assume any pre-determined labels and hence we do not require any additional supervised text/attributes to build the editing framework. We evaluate the effectiveness of the proposed method and demonstrate that extraction of disentangled labeled StyleGAN edit directions is indeed possible, and reveals interesting and non-trivial edit directions.
KartalOl: Transfer learning using deep neural network for iris segmentation and localization: New dataset for iris segmentation
Venue: arXiv cs.CV 2021
Year: 2021

Iris segmentation and localization in unconstrained environments is challenging due to long distances, illumination variations, limited user cooperation, and moving subjects. To address this problem, we present a U-Net with a pre-trained MobileNetV2 deep neural network method. We employ the pre-trained weights given with MobileNetV2 for the ImageNet dataset and fine-tune it on the iris recognition and localization domain. Further, we have introduced a new dataset, called KartalOl, to better evaluate detectors in iris recognition scenarios. To provide domain adaptation, we fine-tune the MobileNetV2 model on the provided data for NIR-ISL 2021 from the CASIA-Iris-Asia, CASIA-Iris-M1, and CASIA-Iris-Africa and our dataset. We also augment the data by performing left-right flips, rotation, zoom, and brightness. We chose the binarization threshold for the binary masks by iterating over the images in the provided dataset. The proposed method is tested and trained in CASIA-Iris-Asia, CASIA-Iris-M1, CASIA-Iris-Africa, along the KartalOl dataset. The experimental results highlight that our method surpasses state-of-the-art methods on mobile-based benchmarks. The codes and evaluation results are publicly available at https://github.com/Jalilnkh/KartalOl-NIR-ISL2021031301.
Long-Range Thermal 3D Perception in Low Contrast Environments
Venue: arXiv cs.CV 2021
Year: 2021

This report discusses the results of SBIR Phase I effort to prove the feasibility of dramatic improvement of the microbolometer-based Long Wave Infrared (LWIR) detectors sensitivity, especially for the 3D measurements. The resulting low SWaP-C thermal depth-sensing system will enable the situational awareness of Autonomous Air Vehicles for Advanced Air Mobility (AAM). It will provide robust 3D information of the surrounding environment, including low-contrast static and moving objects, at far distances in degraded visual conditions and GPS-denied areas. Our multi-sensor 3D perception enabled by COTS uncooled thermal sensors mitigates major weakness of LWIR sensors - low contrast by increasing the system sensitivity over an order of magnitude. There were no available thermal image sets suitable for evaluating this technology, making datasets acquisition our first goal. We discuss the design and construction of the prototype system with sixteen 640pix x 512pix LWIR detectors, camera calibration to subpixel resolution, capture, and process synchronized image. The results show the 3.84x contrast increase for intrascene-only data and an additional 5.5x - with the interscene accumulation, reaching system noise-equivalent temperature difference (NETD) of 1.9 mK with the 40 mK sensors.
SmartCon: Deep Probabilistic Learning Based Intelligent Link-Configuration in Narrowband-IoT Towards 5G and B5G
Venue: arXiv cs.NI 2021
Year: 2021

To enhance the coverage and transmission reliability, repetitions adopted by Narrowband Internet of Things (NB-IoT) allow repeating transmissions several times. However, this results in a waste of radio resources when the signal strength is high. In addition, in low signal quality, the selection of a higher modulation and coding scheme (MCS) level leads to a huge packet loss in the network. Moreover, the number of physical resource blocks (PRBs) per-user needs to be chosen dynamically, such that the utilization of radio resources can be improved on per-user basis. Therefore, in NB-IoT systems, dynamic adaptation of repetitions, MCS, and radio resources, known as auto link-configuration, is crucial. Accordingly, in this paper, we propose SmartCon which is a Generative Adversarial Network (GAN)-based deep learning approach for auto link-configuration during uplink or downlink scheduling, such that the packet loss rate is significantly reduced in NB-IoT networks. For the training purpose of the GAN, we use a Multi-Armed Bandit (MAB)-based reinforcement learning mechanism that intelligently tunes its output depending on the present network condition. The performance of SmartCon is thoroughly evaluated through simulations where it is shown to significantly improve the performance of NB-IoT systems compared to baseline schemes.