OpenRaiser/Intern-Atlas huggingface dataset:
QAnon Propaganda on Twitter as Information Warfare: Influencers, Networks, and Narratives
Venue: arXiv cs.SI 2022
Year: 2022

QAnon refers to a set of far-right, conspiratorial ideologies that have risen in popularity in the U.S. since their initial promotion in 2017 on the 4chan internet message board. A central narrative element of QAnon is that a powerful group of elite, liberal members of the Democratic Party engage in morally reprehensible practices, but that former U.S. President Donald J. Trump was prosecuting them. Five studies investigated the influence and network connectivity of accounts promoting QAnon on Twitter from August, 2020 through January, 2021. Selection of Twitter accounts emphasized on-line influencers and "persons of interest" known or suspected of participation in QAnon propaganda promotion activities. Evidence of large-scale coordination among accounts promoting QAnon was observed, demonstrating rigorous, quantitative evidence of "astroturfing" in QAnon propaganda promotion on Twitter, as opposed to strictly "grassroots" activities of citizens acting independently. Further, evidence was obtained supporting that networks of extreme far-right adherents engaged in organized QAnon propaganda promotion, as revealed by network overlap among accounts promoting far-right extremist (e.g., anti-Semitic) content and insurrectionist themes; New Age, occult, and "esoteric" themes; and internet puzzle games like Cicada 3301 and other "alternate reality games." Based on well-grounded theories and findings from the social sciences, it is argued that QAnon propaganda on Twitter in the months circa the 2020 U.S. Presidential election likely reflected joint participation of multiple actors, including nation-states like Russia, in innovative misuse of social media toward undermining democratic processes by promoting "magical" thinking, ostracism of Democrats and liberals, and salience of White extinction narratives common among otherwise ideologically diverse groups on the extreme far-right.
Dev2vec: Representing Domain Expertise of Developers in an Embedding Space
Venue: arXiv cs.SE 2022
Year: 2022

Accurate assessment of the domain expertise of developers is important for assigning the proper candidate to contribute to a project or to attend a job role. Since the potential candidate can come from a large pool, the automated assessment of this domain expertise is a desirable goal. While previous methods have had some success within a single software project, the assessment of a developer's domain expertise from contributions across multiple projects is more challenging. In this paper, we employ doc2vec to represent the domain expertise of developers as embedding vectors. These vectors are derived from different sources that contain evidence of developers' expertise, such as the description of repositories that they contributed, their issue resolving history, and API calls in their commits. We name it dev2vec and demonstrate its effectiveness in representing the technical specialization of developers. Our results indicate that encoding the expertise of developers in an embedding vector outperforms state-of-the-art methods and improves the F1-score up to 21%. Moreover, our findings suggest that ``issue resolving history'' of developers is the most informative source of information to represent the domain expertise of developers in embedding spaces.
UrduFake@FIRE2021: Shared Track on Fake News Identification in Urdu
Venue: arXiv cs.CL 2022
Year: 2022

This study reports the second shared task named as UrduFake@FIRE2021 on identifying fake news detection in Urdu language. This is a binary classification problem in which the task is to classify a given news article into two classes: (i) real news, or (ii) fake news. In this shared task, 34 teams from 7 different countries (China, Egypt, Israel, India, Mexico, Pakistan, and UAE) registered to participate in the shared task, 18 teams submitted their experimental results and 11 teams submitted their technical reports. The proposed systems were based on various count-based features and used different classifiers as well as neural network architectures. The stochastic gradient descent (SGD) algorithm outperformed other classifiers and achieved 0.679 F-score.
Real-Time And Robust 3D Object Detection with Roadside LiDARs
Venue: arXiv cs.CV 2022
Year: 2022

This work aims to address the challenges in autonomous driving by focusing on the 3D perception of the environment using roadside LiDARs. We design a 3D object detection model that can detect traffic participants in roadside LiDARs in real-time. Our model uses an existing 3D detector as a baseline and improves its accuracy. To prove the effectiveness of our proposed modules, we train and evaluate the model on three different vehicle and infrastructure datasets. To show the domain adaptation ability of our detector, we train it on an infrastructure dataset from China and perform transfer learning on a different dataset recorded in Germany. We do several sets of experiments and ablation studies for each module in the detector that show that our model outperforms the baseline by a significant margin, while the inference speed is at 45 Hz (22 ms). We make a significant contribution with our LiDAR-based 3D detector that can be used for smart city applications to provide connected and automated vehicles with a far-reaching view. Vehicles that are connected to the roadside sensors can get information about other vehicles around the corner to improve their path and maneuver planning and to increase road traffic safety.
Language Models (Mostly) Know What They Know
Venue: arXiv cs.CL 2022
Year: 2022

We study whether language models can evaluate the validity of their own claims and predict which questions they will be able to answer correctly. We first show that larger models are well-calibrated on diverse multiple choice and true/false questions when they are provided in the right format. Thus we can approach self-evaluation on open-ended sampling tasks by asking models to first propose answers, and then to evaluate the probability "P(True)" that their answers are correct. We find encouraging performance, calibration, and scaling for P(True) on a diverse array of tasks. Performance at self-evaluation further improves when we allow models to consider many of their own samples before predicting the validity of one specific possibility. Next, we investigate whether models can be trained to predict "P(IK)", the probability that "I know" the answer to a question, without reference to any particular proposed answer. Models perform well at predicting P(IK) and partially generalize across tasks, though they struggle with calibration of P(IK) on new tasks. The predicted P(IK) probabilities also increase appropriately in the presence of relevant source materials in the context, and in the presence of hints towards the solution of mathematical word problems. We hope these observations lay the groundwork for training more honest models, and for investigating how honesty generalizes to cases where models are trained on objectives other than the imitation of human writing.
Dynamic Proposals for Efficient Object Detection
Venue: arXiv cs.CV 2022
Year: 2022

Object detection is a basic computer vision task to loccalize and categorize objects in a given image. Most state-of-the-art detection methods utilize a fixed number of proposals as an intermediate representation of object candidates, which is unable to adapt to different computational constraints during inference. In this paper, we propose a simple yet effective method which is adaptive to different computational resources by generating dynamic proposals for object detection. We first design a module to make a single query-based model to be able to inference with different numbers of proposals. Further, we extend it to a dynamic model to choose the number of proposals according to the input image, greatly reducing computational costs. Our method achieves significant speed-up across a wide range of detection models including two-stage and query-based models while obtaining similar or even better accuracy.
Cross-Architecture Knowledge Distillation
Venue: arXiv cs.CV 2022
Year: 2022

Transformer attracts much attention because of its ability to learn global relations and superior performance. In order to achieve higher performance, it is natural to distill complementary knowledge from Transformer to convolutional neural network (CNN). However, most existing knowledge distillation methods only consider homologous-architecture distillation, such as distilling knowledge from CNN to CNN. They may not be suitable when applying to cross-architecture scenarios, such as from Transformer to CNN. To deal with this problem, a novel cross-architecture knowledge distillation method is proposed. Specifically, instead of directly mimicking output/intermediate features of the teacher, partially cross attention projector and group-wise linear projector are introduced to align the student features with the teacher's in two projected feature spaces. And a multi-view robust training scheme is further presented to improve the robustness and stability of the framework. Extensive experiments show that the proposed method outperforms 14 state-of-the-arts on both small-scale and large-scale datasets.
Photonic Reconfigurable Accelerators for Efficient Inference of CNNs with Mixed-Sized Tensors
Venue: arXiv cs.AR 2022
Year: 2022

Photonic Microring Resonator (MRR) based hardware accelerators have been shown to provide disruptive speedup and energy-efficiency improvements for processing deep Convolutional Neural Networks (CNNs). However, previous MRR-based CNN accelerators fail to provide efficient adaptability for CNNs with mixed-sized tensors. One example of such CNNs is depthwise separable CNNs. Performing inferences of CNNs with mixed-sized tensors on such inflexible accelerators often leads to low hardware utilization, which diminishes the achievable performance and energy efficiency from the accelerators. In this paper, we present a novel way of introducing reconfigurability in the MRR-based CNN accelerators, to enable dynamic maximization of the size compatibility between the accelerator hardware components and the CNN tensors that are processed using the hardware components. We classify the state-of-the-art MRR-based CNN accelerators from prior works into two categories, based on the layout and relative placements of the utilized hardware components in the accelerators. We then use our method to introduce reconfigurability in accelerators from these two classes, to consequently improve their parallelism, the flexibility of efficiently mapping tensors of different sizes, speed, and overall energy efficiency. We evaluate our reconfigurable accelerators against three prior works for the area proportionate outlook (equal hardware area for all accelerators). Our evaluation for the inference of four modern CNNs indicates that our designed reconfigurable CNN accelerators provide improvements of up to 1.8x in Frames-Per-Second (FPS) and up to 1.5x in FPS/W, compared to an MRR-based accelerator from prior work.
Trusted Multi-Scale Classification Framework for Whole Slide Image
Venue: arXiv cs.CV 2022
Year: 2022

Despite remarkable efforts been made, the classification of gigapixels whole-slide image (WSI) is severely restrained from either the constrained computing resources for the whole slides, or limited utilizing of the knowledge from different scales. Moreover, most of the previous attempts lacked of the ability of uncertainty estimation. Generally, the pathologists often jointly analyze WSI from the different magnifications. If the pathologists are uncertain by using single magnification, then they will change the magnification repeatedly to discover various features of the tissues. Motivated by the diagnose process of the pathologists, in this paper, we propose a trusted multi-scale classification framework for the WSI. Leveraging the Vision Transformer as the backbone for multi branches, our framework can jointly classification modeling, estimating the uncertainty of each magnification of a microscope and integrate the evidence from different magnification. Moreover, to exploit discriminative patches from WSIs and reduce the requirement for computation resources, we propose a novel patch selection schema using attention rollout and non-maximum suppression. To empirically investigate the effectiveness of our approach, empirical experiments are conducted on our WSI classification tasks, using two benchmark databases. The obtained results suggest that the trusted framework can significantly improve the WSI classification performance compared with the state-of-the-art methods.
An outer approximation bi-level framework for mixed categorical structural optimization problems
Venue: arXiv cs.CE 2022
Year: 2022

In this paper, mixed categorical structural optimization problems are investigated. The aim is to minimize the weight of a truss structure with respect to cross-section areas, materials and cross-section type. The proposed methodology consists of using a bi-level decomposition involving two problems: master and slave. The master problem is formulated as a mixed integer linear problem where the linear constraints are incrementally augmented using outer approximations of the slave problem solution. The slave problem addresses the continuous variables of the optimization problem. The proposed methodology is tested on three different structural optimization test cases with increasing complexity. The comparison to state-of-the-art algorithms emphasizes the efficiency of the proposed methodology in terms of the optimum quality, computation cost, as well as its scalability with respect to the problem dimension. A challenging 120-bar dome truss optimization problem with 90 categorical choices per bar is also tested. The obtained results showed that our method is able to solve efficiently large scale mixed categorical structural optimization problems.
CompoundE: Knowledge Graph Embedding with Translation, Rotation and Scaling Compound Operations
Venue: arXiv cs.AI 2022
Year: 2022

Translation, rotation, and scaling are three commonly used geometric manipulation operations in image processing. Besides, some of them are successfully used in developing effective knowledge graph embedding (KGE) models such as TransE and RotatE. Inspired by the synergy, we propose a new KGE model by leveraging all three operations in this work. Since translation, rotation, and scaling operations are cascaded to form a compound one, the new model is named CompoundE. By casting CompoundE in the framework of group theory, we show that quite a few scoring-function-based KGE models are special cases of CompoundE. CompoundE extends the simple distance-based relation to relation-dependent compound operations on head and/or tail entities. To demonstrate the effectiveness of CompoundE, we conduct experiments on three popular KG completion datasets. Experimental results show that CompoundE consistently achieves the state of-the-art performance.
Dual Contrastive Learning for Spatio-temporal Representation
Venue: arXiv cs.CV 2022
Year: 2022

Contrastive learning has shown promising potential in self-supervised spatio-temporal representation learning. Most works naively sample different clips to construct positive and negative pairs. However, we observe that this formulation inclines the model towards the background scene bias. The underlying reasons are twofold. First, the scene difference is usually more noticeable and easier to discriminate than the motion difference. Second, the clips sampled from the same video often share similar backgrounds but have distinct motions. Simply regarding them as positive pairs will draw the model to the static background rather than the motion pattern. To tackle this challenge, this paper presents a novel dual contrastive formulation. Concretely, we decouple the input RGB video sequence into two complementary modes, static scene and dynamic motion. Then, the original RGB features are pulled closer to the static features and the aligned dynamic features, respectively. In this way, the static scene and the dynamic motion are simultaneously encoded into the compact RGB representation. We further conduct the feature space decoupling via activation maps to distill static- and dynamic-related features. We term our method as \textbf{D}ual \textbf{C}ontrastive \textbf{L}earning for spatio-temporal \textbf{R}epresentation (DCLR). Extensive experiments demonstrate that DCLR learns effective spatio-temporal representations and obtains state-of-the-art or comparable performance on UCF-101, HMDB-51, and Diving-48 datasets.
Image and Model Transformation with Secret Key for Vision Transformer
Venue: arXiv cs.CV 2022
Year: 2022

In this paper, we propose a combined use of transformed images and vision transformer (ViT) models transformed with a secret key. We show for the first time that models trained with plain images can be directly transformed to models trained with encrypted images on the basis of the ViT architecture, and the performance of the transformed models is the same as models trained with plain images when using test images encrypted with the key. In addition, the proposed scheme does not require any specially prepared data for training models or network modification, so it also allows us to easily update the secret key. In an experiment, the effectiveness of the proposed scheme is evaluated in terms of performance degradation and model protection performance in an image classification task on the CIFAR-10 dataset.
Occluded Human Body Capture with Self-Supervised Spatial-Temporal Motion Prior
Venue: arXiv cs.CV 2022
Year: 2022

Although significant progress has been achieved on monocular maker-less human motion capture in recent years, it is still hard for state-of-the-art methods to obtain satisfactory results in occlusion scenarios. There are two main reasons: the one is that the occluded motion capture is inherently ambiguous as various 3D poses can map to the same 2D observations, which always results in an unreliable estimation. The other is that no sufficient occluded human data can be used for training a robust model. To address the obstacles, our key-idea is to employ non-occluded human data to learn a joint-level spatial-temporal motion prior for occluded human with a self-supervised strategy. To further reduce the gap between synthetic and real occlusion data, we build the first 3D occluded motion dataset~(OcMotion), which can be used for both training and testing. We encode the motions in 2D maps and synthesize occlusions on non-occluded data for the self-supervised training. A spatial-temporal layer is then designed to learn joint-level correlations. The learned prior reduces the ambiguities of occlusions and is robust to diverse occlusion types, which is then adopted to assist the occluded human motion capture. Experimental results show that our method can generate accurate and coherent human motions from occluded videos with good generalization ability and runtime efficiency. The dataset and code are publicly available at \url{https://github.com/boycehbz/CHOMP}.
Deriving RIP sensing matrices for sparsifying dictionaries
Venue: arXiv cs.IT 2022
Year: 2022

Compressive sensing involves the inversion of a mapping $SD \in \mathbb{R}^{m \times n}$, where $m < n$, $S$ is a sensing matrix, and $D$ is a sparisfying dictionary. The restricted isometry property is a powerful sufficient condition for the inversion that guarantees the recovery of high-dimensional sparse vectors from their low-dimensional embedding into a Euclidean space via convex optimization. However, determining whether $SD$ has the restricted isometry property for a given sparisfying dictionary is an NP-hard problem, hampering the application of compressive sensing. This paper provides a novel approach to resolving this problem. We demonstrate that it is possible to derive a sensing matrix for any sparsifying dictionary with a high probability of retaining the restricted isometry property. In numerical experiments with sensing matrices for K-SVD, Parseval K-SVD, and wavelets, our recovery performance was comparable to that of benchmarks obtained using Gaussian and Bernoulli random sensing matrices for sparse vectors.
Opportunistic Wiretapping/Jamming: A New Attack Model in Millimeter-Wave Wireless Networks
Venue: arXiv cs.CR 2022
Year: 2022

While the millimeter-wave (mmWave) communication is robust against the conventional wiretapping attack due to its short transmission range and directivity, this paper proposes a new opportunistic wiretapping and jamming (OWJ) attack model in mmWave wireless networks. With OWJ, an eavesdropper can opportunistically conduct wiretapping or jamming to initiate a more hazardous attack based on the instantaneous costs of wiretapping and jamming. We also provide three realizations of the OWJ attack, which are mainly determined by the cost models relevant to distance, path loss and received power, respectively. To understand the impact of the new attack on mmWave network security, we first develop novel approximation techniques to characterize the irregular distributions of wiretappers, jammers and interferers under three OWJ realizations. With the help of the results of node distributions, we then derive analytical expressions for the secrecy transmission capacity to depict the network security performance under OWJ. Finally, we provide extensive numerical results to illustrate the effect of OWJ and to demonstrate that the new attack can more significantly degrade the network security performance than the pure wiretapping or jamming attack.
Knowledge Condensation Distillation
Venue: arXiv cs.CV 2022
Year: 2022

Knowledge Distillation (KD) transfers the knowledge from a high-capacity teacher network to strengthen a smaller student. Existing methods focus on excavating the knowledge hints and transferring the whole knowledge to the student. However, the knowledge redundancy arises since the knowledge shows different values to the student at different learning stages. In this paper, we propose Knowledge Condensation Distillation (KCD). Specifically, the knowledge value on each sample is dynamically estimated, based on which an Expectation-Maximization (EM) framework is forged to iteratively condense a compact knowledge set from the teacher to guide the student learning. Our approach is easy to build on top of the off-the-shelf KD methods, with no extra training parameters and negligible computation overhead. Thus, it presents one new perspective for KD, in which the student that actively identifies teacher's knowledge in line with its aptitude can learn to learn more effectively and efficiently. Experiments on standard benchmarks manifest that the proposed KCD can well boost the performance of student model with even higher distillation efficiency. Code is available at https://github.com/dzy3/KCD.