OpenRaiser/Intern-Atlas huggingface dataset:
Incidental Data: Observation of Privacy Compromising Data on Social Media Platforms
Venue: arXiv cs.IR 2022
Year: 2022

Social media plays an important role for a vast majority in one's internet life. Likewise, sharing, publishing and posting content through social media became nearly effortless. This unleashes new threats as unintentionally shared information may be used against oneself or beloved ones. With open source intelligence data and methods, we show how unindented published data can be revealed and further analyze possibilities that can potentially compromise one's privacy. This is backed up by a popular view from interviewed experts from various fields of expertise. We were able to show that only 2 hours of manually fetching data are sufficient in order to unveil private personal information that was not intended to be published by the person. Two distinctive methods are described with several approaches. From our results, we were able to describe a thirteen-step awareness guideline and proposed a change of law within Austrian legislation. Our work has shown that awareness among persons on social media needs to be raised. Critically reflecting on our work has revealed several ethical implications that made countermeasures necessary; however, it can be assumed that criminals do not do that.
Resisting Adversarial Attacks in Deep Neural Networks using Diverse Decision Boundaries
Venue: arXiv cs.LG 2022
Year: 2022

The security of deep learning (DL) systems is an extremely important field of study as they are being deployed in several applications due to their ever-improving performance to solve challenging tasks. Despite overwhelming promises, the deep learning systems are vulnerable to crafted adversarial examples, which may be imperceptible to the human eye, but can lead the model to misclassify. Protections against adversarial perturbations on ensemble-based techniques have either been shown to be vulnerable to stronger adversaries or shown to lack an end-to-end evaluation. In this paper, we attempt to develop a new ensemble-based solution that constructs defender models with diverse decision boundaries with respect to the original model. The ensemble of classifiers constructed by (1) transformation of the input by a method called Split-and-Shuffle, and (2) restricting the significant features by a method called Contrast-Significant-Features are shown to result in diverse gradients with respect to adversarial attacks, which reduces the chance of transferring adversarial examples from the original to the defender model targeting the same class. We present extensive experimentations using standard image classification datasets, namely MNIST, CIFAR-10 and CIFAR-100 against state-of-the-art adversarial attacks to demonstrate the robustness of the proposed ensemble-based defense. We also evaluate the robustness in the presence of a stronger adversary targeting all the models within the ensemble simultaneously. Results for the overall false positives and false negatives have been furnished to estimate the overall performance of the proposed methodology.
Deep Neural Network Approximation of Invariant Functions through Dynamical Systems
Venue: arXiv cs.LG 2022
Year: 2022

We study the approximation of functions which are invariant with respect to certain permutations of the input indices using flow maps of dynamical systems. Such invariant functions includes the much studied translation-invariant ones involving image tasks, but also encompasses many permutation-invariant functions that finds emerging applications in science and engineering. We prove sufficient conditions for universal approximation of these functions by a controlled equivariant dynamical system, which can be viewed as a general abstraction of deep residual networks with symmetry constraints. These results not only imply the universal approximation for a variety of commonly employed neural network architectures for symmetric function approximation, but also guide the design of architectures with approximation guarantees for applications involving new symmetry requirements.
Temporal Up-Sampling for Asynchronous Events
Venue: arXiv cs.CV 2022
Year: 2022

The event camera is a novel bio-inspired vision sensor. When the brightness change exceeds the preset threshold, the sensor generates events asynchronously. The number of valid events directly affects the performance of event-based tasks, such as reconstruction, detection, and recognition. However, when in low-brightness or slow-moving scenes, events are often sparse and accompanied by noise, which poses challenges for event-based tasks. To solve these challenges, we propose an event temporal up-sampling algorithm1 to generate more effective and reliable events. The main idea of our algorithm is to generate up-sampling events on the event motion trajectory. First, we estimate the event motion trajectory by contrast maximization algorithm and then up-sampling the events by temporal point processes. Experimental results show that up-sampling events can provide more effective information and improve the performance of downstream tasks, such as improving the quality of reconstructed images and increasing the accuracy of object detection.
Exploring and Exploiting Multi-Granularity Representations for Machine Reading Comprehension
Venue: arXiv cs.CL 2022
Year: 2022

Recently, the attention-enhanced multi-layer encoder, such as Transformer, has been extensively studied in Machine Reading Comprehension (MRC). To predict the answer, it is common practice to employ a predictor to draw information only from the final encoder layer which generates the coarse-grained representations of the source sequences, i.e., passage and question. The analysis shows that the representation of source sequence becomes more coarse-grained from finegrained as the encoding layer increases. It is generally believed that with the growing number of layers in deep neural networks, the encoding process will gather relevant information for each location increasingly, resulting in more coarse-grained representations, which adds the likelihood of similarity to other locations (referring to homogeneity). Such phenomenon will mislead the model to make wrong judgement and degrade the performance. In this paper, we argue that it would be better if the predictor could exploit representations of different granularity from the encoder, providing different views of the source sequences, such that the expressive power of the model could be fully utilized. To this end, we propose a novel approach called Adaptive Bidirectional Attention-Capsule Network (ABA-Net), which adaptively exploits the source representations of different levels to the predictor. Furthermore, due to the better representations are at the core for boosting MRC performance, the capsule network and self-attention module are carefully designed as the building blocks of our encoders, which provides the capability to explore the local and global representations, respectively. Experimental results on three benchmark datasets, i.e., SQuAD 1.0, SQuAD 2.0 and COQA, demonstrate the effectiveness of our approach. In particular, we set the new state-of-the-art performance on the SQuAD 1.0 dataset
Rate Splitting in MIMO RIS-assisted Systems with Hardware Impairments and Improper Signaling
Venue: arXiv cs.IT 2022
Year: 2022

In this paper, we propose an optimization framework for rate splitting (RS) techniques in multiple-input multiple-output (MIMO) reconfigurable intelligent surface (RIS)-assisted systems, possibly with I/Q imbalance (IQI). This framework can be applied to any optimization problem in which the objective and/or constraints are linear functions of the rates and/or transmit covariance matrices. Such problems include minimum-weighted and weighted-sum rate maximization, total power minimization for a target rate, minimum-weighted energy efficiency (EE) and global EE maximization. The framework may be applied to any interference-limited system with hardware impairments. For the sake of illustration, we consider a multicell MIMO RIS-assisted broadcast channel (BC) in which the base stations (BSs) and/or the users may suffer from IQI. Since IQI generates improper noise, we consider improper Gaussian signaling (IGS) as an interference-management technique that can additionally compensate for IQI. We show that RS when combined with IGS can substantially improve the spectral and energy efficiency of overloaded networks (i.e., when the number of users per cell is larger than the number of transmit/receive antennas).
Blockchain based digital vaccine passport
Venue: arXiv cs.CR 2022
Year: 2022

Travel has been challenging recently since different nations have implemented varied immigration and travel policies. For the time being, immigration officials want proof of each person's immunity to the virus. A vaccine passport serves as evidence that a person has tested negative for or is immune to a particular virus. In terms of COVID-19, those who hold a vaccine passport will be permitted entry into other nations as long as they can provide proof that they have COVID-19 antibodies from prior infections or from full COVID-19 immunizations. To reduce time and effort spent managing data, the vaccination passport system has been digitalized. The process of contact tracing may be facilitated by digitization. The "Blockchain technology" system, which is currently in use, has demonstrated its security and privacy in systems for data exchange among bitcoin users. The Digital Vaccination Passport scheme can use Blockchain technology. The end result would be a decentralized, traceable, transparent, reliable, auditable, secure, and trustworthy solution based on the Ethereum block-chain that would allow tracking of vaccines given and the history of diseases.
An Adjustable Farthest Point Sampling Method for Approximately-sorted Point Cloud Data
Venue: arXiv cs.CV 2022
Year: 2022

Sampling is an essential part of raw point cloud data processing such as in the popular PointNet++ scheme. Farthest Point Sampling (FPS), which iteratively samples the farthest point and performs distance updating, is one of the most popular sampling schemes. Unfortunately it suffers from low efficiency and can become the bottleneck of point cloud applications. We propose adjustable FPS (AFPS), parameterized by M, to aggressively reduce the complexity of FPS without compromising on the sampling performance. Specifically, it divides the original point cloud into M small point clouds and samples M points simultaneously. It exploits the dimensional locality of an approximately sorted point cloud data to minimize its performance degradation. AFPS method can achieve 22 to 30x speedup over original FPS. Furthermore, we propose the nearest-point-distance-updating (NPDU) method to limit the number of distance updates to a constant number. The combined NPDU on AFPS method can achieve a 34-280x speedup on a point cloud with 2K-32K points with algorithmic performance that is comparable to the original FPS. For instance, for the ShapeNet part segmentation task, it achieves 0.8490 instance average mIoU (mean Intersection of Union), which is only 0.0035 drop compared to the original FPS.
Algorithms and Bounds for Complex and Quaternionic Lattices With Application to MIMO Transmission
Venue: arXiv cs.IT 2022
Year: 2022

Lattices are a popular field of study in mathematical research, but also in more practical areas like cryptology or multiple-input/multiple-output (MIMO) transmission. In mathematical theory, most often lattices over real numbers are considered. However, in communications, complex-valued processing is usually of interest. Besides, by the use of dual-polarized transmission as well as by the combination of two time slots or frequencies, four-dimensional (quaternion-valued) approaches become more and more important. Hence, to account for this fact, well-known lattice algorithms and related concepts are generalized in this work. To this end, a brief review of complex arithmetic, including the sets of Gaussian and Eisenstein integers, and an introduction to quaternion-valued numbers, including the sets of Lipschitz and Hurwitz integers, are given. On that basis, generalized variants of two important algorithms are derived: first, of the polynomial-time LLL algorithm, resulting in a reduced basis of a lattice by performing a special variant of the Euclidean algorithm defined for matrices, and second, of an algorithm to calculate the successive minima - the norms of the shortest independent vectors of a lattice - and its related lattice points. Generalized bounds for the quality of the particular results are established and the asymptotic complexities of the algorithms are assessed. These findings are extensively compared to conventional real-valued processing. It is shown that the generalized approaches outperform their real-valued counterparts in complexity and/or quality aspects. Moreover, the application of the generalized algorithms to MIMO communications is studied, particularly in the field of lattice-reduction-aided and integer-forcing equalization.
COPE: End-to-end trainable Constant Runtime Object Pose Estimation
Venue: arXiv cs.CV 2022
Year: 2022

State-of-the-art object pose estimation handles multiple instances in a test image by using multi-model formulations: detection as a first stage and then separately trained networks per object for 2D-3D geometric correspondence prediction as a second stage. Poses are subsequently estimated using the Perspective-n-Points algorithm at runtime. Unfortunately, multi-model formulations are slow and do not scale well with the number of object instances involved. Recent approaches show that direct 6D object pose estimation is feasible when derived from the aforementioned geometric correspondences. We present an approach that learns an intermediate geometric representation of multiple objects to directly regress 6D poses of all instances in a test image. The inherent end-to-end trainability overcomes the requirement of separately processing individual object instances. By calculating the mutual Intersection-over-Unions, pose hypotheses are clustered into distinct instances, which achieves negligible runtime overhead with respect to the number of object instances. Results on multiple challenging standard datasets show that the pose estimation performance is superior to single-model state-of-the-art approaches despite being more than ~35 times faster. We additionally provide an analysis showing real-time applicability (>24 fps) for images where more than 90 object instances are present. Further results show the advantage of supervising geometric-correspondence-based object pose estimation with the 6D pose.
Oh SSH-it, what's my fingerprint? A Large-Scale Analysis of SSH Host Key Fingerprint Verification Records in the DNS
Venue: arXiv cs.CR 2022
Year: 2022

The SSH protocol is commonly used to access remote systems on the Internet, as it provides an encrypted and authenticated channel for communication. If upon establishing a new connection, the presented server key is unknown to the client, the user is asked to verify the key fingerprint manually, which is prone to errors and often blindly trusted. The SSH standard describes an alternative to such manual key verification: using the Domain Name System (DNS) to publish the server key information in SSHFP records. In this paper, we conduct a large-scale Internet study to measure the prevalence of SSHFP records among DNS domain names. We scan the Tranco 1M list and over 500 million names from the certificate transparency log over the course of 26 days. The results show that in two studied populations, about 1 in 10,000 domains has SSHFP records, with more than half of them deployed without using DNSSEC, drastically reducing security benefits.
Brand Celebrity Matching Model Based on Natural Language Processing
Venue: arXiv cs.CL 2022
Year: 2022

Celebrity Endorsement is one of the most significant strategies in brand communication. Nowadays, more and more companies try to build a vivid characteristic for themselves. Therefore, their brand identity communications should accord with some characteristics as humans and regulations. However, the previous works mostly stop by assumptions, instead of proposing a specific way to perform matching between brands and celebrities. In this paper, we propose a brand celebrity matching model (BCM) based on Natural Language Processing (NLP) techniques. Given a brand and a celebrity, we firstly obtain some descriptive documents of them from the Internet, then summarize these documents, and finally calculate a matching degree between the brand and the celebrity to determine whether they are matched. According to the experimental result, our proposed model outperforms the best baselines with a 0.362 F1 score and 6.3% of accuracy, which indicates the effectiveness and application value of our model in the real-world scene. What's more, to our best knowledge, the proposed BCM model is the first work on using NLP to solve endorsement issues, so it can provide some novel research ideas and methodologies for the following works.
NeIF: Representing General Reflectance as Neural Intrinsics Fields for Uncalibrated Photometric Stereo
Venue: arXiv cs.CV 2022
Year: 2022

Uncalibrated photometric stereo (UPS) is challenging due to the inherent ambiguity brought by unknown light. Existing solutions alleviate the ambiguity by either explicitly associating reflectance to light conditions or resolving light conditions in a supervised manner. This paper establishes an implicit relation between light clues and light estimation and solves UPS in an unsupervised manner. The key idea is to represent the reflectance as four neural intrinsics fields, i.e., position, light, specular, and shadow, based on which the neural light field is implicitly associated with light clues of specular reflectance and cast shadow. The unsupervised, joint optimization of neural intrinsics fields can be free from training data bias as well as accumulating error, and fully exploits all observed pixel values for UPS. Our method achieves a superior performance advantage over state-of-the-art UPS methods on public and self-collected datasets, under regular and challenging setups. The code will be released soon.
Conviformers: Convolutionally guided Vision Transformer
Venue: arXiv cs.CV 2022
Year: 2022

Vision transformers are nowadays the de-facto choice for image classification tasks. There are two broad categories of classification tasks, fine-grained and coarse-grained. In fine-grained classification, the necessity is to discover subtle differences due to the high level of similarity between sub-classes. Such distinctions are often lost as we downscale the image to save the memory and computational cost associated with vision transformers (ViT). In this work, we present an in-depth analysis and describe the critical components for developing a system for the fine-grained categorization of plants from herbarium sheets. Our extensive experimental analysis indicated the need for a better augmentation technique and the ability of modern-day neural networks to handle higher dimensional images. We also introduce a convolutional transformer architecture called Conviformer which, unlike the popular Vision Transformer (ConViT), can handle higher resolution images without exploding memory and computational cost. We also introduce a novel, improved pre-processing technique called PreSizer to resize images better while preserving their original aspect ratios, which proved essential for classifying natural plants. With our simple yet effective approach, we achieved SoTA on Herbarium 202x and iNaturalist 2019 dataset.
Learned Indexing in Proteins: Extended Work on Substituting Complex Distance Calculations with Embedding and Clustering Techniques
Venue: arXiv cs.IR 2022
Year: 2022

Despite the constant evolution of similarity searching research, it continues to face the same challenges stemming from the complexity of the data, such as the curse of dimensionality and computationally expensive distance functions. Various machine learning techniques have proven capable of replacing elaborate mathematical models with combinations of simple linear functions, often gaining speed and simplicity at the cost of formal guarantees of accuracy and correctness of querying. The authors explore the potential of this research trend by presenting a lightweight solution for the complex problem of 3D protein structure search. The solution consists of three steps -- (i) transformation of 3D protein structural information into very compact vectors, (ii) use of a probabilistic model to group these vectors and respond to queries by returning a given number of similar objects, and (iii) a final filtering step which applies basic vector distance functions to refine the result.
Communication-Efficient Collaborative Best Arm Identification
Venue: arXiv cs.LG 2022
Year: 2022

We investigate top-$m$ arm identification, a basic problem in bandit theory, in a multi-agent learning model in which agents collaborate to learn an objective function. We are interested in designing collaborative learning algorithms that achieve maximum speedup (compared to single-agent learning algorithms) using minimum communication cost, as communication is frequently the bottleneck in multi-agent learning. We give both algorithmic and impossibility results, and conduct a set of experiments to demonstrate the effectiveness of our algorithms.
Self-Supervised Primal-Dual Learning for Constrained Optimization
Venue: arXiv cs.LG 2022
Year: 2022

This paper studies how to train machine-learning models that directly approximate the optimal solutions of constrained optimization problems. This is an empirical risk minimization under constraints, which is challenging as training must balance optimality and feasibility conditions. Supervised learning methods often approach this challenge by training the model on a large collection of pre-solved instances. This paper takes a different route and proposes the idea of Primal-Dual Learning (PDL), a self-supervised training method that does not require a set of pre-solved instances or an optimization solver for training and inference. Instead, PDL mimics the trajectory of an Augmented Lagrangian Method (ALM) and jointly trains primal and dual neural networks. Being a primal-dual method, PDL uses instance-specific penalties of the constraint terms in the loss function used to train the primal network. Experiments show that, on a set of nonlinear optimization benchmarks, PDL typically exhibits negligible constraint violations and minor optimality gaps, and is remarkably close to the ALM optimization. PDL also demonstrated improved or similar performance in terms of the optimality gaps, constraint violations, and training times compared to existing approaches.
Electronic, Wireless, and Photonic Network-on-Chip Security: Challenges and Countermeasures
Venue: arXiv cs.AR 2022
Year: 2022

Networks-on-chips (NoCs) are an integral part of emerging manycore computing chips. They play a key role in facilitating communication among processing cores and between cores and memory. To meet the aggressive performance and energy-efficiency targets of machine learning and big data applications, NoCs have been evolving to leverage emerging paradigms such as silicon photonics and wireless communication. Increasingly, these NoC fabrics are becoming susceptible to security vulnerabilities, such as from hardware trojans that can snoop, corrupt, or disrupt information transfers on NoCs. This article surveys the landscape of security challenges and countermeasures across electronic, wireless, and photonic NoCs.