OpenRaiser/Intern-Atlas huggingface dataset:
Beamforming Design for Intelligent Reflecting Surface-Enhanced Symbiotic Radio Systems
Venue: arXiv cs.IT 2021
Year: 2021

This paper investigates multiuser multi-input single-output downlink symbiotic radio communication systems assisted by an intelligent reflecting surface (IRS). Different from existing methods ideally assuming the secondary user (SU) can jointly decode information symbols from both the access point (AP) and the IRS via multiuser detection, we consider a more practical SU that only non-coherent detection is available. To characterize the non-coherent decoding performance, a practical upper bound of the average symbol error rate (SER) is derived. Subsequently, we jointly optimize the beamformer at the AP and the phase shifts at the IRS to maximize the average sum-rate of the primary system taking into account the maximum tolerable SER constraint for the SU. To circumvent the couplings of variables, we exploit the Schur complement that facilitates the design of a suboptimal beamforming algorithm based on successive convex approximation. Our simulation results show that compared with various benchmark algorithms, the proposed scheme significantly improves the average sum-rate of the primary system, while guaranteeing the decoding performance of the secondary system.
LMSOC: An Approach for Socially Sensitive Pretraining
Venue: arXiv cs.CL 2021
Year: 2021

While large-scale pretrained language models have been shown to learn effective linguistic representations for many NLP tasks, there remain many real-world contextual aspects of language that current approaches do not capture. For instance, consider a cloze-test "I enjoyed the ____ game this weekend": the correct answer depends heavily on where the speaker is from, when the utterance occurred, and the speaker's broader social milieu and preferences. Although language depends heavily on the geographical, temporal, and other social contexts of the speaker, these elements have not been incorporated into modern transformer-based language models. We propose a simple but effective approach to incorporate speaker social context into the learned representations of large-scale language models. Our method first learns dense representations of social contexts using graph representation learning algorithms and then primes language model pretraining with these social context representations. We evaluate our approach on geographically-sensitive language-modeling tasks and show a substantial improvement (more than 100% relative lift on MRR) compared to baselines.
GTM: Gray Temporal Model for Video Recognition
Venue: arXiv cs.CV 2021
Year: 2021

Data input modality plays an important role in video action recognition. Normally, there are three types of input: RGB, flow stream and compressed data. In this paper, we proposed a new input modality: gray stream. Specifically, taken the stacked consecutive 3 gray images as input, which is the same size of RGB, can not only skip the conversion process from video decoding data to RGB, but also improve the spatio-temporal modeling ability at zero computation and zero parameters. Meanwhile, we proposed a 1D Identity Channel-wise Spatio-temporal Convolution(1D-ICSC) which captures the temporal relationship at channel-feature level within a controllable computation budget(by parameters G & R). Finally, we confirm its effectiveness and efficiency on several action recognition benchmarks, such as Kinetics, Something-Something, HMDB-51 and UCF-101, and achieve impressive results.
Dynamic Multi-Person Mesh Recovery From Uncalibrated Multi-View Cameras
Venue: arXiv cs.CV 2021
Year: 2021

Dynamic multi-person mesh recovery has been a hot topic in 3D vision recently. However, few works focus on the multi-person motion capture from uncalibrated cameras, which mainly faces two challenges: the one is that inter-person interactions and occlusions introduce inherent ambiguities for both camera calibration and motion capture; The other is that a lack of dense correspondences can be used to constrain sparse camera geometries in a dynamic multi-person scene. Our key idea is incorporating motion prior knowledge into simultaneous optimization of extrinsic camera parameters and human meshes from noisy human semantics. First, we introduce a physics-geometry consistency to reduce the low and high frequency noises of the detected human semantics. Then a novel latent motion prior is proposed to simultaneously optimize extrinsic camera parameters and coherent human motions from slightly noisy inputs. Experimental results show that accurate camera parameters and human motions can be obtained through one-stage optimization. The codes will be publicly available at~\url{https://www.yangangwang.com}.
Hierarchical Aspect-guided Explanation Generation for Explainable Recommendation
Venue: arXiv cs.CL 2021
Year: 2021

Explainable recommendation systems provide explanations for recommendation results to improve their transparency and persuasiveness. The existing explainable recommendation methods generate textual explanations without explicitly considering the user's preferences on different aspects of the item. In this paper, we propose a novel explanation generation framework, named Hierarchical Aspect-guided explanation Generation (HAG), for explainable recommendation. Specifically, HAG employs a review-based syntax graph to provide a unified view of the user/item details. An aspect-guided graph pooling operator is proposed to extract the aspect-relevant information from the review-based syntax graphs to model the user's preferences on an item at the aspect level. Then, a hierarchical explanation decoder is developed to generate aspects and aspect-relevant explanations based on the attention mechanism. The experimental results on three real datasets indicate that HAG outperforms state-of-the-art explanation generation methods in both single-aspect and multi-aspect explanation generation tasks, and also achieves comparable or even better preference prediction accuracy than strong baseline methods.
FoV Privacy-aware VR Streaming
Venue: arXiv cs.MM 2021
Year: 2021

Proactive tile-based virtual reality (VR) video streaming can use the trace of FoV and eye movement to predict future requested tiles, then renders and delivers the predicted tiles before playback. The quality of experience (QoE) depends on the combined effect of tile prediction and consumed resources. Recently, it has been found that with the FoV and eye movement data collected for a user, one can infer the identity and preference of the user. Existing works investigate the privacy protection for eye movement, but never address how to protect the privacy in terms of FoV and how the privacy protection affects the QoE. In this paper, we strive to characterize and satisfy the FoV privacy requirement. We consider "trading resources for privacy". We first add camouflaged tile requests around the real FoV and define spatial degree of privacy (SDoP) as a normalized number of camouflaged tile requests. By consuming more resources to ensure SDoP, the real FoVs can be hidden. Then, we proceed to analyze the impacts of SDoP on the QoE by jointly optimizing the durations for prediction, computing, and transmission that maximizes the QoE given arbitrary predictor, configured resources, and SDoP. We find that a larger SDoP requires more resources but degrades the performance of tile prediction. Simulation with state-of-the-art predictors on a real dataset verifies the analysis and shows that a user requiring a larger SDoP can be served with better QoE.
A generalised phase field model for fatigue crack growth in elastic-plastic solids with an efficient monolithic solver
Venue: arXiv cs.CE 2021
Year: 2021

We present a generalised phase field-based formulation for predicting fatigue crack growth in metals. The theoretical framework aims at covering a wide range of material behaviour. Different fatigue degradation functions are considered and their influence is benchmarked against experiments. The phase field constitutive theory accommodates the so-called AT1, AT2 and phase field-cohesive zone (PF-CZM) models. In regards to material deformation, both non-linear kinematic and isotropic hardening are considered, as well as the combination of the two. Moreover, a monolithic solution scheme based on quasi-Newton algorithms is presented and shown to significantly outperform staggered approaches. The potential of the computational framework is demonstrated by investigating several 2D and 3D boundary value problems of particular interest. Constitutive and numerical choices are compared and insight is gained into their differences and similarities. The framework enables predicting fatigue crack growth in arbitrary geometries and for materials exhibiting complex (cyclic) deformation and damage responses. The finite element code developed is made freely available at www.empaneda.com/codes.
Reconstruction of Fragmented Trajectories of Collective Motion using Hadamard Deep Autoencoders
Venue: arXiv cs.LG 2021
Year: 2021

Learning dynamics of collectively moving agents such as fish or humans is an active field in research. Due to natural phenomena such as occlusion and change of illumination, the multi-object methods tracking such dynamics might lose track of the agents where that might result fragmentation in the constructed trajectories. Here, we present an extended deep autoencoder (DA) that we train only on fully observed segments of the trajectories by defining its loss function as the Hadamard product of a binary indicator matrix with the absolute difference between the outputs and the labels. The trajectories of the agents practicing collective motion is low-rank due to mutual interactions and dependencies between the agents that we utilize as the underlying pattern that our Hadamard deep autoencoder (HDA) codes during its training. The performance of our HDA is compared with that of a low-rank matrix completion scheme in the context of fragmented trajectory reconstruction.
Knowledge distillation from language model to acoustic model: a hierarchical multi-task learning approach
Venue: arXiv cs.LG 2021
Year: 2021

The remarkable performance of the pre-trained language model (LM) using self-supervised learning has led to a major paradigm shift in the study of natural language processing. In line with these changes, leveraging the performance of speech recognition systems with massive deep learning-based LMs is a major topic of speech recognition research. Among the various methods of applying LMs to speech recognition systems, in this paper, we focus on a cross-modal knowledge distillation method that transfers knowledge between two types of deep neural networks with different modalities. We propose an acoustic model structure with multiple auxiliary output layers for cross-modal distillation and demonstrate that the proposed method effectively compensates for the shortcomings of the existing label-interpolation-based distillation method. In addition, we extend the proposed method to a hierarchical distillation method using LMs trained in different units (senones, monophones, and subwords) and reveal the effectiveness of the hierarchical distillation method through an ablation study.
A Study On Data Augmentation In Voice Anti-Spoofing
Venue: arXiv cs.SD 2021
Year: 2021

In this paper, we perform an in-depth study of how data augmentation techniques improve synthetic or spoofed audio detection. Specifically, we propose methods to deal with channel variability, different audio compressions, different band-widths, and unseen spoofing attacks, which have all been shown to significantly degrade the performance of audio-based systems and Anti-Spoofing systems. Our results are based on the ASVspoof 2021 challenge, in the Logical Access (LA) and Deep Fake (DF) categories. Our study is Data-Centric, meaning that the models are fixed and we significantly improve the results by making changes in the data. We introduce two forms of data augmentation - compression augmentation for the DF part, compression & channel augmentation for the LA part. In addition, a new type of online data augmentation, SpecAverage, is introduced in which the audio features are masked with their average value in order to improve generalization. Furthermore, we introduce a Log spectrogram feature design that improved the results. Our best single system and fusion scheme both achieve state-of-the-art performance in the DF category, with an EER of 15.46% and 14.46% respectively. Our best system for the LA task reduced the best baseline EER by 50% and the min t-DCF by 16%. Our techniques to deal with spoofed data from a wide variety of distributions can be replicated and can help anti-spoofing and speech-based systems enhance their results.
Detecting and Identifying Optical Signal Attacks on Autonomous Driving Systems
Venue: arXiv cs.CV 2021
Year: 2021

For autonomous driving, an essential task is to detect surrounding objects accurately. To this end, most existing systems use optical devices, including cameras and light detection and ranging (LiDAR) sensors, to collect environment data in real time. In recent years, many researchers have developed advanced machine learning models to detect surrounding objects. Nevertheless, the aforementioned optical devices are vulnerable to optical signal attacks, which could compromise the accuracy of object detection. To address this critical issue, we propose a framework to detect and identify sensors that are under attack. Specifically, we first develop a new technique to detect attacks on a system that consists of three sensors. Our main idea is to: 1) use data from three sensors to obtain two versions of depth maps (i.e., disparity) and 2) detect attacks by analyzing the distribution of disparity errors. In our study, we use real data sets and the state-of-the-art machine learning model to evaluate our attack detection scheme and the results confirm the effectiveness of our detection method. Based on the detection scheme, we further develop an identification model that is capable of identifying up to n-2 attacked sensors in a system with one LiDAR and n cameras. We prove the correctness of our identification scheme and conduct experiments to show the accuracy of our identification method. Finally, we investigate the overall sensitivity of our framework.
FairNet: A Measurement Framework for Traffic Discrimination Detection on the Internet
Venue: arXiv cs.NI 2021
Year: 2021

Network neutrality is related to the non-discriminatory treatment of packets on the Internet. Any deliberate discrimination of traffic of one application while favoring others violates the principle of neutrality. Many countries have enforced laws against such discrimination. To enforce such laws, one requires tools to detect any net neutrality violations. However, detecting such violations is challenging as it is hard to separate any degradation in quality due to natural network effects and selective degradation. Also, legitimate traffic management and deliberate discrimination methods can be technically the same, making it further challenging to distinguish them. We developed an end-to-end measurement framework named FairNet to detect discrimination of traffic. It compares the performance of similar services. Our focus is on HTTPS streaming services which constitute a predominant portion of the Internet traffic. The effect of confounding factors (congestion, traffic management policy, dynamic rate adaptation) is made `similar' on the test services to ensure a fair comparison. FairNet framework uses a ``replay server'' and user-client that exchanges correctly identifiable traffic streams over the Internet. The Server Name Indication (SNI) field in the TLS handshake, which goes in plaintext, ensures that the traffic from the replay server appears to network middle-boxes as that coming from its actual server. We validated that appropriate SNIs results in the correct classification of services using a commercial traffic shaper. FairNet uses two novel algorithms based on application-level throughput and connection status to detect traffic discrimination. We also validated the methodology's effectiveness by collecting network logs through mobile apps over the live Internet and analyzing them.
Investigating Reversibility of Steps in Petri Nets
Venue: arXiv cs.FL 2021
Year: 2021

In reversible computations one is interested in the development of mechanisms allowing to undo the effects of executed actions. The past research has been concerned mainly with reversing single actions. In this paper, we consider the problem of reversing the effect of the execution of groups of actions (steps). Using Petri nets as a system model, we introduce concepts related to this new scenario, generalising notions used in the single action case. We then present properties arising when reverse actions are allowed in place/transition nets (pt-nets). We obtain both positive and negative results, showing that allowing steps makes reversibility more problematic than in the interleaving/sequential case. In particular, we demonstrate that there is a crucial difference between reversing steps which are sets and those which are true multisets. Moreover, in contrast to sequential semantics, splitting reverses does not lead to a general method for reversing bounded pt-nets. We then show that a suitable solution can be obtained by combining split reverses with weighted read arcs.
Few-Shot Temporal Action Localization with Query Adaptive Transformer
Venue: arXiv cs.CV 2021
Year: 2021

Existing temporal action localization (TAL) works rely on a large number of training videos with exhaustive segment-level annotation, preventing them from scaling to new classes. As a solution to this problem, few-shot TAL (FS-TAL) aims to adapt a model to a new class represented by as few as a single video. Exiting FS-TAL methods assume trimmed training videos for new classes. However, this setting is not only unnatural actions are typically captured in untrimmed videos, but also ignores background video segments containing vital contextual cues for foreground action segmentation. In this work, we first propose a new FS-TAL setting by proposing to use untrimmed training videos. Further, a novel FS-TAL model is proposed which maximizes the knowledge transfer from training classes whilst enabling the model to be dynamically adapted to both the new class and each video of that class simultaneously. This is achieved by introducing a query adaptive Transformer in the model. Extensive experiments on two action localization benchmarks demonstrate that our method can outperform all the state of the art alternatives significantly in both single-domain and cross-domain scenarios. The source code can be found in https://github.com/sauradip/fewshotQAT
Why Settle for Just One? Extending EL++ Ontology Embeddings with Many-to-Many Relationships
Venue: arXiv cs.AI 2021
Year: 2021

Knowledge Graph (KG) embeddings provide a low-dimensional representation of entities and relations of a Knowledge Graph and are used successfully for various applications such as question answering and search, reasoning, inference, and missing link prediction. However, most of the existing KG embeddings only consider the network structure of the graph and ignore the semantics and the characteristics of the underlying ontology that provides crucial information about relationships between entities in the KG. Recent efforts in this direction involve learning embeddings for a Description Logic (logical underpinning for ontologies) named EL++. However, such methods consider all the relations defined in the ontology to be one-to-one which severely limits their performance and applications. We provide a simple and effective solution to overcome this shortcoming that allows such methods to consider many-to-many relationships while learning embedding representations. Experiments conducted using three different EL++ ontologies show substantial performance improvement over five baselines. Our proposed solution also paves the way for learning embedding representations for even more expressive description logics such as SROIQ.
Exploring the Relationship Between "Positive Risk Balance" and "Absence of Unreasonable Risk"
Venue: arXiv cs.CY 2021
Year: 2021

International discussions on the overarching topic of how to define and quantify what a "safe enough" Automated Driving System (ADS) is are currently hinged on the question of determining the relationship between "positive risk balance" (PRB) and "absence of unreasonable risk" (AUR). In order to advance the conversation on these important safety topics at the international level, it is first important to start from a shared common understanding, grounded in clear definitions and terminology. To that end, this paper will start with an overview of the notions of PRB and AUR; it will then summarize different positions of the present debate; finally, it will conclude that two possible interpretations exist for PRB, and that failure to distinguish them can lead to misunderstanding different parties' positions. The argumentation in this paper is aimed at showing that the two interpretations for PRB can actually complement each other, but can be considered independently, and can both be subsumed within non-prescriptive guidelines toward ADS safety assurance.
Video Instance Segmentation by Instance Flow Assembly
Venue: arXiv cs.CV 2021
Year: 2021

Instance segmentation is a challenging task aiming at classifying and segmenting all object instances of specific classes. While two-stage box-based methods achieve top performances in the image domain, they cannot easily extend their superiority into the video domain. This is because they usually deal with features or images cropped from the detected bounding boxes without alignment, failing to capture pixel-level temporal consistency. We embrace the observation that bottom-up methods dealing with box-free features could offer accurate spacial correlations across frames, which can be fully utilized for object and pixel level tracking. We first propose our bottom-up framework equipped with a temporal context fusion module to better encode inter-frame correlations. Intra-frame cues for semantic segmentation and object localization are simultaneously extracted and reconstructed by corresponding decoders after a shared backbone. For efficient and robust tracking among instances, we introduce an instance-level correspondence across adjacent frames, which is represented by a center-to-center flow, termed as instance flow, to assemble messy dense temporal correspondences. Experiments demonstrate that the proposed method outperforms the state-of-the-art online methods (taking image-level input) on the challenging Youtube-VIS dataset.
Maximal Information Propagation via Lotteries
Venue: arXiv cs.GT 2021
Year: 2021

Propagating information to more people through their friends is becoming an increasingly important technology used in domains such as blockchain, advertising, and social media. To incentivize people to broadcast the information, the designer may use a monetary rewarding scheme, which specifies who gets how much, to compensate for the propagation. Several properties are desirable for the rewarding scheme, such as budget feasible, individually rational, incentive compatible, and Sybil-proof. In this work, we design a free market with lotteries, where every participant can decide by herself how much of the reward she wants to withhold before propagating to others. We show that in the free market, the participants have a strong incentive to maximally propagate the information and all the above properties are satisfied automatically.