OpenRaiser/Intern-Atlas huggingface dataset:
Dynamical Audio-Visual Navigation: Catching Unheard Moving Sound Sources in Unmapped 3D Environments
Venue: arXiv cs.CV 2022
Year: 2022

Recent work on audio-visual navigation targets a single static sound in noise-free audio environments and struggles to generalize to unheard sounds. We introduce the novel dynamic audio-visual navigation benchmark in which an embodied AI agent must catch a moving sound source in an unmapped environment in the presence of distractors and noisy sounds. We propose an end-to-end reinforcement learning approach that relies on a multi-modal architecture that fuses the spatial audio-visual information from a binaural audio signal and spatial occupancy maps to encode the features needed to learn a robust navigation policy for our new complex task settings. We demonstrate that our approach outperforms the current state-of-the-art with better generalization to unheard sounds and better robustness to noisy scenarios on the two challenging 3D scanned real-world datasets Replica and Matterport3D, for the static and dynamic audio-visual navigation benchmarks. Our novel benchmark will be made available at http://dav-nav.cs.uni-freiburg.de.
Evolutionary Action Selection for Gradient-based Policy Learning
Venue: arXiv cs.NE 2022
Year: 2022

Evolutionary Algorithms (EAs) and Deep Reinforcement Learning (DRL) have recently been integrated to take the advantage of the both methods for better exploration and exploitation.The evolutionary part in these hybrid methods maintains a population of policy networks.However, existing methods focus on optimizing the parameters of policy network, which is usually high-dimensional and tricky for EA.In this paper, we shift the target of evolution from high-dimensional parameter space to low-dimensional action space.We propose Evolutionary Action Selection-Twin Delayed Deep Deterministic Policy Gradient (EAS-TD3), a novel hybrid method of EA and DRL.In EAS, we focus on optimizing the action chosen by the policy network and attempt to obtain high-quality actions to promote policy learning through an evolutionary algorithm. We conduct several experiments on challenging continuous control tasks.The result shows that EAS-TD3 shows superior performance over other state-of-art methods.
Neural Residual Flow Fields for Efficient Video Representations
Venue: arXiv cs.CV 2022
Year: 2022

Neural fields have emerged as a powerful paradigm for representing various signals, including videos. However, research on improving the parameter efficiency of neural fields is still in its early stages. Even though neural fields that map coordinates to colors can be used to encode video signals, this scheme does not exploit the spatial and temporal redundancy of video signals. Inspired by standard video compression algorithms, we propose a neural field architecture for representing and compressing videos that deliberately removes data redundancy through the use of motion information across video frames. Maintaining motion information, which is typically smoother and less complex than color signals, requires a far fewer number of parameters. Furthermore, reusing color values through motion information further improves the network parameter efficiency. In addition, we suggest using more than one reference frame for video frame reconstruction and separate networks, one for optical flows and the other for residuals. Experimental results have shown that the proposed method outperforms the baseline methods by a significant margin. The code is available in https://github.com/daniel03c1/eff_video_representation
PromptBERT: Improving BERT Sentence Embeddings with Prompts
Venue: arXiv cs.CL 2022
Year: 2022

We propose PromptBERT, a novel contrastive learning method for learning better sentence representation. We firstly analyze the drawback of current sentence embedding from original BERT and find that it is mainly due to the static token embedding bias and ineffective BERT layers. Then we propose the first prompt-based sentence embeddings method and discuss two prompt representing methods and three prompt searching methods to make BERT achieve better sentence embeddings. Moreover, we propose a novel unsupervised training objective by the technology of template denoising, which substantially shortens the performance gap between the supervised and unsupervised settings. Extensive experiments show the effectiveness of our method. Compared to SimCSE, PromptBert achieves 2.29 and 2.58 points of improvement based on BERT and RoBERTa in the unsupervised setting.
Video Intelligence as a component of a Global Security system
Venue: arXiv cs.AI 2022
Year: 2022

This paper describes the evolution of our research from video analytics to a global security system with focus on the video surveillance component. Indeed video surveillance has evolved from a commodity security tool up to the most efficient way of tracking perpetrators when terrorism hits our modern urban centers. As number of cameras soars, one could expect the system to leverage the huge amount of data carried through the video streams to provide fast access to video evidences, actionable intelligence for monitoring real-time events and enabling predictive capacities to assist operators in their surveillance tasks. This research explores a hybrid platform for video intelligence capture, automated data extraction, supervised Machine Learning for intelligently assisted urban video surveillance; Extension to other components of a global security system are discussed. Applying Knowledge Management principles in this research helps with deep problem understanding and facilitates the implementation of efficient information and experience sharing decision support systems providing assistance to people on the field as well as in operations centers. The originality of this work is also the creation of "common" human-machine and machine to machine language and a security ontology.
Efficient Modular Graph Transformation Rule Application
Venue: arXiv cs.DM 2022
Year: 2022

Graph transformation formalisms have proven to be suitable tools for the modelling of chemical reactions. They are well established in theoretical studies and increasingly also in practical applications in chemistry. The latter is made feasible via the development of programming frameworks which makes the formalisms executable. The application of such frameworks to large networks of chemical reactions, however, poses unique computational challenges. One such characteristic is the inherent combinatorial nature of the graphs involved. The graphs consist of many connected components, representing individual molecules. While the existing methods for implementing graph transformations can be applied to such graphs, the combinatorics of constructing graph matches quickly becomes a computational bottleneck as the size of the chemical reaction network grows. In this contribution, we develop a new method of enumerating graph matches during graph transformation rule application. The method is designed to improve performance in such scenarios and is based on constructing graph matches in an iterative, component-wise fashion which allows redundant applications to be detected early and pruned. We further extend the algorithm with an efficient heuristic based on local symmetries of the graphs, which allow us to detect and discard isomorphic applications early. Finally, we conduct chemical network generation experiments on real-life as well as synthetic data and compare against the state-of-the-art algorithm in the field.
GateFormer: Speeding Up News Feed Recommendation with Input Gated Transformers
Venue: arXiv cs.IR 2022
Year: 2022

News feed recommendation is an important web service. In recent years, pre-trained language models (PLMs) have been intensively applied to improve the recommendation quality. However, the utilization of these deep models is limited in many aspects, such as lack of explainability and being incompatible with the existing inverted index systems. Above all, the PLMs based recommenders are inefficient, as the encoding of user-side information will take huge computation costs. Although the computation can be accelerated with efficient transformers or distilled PLMs, it is still not enough to make timely recommendations for the active users, who are associated with super long news browsing histories. In this work, we tackle the efficient news recommendation problem from a distinctive perspective. Instead of relying on the entire input (i.e., the collection of news articles a user ever browsed), we argue that the user's interest can be fully captured merely with those representative keywords. Motivated by this, we propose GateFormer, where the input data is gated before feeding into transformers. The gating module is made personalized, lightweight and end-to-end learnable, such that it may perform accurate and efficient filtering of informative user input. GateFormer achieves highly impressive performances in experiments, where it notably outperforms the existing acceleration approaches in both accuracy and efficiency. We also surprisingly find that even with over 10-fold compression of the original input, GateFormer is still able to maintain on-par performances with the SOTA methods.
Path Transitions Tell More:Optimizing Fuzzing Schedules via Runtime Program States
Venue: arXiv cs.CR 2022
Year: 2022

Coverage-guided Greybox Fuzzing (CGF) is one of the most successful and widely-used techniques for bug hunting. Two major approaches are adopted to optimize CGF: (i) to reduce search space of inputs by inferring relationships between input bytes and path constraints; (ii) to formulate fuzzing processes (e.g., path transitions) and build up probability distributions to optimize power schedules, i.e., the number of inputs generated per seed. However, the former is subjective to the inference results which may include extra bytes for a path constraint, thereby limiting the efficiency of path constraints resolution, code coverage discovery, and bugs exposure; the latter formalization, concentrating on power schedules for seeds alone, is inattentive to the schedule for bytes in a seed. In this paper, we propose a lightweight fuzzing approach, Truzz, to optimize existing Coverage-guided Greybox Fuzzers (CGFs). To address two aforementioned challenges, Truzz identifies the bytes related to the validation checks (i.e., the checks guarding error-handling code), and protects those bytes from being frequently mutated, making most generated inputs examine the functionalities of programs, in lieu of being rejected by validation checks. The byte-wise relationship determination mitigates the problem of loading extra bytes when fuzzers infer the byte-constraint relation. Furthermore, the proposed path transition within Truzz can efficiently prioritize the seed as the new path, harvesting many new edges, and the new path likely belongs to a code region with many undiscovered code lines. The experimental results show that on average, Truzz can generate 16.14% more inputs flowing into functional code, in addition to 24.75% more new edges than the vanilla fuzzers. Finally, our approach exposes 13 bugs in 8 target programs, and 6 of them have not been identified by the vanilla fuzzers.
Obstacle avoidance for blind people using a 3D camera and a haptic feedback sleeve
Venue: arXiv cs.HC 2022
Year: 2022

Navigation and obstacle avoidance are some of the hardest tasks for the visually impaired. Recent research projects have proposed technological solutions to tackle this problem. So far most systems fail to provide multidimensional feedback while working under various lighting conditions. We present a novel obstacle avoidance system by combining a 3D camera with a haptic feedback sleeve. Our system uses the distance information of the camera and maps it onto a 2D vibration array on the forearm. In our functionality tests of the haptic feedback sleeve, users were able to correctly identify and localize 98,6% of single motor vibration patterns and 70% of multidirectional and multi-motor vibration patterns. The combined obstacle avoidance system was evaluated on a testing route in the dark, simulating a navigation task. All users were able to complete the task and showed performance improvement over multiple runs. The system is independent of lighting conditions and can be used indoors and outdoors. Therefore, the obstacle avoidance system demonstrates a promising approach towards using technology to enable more independence for the visually impaired.
How Does Data Corruption Affect Natural Language Understanding Models? A Study on GLUE datasets
Venue: arXiv cs.CL 2022
Year: 2022

A central question in natural language understanding (NLU) research is whether high performance demonstrates the models' strong reasoning capabilities. We present an extensive series of controlled experiments where pre-trained language models are exposed to data that have undergone specific corruption transformations. These involve removing instances of specific word classes and often lead to non-sensical sentences. Our results show that performance remains high on most GLUE tasks when the models are fine-tuned or tested on corrupted data, suggesting that they leverage other cues for prediction even in non-sensical contexts. Our proposed data transformations can be used to assess the extent to which a specific dataset constitutes a proper testbed for evaluating models' language understanding capabilities.
Exact learning and test theory
Venue: arXiv cs.CC 2022
Year: 2022

In this paper, based on results of exact learning and test theory, we study arbitrary infinite binary information systems each of which consists of an infinite set of elements and an infinite set of two-valued functions (attributes) defined on the set of elements. We consider the notion of a problem over information system, which is described by a finite number of attributes: for a given element, we should recognize values of these attributes. As algorithms for problem solving, we consider decision trees of two types: (i) using only proper hypotheses (an analog of proper equivalence queries from exact learning), and (ii) using both attributes and proper hypotheses. As time complexity, we study the depth of decision trees. In the worst case, with the growth of the number of attributes in the problem description, the minimum depth of decision trees of both types either is bounded from above by a constant or grows as a logarithm, or linearly. Based on these results and results obtained earlier for attributes and arbitrary hypotheses, we divide the set of all infinite binary information systems into seven complexity classes.
Theme Analysis of Political Facebook Ads in the 2021 Dutch General Election
Venue: arXiv cs.SI 2022
Year: 2022

Social media platforms have been trying to be more transparent about the political ads they run on their platforms, because the Cambridge Analytica scandal revealed that political campaigns are using social media on a large scale. One such transparency effort is the Facebook Ad Library, a public repository of all political ads run on Facebook and Instagram. This library provides journalist and researchers with data to get a better understanding of political advertising and microtargeting on Facebook's platforms. Unfortunately, the Facebook Ad Library only provides estimates and basic information. This paper analyses political ads in more depth, by examining the themes that ads are about. We provide a method to match themes to political Facebook ads and we apply this method to analyse Facebook ad campaigns ran by Dutch political parties during the 2021 Dutch general election.
A Characterization of Approximability for Biased CSPs
Venue: arXiv cs.DS 2022
Year: 2022

A $μ$-biased Max-CSP instance with predicate $ψ:\{0,1\}^r \to \{0,1\}$ is an instance of Constraint Satisfaction Problem (CSP) where the objective is to find a labeling of relative weight at most $μ$ which satisfies the maximum fraction of constraints. Biased CSPs are versatile and express several well studied problems such as Densest-$k$-Sub(Hyper)graph and SmallSetExpansion. In this work, we explore the role played by the bias parameter $μ$ on the approximability of biased CSPs. We show that the approximability of such CSPs can be characterized (up to loss of factors of arity $r$) using the bias-approximation curve of Densest-$k$-SubHypergraph (DkSH). In particular, this gives a tight characterization of predicates which admit approximation guarantees that are independent of the bias parameter $μ$. Motivated by the above, we give new approximation and hardness results for DkSH. In particular, assuming the Small Set Expansion Hypothesis (SSEH), we show that DkSH with arity $r$ and $k = μn$ is NP-hard to approximate to a factor of $Ω(r^3μ^{r-1}\log(1/μ))$ for every $r \geq 2$ and $μ< 2^{-r}$. We also give a $O(μ^{r-1}\log(1/μ))$-approximation algorithm for the same setting. Our upper and lower bounds are tight up to constant factors, when the arity $r$ is a constant, and in particular, imply the first tight approximation bounds for the Densest-$k$-Subgraph problem in the linear bias regime. Furthermore, using the above characterization, our results also imply matching algorithms and hardness for every biased CSP of constant arity.
When Machine Learning Meets Spectrum Sharing Security: Methodologies and Challenges
Venue: arXiv cs.CR 2022
Year: 2022

The exponential growth of internet connected systems has generated numerous challenges, such as spectrum shortage issues, which require efficient spectrum sharing (SS) solutions. Complicated and dynamic SS systems can be exposed to different potential security and privacy issues, requiring protection mechanisms to be adaptive, reliable, and scalable. Machine learning (ML) based methods have frequently been proposed to address those issues. In this article, we provide a comprehensive survey of the recent development of ML based SS methods, the most critical security issues, and corresponding defense mechanisms. In particular, we elaborate the state-of-the-art methodologies for improving the performance of SS communication systems for various vital aspects, including ML based cognitive radio networks (CRNs), ML based database assisted SS networks, ML based LTE-U networks, ML based ambient backscatter networks, and other ML based SS solutions. We also present security issues from the physical layer and corresponding defending strategies based on ML algorithms, including Primary User Emulation (PUE) attacks, Spectrum Sensing Data Falsification (SSDF) attacks, jamming attacks, eavesdropping attacks, and privacy issues. Finally, extensive discussions on open challenges for ML based SS are also given. This comprehensive review is intended to provide the foundation for and facilitate future studies on exploring the potential of emerging ML for coping with increasingly complex SS and their security problems.
CompanyName2Vec: Company Entity Matching Based on Job Ads
Venue: arXiv cs.SI 2022
Year: 2022

Entity Matching is an essential part of all real-world systems that take in structured and unstructured data coming from different sources. Typically no common key is available for connecting records. Massive data cleaning and integration processes require completion before any data analytics, or further processing can be performed. Although record linkage is frequently regarded as a somewhat tedious but necessary step, it reveals valuable insights, supports data visualization, and guides further analytic approaches to the data. Here, we focus on organization entity matching. We introduce CompanyName2Vec, a novel algorithm to solve company entity matching (CEM) using a neural network model to learn company name semantics from a job ad corpus, without relying on any information on the matched company besides its name. Based on a real-world data, we show that CompanyName2Vec outperforms other evaluated methods and solves the CEM challenge with an average success rate of 89.3%.
Deep clustering with fusion autoencoder
Venue: arXiv cs.LG 2022
Year: 2022

Embracing the deep learning techniques for representation learning in clustering research has attracted broad attention in recent years, yielding a newly developed clustering paradigm, viz. the deep clustering (DC). Typically, the DC models capitalize on autoencoders to learn the intrinsic features which facilitate the clustering process in consequence. Nowadays, a generative model named variational autoencoder (VAE) has got wide acceptance in DC studies. Nevertheless, the plain VAE is insufficient to perceive the comprehensive latent features, leading to the deteriorative clustering performance. In this paper, a novel DC method is proposed to address this issue. Specifically, the generative adversarial network and VAE are coalesced into a new autoencoder called fusion autoencoder (FAE) for discerning more discriminative representation that benefits the downstream clustering task. Besides, the FAE is implemented with the deep residual network architecture which further enhances the representation learning ability. Finally, the latent space of the FAE is transformed to an embedding space shaped by a deep dense neural network for pulling away different clusters from each other and collapsing data points within individual clusters. Experiment conducted on several image datasets demonstrate the effectiveness of the proposed DC model against the baseline methods.
Roadside Lidar Vehicle Detection and Tracking Using Range And Intensity Background Subtraction
Venue: arXiv cs.CV 2022
Year: 2022

In this paper, we developed the solution of roadside LiDAR object detection using a combination of two unsupervised learning algorithms. The 3D point clouds are firstly converted into spherical coordinates and filled into the elevation-azimuth matrix using a hash function. After that, the raw LiDAR data were rearranged into new data structures to store the information of range, azimuth, and intensity. Then, the Dynamic Mode Decomposition method is applied to decompose the LiDAR data into low-rank backgrounds and sparse foregrounds based on intensity channel pattern recognition. The Coarse Fine Triangle Algorithm (CFTA) automatically finds the dividing value to separate the moving targets from static background according to range information. After intensity and range background subtraction, the foreground moving objects will be detected using a density-based detector and encoded into the state-space model for tracking. The output of the proposed solution includes vehicle trajectories that can enable many mobility and safety applications. The method was validated at both path and point levels and outperformed the state-of-the-art. In contrast to the previous methods that process directly on the scattered and discrete point clouds, the dynamic classification method can establish the less sophisticated linear relationship of the 3D measurement data, which captures the spatial-temporal structure that we often desire.
Collision Detection: An Improved Deep Learning Approach Using SENet and ResNext
Venue: arXiv cs.CV 2022
Year: 2022

In recent days, with increased population and traffic on roadways, vehicle collision is one of the leading causes of death worldwide. The automotive industry is motivated on developing techniques to use sensors and advancements in the field of computer vision to build collision detection and collision prevention systems to assist drivers. In this article, a deep-learning-based model comprising of ResNext architecture with SENet blocks is proposed. The performance of the model is compared to popular deep learning models like VGG16, VGG19, Resnet50, and stand-alone ResNext. The proposed model outperforms the existing baseline models achieving a ROC-AUC of 0.91 using a significantly less proportion of the GTACrash synthetic data for training, thus reducing the computational overhead.