OpenRaiser/Intern-Atlas huggingface dataset:
Too Global To Be Local: Swarm Consensus in Adversarial Settings
Venue: arXiv cs.MA 2022
Year: 2022

Reaching a consensus in a swarm of robots is one of the fundamental problems in swarm robotics, examining the possibility of reaching an agreement within the swarm members. The recently-introduced contamination problem offers a new perspective of the problem, in which swarm members should reach a consensus in spite of the existence of adversarial members that intentionally act to divert the swarm members towards a different consensus. In this paper, we search for a consensus-reaching algorithm under the contamination problem setting by taking a top-down approach: We transform the problem to a centralized two-player game in which each player controls the behavior of a subset of the swarm, trying to force the entire swarm to converge to an agreement on its own value. We define a performance metric for each players performance, proving a correlation between this metric and the chances of the player to win the game. We then present the globally optimal solution to the game and prove that unfortunately it is unattainable in a distributed setting, due to the challenging characteristics of the swarm members. We therefore examine the problem on a simplified swarm model, and compare the performance of the globally optimal strategy with locally optimal strategies, demonstrating its superiority in rigorous simulation experiments.
ASAP: Adaptive Scheme for Asynchronous Processing of Event-based Vision Algorithms
Venue: arXiv cs.CV 2022
Year: 2022

Event cameras can capture pixel-level illumination changes with very high temporal resolution and dynamic range. They have received increasing research interest due to their robustness to lighting conditions and motion blur. Two main approaches exist in the literature to feed the event-based processing algorithms: packaging the triggered events in event packages and sending them one-by-one as single events. These approaches suffer limitations from either processing overflow or lack of responsivity. Processing overflow is caused by high event generation rates when the algorithm cannot process all the events in real-time. Conversely, lack of responsivity happens in cases of low event generation rates when the event packages are sent at too low frequencies. This paper presents ASAP, an adaptive scheme to manage the event stream through variable-size packages that accommodate to the event package processing times. The experimental results show that ASAP is capable of feeding an asynchronous event-by-event clustering algorithm in a responsive and efficient manner and at the same time prevents overflow.
HGI-SLAM: Loop Closure With Human and Geometric Importance Features
Venue: arXiv cs.RO 2022
Year: 2022

We present Human and Geometric Importance SLAM (HGI-SLAM), a novel approach to loop closure using salient and geometric features. Loop closure is a key element of SLAM, with many established methods for this problem. However, current methods are narrow, using either geometric or salient based features. We merge their successes into a model that outperforms both types of methods alone. Our method utilizes inexpensive monocular cameras and does not depend on depth sensors nor Lidar. HGI-SLAM utilizes geometric and salient features, processes them into descriptors, and optimizes them for a bag of words algorithm. By using a concurrent thread and combing our loop closure detection with ORB-SLAM2, our system is a complete SLAM framework. We present extensive evaluations of HGI loop detection and HGI-SLAM on the KITTI and EuRoC datasets. We also provide a qualitative analysis of our features. Our method runs in real time, and is robust to large viewpoint changes while staying accurate in organic environments. HGI-SLAM is an end-to-end SLAM system that only requires monocular vision and is comparable in performance to state-of-the-art SLAM methods.
Through a fair looking-glass: mitigating bias in image datasets
Venue: arXiv cs.CV 2022
Year: 2022

With the recent growth in computer vision applications, the question of how fair and unbiased they are has yet to be explored. There is abundant evidence that the bias present in training data is reflected in the models, or even amplified. Many previous methods for image dataset de-biasing, including models based on augmenting datasets, are computationally expensive to implement. In this study, we present a fast and effective model to de-bias an image dataset through reconstruction and minimizing the statistical dependence between intended variables. Our architecture includes a U-net to reconstruct images, combined with a pre-trained classifier which penalizes the statistical dependence between target attribute and the protected attribute. We evaluate our proposed model on CelebA dataset, compare the results with a state-of-the-art de-biasing method, and show that the model achieves a promising fairness-accuracy combination.
Dynamic Walking of Bipedal Robots on Uneven Stepping Stones via Adaptive-frequency MPC
Venue: arXiv cs.RO 2022
Year: 2022

This paper presents a novel Adaptive-frequency MPC framework for bipedal locomotion over terrain with uneven stepping stones. In detail, we intend to achieve adaptive foot placement and gait period for bipedal periodic walking gait with this MPC, in order to traverse terrain with discontinuities without slowing down. We pair this adaptive-frequency MPC with a kino-dynamics trajectory optimization for optimal gait periods, center of mass (CoM) trajectory, and foot placements. We use whole-body control (WBC) along with adaptive-frequency MPC to track the optimal trajectories from the offline optimization. In numerical validations, our adaptive-frequency MPC framework with optimization has shown advantages over fixed-frequency MPC. The proposed framework can control the bipedal robot to traverse through uneven stepping stone terrains with perturbed stone heights, widths, and surface shapes while maintaining an average speed of 1.5 m/s.
Semantic Segmentation using Neural Ordinary Differential Equations
Venue: arXiv cs.CV 2022
Year: 2022

The idea of neural Ordinary Differential Equations (ODE) is to approximate the derivative of a function (data model) instead of the function itself. In residual networks, instead of having a discrete sequence of hidden layers, the derivative of the continuous dynamics of hidden state can be parameterized by an ODE. It has been shown that this type of neural network is able to produce the same results as an equivalent residual network for image classification. In this paper, we design a novel neural ODE for the semantic segmentation task. We start by a baseline network that consists of residual modules, then we use the modules to build our neural ODE network. We show that our neural ODE is able to achieve the state-of-the-art results using 57% less memory for training, 42% less memory for testing, and 68% less number of parameters. We evaluate our model on the Cityscapes, CamVid, LIP, and PASCAL-Context datasets.
MECCANO: A Multimodal Egocentric Dataset for Humans Behavior Understanding in the Industrial-like Domain
Venue: arXiv cs.CV 2022
Year: 2022

Wearable cameras allow to acquire images and videos from the user's perspective. These data can be processed to understand humans behavior. Despite human behavior analysis has been thoroughly investigated in third person vision, it is still understudied in egocentric settings and in particular in industrial scenarios. To encourage research in this field, we present MECCANO, a multimodal dataset of egocentric videos to study humans behavior understanding in industrial-like settings. The multimodality is characterized by the presence of gaze signals, depth maps and RGB videos acquired simultaneously with a custom headset. The dataset has been explicitly labeled for fundamental tasks in the context of human behavior understanding from a first person view, such as recognizing and anticipating human-object interactions. With the MECCANO dataset, we explored five different tasks including 1) Action Recognition, 2) Active Objects Detection and Recognition, 3) Egocentric Human-Objects Interaction Detection, 4) Action Anticipation and 5) Next-Active Objects Detection. We propose a benchmark aimed to study human behavior in the considered industrial-like scenario which demonstrates that the investigated tasks and the considered scenario are challenging for state-of-the-art algorithms. To support research in this field, we publicy release the dataset at https://iplab.dmi.unict.it/MECCANO/.
Systematic Constructions of Bent-Negabent Functions, 2-Rotation Symmetric Bent-Negabent Functions and Their Duals
Venue: arXiv cs.IT 2022
Year: 2022

Bent-negabent functions have many important properties for their application in cryptography since they have the flat absolute spectrum under the both Walsh-Hadamard transform and nega-Hadamard transform. In this paper, we present four new systematic constructions of bent-negabent functions on $4k, 8k, 4k+2$ and $8k+2$ variables, respectively, by modifying the truth tables of two classes of quadratic bent-negabent functions with simple form. The algebraic normal forms and duals of these constructed functions are also determined. We further identify necessary and sufficient conditions for those bent-negabent functions which have the maximum algebraic degree. At last, by modifying the truth tables of a class of quadratic 2-rotation symmetric bent-negabent functions, we present a construction of 2-rotation symmetric bent-negabent functions with any possible algebraic degrees. Considering that there are probably no bent-negabent functions in the rotation symmetric class, it is the first significant attempt to construct bent-negabent functions in the generalized rotation symmetric class.
Traffic incident duration prediction via a deep learning framework for text description encoding
Venue: arXiv cs.LG 2022
Year: 2022

Predicting the traffic incident duration is a hard problem to solve due to the stochastic nature of incident occurrence in space and time, a lack of information at the beginning of a reported traffic disruption, and lack of advanced methods in transport engineering to derive insights from past accidents. This paper proposes a new fusion framework for predicting the incident duration from limited information by using an integration of machine learning with traffic flow/speed and incident description as features, encoded via several Deep Learning methods (ANN autoencoder and character-level LSTM-ANN sentiment classifier). The paper constructs a cross-disciplinary modelling approach in transport and data science. The approach improves the incident duration prediction accuracy over the top-performing ML models applied to baseline incident reports. Results show that our proposed method can improve the accuracy by $60\%$ when compared to standard linear or support vector regression models, and a further $7\%$ improvement with respect to the hybrid deep learning auto-encoded GBDT model which seems to outperform all other models. The application area is the city of San Francisco, rich in both traffic incident logs (Countrywide Traffic Accident Data set) and past historical traffic congestion information (5-minute precision measurements from Caltrans Performance Measurement System).
Knowledge-based Analogical Reasoning in Neuro-symbolic Latent Spaces
Venue: arXiv cs.AI 2022
Year: 2022

Analogical Reasoning problems challenge both connectionist and symbolic AI systems as these entail a combination of background knowledge, reasoning and pattern recognition. While symbolic systems ingest explicit domain knowledge and perform deductive reasoning, they are sensitive to noise and require inputs be mapped to preset symbolic features. Connectionist systems on the other hand can directly ingest rich input spaces such as images, text or speech and recognize pattern even with noisy inputs. However, connectionist models struggle to include explicit domain knowledge for deductive reasoning. In this paper, we propose a framework that combines the pattern recognition abilities of neural networks with symbolic reasoning and background knowledge for solving a class of Analogical Reasoning problems where the set of attributes and possible relations across them are known apriori. We take inspiration from the 'neural algorithmic reasoning' approach [DeepMind 2020] and use problem-specific background knowledge by (i) learning a distributed representation based on a symbolic model of the problem (ii) training neural-network transformations reflective of the relations involved in the problem and finally (iii) training a neural network encoder from images to the distributed representation in (i). These three elements enable us to perform search-based reasoning using neural networks as elementary functions manipulating distributed representations. We test this on visual analogy problems in RAVENs Progressive Matrices, and achieve accuracy competitive with human performance and, in certain cases, superior to initial end-to-end neural-network based approaches. While recent neural models trained at scale yield SOTA, our novel neuro-symbolic reasoning approach is a promising direction for this problem, and is arguably more general, especially for problems where domain knowledge is available.
Fairness on Synthetic Visual and Thermal Mask Images
Venue: arXiv cs.CV 2022
Year: 2022

In this paper, we study performance and fairness on visual and thermal images and expand the assessment to masked synthetic images. Using the SpeakingFace and Thermal-Mask dataset, we propose a process to assess fairness on real images and show how the same process can be applied to synthetic images. The resulting process shows a demographic parity difference of 1.59 for random guessing and increases to 5.0 when the recognition performance increases to a precision and recall rate of 99.99\%. We indicate that inherently biased datasets can deeply impact the fairness of any biometric system. A primary cause of a biased dataset is the class imbalance due to the data collection process. To address imbalanced datasets, the classes with fewer samples can be augmented with synthetic images to generate a more balanced dataset resulting in less bias when training a machine learning system. For biometric-enabled systems, fairness is of critical importance, while the related concept of Equity, Diversity, and Inclusion (EDI) is well suited for the generalization of fairness in biometrics, in this paper, we focus on the 3 most common demographic groups age, gender, and ethnicity.
Walk-and-Relate: A Random-Walk-based Algorithm for Representation Learning on Sparse Knowledge Graphs
Venue: arXiv cs.LG 2022
Year: 2022

Knowledge graph (KG) embedding techniques use structured relationships between entities to learn low-dimensional representations of entities and relations. The traditional KG embedding techniques (such as TransE and DistMult) estimate these embeddings via simple models developed over observed KG triplets. These approaches differ in their triplet scoring loss functions. As these models only use the observed triplets to estimate the embeddings, they are prone to suffer through data sparsity that usually occurs in the real-world knowledge graphs, i.e., the lack of enough triplets per entity. To settle this issue, we propose an efficient method to augment the number of triplets to address the problem of data sparsity. We use random walks to create additional triplets, such that the relations carried by these introduced triplets entail the metapath induced by the random walks. We also provide approaches to accurately and efficiently filter out informative metapaths from the possible set of metapaths, induced by the random walks. The proposed approaches are model-agnostic, and the augmented training dataset can be used with any KG embedding approach out of the box. Experimental results obtained on the benchmark datasets show the advantages of the proposed approach.
Persistent Identification and Interlinking of FAIR Scholarly Knowledge
Venue: arXiv cs.DL 2022
Year: 2022

We leverage the Open Research Knowledge Graph - a scholarly infrastructure that supports the creation, curation, and reuse of structured, semantic scholarly knowledge - and present an approach for persistent identification of FAIR scholarly knowledge. We propose a DOI-based persistent identification of ORKG Papers, which are machine-actionable descriptions of the essential information published in scholarly articles. This enables the citability of FAIR scholarly knowledge and its discovery in global scholarly communication infrastructures (e.g., DataCite, OpenAIRE, and ORCID). While publishing, the state of the ORKG Paper is saved and cannot be further edited. To allow for updating published versions, ORKG supports creating new versions, which are linked in provenance chains. We demonstrate the linking of FAIR scholarly knowledge with digital artefacts (articles), agents (researchers) and other objects (organizations). We persistently identify FAIR scholarly knowledge (namely, ORKG Papers and ORKG Comparisons as collections of ORKG Papers) by leveraging DataCite services. Given the existing interoperability between DataCite, Crossref, OpenAIRE and ORCID, sharing metadata with DataCite ensures global findability of FAIR scholarly knowledge in scholarly communication infrastructures.
Probing Spurious Correlations in Popular Event-Based Rumor Detection Benchmarks
Venue: arXiv cs.SI 2022
Year: 2022

As social media becomes a hotbed for the spread of misinformation, the crucial task of rumor detection has witnessed promising advances fostered by open-source benchmark datasets. Despite being widely used, we find that these datasets suffer from spurious correlations, which are ignored by existing studies and lead to severe overestimation of existing rumor detection performance. The spurious correlations stem from three causes: (1) event-based data collection and labeling schemes assign the same veracity label to multiple highly similar posts from the same underlying event; (2) merging multiple data sources spuriously relates source identities to veracity labels; and (3) labeling bias. In this paper, we closely investigate three of the most popular rumor detection benchmark datasets (i.e., Twitter15, Twitter16 and PHEME), and propose event-separated rumor detection as a solution to eliminate spurious cues. Under the event-separated setting, we observe that the accuracy of existing state-of-the-art models drops significantly by over 40%, becoming only comparable to a simple neural classifier. To better address this task, we propose Publisher Style Aggregation (PSA), a generalizable approach that aggregates publisher posting records to learn writing style and veracity stance. Extensive experiments demonstrate that our method outperforms existing baselines in terms of effectiveness, efficiency and generalizability.
T2V-DDPM: Thermal to Visible Face Translation using Denoising Diffusion Probabilistic Models
Venue: arXiv cs.CV 2022
Year: 2022

Modern-day surveillance systems perform person recognition using deep learning-based face verification networks. Most state-of-the-art facial verification systems are trained using visible spectrum images. But, acquiring images in the visible spectrum is impractical in scenarios of low-light and nighttime conditions, and often images are captured in an alternate domain such as the thermal infrared domain. Facial verification in thermal images is often performed after retrieving the corresponding visible domain images. This is a well-established problem often known as the Thermal-to-Visible (T2V) image translation. In this paper, we propose a Denoising Diffusion Probabilistic Model (DDPM) based solution for T2V translation specifically for facial images. During training, the model learns the conditional distribution of visible facial images given their corresponding thermal image through the diffusion process. During inference, the visible domain image is obtained by starting from Gaussian noise and performing denoising repeatedly. The existing inference process for DDPMs is stochastic and time-consuming. Hence, we propose a novel inference strategy for speeding up the inference time of DDPMs, specifically for the problem of T2V image translation. We achieve the state-of-the-art results on multiple datasets. The code and pretrained models are publically available at http://github.com/Nithin-GK/T2V-DDPM
A model-agnostic approach for generating Saliency Maps to explain inferred decisions of Deep Learning Models
Venue: arXiv cs.CV 2022
Year: 2022

The widespread use of black-box AI models has raised the need for algorithms and methods that explain the decisions made by these models. In recent years, the AI research community is increasingly interested in models' explainability since black-box models take over more and more complicated and challenging tasks. Explainability becomes critical considering the dominance of deep learning techniques for a wide range of applications, including but not limited to computer vision. In the direction of understanding the inference process of deep learning models, many methods that provide human comprehensible evidence for the decisions of AI models have been developed, with the vast majority relying their operation on having access to the internal architecture and parameters of these models (e.g., the weights of neural networks). We propose a model-agnostic method for generating saliency maps that has access only to the output of the model and does not require additional information such as gradients. We use Differential Evolution (DE) to identify which image pixels are the most influential in a model's decision-making process and produce class activation maps (CAMs) whose quality is comparable to the quality of CAMs created with model-specific algorithms. DE-CAM achieves good performance without requiring access to the internal details of the model's architecture at the cost of more computational complexity.
Mapping STI ecosystems via Open Data: overcoming the limitations of conflicting taxonomies. A case study for Climate Change Research in Denmark
Venue: arXiv cs.DL 2022
Year: 2022

Science, Technology and Innovation (STI) decision-makers often need to have a clear vision of what is researched and by whom to design effective policies. Such a vision is provided by effective and comprehensive mappings of the research activities carried out within their institutional boundaries. A major challenge to be faced in this context is the difficulty in accessing the relevant data and in combining information coming from different sources: indeed, traditionally, STI data has been confined within closed data sources and, when available, it is categorised with different taxonomies. Here, we present a proof-of-concept study of the use of Open Resources to map the research landscape on the Sustainable Development Goal (SDG) 13-Climate Action, for an entire country, Denmark, and we map it on the 25 ERC panels.
HiMFR: A Hybrid Masked Face Recognition Through Face Inpainting
Venue: arXiv cs.CV 2022
Year: 2022

To recognize the masked face, one of the possible solutions could be to restore the occluded part of the face first and then apply the face recognition method. Inspired by the recent image inpainting methods, we propose an end-to-end hybrid masked face recognition system, namely HiMFR, consisting of three significant parts: masked face detector, face inpainting, and face recognition. The masked face detector module applies a pretrained Vision Transformer (ViT\_b32) to detect whether faces are covered with masked or not. The inpainting module uses a fine-tune image inpainting model based on a Generative Adversarial Network (GAN) to restore faces. Finally, the hybrid face recognition module based on ViT with an EfficientNetB3 backbone recognizes the faces. We have implemented and evaluated our proposed method on four different publicly available datasets: CelebA, SSDMNV2, MAFA, {Pubfig83} with our locally collected small dataset, namely Face5. Comprehensive experimental results show the efficacy of the proposed HiMFR method with competitive performance. Code is available at https://github.com/mdhosen/HiMFR