OpenRaiser/Intern-Atlas huggingface dataset:
e-CLIP: Large-Scale Vision-Language Representation Learning in E-commerce
Venue: arXiv cs.LG 2022
Year: 2022

Understanding vision and language representations of product content is vital for search and recommendation applications in e-commerce. As a backbone for online shopping platforms and inspired by the recent success in representation learning research, we propose a contrastive learning framework that aligns language and visual models using unlabeled raw product text and images. We present techniques we used to train large-scale representation learning models and share solutions that address domain-specific challenges. We study the performance using our pre-trained model as backbones for diverse downstream tasks, including category classification, attribute extraction, product matching, product clustering, and adult product recognition. Experimental results show that our proposed method outperforms the baseline in each downstream task regarding both single modality and multiple modalities.
Bayesian causal inference in automotive software engineering and online evaluation
Venue: arXiv cs.SE 2022
Year: 2022

Randomised field experiments, such as A/B testing, have long been the gold standard for evaluating software changes. In the automotive domain, running randomised field experiments is not always desired, possible, or even ethical. In the face of such limitations, we develop a framework BOAT (Bayesian causal modelling for ObvservAtional Testing), utilising observational studies in combination with Bayesian causal inference, in order to understand real-world impacts from complex automotive software updates and help software development organisations arrive at causal conclusions. In this study, we present three causal inference models in the Bayesian framework and their corresponding cases to address three commonly experienced challenges of software evaluation in the automotive domain. We develop the BOAT framework with our industry collaborator, and demonstrate the potential of causal inference by conducting empirical studies on a large fleet of vehicles. Moreover, we relate the causal assumption theories to their implications in practise, aiming to provide a comprehensive guide on how to apply the causal models in automotive software engineering. We apply Bayesian propensity score matching for producing balanced control and treatment groups when we do not have access to the entire user base, Bayesian regression discontinuity design for identifying covariate dependent treatment assignments and the local treatment effect, and Bayesian difference-in-differences for causal inference of treatment effect overtime and implicitly control unobserved confounding factors. Each one of the demonstrative case has its grounds in practise, and is a scenario experienced when randomisation is not feasible. With the BOAT framework, we enable online software evaluation in the automotive domain without the need of a fully randomised experiment.
High-Performance GPU-to-CPU Transpilation and Optimization via High-Level Parallel Constructs
Venue: arXiv cs.PL 2022
Year: 2022

While parallelism remains the main source of performance, architectural implementations and programming models change with each new hardware generation, often leading to costly application re-engineering. Most tools for performance portability require manual and costly application porting to yet another programming model. We propose an alternative approach that automatically translates programs written in one programming model (CUDA), into another (CPU threads) based on Polygeist/MLIR. Our approach includes a representation of parallel constructs that allows conventional compiler transformations to apply transparently and without modification and enables parallelism-specific optimizations. We evaluate our framework by transpiling and optimizing the CUDA Rodinia benchmark suite for a multi-core CPU and achieve a 76% geomean speedup over handwritten OpenMP code. Further, we show how CUDA kernels from PyTorch can efficiently run and scale on the CPU-only Supercomputer Fugaku without user intervention. Our PyTorch compatibility layer making use of transpiled CUDA PyTorch kernels outperforms the PyTorch CPU native backend by 2.7$\times$.
The Dangers of Computational Law and Cybersecurity; Perspectives from Engineering and the AI Act
Venue: arXiv cs.CY 2022
Year: 2022

Computational Law has begun taking the role in society which has been predicted for some time. Automated decision-making and systems which assist users are now used in various jurisdictions, but with this maturity come certain caveats. Computational Law exists on the platforms which enable it, in this case digital systems, which means that it inherits the same flaws. Cybersecurity addresses these potential weaknesses. In this paper we go through known issues and discuss them in the various levels, from design to the physical realm. We also look at machine-learning specific adversarial problems. Additionally, we make certain considerations regarding computational law and existing and future legislation. Finally, we present three recommendations which are necessary for computational law to function globally, and which follow ideas in safety and security engineering. As indicated, we find that computational law must seriously consider that not only does it face the same risks as other types of software and computer systems, but that failures within it may cause financial or physical damage, as well as injustice. Consequences of Computational Legal systems failing are greater than if they were merely software and hardware. If the system employs machine-learning, it must take note of the very specific dangers which this brings, of which data poisoning is the classic example. Computational law must also be explicitly legislated for, which we show is not the case currently in the EU, and this is also true for the cybersecurity aspects that will be relevant to it. But there is great hope in EU's proposed AI Act, which makes an important attempt at taking the specific problems which Computational Law bring into the legal sphere. Our recommendations for Computational Law and Cybersecurity are: Accommodation of threats, adequate use, and that humans remain in the centre of their deployment.
TopicFM: Robust and Interpretable Topic-Assisted Feature Matching
Venue: arXiv cs.CV 2022
Year: 2022

This study addresses an image-matching problem in challenging cases, such as large scene variations or textureless scenes. To gain robustness to such situations, most previous studies have attempted to encode the global contexts of a scene via graph neural networks or transformers. However, these contexts do not explicitly represent high-level contextual information, such as structural shapes or semantic instances; therefore, the encoded features are still not sufficiently discriminative in challenging scenes. We propose a novel image-matching method that applies a topic-modeling strategy to encode high-level contexts in images. The proposed method trains latent semantic instances called topics. It explicitly models an image as a multinomial distribution of topics, and then performs probabilistic feature matching. This approach improves the robustness of matching by focusing on the same semantic areas between the images. In addition, the inferred topics provide interpretability for matching the results, making our method explainable. Extensive experiments on outdoor and indoor datasets show that our method outperforms other state-of-the-art methods, particularly in challenging cases. The code is available at https://github.com/TruongKhang/TopicFM.
Modelling Users with Item Metadata for Explainable and Interactive Recommendation
Venue: arXiv cs.IR 2022
Year: 2022

Recommender systems are used in many different applications and contexts, however their main goal can always be summarised as "connecting relevant content to interested users". Personalized recommendation algorithms achieve this goal by first building a profile of the user, either implicitly or explicitly, and then matching items with this profile to find relevant content. The more interpretable the profile and this "matching function" are, the easier it is to provide users with accurate and intuitive explanations, and also to let them interact with the system. Indeed, for a user to see what the system has already learned about her interests is of key importance for her to provide feedback to the system and to guide it towards better understanding her preferences. To this end, we propose a linear collaborative filtering recommendation model that builds user profiles within the domain of item metadata, which is arguably the most interpretable domain for end users. Our method is hence inherently transparent and explainable. Moreover, since recommendations are computed as a linear function of item metadata and the interpretable user profile, our method seamlessly supports interactive recommendation. In other words, users can directly tweak the weights of the learned profile for more fine-grained browsing and discovery of content based on their current interests. We demonstrate the interactive aspect of this model in an online application for discovering cultural events in Belgium. Additionally, the performance of the model is evaluated with offline experiments, both static and with simulated feedback, and compared to several state-of-the-art and state-of-practice baselines.
Energy Efficient Routing For Underwater Acoustic Sensor Network Using Genetic Algorithm
Venue: arXiv cs.NI 2022
Year: 2022

In underwater acoustic sensor networks (UWASN), energy-reliable data transmission is a challenging task. This is due to acoustic transmission disturbances caused by excessive noise, exceptionally long propagation delays, a high bit error rate, limited bandwidth capability, and interference. One of the most important issues of UWASN for research is how to extend the life span of data transmission. Data transfer from a source node to a destination node in UWASN is a complicated topic for researchers. Many routing algorithms, such as vector base forwarding and depth base routing, have been developed in past years. We propose a genetic algorithm-based optimization method for improving the energy efficiency of data transmission in the routing path from a source node to a destination node.
Semantic Communications for Future Internet: Fundamentals, Applications, and Challenges
Venue: arXiv cs.NI 2022
Year: 2022

With the increasing demand for intelligent services, the sixth-generation (6G) wireless networks will shift from a traditional architecture that focuses solely on high transmission rate to a new architecture that is based on the intelligent connection of everything. Semantic communication (SemCom), a revolutionary architecture that integrates user as well as application requirements and meaning of information into the data processing and transmission, is predicted to become a new core paradigm in 6G. While SemCom is expected to progress beyond the classical Shannon paradigm, several obstacles need to be overcome on the way to a SemCom-enabled smart wireless Internet. In this paper, we first highlight the motivations and compelling reasons of SemCom in 6G. Then, we outline the major 6G visions and key enabler techniques which lay the foundation of SemCom. Meanwhile, we highlight some benefits of SemCom-empowered 6G and present a SemCom-native 6G network architecture. Next, we show the evolution of SemCom from its introduction to classical SemCom related theory and modern AI-enabled SemCom. Following that, focusing on modern SemCom, we classify SemCom into three categories, i.e., semantic-oriented communication, goal-oriented communication, and semantic-aware communication, and introduce three types of semantic metrics. We then discuss the applications, the challenges and technologies related to semantics and communication. Finally, we introduce future research opportunities. In a nutshell, this paper investigates the fundamentals of SemCom, its applications in 6G networks, and the existing challenges and open issues for further direction.
Counting Dominating Sets in Directed Path Graphs
Venue: arXiv cs.DM 2022
Year: 2022

A dominating set of a graph is a set of vertices such that every vertex not in the set has at least one neighbor in the set. The problem of counting dominating sets is #P-complete for chordal graphs but solvable in polynomial time for its subclass of interval graphs. The complexity status of the corresponding problem is still undetermined for directed path graphs, which are a well-known class of graphs that falls between chordal graphs and interval graphs. This paper reveals that the problem of counting dominating sets remains #P-complete for directed path graphs but a stricter constraint to rooted directed path graphs admits a polynomial-time solution.
Online Reflective Learning for Robust Medical Image Segmentation
Venue: arXiv cs.CV 2022
Year: 2022

Deep segmentation models often face the failure risks when the testing image presents unseen distributions. Improving model robustness against these risks is crucial for the large-scale clinical application of deep models. In this study, inspired by human learning cycle, we propose a novel online reflective learning framework (RefSeg) to improve segmentation robustness. Based on the reflection-on-action conception, our RefSeg firstly drives the deep model to take action to obtain semantic segmentation. Then, RefSeg triggers the model to reflect itself. Because making deep models realize their segmentation failures during testing is challenging, RefSeg synthesizes a realistic proxy image from the semantic mask to help deep models build intuitive and effective reflections. This proxy translates and emphasizes the segmentation flaws. By maximizing the structural similarity between the raw input and the proxy, the reflection-on-action loop is closed with segmentation robustness improved. RefSeg runs in the testing phase and is general for segmentation models. Extensive validation on three medical image segmentation tasks with a public cardiac MR dataset and two in-house large ultrasound datasets show that our RefSeg remarkably improves model robustness and reports state-of-the-art performance over strong competitors.
Panning for gold: Lessons learned from the platform-agnostic automated detection of political content in textual data
Venue: arXiv cs.CL 2022
Year: 2022

The growing availability of data about online information behaviour enables new possibilities for political communication research. However, the volume and variety of these data makes them difficult to analyse and prompts the need for developing automated content approaches relying on a broad range of natural language processing techniques (e.g. machine learning- or neural network-based ones). In this paper, we discuss how these techniques can be used to detect political content across different platforms. Using three validation datasets, which include a variety of political and non-political textual documents from online platforms, we systematically compare the performance of three groups of detection techniques relying on dictionaries, supervised machine learning, or neural networks. We also examine the impact of different modes of data preprocessing (e.g. stemming and stopword removal) on the low-cost implementations of these techniques using a large set (n = 66) of detection models. Our results show the limited impact of preprocessing on model performance, with the best results for less noisy data being achieved by neural network- and machine-learning-based models, in contrast to the more robust performance of dictionary-based models on noisy data.
Enhancing cluster analysis via topological manifold learning
Venue: arXiv cs.LG 2022
Year: 2022

We discuss topological aspects of cluster analysis and show that inferring the topological structure of a dataset before clustering it can considerably enhance cluster detection: theoretical arguments and empirical evidence show that clustering embedding vectors, representing the structure of a data manifold instead of the observed feature vectors themselves, is highly beneficial. To demonstrate, we combine manifold learning method UMAP for inferring the topological structure with density-based clustering method DBSCAN. Synthetic and real data results show that this both simplifies and improves clustering in a diverse set of low- and high-dimensional problems including clusters of varying density and/or entangled shapes. Our approach simplifies clustering because topological pre-processing consistently reduces parameter sensitivity of DBSCAN. Clustering the resulting embeddings with DBSCAN can then even outperform complex methods such as SPECTACL and ClusterGAN. Finally, our investigation suggests that the crucial issue in clustering does not appear to be the nominal dimension of the data or how many irrelevant features it contains, but rather how \textit{separable} the clusters are in the ambient observation space they are embedded in, which is usually the (high-dimensional) Euclidean space defined by the features of the data. Our approach is successful because we perform the cluster analysis after projecting the data into a more suitable space that is optimized for separability, in some sense.
A Survey on Satisfiability Checking for the $μ$-Calculus through Tree Automata
Venue: arXiv cs.LO 2022
Year: 2022

Algorithms for model checking and satisfiability of the modal $μ$-calculus start by converting formulas to alternating parity tree automata. Thus, model checking is reduced to checking acceptance by tree automata and satisfiability to checking their emptiness. The first reduces directly to the solution of parity games but the second is more complicated. We review the non-emptiness checking of alternating tree automata by a reduction to solving parity games of a certain structure, so-called emptiness games. Since the emptiness problem for alternating tree automata is EXPTIME-complete, the size of these games is exponential in the number of states of the input automaton. We show how the construction of the emptiness games combines a (fixed) structural part with (history-)determinization of parity word automata. For tree automata with certain syntactic structures, simpler methods may be used to handle the treatment of the word automata, which then may be asymptotically smaller than in the general case. These results have direct consequences in satisfiability and validity checking for (various fragments of) the modal $μ$-calculus.
Using Machine Learning to Anticipate Tipping Points and Extrapolate to Post-Tipping Dynamics of Non-Stationary Dynamical Systems
Venue: arXiv cs.LG 2022
Year: 2022

In this paper we consider the machine learning (ML) task of predicting tipping point transitions and long-term post-tipping-point behavior associated with the time evolution of an unknown (or partially unknown), non-stationary, potentially noisy and chaotic, dynamical system. We focus on the particularly challenging situation where the past dynamical state time series that is available for ML training predominantly lies in a restricted region of the state space, while the behavior to be predicted evolves on a larger state space set not fully observed by the ML model during training. In this situation, it is required that the ML prediction system have the ability to extrapolate to different dynamics past that which is observed during training. We investigate the extent to which ML methods are capable of accomplishing useful results for this task, as well as conditions under which they fail. In general, we found that the ML methods were surprisingly effective even in situations that were extremely challenging, but do (as one would expect) fail when ``too much" extrapolation is required. For the latter case, we investigate the effectiveness of combining the ML approach with conventional modeling based on scientific knowledge, thus forming a hybrid prediction system which we find can enable useful prediction even when its ML-based and knowledge-based components fail when acting alone. We also found that achieving useful results may require using very carefully selected ML hyperparameters and we propose a hyperparameter optimization strategy to address this problem. The main conclusion of this paper is that ML-based approaches are promising tools for predicting the behavior of non-stationary dynamical systems even in the case where the future evolution (perhaps due to the crossing of a tipping point) includes dynamics on a set outside of that explored by the training data.
Ontology-Based Anomaly Detection for Air Traffic Control Systems
Venue: arXiv cs.CR 2022
Year: 2022

The Automatic Dependent Surveillance-Broadcast (ADS-B) protocol is increasingly being adopted by the aviation industry as a method for aircraft to relay their position to Air Traffic Control (ATC) monitoring systems. ADS-B provides greater precision compared to traditional radar-based technologies, however, it was designed without any encryption or authentication mechanisms and has been shown to be susceptible to spoofing attacks. A capable attacker can transmit falsified ADS-B messages with the intent of causing false information to be shown on ATC displays and threaten the safety of air traffic. Updating the ADS-B protocol will be a lengthy process, therefore, there is a need for systems to detect anomalous ADS-B communications. This paper presents ATC-Sense, an ADS-B anomaly detection system based on ontologies. An ATC ontology is used to model entities in a simulated controlled airspace and is used to detect falsified ADS-B messages by verifying that the entities conform to aviation constraints related to aircraft flight tracks, radar readings, and flight reports. We evaluate the computational performance of the proposed constraints-based detection approach with several ADS-B attack scenarios in a simulated ATC environment. We demonstrate how ontologies can be used for anomaly detection in a real-time environment and call for future work to investigate ways to improve the computational performance of such an approach.
DRESS: Dynamic REal-time Sparse Subnets
Venue: arXiv cs.CV 2022
Year: 2022

The limited and dynamically varied resources on edge devices motivate us to deploy an optimized deep neural network that can adapt its sub-networks to fit in different resource constraints. However, existing works often build sub-networks through searching different network architectures in a hand-crafted sampling space, which not only can result in a subpar performance but also may cause on-device re-configuration overhead. In this paper, we propose a novel training algorithm, Dynamic REal-time Sparse Subnets (DRESS). DRESS samples multiple sub-networks from the same backbone network through row-based unstructured sparsity, and jointly trains these sub-networks in parallel with weighted loss. DRESS also exploits strategies including parameter reusing and row-based fine-grained sampling for efficient storage consumption and efficient on-device adaptation. Extensive experiments on public vision datasets show that DRESS yields significantly higher accuracy than state-of-the-art sub-networks.
VEDLIoT: Very Efficient Deep Learning in IoT
Venue: arXiv cs.AR 2022
Year: 2022

The VEDLIoT project targets the development of energy-efficient Deep Learning for distributed AIoT applications. A holistic approach is used to optimize algorithms while also dealing with safety and security challenges. The approach is based on a modular and scalable cognitive IoT hardware platform. Using modular microserver technology enables the user to configure the hardware to satisfy a wide range of applications. VEDLIoT offers a complete design flow for Next-Generation IoT devices required for collaboratively solving complex Deep Learning applications across distributed systems. The methods are tested on various use-cases ranging from Smart Home to Automotive and Industrial IoT appliances. VEDLIoT is an H2020 EU project which started in November 2020. It is currently in an intermediate stage with the first results available.