OpenRaiser/Intern-Atlas huggingface dataset:
Beyond Double Ascent via Recurrent Neural Tangent Kernel in Sequential Recommendation
Venue: arXiv cs.IR 2022
Year: 2022

Overfitting has long been considered a common issue to large neural network models in sequential recommendation. In our study, an interesting phenomenon is observed that overfitting is temporary. When the model scale is increased, the trend of the performance firstly ascends, then descends (i.e., overfitting) and finally ascends again, which is named as double ascent in this paper. We therefore raise an assumption that a considerably larger model will generalise better with a higher performance. In an extreme case to infinite-width, performance is expected to reach the limit of this specific structure. Unfortunately, it is impractical to directly build a huge model due to the limit of resources. In this paper, we propose the Overparameterised Recommender (OverRec), which utilises a recurrent neural tangent kernel (RNTK) as a similarity measurement for user sequences to successfully bypass the restriction of hardware for huge models. We further prove that the RNTK for the tied input-output embeddings in recommendation is the same as the RNTK for general untied input-output embeddings, which makes RNTK theoretically suitable for recommendation. Since the RNTK is analytically derived, OverRec does not require any training, avoiding physically building the huge model. Extensive experiments are conducted on four datasets, which verifies the state-of-the-art performance of OverRec.
Applying Transformer-based Text Summarization for Keyphrase Generation
Venue: arXiv cs.CL 2022
Year: 2022

Keyphrases are crucial for searching and systematizing scholarly documents. Most current methods for keyphrase extraction are aimed at the extraction of the most significant words in the text. But in practice, the list of keyphrases often includes words that do not appear in the text explicitly. In this case, the list of keyphrases represents an abstractive summary of the source text. In this paper, we experiment with popular transformer-based models for abstractive text summarization using four benchmark datasets for keyphrase extraction. We compare the results obtained with the results of common unsupervised and supervised methods for keyphrase extraction. Our evaluation shows that summarization models are quite effective in generating keyphrases in the terms of the full-match F1-score and BERTScore. However, they produce a lot of words that are absent in the author's list of keyphrases, which makes summarization models ineffective in terms of ROUGE-1. We also investigate several ordering strategies to concatenate target keyphrases. The results showed that the choice of strategy affects the performance of keyphrase generation.
emgr -- EMpirical GRamian Framework Version 5.99
Venue: arXiv cs.CE 2022
Year: 2022

Version 5.99 of the empirical Gramian framework -- "emgr" -- completes a development cycle which focused on parametric model order reduction of gas network models while preserving compatibility to the previous development for the application of combined state and parameter reduction for neuroscience network models. Secondarily, new features concerning empirical Gramian types, perturbation design, and trajectory post-processing, as well as a Python version in addition to the default MATLAB / Octave implementation, have been added. This work summarizes these changes, particularly since "emgr" version 5.4, see Himpe, 2018 [Algorithms 11(7): 91], and gives recent as well as future applications, such as parameter identification in systems biology, based on the current feature set.
Interactive Imitation Learning in Robotics based on Simulations
Venue: arXiv cs.RO 2022
Year: 2022

The transformation towards intelligence in various industries is creating more demand for intelligent and flexible products. In the field of robotics, learning-based methods are increasingly being applied, with the purpose of training robots to learn to deal with complex and changing external environments through data. In this context, reinforcement learning and imitation learning are becoming research hotspots with their respective characteristics. However, the two have their own limitations in some cases, such as the high cost of data acquisition for reinforcement learning. Moreover, it is difficult for imitation learning to provide perfect demonstrations. As a branch of imitation learning, interactive imitation learning aims at transferring human knowledge to the agent through interactions between the demonstrator and the robot, which alleviates the difficulty of teaching. This thesis implements IIL algorithms in four simulation scenarios and conducts extensive experiments, aiming at providing exhaustive information about IIL methods both in action space and state space as well as comparison with RL methods.
Reconstruction Attacks on Aggressive Relaxations of Differential Privacy
Venue: arXiv cs.CR 2022
Year: 2022

Differential privacy is a widely accepted formal privacy definition that allows aggregate information about a dataset to be released while controlling privacy leakage for individuals whose records appear in the data. Due to the unavoidable tension between privacy and utility, there have been many works trying to relax the requirements of differential privacy to achieve greater utility. One class of relaxation, which is starting to gain support outside the privacy community is embodied by the definitions of individual differential privacy (IDP) and bootstrap differential privacy (BDP). The original version of differential privacy defines a set of neighboring database pairs and achieves its privacy guarantees by requiring that each pair of neighbors should be nearly indistinguishable to an attacker. The privacy definitions we study, however, aggressively reduce the set of neighboring pairs that are protected. Both IDP and BDP define a measure of "privacy loss" that satisfies formal privacy properties such as postprocessing invariance and composition, and achieve dramatically better utility than the traditional variants of differential privacy. However, there is a significant downside - we show that they allow a significant portion of the dataset to be reconstructed using algorithms that have arbitrarily low privacy loss under their privacy accounting rules. We demonstrate these attacks using the preferred mechanisms of these privacy definitions. In particular, we design a set of queries that, when protected by these mechanisms with high noise settings (i.e., with claims of very low privacy loss), yield more precise information about the dataset than if they were not protected at all.
Data Management Challenges for Internet-scale 3D Search Engines
Venue: arXiv cs.IR 2022
Year: 2022

This paper describes the most significant data-related challenges involved in building internet-scale 3D search engines. The discussion centers on the most pressing data management issues in this domain, including model acquisition, support for multiple file formats, asset versioning, data integrity errors, the data lifecycle, intellectual property, and the legality of web crawling. The paper also discusses numerous issues that fall under the rubric of trustworthy computing, including privacy, security, inappropriate content, and copying/remixing of assets. The goal of the paper is to provide an overview of these general issues, illustrated by empirical data drawn from the internet's largest operational search engine. While numerous works have been published on 3D information retrieval, this paper is the first to discuss the real-world challenges that arise in building practical search engines at scale.
NeuralFMU: Presenting a workflow for integrating hybrid NeuralODEs into real world applications
Venue: arXiv cs.LG 2022
Year: 2022

The term NeuralODE describes the structural combination of an Artifical Neural Network (ANN) and a numerical solver for Ordinary Differential Equations (ODEs), the former acts as the right-hand side of the ODE to be solved. This concept was further extended by a black-box model in the form of a Functional Mock-up Unit (FMU) to obtain a subclass of NeuralODEs, named NeuralFMUs. The resulting structure features the advantages of first-principle and data-driven modeling approaches in one single simulation model: A higher prediction accuracy compared to conventional First Principle Models (FPMs), while also a lower training effort compared to purely data-driven models. We present an intuitive workflow to setup and use NeuralFMUs, enabling the encapsulation and reuse of existing conventional models exported from common modeling tools. Moreover, we exemplify this concept by deploying a NeuralFMU for a consumption simulation based on a Vehicle Longitudinal Dynamics Model (VLDM), which is a typical use case in automotive industry. Related challenges that are often neglected in scientific use cases, like real measurements (e.g. noise), an unknown system state or high-frequent discontinuities, are handled in this contribution. For the aim to build a hybrid model with a higher prediction quality than the original FPM, we briefly highlight two open-source libraries: FMI.jl for integrating FMUs into the Julia programming environment, as well as an extension to this library called FMIFlux.jl, that allows for the integration of FMUs into a neural network topology to finally obtain a NeuralFMU.
Text-Free Learning of a Natural Language Interface for Pretrained Face Generators
Venue: arXiv cs.CV 2022
Year: 2022

We propose Fast text2StyleGAN, a natural language interface that adapts pre-trained GANs for text-guided human face synthesis. Leveraging the recent advances in Contrastive Language-Image Pre-training (CLIP), no text data is required during training. Fast text2StyleGAN is formulated as a conditional variational autoencoder (CVAE) that provides extra control and diversity to the generated images at test time. Our model does not require re-training or fine-tuning of the GANs or CLIP when encountering new text prompts. In contrast to prior work, we do not rely on optimization at test time, making our method orders of magnitude faster than prior work. Empirically, on FFHQ dataset, our method offers faster and more accurate generation of images from natural language descriptions with varying levels of detail compared to prior work.
What is Software Supply Chain Security?
Venue: arXiv cs.CR 2022
Year: 2022

The software supply chain involves a multitude of tools and processes that enable software developers to write, build, and ship applications. Recently, security compromises of tools or processes has led to a surge in proposals to address these issues. However, these proposals commonly overemphasize specific solutions or conflate goals, resulting in unexpected consequences, or unclear positioning and usage. In this paper, we make the case that developing practical solutions is not possible until the community has a holistic view of the security problem; this view must include both the technical and procedural aspects. To this end, we examine three use cases to identify common security goals, and present a goal-oriented taxonomy of existing solutions demonstrating a holistic overview of software supply chain security.
Privacy of Autonomous Vehicles: Risks, Protection Methods, and Future Directions
Venue: arXiv cs.AI 2022
Year: 2022

Recent advances in machine learning have enabled its wide application in different domains, and one of the most exciting applications is autonomous vehicles (AVs), which have encouraged the development of a number of ML algorithms from perception to prediction to planning. However, training AVs usually requires a large amount of training data collected from different driving environments (e.g., cities) as well as different types of personal information (e.g., working hours and routes). Such collected large data, treated as the new oil for ML in the data-centric AI era, usually contains a large amount of privacy-sensitive information which is hard to remove or even audit. Although existing privacy protection approaches have achieved certain theoretical and empirical success, there is still a gap when applying them to real-world applications such as autonomous vehicles. For instance, when training AVs, not only can individually identifiable information reveal privacy-sensitive information, but also population-level information such as road construction within a city, and proprietary-level commercial secrets of AVs. Thus, it is critical to revisit the frontier of privacy risks and corresponding protection approaches in AVs to bridge this gap. Following this goal, in this work, we provide a new taxonomy for privacy risks and protection methods in AVs, and we categorize privacy in AVs into three levels: individual, population, and proprietary. We explicitly list out recent challenges to protect each of these levels of privacy, summarize existing solutions to these challenges, discuss the lessons and conclusions, and provide potential future directions and opportunities for both researchers and practitioners. We believe this work will help to shape the privacy research in AV and guide the privacy protection technology design.
Audio Analytics-based Human Trafficking Detection Framework for Autonomous Vehicles
Venue: arXiv cs.AI 2022
Year: 2022

Human trafficking is a universal problem, persistent despite numerous efforts to combat it globally. Individuals of any age, race, ethnicity, sex, gender identity, sexual orientation, nationality, immigration status, cultural background, religion, socioeconomic class, and education can be a victim of human trafficking. With the advancements in technology and the introduction of autonomous vehicles (AVs), human traffickers will adopt new ways to transport victims, which could accelerate the growth of organized human trafficking networks, which can make the detection of trafficking in persons more challenging for law enforcement agencies. The objective of this study is to develop an innovative audio analytics-based human trafficking detection framework for autonomous vehicles. The primary contributions of this study are to: (i) define four non-trivial, feasible, and realistic human trafficking scenarios for AVs; (ii) create a new and comprehensive audio dataset related to human trafficking with five classes i.e., crying, screaming, car door banging, car noise, and conversation; and (iii) develop a deep 1-D Convolution Neural Network (CNN) architecture for audio data classification related to human trafficking. We have also conducted a case study using the new audio dataset and evaluated the audio classification performance of the deep 1-D CNN. Our analyses reveal that the deep 1-D CNN can distinguish sound coming from a human trafficking victim from a non-human trafficking sound with an accuracy of 95%, which proves the efficacy of our framework.
Robust and Lossless Fingerprinting of Deep Neural Networks via Pooled Membership Inference
Venue: arXiv cs.CR 2022
Year: 2022

Deep neural networks (DNNs) have already achieved great success in a lot of application areas and brought profound changes to our society. However, it also raises new security problems, among which how to protect the intellectual property (IP) of DNNs against infringement is one of the most important yet very challenging topics. To deal with this problem, recent studies focus on the IP protection of DNNs by applying digital watermarking, which embeds source information and/or authentication data into DNN models by tuning network parameters directly or indirectly. However, tuning network parameters inevitably distorts the DNN and therefore surely impairs the performance of the DNN model on its original task regardless of the degree of the performance degradation. It has motivated the authors in this paper to propose a novel technique called pooled membership inference (PMI) so as to protect the IP of the DNN models. The proposed PMI neither alters the network parameters of the given DNN model nor fine-tunes the DNN model with a sequence of carefully crafted trigger samples. Instead, it leaves the original DNN model unchanged, but can determine the ownership of the DNN model by inferring which mini-dataset among multiple mini-datasets was once used to train the target DNN model, which differs from previous arts and has remarkable potential in practice. Experiments also have demonstrated the superiority and applicability of this work.
FLInt: Exploiting Floating Point Enabled Integer Arithmetic for Efficient Random Forest Inference
Venue: arXiv cs.LG 2022
Year: 2022

In many machine learning applications, e.g., tree-based ensembles, floating point numbers are extensively utilized due to their expressiveness. Nowadays performing data analysis on embedded devices from dynamic data masses becomes available, but such systems often lack hardware capabilities to process floating point numbers, introducing large overheads for their processing. Even if such hardware is present in general computing systems, using integer operations instead of floating point operations promises to reduce operation overheads and improve the performance. In this paper, we provide \mdname, a full precision floating point comparison for random forests, by only using integer and logic operations. To ensure the same functionality preserves, we formally prove the correctness of this comparison. Since random forests only require comparison of floating point numbers during inference, we implement \mdname~in low level realizations and therefore eliminate the need for floating point hardware entirely, by keeping the model accuracy unchanged. The usage of \mdname~basically boils down to a one-by-one replacement of conditions: For instance, a comparison statement in C: if(pX[3]<=(float)10.074347) becomes if((*(((int*)(pX))+3))<=((int)(0x41213087))). Experimental evaluation on X86 and ARMv8 desktop and server class systems shows that the execution time can be reduced by up to $\approx 30\%$ with our novel approach.
Generative Deformable Radiance Fields for Disentangled Image Synthesis of Topology-Varying Objects
Venue: arXiv cs.CV 2022
Year: 2022

3D-aware generative models have demonstrated their superb performance to generate 3D neural radiance fields (NeRF) from a collection of monocular 2D images even for topology-varying object categories. However, these methods still lack the capability to separately control the shape and appearance of the objects in the generated radiance fields. In this paper, we propose a generative model for synthesizing radiance fields of topology-varying objects with disentangled shape and appearance variations. Our method generates deformable radiance fields, which builds the dense correspondence between the density fields of the objects and encodes their appearances in a shared template field. Our disentanglement is achieved in an unsupervised manner without introducing extra labels to previous 3D-aware GAN training. We also develop an effective image inversion scheme for reconstructing the radiance field of an object in a real monocular image and manipulating its shape and appearance. Experiments show that our method can successfully learn the generative model from unstructured monocular images and well disentangle the shape and appearance for objects (e.g., chairs) with large topological variance. The model trained on synthetic data can faithfully reconstruct the real object in a given single image and achieve high-quality texture and shape editing results.
Talking Head from Speech Audio using a Pre-trained Image Generator
Venue: arXiv cs.CV 2022
Year: 2022

We propose a novel method for generating high-resolution videos of talking-heads from speech audio and a single 'identity' image. Our method is based on a convolutional neural network model that incorporates a pre-trained StyleGAN generator. We model each frame as a point in the latent space of StyleGAN so that a video corresponds to a trajectory through the latent space. Training the network is in two stages. The first stage is to model trajectories in the latent space conditioned on speech utterances. To do this, we use an existing encoder to invert the generator, mapping from each video frame into the latent space. We train a recurrent neural network to map from speech utterances to displacements in the latent space of the image generator. These displacements are relative to the back-projection into the latent space of an identity image chosen from the individuals depicted in the training dataset. In the second stage, we improve the visual quality of the generated videos by tuning the image generator on a single image or a short video of any chosen identity. We evaluate our model on standard measures (PSNR, SSIM, FID and LMD) and show that it significantly outperforms recent state-of-the-art methods on one of two commonly used datasets and gives comparable performance on the other. Finally, we report on ablation experiments that validate the components of the model. The code and videos from experiments can be found at https://mohammedalghamdi.github.io/talking-heads-acm-mm
Shapley value-based approaches to explain the robustness of classifiers in machine learning
Venue: arXiv cs.LG 2022
Year: 2022

The use of algorithm-agnostic approaches is an emerging area of research for explaining the contribution of individual features towards the predicted outcome. Whilst there is a focus on explaining the prediction itself, a little has been done on explaining the robustness of these models, that is, how each feature contributes towards achieving that robustness. In this paper, we propose the use of Shapley values to explain the contribution of each feature towards the model's robustness, measured in terms of Receiver-operating Characteristics (ROC) curve and the Area under the ROC curve (AUC). With the help of an illustrative example, we demonstrate the proposed idea of explaining the ROC curve, and visualising the uncertainties in these curves. For imbalanced datasets, the use of Precision-Recall Curve (PRC) is considered more appropriate, therefore we also demonstrate how to explain the PRCs with the help of Shapley values. The explanation of robustness can help analysts in a number of ways, for example, it can help in feature selection by identifying the irrelevant features that can be removed to reduce the computational complexity. It can also help in identifying the features having critical contributions or negative contributions towards robustness.
Separating bichromatic point sets in the plane by restricted orientation convex hulls
Venue: arXiv cs.CG 2022
Year: 2022

We explore the separability of point sets in the plane by a restricted-orientation convex hull, which is an orientation-dependent, possibly disconnected, and non-convex enclosing shape that generalizes the convex hull. Let $R$ and $B$ be two disjoint sets of red and blue points in the plane, and $\mathcal{O}$ be a set of $k \geq 2$ lines passing through the origin. We study the problem of computing the set of orientations of the lines of $\mathcal{O}$ for which the $\mathcal{O}$-convex hull of $R$ contains no points of $B$. For $k=2$ orthogonal lines we have the rectilinear convex hull. In optimal $O(n \log n)$ time and $O(n)$ space, $n = \vert R \vert + \vert B \vert$, we compute the set of rotation angles such that, after simultaneously rotating the lines of $\mathcal{O}$ around the origin in the same direction, the rectilinear convex hull of $R$ contains no points of $B$. We generalize this result to the case where $\mathcal{O}$ is formed by $k \geq 2$ lines with arbitrary orientations. In the counter-clockwise circular order of the lines of $\mathcal{O}$, let $α_i$ be the angle required to clockwise rotate the $i$th line so it coincides with its successor. We solve the problem in this case in $O(1/Θ\cdot N \log N)$ time and $O(1/Θ\cdot N)$ space, where $Θ= \min \{ α_1,\ldots,α_k \}$ and $N=\max\{k,\vert R \vert + \vert B \vert \}$. We finally consider the case in which $\mathcal{O}$ is formed by $k=2$ lines, one of the lines is fixed, and the second line rotates by an angle that goes from $0$ to $π$. We show that this last case can also be solved in optimal $O(n\log n)$ time and $O(n)$ space, where $n = \vert R \vert + \vert B \vert$.