OpenRaiser/Intern-Atlas huggingface dataset:
Kernel-based Substructure Exploration for Next POI Recommendation
Venue: arXiv cs.LG 2022
Year: 2022

Point-of-Interest (POI) recommendation, which benefits from the proliferation of GPS-enabled devices and location-based social networks (LBSNs), plays an increasingly important role in recommender systems. It aims to provide users with the convenience to discover their interested places to visit based on previous visits and current status. Most existing methods usually merely leverage recurrent neural networks (RNNs) to explore sequential influences for recommendation. Despite the effectiveness, these methods not only neglect topological geographical influences among POIs, but also fail to model high-order sequential substructures. To tackle the above issues, we propose a Kernel-Based Graph Neural Network (KBGNN) for next POI recommendation, which combines the characteristics of both geographical and sequential influences in a collaborative way. KBGNN consists of a geographical module and a sequential module. On the one hand, we construct a geographical graph and leverage a message passing neural network to capture the topological geographical influences. On the other hand, we explore high-order sequential substructures in the user-aware sequential graph using a graph kernel neural network to capture user preferences. Finally, a consistency learning framework is introduced to jointly incorporate geographical and sequential information extracted from two separate graphs. In this way, the two modules effectively exchange knowledge to mutually enhance each other. Extensive experiments conducted on two real-world LBSN datasets demonstrate the superior performance of our proposed method over the state-of-the-arts. Our codes are available at https://github.com/Fang6ang/KBGNN.
Towards the Detection of Malicious Java Packages
Venue: arXiv cs.CR 2022
Year: 2022

Open-source software supply chain attacks aim at infecting downstream users by poisoning open-source packages. The common way of consuming such artifacts is through package repositories and the development of vetting strategies to detect such attacks is ongoing research. Despite its popularity, the Java ecosystem is the less explored one in the context of supply chain attacks. In this paper we present indicators of malicious behavior that can be observed statically through the analysis of Java bytecode. Then we evaluate how such indicators and their combinations perform when detecting malicious code injections. We do so by injecting three malicious payloads taken from real-world examples into the Top-10 most popular Java libraries from libraries.io. We found that the analysis of strings in the constant pool and of sensitive APIs in the bytecode instructions aid in the task of detecting malicious Java packages by significantly reducing the information, thus, making also manual triage possible.
AdaptivePose++: A Powerful Single-Stage Network for Multi-Person Pose Regression
Venue: arXiv cs.CV 2022
Year: 2022

Multi-person pose estimation generally follows top-down and bottom-up paradigms. Both of them use an extra stage ($\boldsymbol{e.g.,}$ human detection in top-down paradigm or grouping process in bottom-up paradigm) to build the relationship between the human instance and corresponding keypoints, thus leading to the high computation cost and redundant two-stage pipeline. To address the above issue, we propose to represent the human parts as adaptive points and introduce a fine-grained body representation method. The novel body representation is able to sufficiently encode the diverse pose information and effectively model the relationship between the human instance and corresponding keypoints in a single-forward pass. With the proposed body representation, we further deliver a compact single-stage multi-person pose regression network, termed as AdaptivePose. During inference, our proposed network only needs a single-step decode operation to form the multi-person pose without complex post-processes and refinements. We employ AdaptivePose for both 2D/3D multi-person pose estimation tasks to verify the effectiveness of AdaptivePose. Without any bells and whistles, we achieve the most competitive performance on MS COCO and CrowdPose in terms of accuracy and speed. Furthermore, the outstanding performance on MuCo-3DHP and MuPoTS-3D further demonstrates the effectiveness and generalizability on 3D scenes. Code is available at https://github.com/buptxyb666/AdaptivePose.
Reliability of fault-tolerant system architectures for automated driving systems
Venue: arXiv cs.CY 2022
Year: 2022

Automated driving functions at high levels of autonomy operate without driver supervision. The system itself must provide suitable responses in case of hardware element failures. This requires fault-tolerant approaches using domain ECUs and multicore processors operating in lockstep mode. The selection of a suitable architecture for fault-tolerant vehicle systems is currently challenging. Lockstep CPUs enable the implementation of majority redundancy or M-out-of-N ($M$oo$N$) architectures. In addition to structural redundancy, diversity redundancy in the ECU architecture is also relevant to fault tolerance. Two fault-tolerant ECU architecture groups exist: architectures with one ECU (system on a chip) and architectures consisting of multiple communicating ECUs. The single-ECU systems achieve higher reliability, whereas the multi-ECU systems are more robust against dependent failures, such as common-cause or cascading failures, due to their increased potential for diversity redundancy. Yet, it remains not fully understood how different types of architectures influence the system reliability. The work aims to design architectures with respect to CPU and sensor number, $M$oo$N$ expression, and hardware element reliability. The results enable a direct comparison of different architecture types. We calculate their reliability and quantify the effort to achieve high safety requirements. Markov processes allow comparing sensor and CPU architectures by varying the number of components and failure rates. The objective is to evaluate systems' survival probability and fault tolerance and design suitable sensor-CPU architectures. The results show that the system architecture strongly influences the reliability. However, a suitable system architecture must have a trade-off between reliability and self-diagnostics that parallel systems without majority redundancies do not provide.
AI and ML Accelerator Survey and Trends
Venue: arXiv cs.AR 2022
Year: 2022

This paper updates the survey of AI accelerators and processors from past three years. This paper collects and summarizes the current commercial accelerators that have been publicly announced with peak performance and power consumption numbers. The performance and power values are plotted on a scatter graph, and a number of dimensions and observations from the trends on this plot are again discussed and analyzed. Two new trends plots based on accelerator release dates are included in this year's paper, along with the additional trends of some neuromorphic, photonic, and memristor-based inference accelerators.
Drowsiness detection in drivers with a smartwatch
Venue: arXiv cs.CR 2022
Year: 2022

The main objective of this work is to detect early if a driver shows symptoms of sleepiness that indicate that he/she is falling asleep and, in that case, generate an alert to wake him/her up. To solve this problem, an application has been designed that collects various parameters, through a smartwatch while driving. First, the application detects the driving action. Then, it collects information about the most significant physiological variables of a person while driving. On the other hand, given the high level of sensitivity of the data managed in the designed application, in this work special attention has been paid to the security of the implementation. The proposed solution improves road safety, reducing the number of accidents caused by drowsiness while driving.
APUD(1,1) Recognition in Polynomial Time
Venue: arXiv cs.CG 2022
Year: 2022

A unit disk graph is the intersection graph of a set of disk of unit radius in the Euclidean plane. In 1998, Breu and Kirkpatrick showed that the recognition problem for unit disk graphs is NP-hard. Given $k$ horizontal and $m$ vertical lines, an APUD($k,m$) is a unit disk graph such that each unit disk is centered either on a given horizontal or vertical line. Çağırıcı showed in 2020 that APUD($k,m$) recognition is NP-hard. In this paper, we show that APUD($1,1$) recognition is polynomial time solvable.
PropertyDAG: Multi-objective Bayesian optimization of partially ordered, mixed-variable properties for biological sequence design
Venue: arXiv cs.LG 2022
Year: 2022

Bayesian optimization offers a sample-efficient framework for navigating the exploration-exploitation trade-off in the vast design space of biological sequences. Whereas it is possible to optimize the various properties of interest jointly using a multi-objective acquisition function, such as the expected hypervolume improvement (EHVI), this approach does not account for objectives with a hierarchical dependency structure. We consider a common use case where some regions of the Pareto frontier are prioritized over others according to a specified $\textit{partial ordering}$ in the objectives. For instance, when designing antibodies, we would like to maximize the binding affinity to a target antigen only if it can be expressed in live cell culture -- modeling the experimental dependency in which affinity can only be measured for antibodies that can be expressed and thus produced in viable quantities. In general, we may want to confer a partial ordering to the properties such that each property is optimized conditioned on its parent properties satisfying some feasibility condition. To this end, we present PropertyDAG, a framework that operates on top of the traditional multi-objective BO to impose this desired ordering on the objectives, e.g. expression $\rightarrow$ affinity. We demonstrate its performance over multiple simulated active learning iterations on a penicillin production task, toy numerical problem, and a real-world antibody design task.
Training Deep Learning Algorithms on Synthetic Forest Images for Tree Detection
Venue: arXiv cs.CV 2022
Year: 2022

Vision-based segmentation in forested environments is a key functionality for autonomous forestry operations such as tree felling and forwarding. Deep learning algorithms demonstrate promising results to perform visual tasks such as object detection. However, the supervised learning process of these algorithms requires annotations from a large diversity of images. In this work, we propose to use simulated forest environments to automatically generate 43 k realistic synthetic images with pixel-level annotations, and use it to train deep learning algorithms for tree detection. This allows us to address the following questions: i) what kind of performance should we expect from deep learning in harsh synthetic forest environments, ii) which annotations are the most important for training, and iii) what modality should be used between RGB and depth. We also report the promising transfer learning capability of features learned on our synthetic dataset by directly predicting bounding box, segmentation masks and keypoints on real images. Code available on GitHub (https://github.com/norlab-ulaval/PercepTreeV1).
Comparing Computational Architectures for Automated Journalism
Venue: arXiv cs.CL 2022
Year: 2022

The majority of NLG systems have been designed following either a template-based or a pipeline-based architecture. Recent neural models for data-to-text generation have been proposed with an end-to-end deep learning flavor, which handles non-linguistic input in natural language without explicit intermediary representations. This study compares the most often employed methods for generating Brazilian Portuguese texts from structured data. Results suggest that explicit intermediate steps in the generation process produce better texts than the ones generated by neural end-to-end architectures, avoiding data hallucination while better generalizing to unseen inputs. Code and corpus are publicly available.
Towards Real-Time Temporal Graph Learning
Venue: arXiv cs.LG 2022
Year: 2022

In recent years, graph representation learning has gained significant popularity, which aims to generate node embeddings that capture features of graphs. One of the methods to achieve this is employing a technique called random walks that captures node sequences in a graph and then learns embeddings for each node using a natural language processing technique called Word2Vec. These embeddings are then used for deep learning on graph data for classification tasks, such as link prediction or node classification. Prior work operates on pre-collected temporal graph data and is not designed to handle updates on a graph in real-time. Real world graphs change dynamically and their entire temporal updates are not available upfront. In this paper, we propose an end-to-end graph learning pipeline that performs temporal graph construction, creates low-dimensional node embeddings, and trains multi-layer neural network models in an online setting. The training of the neural network models is identified as the main performance bottleneck as it performs repeated matrix operations on many sequentially connected low-dimensional kernels. We propose to unlock fine-grain parallelism in these low-dimensional kernels to boost performance of model training.
Deep Clustering: A Comprehensive Survey
Venue: arXiv cs.LG 2022
Year: 2022

Cluster analysis plays an indispensable role in machine learning and data mining. Learning a good data representation is crucial for clustering algorithms. Recently, deep clustering, which can learn clustering-friendly representations using deep neural networks, has been broadly applied in a wide range of clustering tasks. Existing surveys for deep clustering mainly focus on the single-view fields and the network architectures, ignoring the complex application scenarios of clustering. To address this issue, in this paper we provide a comprehensive survey for deep clustering in views of data sources. With different data sources and initial conditions, we systematically distinguish the clustering methods in terms of methodology, prior knowledge, and architecture. Concretely, deep clustering methods are introduced according to four categories, i.e., traditional single-view deep clustering, semi-supervised deep clustering, deep multi-view clustering, and deep transfer clustering. Finally, we discuss the open challenges and potential future opportunities in different fields of deep clustering.
Text detection and recognition based on a lensless imaging system
Venue: arXiv cs.CV 2022
Year: 2022

Lensless cameras are characterized by several advantages (e.g., miniaturization, ease of manufacture, and low cost) as compared with conventional cameras. However, they have not been extensively employed due to their poor image clarity and low image resolution, especially for tasks that have high requirements on image quality and details such as text detection and text recognition. To address the problem, a framework of deep-learning-based pipeline structure was built to recognize text with three steps from raw data captured by employing lensless cameras. This pipeline structure consisted of the lensless imaging model U-Net, the text detection model connectionist text proposal network (CTPN), and the text recognition model convolutional recurrent neural network (CRNN). Compared with the method focusing only on image reconstruction, UNet in the pipeline was able to supplement the imaging details by enhancing factors related to character categories in the reconstruction process, so the textual information can be more effectively detected and recognized by CTPN and CRNN with fewer artifacts and high-clarity reconstructed lensless images. By performing experiments on datasets of different complexities, the applicability to text detection and recognition on lensless cameras was verified. This study reasonably demonstrates text detection and recognition tasks in the lensless camera system,and develops a basic method for novel applications.
Unsupervised Cross-Modality Domain Adaptation for Vestibular Schwannoma Segmentation and Koos Grade Prediction based on Semi-Supervised Contrastive Learning
Venue: arXiv cs.CV 2022
Year: 2022

Domain adaptation has been widely adopted to transfer styles across multi-vendors and multi-centers, as well as to complement the missing modalities. In this challenge, we proposed an unsupervised domain adaptation framework for cross-modality vestibular schwannoma (VS) and cochlea segmentation and Koos grade prediction. We learn the shared representation from both ceT1 and hrT2 images and recover another modality from the latent representation, and we also utilize proxy tasks of VS segmentation and brain parcellation to restrict the consistency of image structures in domain adaptation. After generating missing modalities, the nnU-Net model is utilized for VS and cochlea segmentation, while a semi-supervised contrastive learning pre-train approach is employed to improve the model performance for Koos grade prediction. On CrossMoDA validation phase Leaderboard, our method received rank 4 in task1 with a mean Dice score of 0.8394 and rank 2 in task2 with Macro-Average Mean Square Error of 0.3941. Our code is available at https://github.com/fiy2W/cmda2022.superpolymerization.
Boost Event-Driven Tactile Learning with Location Spiking Neurons
Venue: arXiv cs.NE 2022
Year: 2022

Tactile sensing is essential for a variety of daily tasks. And recent advances in event-driven tactile sensors and Spiking Neural Networks (SNNs) spur the research in related fields. However, SNN-enabled event-driven tactile learning is still in its infancy due to the limited representation abilities of existing spiking neurons and high spatio-temporal complexity in the event-driven tactile data. In this paper, to improve the representation capability of existing spiking neurons, we propose a novel neuron model called "location spiking neuron", which enables us to extract features of event-based data in a novel way. Specifically, based on the classical Time Spike Response Model (TSRM), we develop the Location Spike Response Model (LSRM). In addition, based on the most commonly-used Time Leaky Integrate-and-Fire (TLIF) model, we develop the Location Leaky Integrate-and-Fire (LLIF) model. Moreover, to demonstrate the representation effectiveness of our proposed neurons and capture the complex spatio-temporal dependencies in the event-driven tactile data, we exploit the location spiking neurons to propose two hybrid models for event-driven tactile learning. Specifically, the first hybrid model combines a fully-connected SNN with TSRM neurons and a fully-connected SNN with LSRM neurons. And the second hybrid model fuses the spatial spiking graph neural network with TLIF neurons and the temporal spiking graph neural network with LLIF neurons. Extensive experiments demonstrate the significant improvements of our models over the state-of-the-art methods on event-driven tactile learning. Moreover, compared to the counterpart artificial neural networks (ANNs), our SNN models are 10x to 100x energy-efficient, which shows the superior energy efficiency of our models and may bring new opportunities to the spike-based learning community and neuromorphic engineering.
Privacy-preserving Intelligent Resource Allocation for Federated Edge Learning in Quantum Internet
Venue: arXiv cs.NI 2022
Year: 2022

Federated edge learning (FEL) is a promising paradigm of distributed machine learning that can preserve data privacy while training the global model collaboratively. However, FEL is still facing model confidentiality issues due to eavesdropping risks of exchanging cryptographic keys through traditional encryption schemes. Therefore, in this paper, we propose a hierarchical architecture for quantum-secured FEL systems with ideal security based on the quantum key distribution (QKD) to facilitate public key and model encryption against eavesdropping attacks. Specifically, we propose a stochastic resource allocation model for efficient QKD to encrypt FEL keys and models. In FEL systems, remote FEL workers are connected to cluster heads via quantum-secured channels to train an aggregated global model collaboratively. However, due to the unpredictable number of workers at each location, the demand for secret-key rates to support secure model transmission to the server is unpredictable. The proposed systems need to efficiently allocate limited QKD resources (i.e., wavelengths) such that the total cost is minimized in the presence of stochastic demand by formulating the optimization problem for the proposed architecture as a stochastic programming model. To this end, we propose a federated reinforcement learning-based resource allocation scheme to solve the proposed model without complete state information. The proposed scheme enables QKD managers and controllers to train a global QKD resource allocation policy while keeping their private experiences local. Numerical results demonstrate that the proposed schemes can successfully achieve the cost-minimizing objective under uncertain demand while improving the training efficiency by about 50\% compared to state-of-the-art schemes.
QAScore -- An Unsupervised Unreferenced Metric for the Question Generation Evaluation
Venue: arXiv cs.CL 2022
Year: 2022

Question Generation (QG) aims to automate the task of composing questions for a passage with a set of chosen answers found within the passage. In recent years, the introduction of neural generation models has resulted in substantial improvements of automatically generated questions in terms of quality, especially compared to traditional approaches that employ manually crafted heuristics. However, the metrics commonly applied in QG evaluations have been criticized for their low agreement with human judgement. We therefore propose a new reference-free evaluation metric that has the potential to provide a better mechanism for evaluating QG systems, called QAScore. Instead of fine-tuning a language model to maximize its correlation with human judgements, QAScore evaluates a question by computing the cross entropy according to the probability that the language model can correctly generate the masked words in the answer to that question. Furthermore, we conduct a new crowd-sourcing human evaluation experiment for the QG evaluation to investigate how QAScore and other metrics can correlate with human judgements. Experiments show that QAScore obtains a stronger correlation with the results of our proposed human evaluation method compared to existing traditional word-overlap-based metrics such as BLEU and ROUGE, as well as the existing pretrained-model-based metric BERTScore.
Quantifying Social Biases Using Templates is Unreliable
Venue: arXiv cs.CL 2022
Year: 2022

Recently, there has been an increase in efforts to understand how large language models (LLMs) propagate and amplify social biases. Several works have utilized templates for fairness evaluation, which allow researchers to quantify social biases in the absence of test sets with protected attribute labels. While template evaluation can be a convenient and helpful diagnostic tool to understand model deficiencies, it often uses a simplistic and limited set of templates. In this paper, we study whether bias measurements are sensitive to the choice of templates used for benchmarking. Specifically, we investigate the instability of bias measurements by manually modifying templates proposed in previous works in a semantically-preserving manner and measuring bias across these modifications. We find that bias values and resulting conclusions vary considerably across template modifications on four tasks, ranging from an 81% reduction (NLI) to a 162% increase (MLM) in (task-specific) bias measurements. Our results indicate that quantifying fairness in LLMs, as done in current practice, can be brittle and needs to be approached with more care and caution.