OpenRaiser/Intern-Atlas huggingface dataset:
Contact-Aware Non-prehensile Robotic Manipulation for Object Retrieval in Cluttered Environments
Venue: arXiv cs.RO 2023
Year: 2023

Non-prehensile manipulation methods usually use a simple end effector, e.g., a single rod, to manipulate the object. Compared to the grasping method, such an end effector is compact and flexible, and hence it can perform tasks in a constrained workspace; As a trade-off, it has relatively few degrees of freedom (DoFs), resulting in an under-actuation problem with complex constraints for planning and control. This paper proposes a new non-prehensile manipulation method for the task of object retrieval in cluttered environments, using a rod-like pusher. Specifically, a candidate trajectory in a cluttered environment is first generated with an improved Rapidly-Exploring Random Tree (RRT) planner; Then, a Model Predictive Control (MPC) scheme is applied to stabilize the slider's poses through necessary contact with obstacles. Different from existing methods, the proposed approach is with the contact-aware feature, which enables the synthesized effect of active removal of obstacles, avoidance behavior, and switching contact face for improved dexterity. Hence both the feasibility and efficiency of the task are greatly promoted. The performance of the proposed method is validated in a planar object retrieval task, where the target object, surrounded by many fixed or movable obstacles, is manipulated and isolated. Both simulation and experimental results are presented.
Collaboration with Conversational AI Assistants for UX Evaluation: Questions and How to Ask them (Voice vs. Text)
Venue: arXiv cs.HC 2023
Year: 2023

AI is promising in assisting UX evaluators with analyzing usability tests, but its judgments are typically presented as non-interactive visualizations. Evaluators may have questions about test recordings, but have no way of asking them. Interactive conversational assistants provide a Q&A dynamic that may improve analysis efficiency and evaluator autonomy. To understand the full range of analysis-related questions, we conducted a Wizard-of-Oz design probe study with 20 participants who interacted with simulated AI assistants via text or voice. We found that participants asked for five categories of information: user actions, user mental model, help from the AI assistant, product and task information, and user demographics. Those who used the text assistant asked more questions, but the question lengths were similar. The text assistant was perceived as significantly more efficient, but both were rated equally in satisfaction and trust. We also provide design considerations for future conversational AI assistants for UX evaluation.
F2BEV: Bird's Eye View Generation from Surround-View Fisheye Camera Images for Automated Driving
Venue: arXiv cs.CV 2023
Year: 2023

Bird's Eye View (BEV) representations are tremendously useful for perception-related automated driving tasks. However, generating BEVs from surround-view fisheye camera images is challenging due to the strong distortions introduced by such wide-angle lenses. We take the first step in addressing this challenge and introduce a baseline, F2BEV, to generate discretized BEV height maps and BEV semantic segmentation maps from fisheye images. F2BEV consists of a distortion-aware spatial cross attention module for querying and consolidating spatial information from fisheye image features in a transformer-style architecture followed by a task-specific head. We evaluate single-task and multi-task variants of F2BEV on our synthetic FB-SSEM dataset, all of which generate better BEV height and segmentation maps (in terms of the IoU) than a state-of-the-art BEV generation method operating on undistorted fisheye images. We also demonstrate discretized height map generation from real-world fisheye images using F2BEV. Our dataset is publicly available at https://github.com/volvo-cars/FB-SSEM-dataset
MPool: Motif-Based Graph Pooling
Venue: arXiv cs.LG 2023
Year: 2023

Graph Neural networks (GNNs) have recently become a powerful technique for many graph-related tasks including graph classification. Current GNN models apply different graph pooling methods that reduce the number of nodes and edges to learn the higher-order structure of the graph in a hierarchical way. All these methods primarily rely on the one-hop neighborhood. However, they do not consider the higher- order structure of the graph. In this work, we propose a multi-channel Motif-based Graph Pooling method named (MPool) captures the higher-order graph structure with motif and local and global graph structure with a combination of selection and clustering-based pooling operations. As the first channel, we develop node selection-based graph pooling by designing a node ranking model considering the motif adjacency of nodes. As the second channel, we develop cluster-based graph pooling by designing a spectral clustering model using motif adjacency. As the final layer, the result of each channel is aggregated into the final graph representation. We perform extensive experiments on eight benchmark datasets and show that our proposed method shows better accuracy than the baseline methods for graph classification tasks.
Logit Margin Matters: Improving Transferable Targeted Adversarial Attack by Logit Calibration
Venue: arXiv cs.CV 2023
Year: 2023

Previous works have extensively studied the transferability of adversarial samples in untargeted black-box scenarios. However, it still remains challenging to craft targeted adversarial examples with higher transferability than non-targeted ones. Recent studies reveal that the traditional Cross-Entropy (CE) loss function is insufficient to learn transferable targeted adversarial examples due to the issue of vanishing gradient. In this work, we provide a comprehensive investigation of the CE loss function and find that the logit margin between the targeted and untargeted classes will quickly obtain saturation in CE, which largely limits the transferability. Therefore, in this paper, we devote to the goal of continually increasing the logit margin along the optimization to deal with the saturation issue and propose two simple and effective logit calibration methods, which are achieved by downscaling the logits with a temperature factor and an adaptive margin, respectively. Both of them can effectively encourage optimization to produce a larger logit margin and lead to higher transferability. Besides, we show that minimizing the cosine distance between the adversarial examples and the classifier weights of the target class can further improve the transferability, which is benefited from downscaling logits via L2-normalization. Experiments conducted on the ImageNet dataset validate the effectiveness of the proposed methods, which outperform the state-of-the-art methods in black-box targeted attacks. The source code is available at \href{https://github.com/WJJLL/Target-Attack/}{Link}
EavesDroid: Eavesdropping User Behaviors via OS Side-Channels on Smartphones
Venue: arXiv cs.CR 2023
Year: 2023

As the Internet of Things (IoT) continues to evolve, smartphones have become essential components of IoT systems. However, with the increasing amount of personal information stored on smartphones, user privacy is at risk of being compromised by malicious attackers. Although malware detection engines are commonly installed on smartphones against these attacks, attacks that can evade these defenses may still emerge. In this paper, we analyze the return values of system calls on Android smartphones and find two never-disclosed vulnerable return values that can leak fine-grained user behaviors. Based on this observation, we present EavesDroid, an application-embedded side-channel attack on Android smartphones that allows unprivileged attackers to accurately identify fine-grained user behaviors (e.g., viewing messages and playing videos) via on-screen operations. Our attack relies on the correlation between user behaviors and the return values associated with hardware and system resources. While this attack is challenging since these return values are susceptible to fluctuation and misalignment caused by many factors, we show that attackers can eavesdrop on fine-grained user behaviors using a CNN-GRU classification model that adopts min-max normalization and multiple return value fusion. Our experiments on different models and versions of Android smartphones demonstrate that EavesDroid can achieve 98% and 86% inference accuracy for 17 classes of user behaviors in the test set and real-world settings, highlighting the risk of our attack on user privacy. Finally, we recommend effective malware detection, carefully designed obfuscation methods, or restrictions on reading vulnerable return values to mitigate this attack.
Challenges of the Creation of a Dataset for Vision Based Human Hand Action Recognition in Industrial Assembly
Venue: arXiv cs.CV 2023
Year: 2023

This work presents the Industrial Hand Action Dataset V1, an industrial assembly dataset consisting of 12 classes with 459,180 images in the basic version and 2,295,900 images after spatial augmentation. Compared to other freely available datasets tested, it has an above-average duration and, in addition, meets the technical and legal requirements for industrial assembly lines. Furthermore, the dataset contains occlusions, hand-object interaction, and various fine-grained human hand actions for industrial assembly tasks that were not found in combination in examined datasets. The recorded ground truth assembly classes were selected after extensive observation of real-world use cases. A Gated Transformer Network, a state-of-the-art model from the transformer domain was adapted, and proved with a test accuracy of 86.25% before hyperparameter tuning by 18,269,959 trainable parameters, that it is possible to train sequential deep learning models with this dataset.
Fuzzy Logic and Markov Kernels
Venue: arXiv cs.LO 2023
Year: 2023

Fuzzy logic is a way to argue with boolean predicates for which we only have a confidence value between 0 and 1 rather than a well defined truth value. It is tempting to interpret such a confidence as a probability. We use Markov kernels, parametrised probability distributions, to do just that. As a consequence we get general fuzzy logic connectives from probabilistic computations on products of the booleans, stressing the importance of joint confidence functions. We discuss binary logic connectives in detail and recover the "classic" fuzzy connectives as bounds for the confidence for general connectives. We push multivariable logic formulas as far as being able to define fuzzy quantifiers and estimate the confidence.
Preparing the Vuk'uzenzele and ZA-gov-multilingual South African multilingual corpora
Venue: arXiv cs.CL 2023
Year: 2023

This paper introduces two multilingual government themed corpora in various South African languages. The corpora were collected by gathering the South African Government newspaper (Vuk'uzenzele), as well as South African government speeches (ZA-gov-multilingual), that are translated into all 11 South African official languages. The corpora can be used for a myriad of downstream NLP tasks. The corpora were created to allow researchers to study the language used in South African government publications, with a focus on understanding how South African government officials communicate with their constituents. In this paper we highlight the process of gathering, cleaning and making available the corpora. We create parallel sentence corpora for Neural Machine Translation (NMT) tasks using Language-Agnostic Sentence Representations (LASER) embeddings. With these aligned sentences we then provide NMT benchmarks for 9 indigenous languages by fine-tuning a massively multilingual pre-trained language model.
Can Decentralized Learning be more robust than Federated Learning?
Venue: arXiv cs.LG 2023
Year: 2023

Decentralized Learning (DL) is a peer--to--peer learning approach that allows a group of users to jointly train a machine learning model. To ensure correctness, DL should be robust, i.e., Byzantine users must not be able to tamper with the result of the collaboration. In this paper, we introduce two \textit{new} attacks against DL where a Byzantine user can: make the network converge to an arbitrary model of their choice, and exclude an arbitrary user from the learning process. We demonstrate our attacks' efficiency against Self--Centered Clipping, the state--of--the--art robust DL protocol. Finally, we show that the capabilities decentralization grants to Byzantine users result in decentralized learning \emph{always} providing less robustness than federated learning.
MAP-Elites with Descriptor-Conditioned Gradients and Archive Distillation into a Single Policy
Venue: arXiv cs.NE 2023
Year: 2023

Quality-Diversity algorithms, such as MAP-Elites, are a branch of Evolutionary Computation generating collections of diverse and high-performing solutions, that have been successfully applied to a variety of domains and particularly in evolutionary robotics. However, MAP-Elites performs a divergent search based on random mutations originating from Genetic Algorithms, and thus, is limited to evolving populations of low-dimensional solutions. PGA-MAP-Elites overcomes this limitation by integrating a gradient-based variation operator inspired by Deep Reinforcement Learning which enables the evolution of large neural networks. Although high-performing in many environments, PGA-MAP-Elites fails on several tasks where the convergent search of the gradient-based operator does not direct mutations towards archive-improving solutions. In this work, we present two contributions: (1) we enhance the Policy Gradient variation operator with a descriptor-conditioned critic that improves the archive across the entire descriptor space, (2) we exploit the actor-critic training to learn a descriptor-conditioned policy at no additional cost, distilling the knowledge of the archive into one single versatile policy that can execute the entire range of behaviors contained in the archive. Our algorithm, DCG-MAP-Elites improves the QD score over PGA-MAP-Elites by 82% on average, on a set of challenging locomotion tasks.
A Challenging Benchmark for Low-Resource Learning
Venue: arXiv cs.CL 2023
Year: 2023

With promising yet saturated results in high-resource settings, low-resource datasets have gradually become popular benchmarks for evaluating the learning ability of advanced neural networks (e.g., BigBench, superGLUE). Some models even surpass humans according to benchmark test results. However, we find that there exists a set of hard examples in low-resource settings that challenge neural networks but are not well evaluated, which causes over-estimated performance. We first give a theoretical analysis on which factors bring the difficulty of low-resource learning. It then motivate us to propose a challenging benchmark hardBench to better evaluate the learning ability, which covers 11 datasets, including 3 computer vision (CV) datasets and 8 natural language process (NLP) datasets. Experiments on a wide range of models show that neural networks, even pre-trained language models, have sharp performance drops on our benchmark, demonstrating the effectiveness on evaluating the weaknesses of neural networks. On NLP tasks, we surprisingly find that despite better results on traditional low-resource benchmarks, pre-trained networks, does not show performance improvements on our benchmarks. These results demonstrate that there are still a large robustness gap between existing models and human-level performance.
CBIM: object-level cloud collaboration platform for supporting across-domain asynchronous design
Venue: arXiv cs.SE 2023
Year: 2023

The conventional approach of designing BIM projects requires packaging of building information as files to exchange designs. This study develops a series of components to implement a previously established Cloud BIM (CBIM) platform that facilitates fileless cloud collaboration across BIM disciplines. A CBIM connector was developed to synchronize design changes from one discipline client to a CBIM server. The server processes the modifications and propagates relevant reference geometries to affected disciplines to assist their designs. The success of the case study demonstrates a fileless approach for multidisciplinary BIM collaboration and validates the practicality and capabilities of the CBIM paradigm.
A survey on automated detection and classification of acute leukemia and WBCs in microscopic blood cells
Venue: arXiv cs.CV 2023
Year: 2023

Leukemia (blood cancer) is an unusual spread of White Blood Cells or Leukocytes (WBCs) in the bone marrow and blood. Pathologists can diagnose leukemia by looking at a person's blood sample under a microscope. They identify and categorize leukemia by counting various blood cells and morphological features. This technique is time-consuming for the prediction of leukemia. The pathologist's professional skills and experiences may be affecting this procedure, too. In computer vision, traditional machine learning and deep learning techniques are practical roadmaps that increase the accuracy and speed in diagnosing and classifying medical images such as microscopic blood cells. This paper provides a comprehensive analysis of the detection and classification of acute leukemia and WBCs in the microscopic blood cells. First, we have divided the previous works into six categories based on the output of the models. Then, we describe various steps of detection and classification of acute leukemia and WBCs, including Data Augmentation, Preprocessing, Segmentation, Feature Extraction, Feature Selection (Reduction), Classification, and focus on classification step in the methods. Finally, we divide automated detection and classification of acute leukemia and WBCs into three categories, including traditional, Deep Neural Network (DNN), and mixture (traditional and DNN) methods based on the type of classifier in the classification step and analyze them. The results of this study show that in the diagnosis and classification of acute leukemia and WBCs, the Support Vector Machine (SVM) classifier in traditional machine learning models and Convolutional Neural Network (CNN) classifier in deep learning models have widely employed. The performance metrics of the models that use these classifiers compared to the others model are higher.
Approximate degree lower bounds for oracle identification problems
Venue: arXiv cs.CC 2023
Year: 2023

The approximate degree of a Boolean function is the minimum degree of real polynomial that approximates it pointwise. For any Boolean function, its approximate degree serves as a lower bound on its quantum query complexity, and generically lifts to a quantum communication lower bound for a related function. We introduce a framework for proving approximate degree lower bounds for certain oracle identification problems, where the goal is to recover a hidden binary string $x \in \{0, 1\}^n$ given possibly non-standard oracle access to it. Our lower bounds apply to decision versions of these problems, where the goal is to compute the parity of $x$. We apply our framework to the ordered search and hidden string problems, proving nearly tight approximate degree lower bounds of $Ω(n/\log^2 n)$ for each. These lower bounds generalize to the weakly unbounded error setting, giving a new quantum query lower bound for the hidden string problem in this regime. Our lower bounds are driven by randomized communication upper bounds for the greater-than and equality functions.
Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modeling
Venue: arXiv cs.CL 2023
Year: 2023

We propose a cross-lingual neural codec language model, VALL-E X, for cross-lingual speech synthesis. Specifically, we extend VALL-E and train a multi-lingual conditional codec language model to predict the acoustic token sequences of the target language speech by using both the source language speech and the target language text as prompts. VALL-E X inherits strong in-context learning capabilities and can be applied for zero-shot cross-lingual text-to-speech synthesis and zero-shot speech-to-speech translation tasks. Experimental results show that it can generate high-quality speech in the target language via just one speech utterance in the source language as a prompt while preserving the unseen speaker's voice, emotion, and acoustic environment. Moreover, VALL-E X effectively alleviates the foreign accent problems, which can be controlled by a language ID. Audio samples are available at \url{https://aka.ms/vallex}.
Mental state attribution to educational robots: an experience with children in primary school
Venue: arXiv cs.RO 2023
Year: 2023

The work presented in this paper was carried out in the context of the project Girls and boys: one day at university promoted by the City of Turin together with the University of Turin. We were responsible for two educational activities on robotics and coding hosted at the Computer Science Department, which made one of its laboratories available for this kind of lesson. At the conclusion of the lab's sessions, children compiled the Attribution of Mental State (AMS) questionnaire, which is a measure of mental states that participants attribute to robots, namely the user's perception of the robot's mental qualities as compared to humans. We distributed the questionnaires both to children attending the educational robotics lab and to children performing coding activities. Results show that the first group attributed higher mental qualities to the robots, compared to the attribution given by children that did not have a direct experience with a robot.
Pseudo Labels Regularization for Imbalanced Partial-Label Learning
Venue: arXiv cs.LG 2023
Year: 2023

Partial-label learning (PLL) is an important branch of weakly supervised learning where the single ground truth resides in a set of candidate labels, while the research rarely considers the label imbalance. A recent study for imbalanced partial-Label learning proposed that the combinatorial challenge of partial-label learning and long-tail learning lies in matching between a decent marginal prior distribution with drawing the pseudo labels. However, we believe that even if the pseudo label matches the prior distribution, the tail classes will still be difficult to learn because the total weight is too small. Therefore, we propose a pseudo-label regularization technique specially designed for PLL. By punishing the pseudo labels of head classes, our method implements state-of-art under the standardized benchmarks compared to the previous PLL methods.
Certified Compilation of Choreographies with hacc
Venue: arXiv cs.PL 2023
Year: 2023

Programming communicating processes is challenging, because it requires writing separate programs that perform compatible send and receive actions at the right time during execution. Leaving this task to the programmer can easily lead to bugs. Choreographic programming addresses this challenge by equipping developers with high-level abstractions for codifying the desired communication structures from a global viewpoint. Given a choreography, implementations of the involved processes can be automatically generated by endpoint projection (EPP). While choreographic programming prevents manual mistakes in the implementation of communications, the correctness of a choreographic programming framework crucially hinges on the correctness of its complex compiler, which has motivated formalisation of theories of choreographic programming in theorem provers. In this paper, we build upon one of these formalisations to construct a toolchain that produces executable code from a choreography.