OpenRaiser/Intern-Atlas huggingface dataset:
Controllable Music Production with Diffusion Models and Guidance Gradients
Venue: arXiv cs.SD 2023
Year: 2023

We demonstrate how conditional generation from diffusion models can be used to tackle a variety of realistic tasks in the production of music in 44.1kHz stereo audio with sampling-time guidance. The scenarios we consider include continuation, inpainting and regeneration of musical audio, the creation of smooth transitions between two different music tracks, and the transfer of desired stylistic characteristics to existing audio clips. We achieve this by applying guidance at sampling time in a simple framework that supports both reconstruction and classification losses, or any combination of the two. This approach ensures that generated audio can match its surrounding context, or conform to a class distribution or latent representation specified relative to any suitable pre-trained classifier or embedding model. Audio samples are available at https://machinelearning.apple.com/research/controllable-music
What User Behaviors Make the Differences During the Process of Visual Analytics?
Venue: arXiv cs.HC 2023
Year: 2023

The understanding of visual analytics process can benefit visualization researchers from multiple aspects, including improving visual designs and developing advanced interaction functions. However, the log files of user behaviors are still hard to analyze due to the complexity of sensemaking and our lack of knowledge on the related user behaviors. This work presents a study on a comprehensive data collection of user behaviors, and our analysis approach with time-series classification methods. We have chosen a classical visualization application, Covid-19 data analysis, with common analysis tasks covering geo-spatial, time-series and multi-attributes. Our user study collects user behaviors on a diverse set of visualization tasks with two comparable systems, desktop and immersive visualizations. We summarize the classification results with three time-series machine learning algorithms at two scales, and explore the influences of behavior features. Our results reveal that user behaviors can be distinguished during the process of visual analytics and there is a potentially strong association between the physical behaviors of users and the visualization tasks they perform. We also demonstrate the usage of our models by interpreting open sessions of visual analytics, which provides an automatic way to study sensemaking without tedious manual annotations.
On Task-personalized Multimodal Few-shot Learning for Visually-rich Document Entity Retrieval
Venue: arXiv cs.AI 2023
Year: 2023

Visually-rich document entity retrieval (VDER), which extracts key information (e.g. date, address) from document images like invoices and receipts, has become an important topic in industrial NLP applications. The emergence of new document types at a constant pace, each with its unique entity types, presents a unique challenge: many documents contain unseen entity types that occur only a couple of times. Addressing this challenge requires models to have the ability of learning entities in a few-shot manner. However, prior works for Few-shot VDER mainly address the problem at the document level with a predefined global entity space, which doesn't account for the entity-level few-shot scenario: target entity types are locally personalized by each task and entity occurrences vary significantly among documents. To address this unexplored scenario, this paper studies a novel entity-level few-shot VDER task. The challenges lie in the uniqueness of the label space for each task and the increased complexity of out-of-distribution (OOD) contents. To tackle this novel task, we present a task-aware meta-learning based framework, with a central focus on achieving effective task personalization that distinguishes between in-task and out-of-task distribution. Specifically, we adopt a hierarchical decoder (HC) and employ contrastive learning (ContrastProtoNet) to achieve this goal. Furthermore, we introduce a new dataset, FewVEX, to boost future research in the field of entity-level few-shot VDER. Experimental results demonstrate our approaches significantly improve the robustness of popular meta-learning baselines.
Enhancing Chemistry Learning with ChatGPT, Bing Chat, Bard, and Claude as Agents-to-Think-With: A Comparative Case Study
Venue: arXiv cs.HC 2023
Year: 2023

This research delves into the comparative advantages of Generative AI chatbots (GenAIbots) -- ChatGPT, Bing Chat, Bard, and Claude -- in the context of Chemistry education, framed within a constructivist perspective. Our primary objective was to identify which of these four AI tools is more effective for enhancing Chemistry learning. Employing a single-case study approach, we scrutinised interaction logs between the AI systems and a simulated student persona during Chemistry learning simulations, incorporating Content Analysis methodology to delve deeper into the discourse. Our findings underscore these tools' potential as "agents-to-think-with", enhancing critical thinking, problem-solving, comprehension, creativity, and tailored learning. Especially noteworthy is their ability to stimulate learners through Socratic-like questioning, aligning with constructionist principles. The research emphasises the pivotal role of prompt crafting to coax desired responses from GenAIbots, engendering iterative reflections. It also highlights the need for robust educator training to infuse these technologies into educational settings. Conclusively, while ChatGPT, Bing Chat, Bard, and Claude are poised to enrich Chemistry education by fostering dynamic, inclusive learning experiences, ChatGPT stood out, decisively surpassing Bing Chat in its performance. Bard and Claude trailed closely, with all three showcasing a more in-depth, precise, and nuanced understanding, underscoring ChatGPT's adeptness at contextual comprehension.
Chat GPT Integrated with Voice Assistant as Learning Oral Chat-based Constructive Communication to Improve Communicative Competence for EFL earners
Venue: arXiv cs.HC 2023
Year: 2023

Chat GPT belongs to the category of Generative Pre-trained Transformer (GPT) language models, which have received specialized training to produce text based on natural language inputs. Its purpose is to imitate human-like conversation and can be implemented in multiple applications, such as chatbots, virtual assistants, and language translation systems, starting with an introduction to the new trends and differences between artificial intelligence, machine learning, and artificial neural networks, and highlighting the rigorous language logic and powerful text generation capabilities of Chat GPT. This paper delves into how advances in artificial intelligence will shape e-learning in the coming decades, particularly in terms of Chat- GPT's ability to improve learners' Communicative Competence when English is a second language. The combination of new trends in artificial intelligence, mainly in the particular case of English as a second language, and, at the academic level, chatbot technology, will be the next step in the replacement of the human academic community by virtual assistants, apparently until a certain point. Despite the controversy, this very innovative solution will be able to bridge the gap between technology and education. Moreover, such innovative practices facilitate communication by enabling its inclusion in various applications, including virtual assistants, chatbots, and language education. Keyword: Chat GPT, artificial intelligence, Communicative Competence, Communicative Language Teaching (CLT)
Fraud Analytics Using Machine-learning & Engineering on Big Data (FAME) for Telecom
Venue: arXiv cs.LG 2023
Year: 2023

Telecom industries lose globally 46.3 Billion USD due to fraud. Data mining and machine learning techniques (apart from rules oriented approach) have been used in past, but efficiency has been low as fraud pattern changes very rapidly. This paper presents an industrialized solution approach with self adaptive data mining technique and application of big data technologies to detect fraud and discover novel fraud patterns in accurate, efficient and cost effective manner. Solution has been successfully demonstrated to detect International Revenue Share Fraud with <5% false positive. More than 1 Terra Bytes of Call Detail Record from a reputed wholesale carrier and overseas telecom transit carrier has been used to conduct this study.
PET Tracer Conversion among Brain PET via Variable Augmented Invertible Network
Venue: arXiv cs.LG 2023
Year: 2023

Positron emission tomography (PET) serves as an essential tool for diagnosis of encephalopathy and brain science research. However, it suffers from the limited choice of tracers. Nowadays, with the wide application of PET imaging in neuropsychiatric treatment, 6-18F-fluoro-3, 4-dihydroxy-L-phenylalanine (DOPA) has been found to be more effective than 18F-labeled fluorine-2-deoxyglucose (FDG) in the field. Nevertheless, due to the complexity of its preparation and other limitations, DOPA is far less widely used than FDG. To address this issue, a tracer conversion invertible neural network (TC-INN) for image projection is developed to map FDG images to DOPA images through deep learning. More diagnostic information is obtained by generating PET images from FDG to DOPA. Specifically, the proposed TC-INN consists of two separate phases, one for training traceable data, the other for rebuilding new data. The reference DOPA PET image is used as a learning target for the corresponding network during the training process of tracer conversion. Meanwhile, the invertible network iteratively estimates the resultant DOPA PET data and compares it to the reference DOPA PET data. Notably, the reversible model employs variable enhancement technique to achieve better power generation. Moreover, image registration needs to be performed before training due to the angular deviation of the acquired FDG and DOPA data information. Experimental results exhibited excellent generation capability in mapping between FDG and DOPA, suggesting that PET tracer conversion has great potential in the case of limited tracer applications.
Shortest paths on polymatroids and hypergraphic polytopes
Venue: arXiv cs.DS 2023
Year: 2023

Base polytopes of polymatroids, also known as generalized permutohedra, are polytopes whose edges are parallel to a vector of the form $\mathbf{e}_i - \mathbf{e}_j$. We consider the following computational problem: Given two vertices of a generalized permutohedron $P$, find a shortest path between them on the skeleton of $P$. This captures many known flip distance problems, such as computing the minimum number of exchanges between two spanning trees of a graph, the rotation distance between binary search trees, the flip distance between acyclic orientations of a graph, or rectangulations of a square. We prove that this problem is $NP$-hard, even when restricted to very simple polymatroids in $\mathbb{R}^n$ defined by $O(n)$ inequalities. Assuming $P\not= NP$, this rules out the existence of an efficient simplex pivoting rule that performs a minimum number of nondegenerate pivoting steps to an optimal solution of a linear program, even when the latter defines a polymatroid. We also prove that the shortest path problem is inapproximable when the polymatroid is specified via an evaluation oracle for a corresponding submodular function, strengthening a recent result by Ito et al. (ICALP'23). More precisely, we prove the $APX$-hardness of the shortest path problem when the polymatroid is a hypergraphic polytope, whose vertices are in bijection with acyclic orientations of a given hypergraph. The shortest path problem then amounts to computing the flip distance between two acyclic orientations of a hypergraph. On the positive side, we provide a polynomial-time approximation algorithm for the problem of computing the flip distance between two acyclic orientations of a hypergraph, where the approximation factor is the maximum codegree of the hypergraph. Our result implies an exact polynomial-time algorithm for the flip distance between two acyclic orientations of any linear hypergraph.
Code Sparsification and its Applications
Venue: arXiv cs.DS 2023
Year: 2023

We introduce a notion of code sparsification that generalizes the notion of cut sparsification in graphs. For a (linear) code $\mathcal{C} \subseteq \mathbb{F}_q^n$ of dimension $k$ a $(1 \pm ε)$-sparsification of size $s$ is given by a weighted set $S \subseteq [n]$ with $|S| \leq s$ such that for every codeword $c \in \mathcal{C}$ the projection $c|_S$ of $c$ to the set $S$ has (weighted) hamming weight which is a $(1 \pm ε)$ approximation of the hamming weight of $c$. We show that for every code there exists a $(1 \pm ε)$-sparsification of size $s = \widetilde{O}(k \log (q) / ε^2)$. This immediately implies known results on graph and hypergraph cut sparsification up to polylogarithmic factors (with a simple unified proof). One application of our result is near-linear size sparsifiers for constraint satisfaction problems (CSPs) over $\mathbb{F}_p$-valued variables whose unsatisfying assignments can be expressed as the zeros of a linear equation modulo a prime $p$. Building on this, we obtain a complete characterization of ternary Boolean CSPs that admit near-linear size sparsification. Finally, by connections between the eigenvalues of the Laplacians of Cayley graphs over $\mathbb{F}_2^k$ to the weights of codewords, we also give the first proof of the existence of spectral Cayley graph sparsifiers over $\mathbb{F}_2^k$ by Cayley graphs, i.e., where we sparsify the set of generators to nearly-optimal size.
Where to Deploy an Airborne Relay in Unknown Environments: Feasible Locations for Throughput and LoS Enhancement
Venue: arXiv cs.MA 2023
Year: 2023

The deployment of heterogeneous teams of both air and ground mobile assets combines the advantages of mobility, sensing capability, and operational duration when performing complex tasks. Air assets in such teams act to relay information between ground assets but must maintain unblocked paths to enable high-capacity communication modes. Obstacles in the operational environment may block the line of sight (LoS) between air assets and ground assets depending on their locations and heights. In this paper, we analyze the probability of spanning a two-hop communication between a pair of ground assets deployed in an environment with obstacles at random locations and with random heights (i.e. a Poisson Forest) using an air asset at any location near the ground assets. We provide a closed-form expression of the LoS probability based on the 3-dimensional locations of the air asset. We then compute a 3-D manifold of the air asset locations that satisfy a given LoS probability constraint. We further consider throughput as a measure of communication quality, and use it as an optimization objective.
Mahalanobis-Aware Training for Out-of-Distribution Detection
Venue: arXiv cs.LG 2023
Year: 2023

While deep learning models have seen widespread success in controlled environments, there are still barriers to their adoption in open-world settings. One critical task for safe deployment is the detection of anomalous or out-of-distribution samples that may require human intervention. In this work, we present a novel loss function and recipe for training networks with improved density-based out-of-distribution sensitivity. We demonstrate the effectiveness of our method on CIFAR-10, notably reducing the false-positive rate of the relative Mahalanobis distance method on far-OOD tasks by over 50%.
PIAug -- Physics Informed Augmentation for Learning Vehicle Dynamics for Off-Road Navigation
Venue: arXiv cs.RO 2023
Year: 2023

Modeling the precise dynamics of off-road vehicles is a complex yet essential task due to the challenging terrain they encounter and the need for optimal performance and safety. Recently, there has been a focus on integrating nominal physics-based models alongside data-driven neural networks using Physics Informed Neural Networks. These approaches often assume the availability of a well-distributed dataset; however, this assumption may not hold due to regions in the physical distribution that are hard to collect, such as high-speed motions and rare terrains. Therefore, we introduce a physics-informed data augmentation methodology called PIAug. We show an example use case of the same by modeling high-speed and aggressive motion predictions, given a dataset with only low-speed data. During the training phase, we leverage the nominal model for generating target domain (medium and high velocity) data using the available source data (low velocity). Subsequently, we employ a physics-inspired loss function with this augmented dataset to incorporate prior knowledge of physics into the neural network. Our methodology results in up to 67% less mean error in trajectory prediction in comparison to a standalone nominal model, especially during aggressive maneuvers at speeds outside the training domain. In real-life navigation experiments, our model succeeds in 4x tighter waypoint tracking constraints than the Kinematic Bicycle Model (KBM) at out-of-domain velocities.
healthAIChain: Improving security and safety using Blockchain Technology applications in AI-based healthcare systems
Venue: arXiv cs.CR 2023
Year: 2023

Blockchain as a digital ledger for keeping records of digital transactions and other information, it is secure and decentralized technology. The globally growing number of digital population every day possesses a significant threat to online data including the medical and patients data. After bitcoin, blockchain technology has emerged into a general-purpose technology with applications in medical industries and healthcare. Blockchain can promote highly configurable openness while retaining the highest security standards for critical data of medical patients. Referred to as distributed record keeping for healthcare systems which makes digital assets unalterable and transparent via a cryptographic hash and decentralized network. The study delves into the security and safety improvement associated with implementing blockchain in AI-based healthcare systems. Blockchain-enabled AI tackles the existing issues related to security, performance efficiencies, and safety in healthcare systems. We have also examined the Artificial Intelligence in healthcare and medical industry, potential areas, open questions concerning the blockchain in healthcare systems. Finally, the article proposed an AI-based healthcare blockchain model (healthAIChain) to improve patients data and security.
$2$-Fault-Tolerant Strong Connectivity Oracles
Venue: arXiv cs.DS 2023
Year: 2023

We study the problem of efficiently answering strong connectivity queries under two vertex failures. Given a directed graph $G$ with $n$ vertices, we provide a data structure with $O(nh)$ space and $O(h)$ query time, where $h$ is the height of a decomposition tree of $G$ into strongly connected subgraphs. This immediately implies data structures with $O(n \log{n})$ space and $O(\log{n})$ query time for graphs of constant treewidth, and $O(n^{3/2})$ space and $O(\sqrt{n})$ query time for planar graphs. For general directed graphs, we give a refined version of our data structure that achieves $O(n\sqrt{m})$ space and $O(\sqrt{m})$ query time, where $m$ is the number of edges of the graph. We also provide some simple BFS-based heuristics that seem to work remarkably well in practice. In the experimental part, we first evaluate various methods to construct a decomposition tree with small height $h$ in practice. Then we provide efficient implementations of our data structures, and evaluate their empirical performance by conducting an extensive experimental study on graphs taken from real-world applications.
Artificial Intelligence Ethics Education in Cybersecurity: Challenges and Opportunities: a focus group report
Venue: arXiv cs.CR 2023
Year: 2023

The emergence of AI tools in cybersecurity creates many opportunities and uncertainties. A focus group with advanced graduate students in cybersecurity revealed the potential depth and breadth of the challenges and opportunities. The salient issues are access to open source or free tools, documentation, curricular diversity, and clear articulation of ethical principles for AI cybersecurity education. Confronting the "black box" mentality in AI cybersecurity work is also of the greatest importance, doubled by deeper and prior education in foundational AI work. Systems thinking and effective communication were considered relevant areas of educational improvement. Future AI educators and practitioners need to address these issues by implementing rigorous technical training curricula, clear documentation, and frameworks for ethically monitoring AI combined with critical and system's thinking and communication skills.
The Power of the Senses: Generalizable Manipulation from Vision and Touch through Masked Multimodal Learning
Venue: arXiv cs.RO 2023
Year: 2023

Humans rely on the synergy of their senses for most essential tasks. For tasks requiring object manipulation, we seamlessly and effectively exploit the complementarity of our senses of vision and touch. This paper draws inspiration from such capabilities and aims to find a systematic approach to fuse visual and tactile information in a reinforcement learning setting. We propose Masked Multimodal Learning (M3L), which jointly learns a policy and visual-tactile representations based on masked autoencoding. The representations jointly learned from vision and touch improve sample efficiency, and unlock generalization capabilities beyond those achievable through each of the senses separately. Remarkably, representations learned in a multimodal setting also benefit vision-only policies at test time. We evaluate M3L on three simulated environments with both visual and tactile observations: robotic insertion, door opening, and dexterous in-hand manipulation, demonstrating the benefits of learning a multimodal policy. Code and videos of the experiments are available at https://sferrazza.cc/m3l_site.
Workspace optimization of 1T2R parallel manipulators with a dimensionally homogeneous constraint-embedded Jacobian
Venue: arXiv cs.RO 2023
Year: 2023

This paper presents the workspace optimization of one-translational two-rotational (1T2R) parallel manipulators using a dimensionally homogeneous constraint-embedded Jacobian. The mixed degrees of freedom of 1T2R parallel manipulators, which cause dimensional inconsistency, make it difficult to optimize their architectural parameters. To solve this problem, a point-based approach with a shifting property, selection matrix, and constraint-embedded inverse Jacobian is proposed. A simplified formulation is provided, eliminating the complex partial differentiation required in previous approaches. The dimensional homogeneity of the proposed method was analytically proven, and its validity was confirmed by comparing it with the conventional point-based method using a 3-PRS manipulator. Furthermore, the approach was applied to an asymmetric 2-RRS/RRRU manipulator with no parasitic motion. This mechanism has a T-shape combination of limbs with different kinematic parameters, making it challenging to derive a dimensionally homogeneous Jacobian using the conventional method. Finally, optimization was performed, and the results show that the proposed method is more efficient than the conventional approach. The efficiency and simplicity of the proposed method were verified using two distinct parallel manipulators.