OpenRaiser/Intern-Atlas huggingface dataset:
FusionMind -- Improving question and answering with external context fusion
Venue: arXiv cs.CL 2023
Year: 2023

Answering questions using pre-trained language models (LMs) and knowledge graphs (KGs) presents challenges in identifying relevant knowledge and performing joint reasoning.We compared LMs (fine-tuned for the task) with the previously published QAGNN method for the Question-answering (QA) objective and further measured the impact of additional factual context on the QAGNN performance. The QAGNN method employs LMs to encode QA context and estimate KG node importance, and effectively update the question choice entity representations using Graph Neural Networks (GNNs). We further experimented with enhancing the QA context encoding by incorporating relevant knowledge facts for the question stem. The models are trained on the OpenbookQA dataset, which contains ~6000 4-way multiple choice questions and is widely used as a benchmark for QA tasks. Through our experimentation, we found that incorporating knowledge facts context led to a significant improvement in performance. In contrast, the addition of knowledge graphs to language models resulted in only a modest increase. This suggests that the integration of contextual knowledge facts may be more impactful for enhancing question answering performance compared to solely adding knowledge graphs.
Exploring the Need of Accessibility Education in the Software Industry: Insights from a Survey of Software Professionals in India
Venue: arXiv cs.SE 2023
Year: 2023

A UserWay study in 2021 indicates that an annual global e-commerce revenue loss of approximately $16 billion can be attributed to inaccessible websites and applications. According to the 2023 WebAIM study, only 3.7% of the world's top one million website homepages are fully accessible. This shows that many software developers use poor coding practices that don't adhere to the Web Content Accessibility Guidelines (WCAG). This research centers on software professionals and their role in addressing accessibility. This work seeks to understand (a) who within the software development community actively practices accessibility, (b) when and how accessibility is considered in the software development lifecycle, (c) the various challenges encountered in building accessible software, and (d) the resources required by software professionals to enhance product accessibility. Our survey of 269 software professionals from India sheds light on the pressing need for accessibility education within the software industry. A substantial majority (69.9%, N=269) of respondents express the need for training materials, workshops, and bootcamps to enhance their accessibility skills. We present a list of actionable recommendations that can be implemented within the industry to promote accessibility awareness and skills. We also open source our raw data for further research, encouraging continued exploration in this domain.
HSC-GPT: A Large Language Model for Human Settlements Construction
Venue: arXiv cs.CL 2023
Year: 2023

The field of human settlement construction encompasses a range of spatial designs and management tasks, including urban planning and landscape architecture design. These tasks involve a plethora of instructions and descriptions presented in natural language, which are essential for understanding design requirements and producing effective design solutions. Recent research has sought to integrate natural language processing (NLP) and generative artificial intelligence (AI) into human settlement construction tasks. Due to the efficient processing and analysis capabilities of AI with data, significant successes have been achieved in design within this domain. However, this task still faces several fundamental challenges. The semantic information involved includes complex spatial details, diverse data source formats, high sensitivity to regional culture, and demanding requirements for innovation and rigor in work scenarios. These factors lead to limitations when applying general generative AI in this field, further exacerbated by a lack of high-quality data for model training. To address these challenges, this paper first proposes HSC-GPT, a large-scale language model framework specifically designed for tasks in human settlement construction, considering the unique characteristics of this domain.
Balanced Multi-modal Federated Learning via Cross-Modal Infiltration
Venue: arXiv cs.LG 2023
Year: 2023

Federated learning (FL) underpins advancements in privacy-preserving distributed computing by collaboratively training neural networks without exposing clients' raw data. Current FL paradigms primarily focus on uni-modal data, while exploiting the knowledge from distributed multimodal data remains largely unexplored. Existing multimodal FL (MFL) solutions are mainly designed for statistical or modality heterogeneity from the input side, however, have yet to solve the fundamental issue,"modality imbalance", in distributed conditions, which can lead to inadequate information exploitation and heterogeneous knowledge aggregation on different modalities.In this paper, we propose a novel Cross-Modal Infiltration Federated Learning (FedCMI) framework that effectively alleviates modality imbalance and knowledge heterogeneity via knowledge transfer from the global dominant modality. To avoid the loss of information in the weak modality due to merely imitating the behavior of dominant modality, we design the two-projector module to integrate the knowledge from dominant modality while still promoting the local feature exploitation of weak modality. In addition, we introduce a class-wise temperature adaptation scheme to achieve fair performance across different classes. Extensive experiments over popular datasets are conducted and give us a gratifying confirmation of the proposed framework for fully exploring the information of each modality in MFL.
Does Few-shot Learning Suffer from Backdoor Attacks?
Venue: arXiv cs.CR 2023
Year: 2023

The field of few-shot learning (FSL) has shown promising results in scenarios where training data is limited, but its vulnerability to backdoor attacks remains largely unexplored. We first explore this topic by first evaluating the performance of the existing backdoor attack methods on few-shot learning scenarios. Unlike in standard supervised learning, existing backdoor attack methods failed to perform an effective attack in FSL due to two main issues. Firstly, the model tends to overfit to either benign features or trigger features, causing a tough trade-off between attack success rate and benign accuracy. Secondly, due to the small number of training samples, the dirty label or visible trigger in the support set can be easily detected by victims, which reduces the stealthiness of attacks. It seemed that FSL could survive from backdoor attacks. However, in this paper, we propose the Few-shot Learning Backdoor Attack (FLBA) to show that FSL can still be vulnerable to backdoor attacks. Specifically, we first generate a trigger to maximize the gap between poisoned and benign features. It enables the model to learn both benign and trigger features, which solves the problem of overfitting. To make it more stealthy, we hide the trigger by optimizing two types of imperceptible perturbation, namely attractive and repulsive perturbation, instead of attaching the trigger directly. Once we obtain the perturbations, we can poison all samples in the benign support set into a hidden poisoned support set and fine-tune the model on it. Our method demonstrates a high Attack Success Rate (ASR) in FSL tasks with different few-shot learning paradigms while preserving clean accuracy and maintaining stealthiness. This study reveals that few-shot learning still suffers from backdoor attacks, and its security should be given attention.
Neuronal Auditory Machine Intelligence (NEURO-AMI) In Perspective
Venue: arXiv cs.NE 2023
Year: 2023

The recent developments in soft computing cannot be complete without noting the contributions of artificial neural machine learning systems that draw inspiration from real cortical tissue or processes that occur in human brain. The universal approximability of such neural systems has led to its wide spread use, and novel developments in this evolving technology has shown that there is a bright future for such Artificial Intelligent (AI) techniques in the soft computing field. Indeed, the proliferation of large and very deep networks of artificial neural systems and the corresponding enhancement and development of neural machine learning algorithms have contributed immensely to the development of the modern field of Deep Learning as may be found in the well documented research works of Lecun, Bengio and Hinton. However, the key requirements of end user affordability in addition to reduced complexity and reduced data learning size requirement means there still remains a need for the synthesis of more cost-efficient and less data-hungry artificial neural systems. In this report, we present an overview of a new competing bio-inspired continual learning neural tool Neuronal Auditory Machine Intelligence (Neuro-AMI) as a predictor detailing its functional and structural details, important aspects on right applicability, some recent application use cases and future research directions for current and prospective machine learning experts and data scientists.
Point Location in Constant Time
Venue: arXiv cs.CG 2023
Year: 2023

We preprocess the input subdivision with $n$ points on the plane in $O(n\sqrt{\log n})$ time to facilitate point location in constant time. Previously the preprocessing time is $O(n\log n)$ and point location takes $O(\log n)$ time.
Perceptual and technical barriers in sharing and formatting metadata accompanying omics studies
Venue: arXiv cs.DL 2023
Year: 2023

Metadata, often termed "data about data," is crucial for organizing, understanding, and managing vast omics datasets. It aids in efficient data discovery, integration, and interpretation, enabling users to access, comprehend, and utilize data effectively. Its significance spans the domains of scientific research, facilitating data reproducibility, reusability, and secondary analysis. However, numerous perceptual and technical barriers hinder the sharing of metadata among researchers. These barriers compromise the reliability of research results and hinder integrative meta-analyses of omics studies . This study highlights the key barriers to metadata sharing, including the lack of uniform standards, privacy and legal concerns, limitations in study design, limited incentives, inadequate infrastructure, and the dearth of well-trained personnel for metadata management and reuse. Proposed solutions include emphasizing the promotion of standardization, educational efforts, the role of journals and funding agencies, incentives and rewards, and the improvement of infrastructure. More accurate, reliable, and impactful research outcomes are achievable if the scientific community addresses these barriers, facilitating more accurate, reliable, and impactful research outcomes.
Uncovering Regulatory Affairs Complexity in Medical Products: A Qualitative Assessment Utilizing Open Coding and Natural Language Processing (NLP)
Venue: arXiv cs.CY 2023
Year: 2023

This study investigates the complexity of regulatory affairs in the medical device industry, a critical factor influencing market access and patient care. Through qualitative research, we sought expert insights to understand the factors contributing to this complexity. The study involved semi-structured interviews with 28 professionals from medical device companies, specializing in various aspects of regulatory affairs. These interviews were analyzed using open coding and Natural Language Processing (NLP) techniques. The findings reveal key sources of complexity within the regulatory landscape, divided into five domains: (A) Regulatory language complexity, (B) Intricacies within the regulatory process, (C) Global-level complexities, (D) Database-related considerations, and (E) Product-level issues. The participants highlighted the need for strategies to streamline regulatory compliance, enhance interactions between regulatory bodies and industry players, and develop adaptable frameworks for rapid technological advancements. Emphasizing interdisciplinary collaboration and increased transparency, the study concludes that these elements are vital for establishing coherent and effective regulatory procedures in the medical device sector.
Measured and Modeled Outdoor Indoor Coverage at 28 GHz into High Thermal Efficiency Buildings
Venue: arXiv cs.NI 2023
Year: 2023

28 GHz outdoor-indoor coverage into modern office buildings with high thermal efficiency windows is found to be severely limited due to 46 dB median penetration loss at normal incidence and additional 15 dB median oblique incidence loss. The study is based on measurements of path gain over 280 outdoor-indoor links, at ranges up to 100 m. A simple theoretical path gain model is extended to include building penetration through multiple sides of the building as well as a reflection from another building. The theoretical model accounts for the building orientation relative to the source, resulting in 4.9 dB RMSE relative to data, as compared to 5.7 dB RMSE from a linear fit and 14.7 dB RMSE for the 3GPP recommended model. Only coarse description of the buildings is required: building orientation and exterior wall composition, without any interior details. Coverage range for SNR>-8 dB from an outdoor base to a terminal just inside a high-efficiency building is under 35 m
Symbolic Regression of Dynamic Network Models
Venue: arXiv cs.NE 2023
Year: 2023

Growing interest in modelling complex systems from brains to societies to cities using networks has led to increased efforts to describe generative processes that explain those networks. Recent successes in machine learning have prompted the usage of evolutionary computation, especially genetic programming to evolve computer programs that effectively forage a multidimensional search space to iteratively find better solutions that explain network structure. Symbolic regression contributes to these approaches by replicating network morphologies using both structure and processes, all while not relying on the scientists intuition or expertise. It distinguishes itself by introducing a novel formulation of a network generator and a parameter-free fitness function to evaluate the generated network and is found to consistently retrieve synthetically generated growth processes as well as simple, interpretable rules for a range of empirical networks. We extend this approach by modifying generator semantics to create and retrieve rules for time-varying networks. Lexicon to study networks created dynamically in multiple stages is introduced. The framework was improved using methods from the genetic programming toolkit (recombination) and computational improvements (using heuristic distance measures) and used to test the consistency and robustness of the upgrades to the semantics using synthetically generated networks. Using recombination was found to improve retrieval rate and fitness of the solutions. The framework was then used on three empirical datasets - subway networks of major cities, regions of street networks and semantic co-occurrence networks of literature in Artificial Intelligence to illustrate the possibility of obtaining interpretable, decentralised growth processes from complex networks.
A debiasing technique for place-based algorithmic patrol management
Venue: arXiv cs.CY 2023
Year: 2023

In recent years, there has been a revolution in data-driven policing. With that has come scrutiny on how bias in historical data affects algorithmic decision making. In this exploratory work, we introduce a debiasing technique for place-based algorithmic patrol management systems. We show that the technique efficiently eliminates racially biased features while retaining high accuracy in the models. Finally, we provide a lengthy list of potential future research in the realm of fairness and data-driven policing which this work uncovered.
Initial Analysis of Data-Driven Haptic Search for the Smart Suction Cup
Venue: arXiv cs.RO 2023
Year: 2023

Suction cups offer a useful gripping solution, particularly in industrial robotics and warehouse applications. Vision-based grasp algorithms, like Dex-Net, show promise but struggle to accurately perceive dark or reflective objects, sub-resolution features, and occlusions, resulting in suction cup grip failures. In our prior work, we designed the Smart Suction Cup, which estimates the flow state within the cup and provides a mechanically resilient end-effector that can inform arm feedback control through a sense of touch. We then demonstrated how this cup's signals enable haptically-driven search behaviors for better grasping points on adversarial objects. This prior work uses a model-based approach to predict the desired motion direction, which opens up the question: does a data-driven approach perform better? This technical report provides an initial analysis harnessing the data previously collected. Specifically, we compare the model-based method with a preliminary data-driven approach to accurately estimate lateral pose adjustment direction for improved grasp success.
Visions Of Destruction: Exploring Human Impact on Nature by Navigating the Latent Space of a Diffusion Model via Gaze
Venue: arXiv cs.CY 2023
Year: 2023

This paper discusses the artwork "Visions of Destruction", with a primary conceptual focus on the Anthropocene, which is communicated through audience interaction and generative AI as artistic research methods. Gaze-based interaction transitions the audience from mere observers to agents of landscape transformation, fostering a profound, on-the-edge engagement with pressing issues such as climate change and planetary destruction. The paper looks into early references of interactive art history that deploy eye-tracking as a method for audience interaction, and presents recent AI-aided artworks that demonstrate interactive latent space navigation.
Learning with Chemical versus Electrical Synapses -- Does it Make a Difference?
Venue: arXiv cs.NE 2023
Year: 2023

Bio-inspired neural networks have the potential to advance our understanding of neural computation and improve the state-of-the-art of AI systems. Bio-electrical synapses directly transmit neural signals, by enabling fast current flow between neurons. In contrast, bio-chemical synapses transmit neural signals indirectly, through neurotransmitters. Prior work showed that interpretable dynamics for complex robotic control, can be achieved by using chemical synapses, within a sparse, bio-inspired architecture, called Neural Circuit Policies (NCPs). However, a comparison of these two synaptic models, within the same architecture, remains an unexplored area. In this work we aim to determine the impact of using chemical synapses compared to electrical synapses, in both sparse and all-to-all connected networks. We conduct experiments with autonomous lane-keeping through a photorealistic autonomous driving simulator to evaluate their performance under diverse conditions and in the presence of noise. The experiments highlight the substantial influence of the architectural and synaptic-model choices, respectively. Our results show that employing chemical synapses yields noticeable improvements compared to electrical synapses, and that NCPs lead to better results in both synaptic models.
Typification of Driver Models Using Clustering Methods
Venue: arXiv cs.RO 2023
Year: 2023

The rapid development of automated driving systems in recent years has led to improvements in road safety and travel comfort. One typical function of these systems is Lane Keep Assist, which generally does not take human driving preferences into account. In our previous work, we have demonstrated that it is possible to implement a Lane Keep Assist function that is appropriate to human preferences using a trajectory planning algorithm based on a linear driving model. In our current work, we investigated how to separate the driving styles of individual drivers. We assumed that there are three driving styles: sporty, neutral and defensive. To prove these relations, clustering methods were applied to previously recorded measurements . Simulations with parameters describing the average behaviour of the classes (re-simulated with clustered types) showed that the resulting paths successfully classified drivers, that the 3 classes are distinct in their behaviour and that our model reproduces these behaviours.
DanceMeld: Unraveling Dance Phrases with Hierarchical Latent Codes for Music-to-Dance Synthesis
Venue: arXiv cs.OH 2023
Year: 2023

In the realm of 3D digital human applications, music-to-dance presents a challenging task. Given the one-to-many relationship between music and dance, previous methods have been limited in their approach, relying solely on matching and generating corresponding dance movements based on music rhythm. In the professional field of choreography, a dance phrase consists of several dance poses and dance movements. Dance poses composed of a series of basic meaningful body postures, while dance movements can reflect dynamic changes such as the rhythm, melody, and style of dance. Taking inspiration from these concepts, we introduce an innovative dance generation pipeline called DanceMeld, which comprising two stages, i.e., the dance decouple stage and the dance generation stage. In the decouple stage, a hierarchical VQ-VAE is used to disentangle dance poses and dance movements in different feature space levels, where the bottom code represents dance poses, and the top code represents dance movements. In the generation stage, we utilize a diffusion model as a prior to model the distribution and generate latent codes conditioned on music features. We have experimentally demonstrated the representational capabilities of top code and bottom code, enabling the explicit decoupling expression of dance poses and dance movements. This disentanglement not only provides control over motion details, styles, and rhythm but also facilitates applications such as dance style transfer and dance unit editing. Our approach has undergone qualitative and quantitative experiments on the AIST++ dataset, demonstrating its superiority over other methods.
Building a Reusable and Extensible Automatic Compiler Infrastructure for Reconfigurable Devices
Venue: arXiv cs.AR 2023
Year: 2023

Multi-Level Intermediate Representation (MLIR) is gaining increasing attention in reconfigurable hardware communities due to its capability to represent various abstract levels for software compilers. This project aims to be the first to provide an end-to-end framework that leverages open-source, cross-platform compilation technology to generate MLIR from SYCL. Additionally, it aims to explore a lowering pipeline that converts MLIR to RTL using open-source hardware intermediate representation (IR) and compilers. Furthermore, it aims to couple the generated hardware module with the host CPU using vendor-specific crossbars. Our preliminary results demonstrated the feasibility of lowering customized MLIR to RTL, thus paving the way for an end-to-end compilation.