OpenRaiser/Intern-Atlas huggingface dataset:
Artificial Intelligence for Technical Debt Management in Software Development
Venue: arXiv cs.SE 2023
Year: 2023

Technical debt is a well-known challenge in software development, and its negative impact on software quality, maintainability, and performance is widely recognized. In recent years, artificial intelligence (AI) has proven to be a promising approach to assist in managing technical debt. This paper presents a comprehensive literature review of existing research on the use of AI powered tools for technical debt avoidance in software development. In this literature review we analyzed 15 related research papers which covers various AI-powered techniques, such as code analysis and review, automated testing, code refactoring, predictive maintenance, code generation, and code documentation, and explores their effectiveness in addressing technical debt. The review also discusses the benefits and challenges of using AI for technical debt management, provides insights into the current state of research, and highlights gaps and opportunities for future research. The findings of this review suggest that AI has the potential to significantly improve technical debt management in software development, and that existing research provides valuable insights into how AI can be leveraged to address technical debt effectively and efficiently. However, the review also highlights several challenges and limitations of current approaches, such as the need for high-quality data and ethical considerations and underscores the importance of further research to address these issues. The paper provides a comprehensive overview of the current state of research on AI for technical debt avoidance and offers practical guidance for software development teams seeking to leverage AI in their development processes to mitigate technical debt effectively
Privacy-Enhancing Technologies for Financial Data Sharing
Venue: arXiv cs.CR 2023
Year: 2023

Today, financial institutions (FIs) store and share consumers' financial data for various reasons such as offering loans, processing payments, and protecting against fraud and financial crime. Such sharing of sensitive data have been subject to data breaches in the past decade. While some regulations (e.g., GDPR, FCRA, and CCPA) help to prevent institutions from freely sharing clients' sensitive information, some regulations (e.g., BSA 1970) require FIs to share certain financial data with government agencies to combat financial crime. This creates an inherent tension between the privacy and the integrity of financial transactions. In the past decade, significant progress has been made in building efficient privacy-enhancing technologies that allow computer systems and networks to validate encrypted data automatically. In this paper, we investigate some of these technologies to identify the benefits and limitations of each, in particular, for use in data sharing among FIs. As a case study, we look into the emerging area of Central Bank Digital Currencies (CBDCs) and how privacy-enhancing technologies can be integrated into the CBDC architecture. Our study, however, is not limited to CBDCs and can be applied to other financial scenarios with tokenized bank deposits such as cross-border payments, real-time settlements, and card payments.
Learning Space-Time Semantic Correspondences
Venue: arXiv cs.CV 2023
Year: 2023

We propose a new task of space-time semantic correspondence prediction in videos. Given a source video, a target video, and a set of space-time key-points in the source video, the task requires predicting a set of keypoints in the target video that are the semantic correspondences of the provided source keypoints. We believe that this task is important for fine-grain video understanding, potentially enabling applications such as activity coaching, sports analysis, robot imitation learning, and more. Our contributions in this paper are: (i) proposing a new task and providing annotations for space-time semantic correspondences on two existing benchmarks: Penn Action and Pouring; and (ii) presenting a comprehensive set of baselines and experiments to gain insights about the new problem. Our main finding is that the space-time semantic correspondence prediction problem is best approached jointly in space and time rather than in their decomposed sub-problems: time alignment and spatial correspondences.
GLIMMER: generalized late-interaction memory reranker
Venue: arXiv cs.CL 2023
Year: 2023

Memory-augmentation is a powerful approach for efficiently incorporating external information into language models, but leads to reduced performance relative to retrieving text. Recent work introduced LUMEN, a memory-retrieval hybrid that partially pre-computes memory and updates memory representations on the fly with a smaller live encoder. We propose GLIMMER, which improves on this approach through 1) exploiting free access to the powerful memory representations by applying a shallow reranker on top of memory to drastically improve retrieval quality at low cost, and 2) incorporating multi-task training to learn a general and higher quality memory and live encoder. GLIMMER achieves strong gains in performance at faster speeds compared to LUMEN and FiD on the KILT benchmark of knowledge-intensive tasks.
Geometric Mechanics of Contact-Switching Systems
Venue: arXiv cs.RO 2023
Year: 2023

Discrete and periodic contact switching is a key characteristic of steady-state legged locomotion. This paper introduces a framework for modeling and analyzing this contact-switching behavior through the framework of geometric mechanics on a toy robot model that can make continuous limb swings and discrete contact switches. The kinematics of this model form a hybrid shape-space and by extending the generalized Stokes' theorem to compute discrete curvature functions called \textit{stratified panels}, we determine average locomotion generated by gaits spanning multiple contact modes. Using this tool, we also demonstrate the ability to optimize gaits based on the system's locomotion constraints and perform gait reduction on a complex gait spanning multiple contact modes to highlight the method's scalability to multilegged systems.
Adaptive Strategies in Non-convex Optimization
Venue: arXiv cs.LG 2023
Year: 2023

An algorithm is said to be adaptive to a certain parameter (of the problem) if it does not need a priori knowledge of such a parameter but performs competitively to those that know it. This dissertation presents our work on adaptive algorithms in following scenarios: 1. In the stochastic optimization setting, we only receive stochastic gradients and the level of noise in evaluating them greatly affects the convergence rate. Tuning is typically required when without prior knowledge of the noise scale in order to achieve the optimal rate. Considering this, we designed and analyzed noise-adaptive algorithms that can automatically ensure (near)-optimal rates under different noise scales without knowing it. 2. In training deep neural networks, the scales of gradient magnitudes in each coordinate can scatter across a very wide range unless normalization techniques, like BatchNorm, are employed. In such situations, algorithms not addressing this problem of gradient scales can behave very poorly. To mitigate this, we formally established the advantage of scale-free algorithms that adapt to the gradient scales and presented its real benefits in empirical experiments. 3. Traditional analyses in non-convex optimization typically rely on the smoothness assumption. Yet, this condition does not capture the properties of some deep learning objective functions, including the ones involving Long Short-Term Memory networks and Transformers. Instead, they satisfy a much more relaxed condition, with potentially unbounded smoothness. Under this condition, we show that a generalized SignSGD algorithm can theoretically match the best-known convergence rates obtained by SGD with gradient clipping but does not need explicit clipping at all, and it can empirically match the performance of Adam and beat others. Moreover, it can also be made to automatically adapt to the unknown relaxed smoothness.
Managing health insurance using blockchain technology
Venue: arXiv cs.CY 2023
Year: 2023

Health insurance plays a significant role in ensuring quality healthcare. In response to the escalating costs of the medical industry, the demand for health insurance is soaring. Additionally, those with health insurance are more likely to receive preventative care than those without health insurance. However, from granting health insurance to delivering services to insured individuals, the health insurance industry faces numerous obstacles. Fraudulent actions, false claims, a lack of transparency and data privacy, reliance on human effort and dishonesty from consumers, healthcare professionals, or even the insurer party itself, are the most common and important hurdles towards success. Given these constraints, this chapter briefly covers the most immediate concerns in the health insurance industry and provides insight into how blockchain technology integration can contribute to resolving these issues. This chapter finishes by highlighting existing limitations as well as potential future directions.
An analysis of Universal Differential Equations for data-driven discovery of Ordinary Differential Equations
Venue: arXiv cs.LG 2023
Year: 2023

In the last decade, the scientific community has devolved its attention to the deployment of data-driven approaches in scientific research to provide accurate and reliable analysis of a plethora of phenomena. Most notably, Physics-informed Neural Networks and, more recently, Universal Differential Equations (UDEs) proved to be effective both in system integration and identification. However, there is a lack of an in-depth analysis of the proposed techniques. In this work, we make a contribution by testing the UDE framework in the context of Ordinary Differential Equations (ODEs) discovery. In our analysis, performed on two case studies, we highlight some of the issues arising when combining data-driven approaches and numerical solvers, and we investigate the importance of the data collection process. We believe that our analysis represents a significant contribution in investigating the capabilities and limitations of Physics-informed Machine Learning frameworks.
Text-Driven Foley Sound Generation With Latent Diffusion Model
Venue: arXiv cs.SD 2023
Year: 2023

Foley sound generation aims to synthesise the background sound for multimedia content. Previous models usually employ a large development set with labels as input (e.g., single numbers or one-hot vector). In this work, we propose a diffusion model based system for Foley sound generation with text conditions. To alleviate the data scarcity issue, our model is initially pre-trained with large-scale datasets and fine-tuned to this task via transfer learning using the contrastive language-audio pertaining (CLAP) technique. We have observed that the feature embedding extracted by the text encoder can significantly affect the performance of the generation model. Hence, we introduce a trainable layer after the encoder to improve the text embedding produced by the encoder. In addition, we further refine the generated waveform by generating multiple candidate audio clips simultaneously and selecting the best one, which is determined in terms of the similarity score between the embedding of the candidate clips and the embedding of the target text label. Using the proposed method, our system ranks ${1}^{st}$ among the systems submitted to DCASE Challenge 2023 Task 7. The results of the ablation studies illustrate that the proposed techniques significantly improve sound generation performance. The codes for implementing the proposed system are available online.
Algorithms for Multiple Drone-Delivery Scheduling Problem (MDSP)
Venue: arXiv cs.DS 2023
Year: 2023

The Multiple Drone-Delivery Scheduling Problem (MDSP) is a scheduling problem that optimizes the maximum reward earned by a set of $m$ drones executing a sequence of deliveries on a truck delivery route. The current best-known approximation algorithm for the problem is a $\frac{1}{4}$-approximation algorithm developed by Jana and Mandal (2022). In this paper, we propose exact and approximation algorithms for the general MDSP, as well as a unit-cost variant. We first propose a greedy algorithm which we show to be a $\frac{1}{3}$-approximation algorithm for the general MDSP problem formulation, provided the number of conflicting intervals is less than the number of drones. We then introduce a unit-cost variant of MDSP and we devise an exact dynamic programming algorithm that runs in polynomial time when the number of drones $m$ can be assumed to be a constant.
The CUFF, Clenching Upper-limb Force Feedback wearable device: design, characterization and validation
Venue: arXiv cs.HC 2023
Year: 2023

This paper presents the design, characterization and validation of a wearable haptic device able to convey skin stretch, force feedback, and a combination of both, to the user's arm. In this work, we carried out physical and perceptual characterization with eleven able-bodied participants as well as two experiments of discrimination and manipulation task hiring a total of 32 participants. In both the experiments the CUFF was used in conjunction with the Pisa/IIT SoftHand. The first experiment was a discrimination task where the subjects had to recognize the dimension and the softness between pair of cylinder. in the second experiment the subjects were asked to control the robotic hand for grasping objects. After the experiments the subjects underwent to a subjective evaluation of the device. Results of the experiments and questionnaire showed the effectiveness of the proposed device. Thank to its versatility and structure, the device could be a viable solution for teleoperation application, guidance and rehabilitation tasks, including prosthesis applications.
Generation of Radiology Findings in Chest X-Ray by Leveraging Collaborative Knowledge
Venue: arXiv cs.CV 2023
Year: 2023

Among all the sub-sections in a typical radiology report, the Clinical Indications, Findings, and Impression often reflect important details about the health status of a patient. The information included in Impression is also often covered in Findings. While Findings and Impression can be deduced by inspecting the image, Clinical Indications often require additional context. The cognitive task of interpreting medical images remains the most critical and often time-consuming step in the radiology workflow. Instead of generating an end-to-end radiology report, in this paper, we focus on generating the Findings from automated interpretation of medical images, specifically chest X-rays (CXRs). Thus, this work focuses on reducing the workload of radiologists who spend most of their time either writing or narrating the Findings. Unlike past research, which addresses radiology report generation as a single-step image captioning task, we have further taken into consideration the complexity of interpreting CXR images and propose a two-step approach: (a) detecting the regions with abnormalities in the image, and (b) generating relevant text for regions with abnormalities by employing a generative large language model (LLM). This two-step approach introduces a layer of interpretability and aligns the framework with the systematic reasoning that radiologists use when reviewing a CXR.
IF2Net: Innately Forgetting-Free Networks for Continual Learning
Venue: arXiv cs.LG 2023
Year: 2023

Continual learning can incrementally absorb new concepts without interfering with previously learned knowledge. Motivated by the characteristics of neural networks, in which information is stored in weights on connections, we investigated how to design an Innately Forgetting-Free Network (IF2Net) for continual learning context. This study proposed a straightforward yet effective learning paradigm by ingeniously keeping the weights relative to each seen task untouched before and after learning a new task. We first presented the novel representation-level learning on task sequences with random weights. This technique refers to tweaking the drifted representations caused by randomization back to their separate task-optimal working states, but the involved weights are frozen and reused (opposite to well-known layer-wise updates of weights). Then, sequential decision-making without forgetting can be achieved by projecting the output weight updates into the parsimonious orthogonal space, making the adaptations not disturb old knowledge while maintaining model plasticity. IF2Net allows a single network to inherently learn unlimited mapping rules without telling task identities at test time by integrating the respective strengths of randomization and orthogonalization. We validated the effectiveness of our approach in the extensive theoretical analysis and empirical study.
Structure-Sensitive Graph Dictionary Embedding for Graph Classification
Venue: arXiv cs.LG 2023
Year: 2023

Graph structure expression plays a vital role in distinguishing various graphs. In this work, we propose a Structure-Sensitive Graph Dictionary Embedding (SS-GDE) framework to transform input graphs into the embedding space of a graph dictionary for the graph classification task. Instead of a plain use of a base graph dictionary, we propose the variational graph dictionary adaptation (VGDA) to generate a personalized dictionary (named adapted graph dictionary) for catering to each input graph. In particular, for the adaptation, the Bernoulli sampling is introduced to adjust substructures of base graph keys according to each input, which increases the expression capacity of the base dictionary tremendously. To make cross-graph measurement sensitive as well as stable, multi-sensitivity Wasserstein encoding is proposed to produce the embeddings by designing multi-scale attention on optimal transport. To optimize the framework, we introduce mutual information as the objective, which further deduces to variational inference of the adapted graph dictionary. We perform our SS-GDE on multiple datasets of graph classification, and the experimental results demonstrate the effectiveness and superiority over the state-of-the-art methods.
Boosting the Performance of Degraded Reads in RS-coded Distributed Storage Systems
Venue: arXiv cs.DC 2023
Year: 2023

Reed-Solomon (RS) codes have been increasingly adopted by distributed storage systems in place of replication,because they provide the same level of availability with much lower storage overhead. However, a key drawback of those RS-coded distributed storage systems is the poor latency of degraded reads, which can be incurred by data failures or hot spots,and are not rare in production environments. To address this issue, we propose a novel parallel reconstruction solution called APLS. APLS leverages all surviving source nodes to send the data needed by degraded reads and chooses light-loaded starter nodes to receive the reconstructed data of those degraded reads. Hence, the latency of the degraded reads can be improved.Prototyping-based experiments are conducted to compare APLS with ECPipe, the state-of-the-art solution of improving the latency of degraded reads. The experimental results demonstrate that APLS effectively reduces the latency, particularly under heavy or medium workloads.
Gender Bias in Transformer Models: A comprehensive survey
Venue: arXiv cs.CL 2023
Year: 2023

Gender bias in artificial intelligence (AI) has emerged as a pressing concern with profound implications for individuals' lives. This paper presents a comprehensive survey that explores gender bias in Transformer models from a linguistic perspective. While the existence of gender bias in language models has been acknowledged in previous studies, there remains a lack of consensus on how to effectively measure and evaluate this bias. Our survey critically examines the existing literature on gender bias in Transformers, shedding light on the diverse methodologies and metrics employed to assess bias. Several limitations in current approaches to measuring gender bias in Transformers are identified, encompassing the utilization of incomplete or flawed metrics, inadequate dataset sizes, and a dearth of standardization in evaluation methods. Furthermore, our survey delves into the potential ramifications of gender bias in Transformers for downstream applications, including dialogue systems and machine translation. We underscore the importance of fostering equity and fairness in these systems by emphasizing the need for heightened awareness and accountability in developing and deploying language technologies. This paper serves as a comprehensive overview of gender bias in Transformer models, providing novel insights and offering valuable directions for future research in this critical domain.
UniMC: A Unified Framework for Long-Term Memory Conversation via Relevance Representation Learning
Venue: arXiv cs.CL 2023
Year: 2023

Open-domain long-term memory conversation can establish long-term intimacy with humans, and the key is the ability to understand and memorize long-term dialogue history information. Existing works integrate multiple models for modelling through a pipeline, which ignores the coupling between different stages. In this paper, we propose a Unified framework for Long-term Memory Conversations (UniMC), which increases the connection between different stages by learning relevance representation. Specifically, we decompose the main task into three subtasks based on probability graphs: 1) conversation summarization, 2) memory retrieval, 3) memory-augmented generation. Each subtask involves learning a representation for calculating the relevance between the query and memory, which is modelled by inserting a special token at the beginning of the decoder input. The relevance representation learning strengthens the connection across subtasks through parameter sharing and joint training. Extensive experimental results show that the proposed method consistently improves over strong baselines and yields better dialogue consistency and engagingness.
Optical Integrated Sensing and Communication for Cooperative Mobile Robotics Design and Experiments
Venue: arXiv cs.NI 2023
Year: 2023

Integrated Sensing and Communication (ISAC) is an emerging technology that integrates wireless sensing and communication into a single system, transforming many applications, including cooperative mobile robotics. However, in scenarios where radio communications are unavailable, alternative approaches are needed. In this paper, we propose a new optical ISAC (OISAC) scheme for cooperative mobile robots by integrating camera sensing and screen-camera communication (SCC). Unlike previous throughput-oriented SCC designs that work with stationary SCC links, our OISAC scheme is designed for real-time control of mobile robots. It addresses new problems such as image blur and long image display delay. As a case study, we consider the leader-follower formation control problem, an essential part of cooperative mobile robotics. The proposed OISAC scheme enables the follower robot to simultaneously acquire the information shared by the leader and sense the relative pose to the leader using only RGB images captured by its onboard camera. We then design a new control law that can leverage all the information acquired by the camera to achieve stable and accurate formations. We design and conduct real-world experiments involving uniform and nonuniform motions to evaluate the proposed system and demonstrate the advantages of applying OISAC over a benchmark approach that uses extended Kalman filtering (EKF) to estimate the leader's states. Our results show that the proposed OISAC-augmented leader-follower formation system achieves better performance in terms of accuracy, stability, and robustness.