OpenRaiser/Intern-Atlas huggingface dataset:
GraSSRep: Graph-Based Self-Supervised Learning for Repeat Detection in Metagenomic Assembly
Venue: arXiv cs.LG 2024
Year: 2024

Repetitive DNA (repeats) poses significant challenges for accurate and efficient genome assembly and sequence alignment. This is particularly true for metagenomic data, where genome dynamics such as horizontal gene transfer, gene duplication, and gene loss/gain complicate accurate genome assembly from metagenomic communities. Detecting repeats is a crucial first step in overcoming these challenges. To address this issue, we propose GraSSRep, a novel approach that leverages the assembly graph's structure through graph neural networks (GNNs) within a self-supervised learning framework to classify DNA sequences into repetitive and non-repetitive categories. Specifically, we frame this problem as a node classification task within a metagenomic assembly graph. In a self-supervised fashion, we rely on a high-precision (but low-recall) heuristic to generate pseudo-labels for a small proportion of the nodes. We then use those pseudo-labels to train a GNN embedding and a random forest classifier to propagate the labels to the remaining nodes. In this way, GraSSRep combines sequencing features with pre-defined and learned graph features to achieve state-of-the-art performance in repeat detection. We evaluate our method using simulated and synthetic metagenomic datasets. The results on the simulated data highlight our GraSSRep's robustness to repeat attributes, demonstrating its effectiveness in handling the complexity of repeated sequences. Additionally, our experiments with synthetic metagenomic datasets reveal that incorporating the graph structure and the GNN enhances our detection performance. Finally, in comparative analyses, GraSSRep outperforms existing repeat detection tools with respect to precision and recall.
An Enhanced Analysis of Traffic Intelligence in Smart Cities Using Sustainable Deep Radial Function
Venue: arXiv cs.LG 2024
Year: 2024

Smart cities have revolutionized urban living by incorporating sophisticated technologies to optimize various aspects of urban infrastructure, such as transportation systems. Effective traffic management is a crucial component of smart cities, as it has a direct impact on the quality of life of residents and tourists. Utilizing deep radial basis function (RBF) networks, this paper describes a novel strategy for enhancing traffic intelligence in smart cities. Traditional methods of traffic analysis frequently rely on simplistic models that are incapable of capturing the intricate patterns and dynamics of urban traffic systems. Deep learning techniques, such as deep RBF networks, have the potential to extract valuable insights from traffic data and enable more precise predictions and decisions. In this paper, we propose an RBF based method for enhancing smart city traffic intelligence. Deep RBF networks combine the adaptability and generalization capabilities of deep learning with the discriminative capability of radial basis functions. The proposed method can effectively learn intricate relationships and nonlinear patterns in traffic data by leveraging the hierarchical structure of deep neural networks. The deep RBF model can learn to predict traffic conditions, identify congestion patterns, and make informed recommendations for optimizing traffic management strategies by incorporating these rich and diverse data To evaluate the efficacy of our proposed method, extensive experiments and comparisons with real world traffic datasets from a smart city environment were conducted. In terms of prediction accuracy and efficiency, the results demonstrate that the deep RBF based approach outperforms conventional traffic analysis methods. Smart city traffic intelligence is enhanced by the model capacity to capture nonlinear relationships and manage large scale data sets.
Managing Household Waste through Transfer Learning
Venue: arXiv cs.CV 2024
Year: 2024

As the world continues to face the challenges of climate change, it is crucial to consider the environmental impact of the technologies we use. In this study, we investigate the performance and computational carbon emissions of various transfer learning models for garbage classification. We examine the MobileNet, ResNet50, ResNet101, and EfficientNetV2S and EfficientNetV2M models. Our findings indicate that the EfficientNetV2 family achieves the highest accuracy, recall, f1-score, and IoU values. However, the EfficientNetV2M model requires more time and produces higher carbon emissions. ResNet50 outperforms ResNet110 in terms of accuracy, recall, f1-score, and IoU, but it has a larger carbon footprint. We conclude that EfficientNetV2S is the most sustainable and accurate model with 96.41% accuracy. Our research highlights the significance of considering the ecological impact of machine learning models in garbage classification.
LiveDocs: Crafting Interactive Development Environments From Research Findings
Venue: arXiv cs.HC 2024
Year: 2024

Open Science is a recurrent topic in scientific discussion, and there is a current effort to make research more accessible to a broader audience. A focus on delivering research findings that are reproducible, or even re-usable has been proposed as one way of achieving such accessibility goals. In this work, we present the LiveDocs initiative, an effort of the ``Collaborative Research Center 1456 - Mathematics of Experiment'' on tackling common issues of reproducibility and re-usability in scientific publications. The LiveDocs initiative is proposed as a concept alongside a collection of methods that enable scientists to provide research findings under an interactive development environment. This environment allows users from a broader audience to easily reproduce research findings by re-running scripts, for instance, those that generate figures, tables, and other elements from scientific publications. Moreover, LiveDocs also allow the audience to interact with code and data in such environments, thus allowing users to explore algorithms, datasets and software interfaces. This directly lowers the barriers to access and comprehend research methods and findings, which facilitates more scientific exchange and fosters knowledge advancement.
PMGDA: A Preference-based Multiple Gradient Descent Algorithm
Venue: arXiv cs.LG 2024
Year: 2024

It is desirable in many multi-objective machine learning applications, such as multi-task learning with conflicting objectives and multi-objective reinforcement learning, to find a Pareto solution that can match a given preference of a decision maker. These problems are often large-scale with available gradient information but cannot be handled very well by the existing algorithms. To tackle this critical issue, this paper proposes a novel predict-and-correct framework for locating a Pareto solution that fits the preference of a decision maker. In the proposed framework, a constraint function is introduced in the search progress to align the solution with a user-specific preference, which can be optimized simultaneously with multiple objective functions. Experimental results show that our proposed method can efficiently find a particular Pareto solution under the demand of a decision maker for standard multiobjective benchmark, multi-task learning, and multi-objective reinforcement learning problems with more than thousands of decision variables. Code is available at: https://github.com/xzhang2523/pmgda. Our code is current provided in the pgmda.rar attached file and will be open-sourced after publication.}
Detection of the most influential variables for preventing postpartum urinary incontinence using machine learning techniques
Venue: arXiv cs.AI 2024
Year: 2024

Background: Postpartum urinary incontinence (PUI) is a common issue among postnatal women. Previous studies identified potential related variables, but lacked analysis on certain intrinsic and extrinsic patient variables during pregnancy. Objective: The study aims to evaluate the most influential variables in PUI using machine learning, focusing on intrinsic, extrinsic, and combined variable groups. Methods: Data from 93 pregnant women were analyzed using machine learning and oversampling techniques. Four key variables were predicted: occurrence, frequency, intensity of urinary incontinence, and stress urinary incontinence. Results: Models using extrinsic variables were most accurate, with 70% accuracy for urinary incontinence, 77% for frequency, 71% for intensity, and 93% for stress urinary incontinence. Conclusions: The study highlights extrinsic variables as significant predictors of PUI issues. This suggests that PUI prevention might be achievable through healthy habits during pregnancy, although further research is needed for confirmation.
Learning From Lessons Learned: Preliminary Findings From a Study of Learning From Failure
Venue: arXiv cs.SE 2024
Year: 2024

Due to various sources of uncertainty, emergent behavior, and ongoing changes, the reliability of many socio-technical systems depends on an iterative and collaborative process in which organizations (1) analyze and learn from system failures, and then (2) co-evolve both the technical and human parts of their systems based on what they learn. Many organizations have defined processes for learning from failure, often involving postmortem analyses conducted after any system failures that are judged to be sufficiently severe. Despite established processes and tool support, our preliminary research, and professional experience, suggest that it is not straightforward to take what was learned from a failure and successfully improve the reliability of the socio-technical system. To better understand this collaborative process and the associated challenges, we are conducting a study of how teams learn from failure. We are gathering incident reports from multiple organizations and conducting interviews with engineers and managers with relevant experience. Our analytic interest is in what is learned by teams as they reflect on failures, the learning processes involved, and how they use what is learned. Our data collection and analysis are not yet complete, but we have so far analyzed 13 incident reports and seven interviews. In this short paper we (1) present our preliminary findings, and (2) outline our broader research plans.
Dataset Clustering for Improved Offline Policy Learning
Venue: arXiv cs.LG 2024
Year: 2024

Offline policy learning aims to discover decision-making policies from previously-collected datasets without additional online interactions with the environment. As the training dataset is fixed, its quality becomes a crucial determining factor in the performance of the learned policy. This paper studies a dataset characteristic that we refer to as multi-behavior, indicating that the dataset is collected using multiple policies that exhibit distinct behaviors. In contrast, a uni-behavior dataset would be collected solely using one policy. We observed that policies learned from a uni-behavior dataset typically outperform those learned from multi-behavior datasets, despite the uni-behavior dataset having fewer examples and less diversity. Therefore, we propose a behavior-aware deep clustering approach that partitions multi-behavior datasets into several uni-behavior subsets, thereby benefiting downstream policy learning. Our approach is flexible and effective; it can adaptively estimate the number of clusters while demonstrating high clustering accuracy, achieving an average Adjusted Rand Index of 0.987 across various continuous control task datasets. Finally, we present improved policy learning examples using dataset clustering and discuss several potential scenarios where our approach might benefit the offline policy learning community.
Enhancing Source Code Representations for Deep Learning with Static Analysis
Venue: arXiv cs.SE 2024
Year: 2024

Deep learning techniques applied to program analysis tasks such as code classification, summarization, and bug detection have seen widespread interest. Traditional approaches, however, treat programming source code as natural language text, which may neglect significant structural or semantic details. Additionally, most current methods of representing source code focus solely on the code, without considering beneficial additional context. This paper explores the integration of static analysis and additional context such as bug reports and design patterns into source code representations for deep learning models. We use the Abstract Syntax Tree-based Neural Network (ASTNN) method and augment it with additional context information obtained from bug reports and design patterns, creating an enriched source code representation that significantly enhances the performance of common software engineering tasks such as code classification and code clone detection. Utilizing existing open-source code data, our approach improves the representation and processing of source code, thereby improving task performance.
Reconstructing the Geometry of Random Geometric Graphs
Venue: arXiv cs.LG 2024
Year: 2024

Random geometric graphs are random graph models defined on metric spaces. Such a model is defined by first sampling points from a metric space and then connecting each pair of sampled points with probability that depends on their distance, independently among pairs. In this work, we show how to efficiently reconstruct the geometry of the underlying space from the sampled graph under the manifold assumption, i.e., assuming that the underlying space is a low dimensional manifold and that the connection probability is a strictly decreasing function of the Euclidean distance between the points in a given embedding of the manifold in $\mathbb{R}^N$. Our work complements a large body of work on manifold learning, where the goal is to recover a manifold from sampled points sampled in the manifold along with their (approximate) distances.
A Web-Based Tool for Automatic Data Collection, Curation, and Visualization of Complex Healthcare Survey Studies including Social Network Analysis
Venue: arXiv cs.AI 2024
Year: 2024

There is a great concern nowadays regarding alcohol consumption and drug abuse, especially in young people. Analyzing the social environment where these adolescents are immersed, as well as a series of measures determining the alcohol abuse risk or personal situation and perception using a number of questionnaires like AUDIT, FAS, KIDSCREEN, and others, it is possible to gain insight into the current situation of a given individual regarding his/her consumption behavior. But this analysis, in order to be achieved, requires the use of tools that can ease the process of questionnaire creation, data gathering, curation and representation, and later analysis and visualization to the user. This research presents the design and construction of a web-based platform able to facilitate each of the mentioned processes by integrating the different phases into an intuitive system with a graphical user interface that hides the complexity underlying each of the questionnaires and techniques used and presenting the results in a flexible and visual way, avoiding any manual handling of data during the process. Advantages of this approach are shown and compared to the previous situation where some of the tasks were accomplished by time consuming and error prone manipulations of data.
Correctly Communicating Software: Distributed, Asynchronous, and Beyond (extended version)
Venue: arXiv cs.LO 2024
Year: 2024

Much of the software we use in everyday life consists of distributed components (running on separate cores or even computers) that collaborate through communication (by exchanging messages). It is crucial to develop robust methods that can give reliable guarantees about the behavior of such message-passing software. With a focus on session types as communication protocols and their foundations in logic, this thesis revolves around the following question: How can we push the boundaries of the logical foundations of session types (binary and multiparty), extending their expressiveness and applicability, while preserving fundamental correctness properties? In this context, this thesis studies several intertwined aspects of message-passing.
Smart Information Exchange for Unsupervised Federated Learning via Reinforcement Learning
Venue: arXiv cs.LG 2024
Year: 2024

One of the main challenges of decentralized machine learning paradigms such as Federated Learning (FL) is the presence of local non-i.i.d. datasets. Device-to-device transfers (D2D) between distributed devices has been shown to be an effective tool for dealing with this problem and robust to stragglers. In an unsupervised case, however, it is not obvious how data exchanges should take place due to the absence of labels. In this paper, we propose an approach to create an optimal graph for data transfer using Reinforcement Learning. The goal is to form links that will provide the most benefit considering the environment's constraints and improve convergence speed in an unsupervised FL environment. Numerical analysis shows the advantages in terms of convergence speed and straggler resilience of the proposed method to different available FL schemes and benchmark datasets.
Foul prediction with estimated poses from soccer broadcast video
Venue: arXiv cs.CV 2024
Year: 2024

Recent advances in computer vision have made significant progress in tracking and pose estimation of sports players. However, there have been fewer studies on behavior prediction with pose estimation in sports, in particular, the prediction of soccer fouls is challenging because of the smaller image size of each player and of difficulty in the usage of e.g., the ball and pose information. In our research, we introduce an innovative deep learning approach for anticipating soccer fouls. This method integrates video data, bounding box positions, image details, and pose information by curating a novel soccer foul dataset. Our model utilizes a combination of convolutional and recurrent neural networks (CNNs and RNNs) to effectively merge information from these four modalities. The experimental results show that our full model outperformed the ablated models, and all of the RNN modules, bounding box position and image, and estimated pose were useful for the foul prediction. Our findings have important implications for a deeper understanding of foul play in soccer and provide a valuable reference for future research and practice in this area.
Practitioners' Challenges and Perceptions of CI Build Failure Predictions at Atlassian
Venue: arXiv cs.SE 2024
Year: 2024

Continuous Integration (CI) build failures could significantly impact the software development process and teams, such as delaying the release of new features and reducing developers' productivity. In this work, we report on an empirical study that investigates CI build failures throughout product development at Atlassian. Our quantitative analysis found that the repository dimension is the key factor influencing CI build failures. In addition, our qualitative survey revealed that Atlassian developers perceive CI build failures as challenging issues in practice. Furthermore, we found that the CI build prediction can not only provide proactive insight into CI build failures but also facilitate the team's decision-making. Our study sheds light on the challenges and expectations involved in integrating CI build prediction tools into the Bitbucket environment, providing valuable insights for enhancing CI processes.
The Butterfly Effect of Model Editing: Few Edits Can Trigger Large Language Models Collapse
Venue: arXiv cs.AI 2024
Year: 2024

Although model editing has shown promise in revising knowledge in Large Language Models (LLMs), its impact on the inherent capabilities of LLMs is often overlooked. In this work, we reveal a critical phenomenon: even a single edit can trigger model collapse, manifesting as significant performance degradation in various benchmark tasks. However, benchmarking LLMs after each edit, while necessary to prevent such collapses, is impractically time-consuming and resource-intensive. To mitigate this, we propose using perplexity as a surrogate metric, validated by extensive experiments demonstrating changes in an edited model's perplexity are strongly correlated with its downstream task performances. We further conduct an in-depth study on sequential editing, a practical setting for real-world scenarios, across various editing methods and LLMs, focusing on hard cases from our previous single edit studies. The results indicate that nearly all examined editing methods result in model collapse after only few edits. To facilitate further research, we have utilized GPT-3.5 to develop a new dataset, HardEdit, based on those hard cases. This dataset aims to establish the foundation for pioneering research in reliable model editing and the mechanisms underlying editing-induced model collapse. We hope this work can draw the community's attention to the potential risks inherent in model editing practices.
EntailE: Introducing Textual Entailment in Commonsense Knowledge Graph Completion
Venue: arXiv cs.CL 2024
Year: 2024

Commonsense knowledge graph completion is a new challenge for commonsense knowledge graph construction and application. In contrast to factual knowledge graphs such as Freebase and YAGO, commonsense knowledge graphs (CSKGs; e.g., ConceptNet) utilize free-form text to represent named entities, short phrases, and events as their nodes. Such a loose structure results in large and sparse CSKGs, which makes the semantic understanding of these nodes more critical for learning rich commonsense knowledge graph embedding. While current methods leverage semantic similarities to increase the graph density, the semantic plausibility of the nodes and their relations are under-explored. Previous works adopt conceptual abstraction to improve the consistency of modeling (event) plausibility, but they are not scalable enough and still suffer from data sparsity. In this paper, we propose to adopt textual entailment to find implicit entailment relations between CSKG nodes, to effectively densify the subgraph connecting nodes within the same conceptual class, which indicates a similar level of plausibility. Each node in CSKG finds its top entailed nodes using a finetuned transformer over natural language inference (NLI) tasks, which sufficiently capture textual entailment signals. The entailment relation between these nodes are further utilized to: 1) build new connections between source triplets and entailed nodes to densify the sparse CSKGs; 2) enrich the generalization ability of node representations by comparing the node embeddings with a contrastive loss. Experiments on two standard CSKGs demonstrate that our proposed framework EntailE can improve the performance of CSKG completion tasks under both transductive and inductive settings.
SpaceMeta: Global-Scale Massive Multi-User Virtual Interaction over LEO Satellite Constellations
Venue: arXiv cs.MM 2024
Year: 2024

Low latency and high synchronization among users are critical for emerging multi-user virtual interaction applications. However, the existing ground-based cloud solutions are naturally limited by the complex ground topology and fiber speeds, making it difficult to pace with the requirement of multi-user virtual interaction. The growth of low earth orbit (LEO) satellite constellations becomes a promising alternative to ground solutions. To fully exploit the potential of the LEO satellite, in this paper, we study the satellite server selection problem for global-scale multi-user interaction applications over LEO constellations. We propose an effective server selection framework, called SpaceMeta, that jointly selects the ingress satellite servers and relay servers on the communication path to minimize latency and latency discrepancy among users. Extensive experiments using real-world Starlink topology demonstrate that SpaceMeta reduces the latency by 6.72% and the interquartile range (IQR) of user latency by 39.50% compared with state-of-the-art methods.