OpenRaiser/Intern-Atlas huggingface dataset:
WrappingNet: Mesh Autoencoder via Deep Sphere Deformation
Venue: arXiv cs.CV 2023
Year: 2023

There have been recent efforts to learn more meaningful representations via fixed length codewords from mesh data, since a mesh serves as a complete model of underlying 3D shape compared to a point cloud. However, the mesh connectivity presents new difficulties when constructing a deep learning pipeline for meshes. Previous mesh unsupervised learning approaches typically assume category-specific templates, e.g., human face/body templates. It restricts the learned latent codes to only be meaningful for objects in a specific category, so the learned latent spaces are unable to be used across different types of objects. In this work, we present WrappingNet, the first mesh autoencoder enabling general mesh unsupervised learning over heterogeneous objects. It introduces a novel base graph in the bottleneck dedicated to representing mesh connectivity, which is shown to facilitate learning a shared latent space representing object shape. The superiority of WrappingNet mesh learning is further demonstrated via improved reconstruction quality and competitive classification compared to point cloud learning, as well as latent interpolation between meshes of different categories.
Input margins can predict generalization too
Venue: arXiv cs.LG 2023
Year: 2023

Understanding generalization in deep neural networks is an active area of research. A promising avenue of exploration has been that of margin measurements: the shortest distance to the decision boundary for a given sample or its representation internal to the network. While margins have been shown to be correlated with the generalization ability of a model when measured at its hidden representations (hidden margins), no such link between large margins and generalization has been established for input margins. We show that while input margins are not generally predictive of generalization, they can be if the search space is appropriately constrained. We develop such a measure based on input margins, which we refer to as `constrained margins'. The predictive power of this new measure is demonstrated on the 'Predicting Generalization in Deep Learning' (PGDL) dataset and contrasted with hidden representation margins. We find that constrained margins achieve highly competitive scores and outperform other margin measurements in general. This provides a novel insight on the relationship between generalization and classification margins, and highlights the importance of considering the data manifold for investigations of generalization in DNNs.
Graph Theory and its Uses in Graph Algorithms and Beyond
Venue: arXiv cs.DS 2023
Year: 2023

Graphs are fundamental objects that find widespread applications across computer science and beyond. Graph Theory has yielded deep insights about structural properties of various families of graphs, which are leveraged in the design and analysis of algorithms for graph optimization problems and other computational optimization problems. These insights have also proved helpful in understanding the limits of efficient computation by providing constructions of hard problem instances. At the same time, algorithmic tools and techniques provide a fresh perspective on graph theoretic problems, often leading to novel discoveries. In this thesis, we exploit this symbiotic relationship between graph theory and algorithms for graph optimization problems and beyond. This thesis consists of three parts. In the first part, we study a graph routing problem called the Node-Disjoint Paths (NDP) problem. Given a graph and a set of source-destination pairs of its vertices, the goal is to route the maximum number of pairs via node-disjoint paths. We come close to resolving the approximability of NDP by showing that it is $n^{Ω(1/poly\log\log n)}$-hard to approximate, even on grid graphs, where n is the number of vertices. In the second part of this thesis, we use graph decomposition techniques developed for efficient algorithms to derive a graph theoretic result. We show that for every n-vertex expander graph G, if H is any graph with at most $O(n/\log n)$ vertices and edges, then H is a minor of G. In the last part, we show that the graph theoretic tools and graph algorithmic techniques can shed light on problems seemingly unrelated to graphs. We show that the randomized space complexity of the Longest Increasing Subsequence (LIS) problem in the streaming model is intrinsically tied to the query-complexity of the Non-Crossing Matching problem on graphs in a new model of computation that we define.
3D Adversarial Augmentations for Robust Out-of-Domain Predictions
Venue: arXiv cs.CV 2023
Year: 2023

Since real-world training datasets cannot properly sample the long tail of the underlying data distribution, corner cases and rare out-of-domain samples can severely hinder the performance of state-of-the-art models. This problem becomes even more severe for dense tasks, such as 3D semantic segmentation, where points of non-standard objects can be confidently associated to the wrong class. In this work, we focus on improving the generalization to out-of-domain data. We achieve this by augmenting the training set with adversarial examples. First, we learn a set of vectors that deform the objects in an adversarial fashion. To prevent the adversarial examples from being too far from the existing data distribution, we preserve their plausibility through a series of constraints, ensuring sensor-awareness and shapes smoothness. Then, we perform adversarial augmentation by applying the learned sample-independent vectors to the available objects when training a model. We conduct extensive experiments across a variety of scenarios on data from KITTI, Waymo, and CrashD for 3D object detection, and on data from SemanticKITTI, Waymo, and nuScenes for 3D semantic segmentation. Despite training on a standard single dataset, our approach substantially improves the robustness and generalization of both 3D object detection and 3D semantic segmentation methods to out-of-domain data.
The Relationship between Loneliness and Depression among College Students: Mining data derived from Passive Sensing
Venue: arXiv cs.HC 2023
Year: 2023

Loneliness and depression are interrelated mental health issues affecting students well-being. Using passive sensing data provides a novel approach to examine the granular behavioural indicators differentiating loneliness and depression, and the mediators in their relationship. This study aimed to investigate associations between behavioural features and loneliness and depression among students, exploring the complex relationships between these mental health conditions and associated behaviours. This study combined regression analysis, mediation analysis, and machine learning analysis to explore relationships between behavioural features, loneliness, and depression using passive sensing data, capturing daily life behaviours such as physical activity, phone usage, sleep patterns, and social interactions. Results revealed significant associations between behavioural features and loneliness and depression, emphasizing their interconnected nature. Increased activity and sleep duration were identified as protective factors. Distinct behavioural features for each condition were also found. Mediation analysis highlighted significant indirect effects in the relationship between loneliness and depression. The XGBoost model achieved the highest accuracy in predicting these conditions. This study demonstrated the importance of using passive sensing data and a multi-method approach to understand the complex relationship between loneliness, depression, and associated behaviours. Identifying specific behavioural features and mediators contributes to a deeper understanding of factors influencing loneliness and depression among students. This comprehensive perspective emphasizes the importance of interdisciplinary collaboration for a more nuanced understanding of complex human experiences.
Dimensionality Reduction Using pseudo-Boolean polynomials For Cluster Analysis
Venue: arXiv cs.IR 2023
Year: 2023

We introduce usage of a reduction property of penalty-based formulation of pseudo-Boolean polynomials as a mechanism for invariant dimensionality reduction in cluster analysis processes. In our experiments, we show that multidimensional data, like 4-dimensional Iris Flower dataset can be reduced to 2-dimensional space while the 30-dimensional Wisconsin Diagnostic Breast Cancer (WDBC) dataset can be reduced to 3-dimensional space, and by searching lines or planes that lie between reduced samples we can extract clusters in a linear and unbiased manner with competitive accuracies, reproducibility and clear interpretation.
Blind Cyclic Prefix-based CFO Estimation in MIMO-OFDM Systems
Venue: arXiv cs.IT 2023
Year: 2023

Low-complexity estimation and correction of carrier frequency offset (CFO) are essential in orthogonal frequency division multiplexing (OFDM). In this paper, we propose a low-overhead blind CFO estimation technique based on cyclic prefix (CP), in multi-input multi-output (MIMO)-OFDM systems. We propose to use antenna diversity for CFO estimation. Given that the RF chains for all antenna elements at a communication node share the same clock, the carrier frequency offset (CFO) between two points may be estimated by using the combination of the received signal at all antennas. We improve our method by combining the antenna diversity with time diversity by considering the CP for multiple OFDM symbols. We provide a closed-form expression for CFO estimation and present algorithms that can considerably improve the CFO estimation performance at the expense of a linear increase in computational complexity. We validate the effectiveness of our estimation scheme via extensive numerical analysis.
A Survey on Multi-Behavior Sequential Recommendation
Venue: arXiv cs.IR 2023
Year: 2023

Recommender systems is set up to address the issue of information overload in traditional information retrieval systems, which is focused on recommending information that is of most interest to users from massive information. Generally, there is a sequential nature and heterogeneity to the behavior of a person interacting with a system, leading to the proposal of multi-behavior sequential recommendation (MBSR). MBSR is a relatively new and worthy direction for in-depth research, which can achieve state-of-the-art recommendation through suitable modeling, and some related works have been proposed. This survey aims to shed light on the MBSR problem. Firstly, we introduce MBSR in detail, including its problem definition, application scenarios and challenges faced. Secondly, we detail the classification of MBSR, including neighborhood-based methods, matrix factorization-based methods and deep learning-based methods, where we further classify the deep learning-based methods into different learning architectures based on RNN, GNN, Transformer, and generic architectures as well as architectures that integrate hybrid techniques. In each method, we present related works based on the data perspective and the modeling perspective, as well as analyze the strengths, weaknesses and features of these works. Finally, we discuss some promising future research directions to address the challenges and improve the current status of MBSR.
Quantifying and Analyzing Entity-level Memorization in Large Language Models
Venue: arXiv cs.CL 2023
Year: 2023

Large language models (LLMs) have been proven capable of memorizing their training data, which can be extracted through specifically designed prompts. As the scale of datasets continues to grow, privacy risks arising from memorization have attracted increasing attention. Quantifying language model memorization helps evaluate potential privacy risks. However, prior works on quantifying memorization require access to the precise original data or incur substantial computational overhead, making it difficult for applications in real-world language models. To this end, we propose a fine-grained, entity-level definition to quantify memorization with conditions and metrics closer to real-world scenarios. In addition, we also present an approach for efficiently extracting sensitive entities from autoregressive language models. We conduct extensive experiments based on the proposed, probing language models' ability to reconstruct sensitive entities under different settings. We find that language models have strong memorization at the entity level and are able to reproduce the training data even with partial leakages. The results demonstrate that LLMs not only memorize their training data but also understand associations between entities. These findings necessitate that trainers of LLMs exercise greater prudence regarding model memorization, adopting memorization mitigation techniques to preclude privacy violations.
Computing Geodesic Paths Encoding a Curvature Prior
Venue: arXiv cs.CG 2023
Year: 2023

In this paper, we introduce an efficient method for computing curves minimizing a variant of the Euler-Mumford elastica energy, with fixed endpoints and tangents at these endpoints, where the bending energy is enhanced with a user defined and data-driven scalar-valued term referred to as the curvature prior. In order to guarantee that the globally optimal curve is extracted, the proposed method involves the numerical computation of the viscosity solution to a specific static Hamilton-Jacobi-Bellman (HJB) partial differential equation (PDE). For that purpose, we derive the explicit Hamiltonian associated to this variant model equipped with a curvature prior, discretize the resulting HJB PDE using an adaptive finite difference scheme, and solve it in a single pass using a generalized Fast-Marching method. In addition, we also present a practical method for estimating the curvature prior values from image data, designed for the task of accurately tracking curvilinear structure centerlines. Numerical experiments on synthetic and real image data illustrate the advantages of the considered variant of the elastica model with a prior curvature enhancement in complex scenarios where challenging geometric structures appear.
Task-Based MoE for Multitask Multilingual Machine Translation
Venue: arXiv cs.CL 2023
Year: 2023

Mixture-of-experts (MoE) architecture has been proven a powerful method for diverse tasks in training deep models in many applications. However, current MoE implementations are task agnostic, treating all tokens from different tasks in the same manner. In this work, we instead design a novel method that incorporates task information into MoE models at different granular levels with shared dynamic task-based adapters. Our experiments and analysis show the advantages of our approaches over the dense and canonical MoE models on multi-task multilingual machine translations. With task-specific adapters, our models can additionally generalize to new tasks efficiently.
Federated Two Stage Decoupling With Adaptive Personalization Layers
Venue: arXiv cs.LG 2023
Year: 2023

Federated learning has gained significant attention due to its groundbreaking ability to enable distributed learning while maintaining privacy constraints. However, as a consequence of data heterogeneity among decentralized devices, it inherently experiences significant learning degradation and slow convergence speed. Therefore, it is natural to employ the concept of clustering homogeneous clients into the same group, allowing only the model weights within each group to be aggregated. While most existing clustered federated learning methods employ either model gradients or inference outputs as metrics for client partitioning, with the goal of grouping similar devices together, may still have heterogeneity within each cluster. Moreover, there is a scarcity of research exploring the underlying reasons for determining the appropriate timing for clustering, resulting in the common practice of assigning each client to its own individual cluster, particularly in the context of highly non independent and identically distributed (Non-IID) data. In this paper, we introduce a two-stage decoupling federated learning algorithm with adaptive personalization layers named FedTSDP, where client clustering is performed twice according to inference outputs and model weights, respectively. Hopkins amended sampling is adopted to determine the appropriate timing for clustering and the sampling weight of public unlabeled data. In addition, a simple yet effective approach is developed to adaptively adjust the personalization layers based on varying degrees of data skew. Experimental results show that our proposed method has reliable performance on both IID and non-IID scenarios.
Pandemic Pedagogy: Evaluating Remote Education Strategies during COVID-19
Venue: arXiv cs.CY 2023
Year: 2023

The COVID-19 pandemic precipitated an abrupt shift in the educational landscape, compelling universities to transition from in-person to online instruction. This sudden shift left many university instructors grappling with the intricacies of remote teaching. Now, with the pandemic behind us, we present a retrospective study aimed at understanding and evaluating the remote teaching practices employed during that period. Drawing from a cross-sectional analysis of 300 computer science students who underwent a full year of online education during the lockdown, our findings indicate that while remote teaching practices moderately influenced students' learning outcomes, they had a pronounced positive impact on student satisfaction. Remarkably, these outcomes were consistent across various demographics, including country, gender, and educational level. As we reflect on the lessons from this global event, this research offers evidence-based recommendations that could inform educational strategies in unwelcoming future scenarios of a similar nature, ensuring both student satisfaction and effective learning outcomes in online settings.
Inductive Learning of Declarative Domain-Specific Heuristics for ASP
Venue: arXiv cs.AI 2023
Year: 2023

Domain-specific heuristics are a crucial technique for the efficient solving of problems that are large or computationally hard. Answer Set Programming (ASP) systems support declarative specifications of domain-specific heuristics to improve solving performance. However, such heuristics must be invented manually so far. Inventing domain-specific heuristics for answer-set programs requires expertise with the domain under consideration and familiarity with ASP syntax, semantics, and solving technology. The process of inventing useful heuristics would highly profit from automatic support. This paper presents a novel approach to the automatic learning of such heuristics. We use Inductive Logic Programming (ILP) to learn declarative domain-specific heuristics from examples stemming from (near-)optimal answer sets of small but representative problem instances. Our experimental results indicate that the learned heuristics can improve solving performance and solution quality when solving larger, harder instances of the same problem.
Sarrus-inspired Deployable Polyhedral Mechanisms
Venue: arXiv cs.RO 2023
Year: 2023

Deployable polyhedral mechanisms (DPMs) have witnessed flourishing growth in recent years because of their potential applications in robotics, space exploration, structure engineering, etc. This paper firstly presents the construction, mobility and kinematics of a family of Sarrus-inspired deployable polyhedral mechanisms. By carrying out expansion operation and implanting Sarrus linkages along the straight-line motion paths, deployable tetrahedral, cubic and dodecahedral mechanisms are identified and constructed following tetrahedral, octahedral and icosahedral symmetry, respectively. Three paired transformations with synchronized radial motion between Platonic and Archimedean polyhedrons are revealed, and their significant symmetric properties are perfectly remained in each work configuration. Subsequently, with assistant of equivalent prismatic joints, the equivalent analysis strategy for mobility of multiloop polyhedral mechanisms is proposed to significantly simplify the calculation process. This paper hence presents the construction method and equivalent analysis of the Sarrus-inspired DPMs that are not only valuable in theoretical investigation, but also have great potential in practical applications such as mechanical metamaterials, deployable architectures and space exploration.
A Dataflow Analysis for Comparing and Reordering Predicate Arguments
Venue: arXiv cs.LO 2023
Year: 2023

In this work, which is done in the context of a (moded) logic programming language, we devise a data-flow analysis dedicated to computing what we call argument profiles. Such a profile essentially describes, for each argument of a predicate, its functionality, i.e. the operations in which the argument can be involved during an evaluation of the predicate, as well as how the argument contributes to the consumption and/or construction of data values. While the computed argument profiles can be useful for applications in the context of program understanding (as each profile essentially provides a way to better understand the role of the argument), they more importantly provide a way to discern between arguments in a manner that is more fine-grained than what can be done with other abstract characterizations such as types and modes. This is important for applications where one needs to identify correspondences between the arguments of two or more different predicates that need to be compared, such as during clone detection. Moreover, since a total order can be defined on the abstract domain of profiles, our analysis can be used for rearranging predicate arguments and order them according to their functionality, constituting as such an essential ingredient for predicate normalization techniques.
Explainable Answer-set Programming
Venue: arXiv cs.AI 2023
Year: 2023

The interest in explainability in artificial intelligence (AI) is growing vastly due to the near ubiquitous state of AI in our lives and the increasing complexity of AI systems. Answer-set Programming (ASP) is used in many areas, among them are industrial optimisation, knowledge management or life sciences, and thus of great interest in the context of explainability. To ensure the successful application of ASP as a problem-solving paradigm in the future, it is thus crucial to investigate explanations for ASP solutions. Such an explanation generally tries to give an answer to the question of why something is, respectively is not, part of the decision produced or solution to the formulated problem. Although several explanation approaches for ASP exist, almost all of them lack support for certain language features that are used in practice. Most notably, this encompasses the various ASP extensions that have been developed in the recent years to enable reasoning over theories, external computations, or neural networks. This project aims to fill some of these gaps and contribute to the state of the art in explainable ASP. We tackle this by extending the language support of existing approaches but also by the development of novel explanation formalisms, like contrastive explanations.