OpenRaiser/Intern-Atlas huggingface dataset:
Enhancing DP-SGD through Non-monotonous Adaptive Scaling Gradient Weight
Venue: arXiv cs.LG 2024
Year: 2024

In the domain of deep learning, the challenge of protecting sensitive data while maintaining model utility is significant. Traditional Differential Privacy (DP) techniques such as Differentially Private Stochastic Gradient Descent (DP-SGD) typically employ strategies like direct or per-sample adaptive gradient clipping. These methods, however, compromise model accuracy due to their critical influence on gradient handling, particularly neglecting the significant contribution of small gradients during later training stages. In this paper, we introduce an enhanced version of DP-SGD, named Differentially Private Per-sample Adaptive Scaling Clipping (DP-PSASC). This approach replaces traditional clipping with non-monotonous adaptive gradient scaling, which alleviates the need for intensive threshold setting and rectifies the disproportionate weighting of smaller gradients. Our contribution is twofold. First, we develop a novel gradient scaling technique that effectively assigns proper weights to gradients, particularly small ones, thus improving learning under differential privacy. Second, we integrate a momentum-based method into DP-PSASC to reduce bias from stochastic sampling, enhancing convergence rates. Our theoretical and empirical analyses confirm that DP-PSASC preserves privacy and delivers superior performance across diverse datasets, setting new standards for privacy-sensitive applications.
HFGaussian: Learning Generalizable Gaussian Human with Integrated Human Features
Venue: arXiv cs.CV 2024
Year: 2024

Recent advancements in radiance field rendering show promising results in 3D scene representation, where Gaussian splatting-based techniques emerge as state-of-the-art due to their quality and efficiency. Gaussian splatting is widely used for various applications, including 3D human representation. However, previous 3D Gaussian splatting methods either use parametric body models as additional information or fail to provide any underlying structure, like human biomechanical features, which are essential for different applications. In this paper, we present a novel approach called HFGaussian that can estimate novel views and human features, such as the 3D skeleton, 3D key points, and dense pose, from sparse input images in real time at 25 FPS. The proposed method leverages generalizable Gaussian splatting technique to represent the human subject and its associated features, enabling efficient and generalizable reconstruction. By incorporating a pose regression network and the feature splatting technique with Gaussian splatting, HFGaussian demonstrates improved capabilities over existing 3D human methods, showcasing the potential of 3D human representations with integrated biomechanics. We thoroughly evaluate our HFGaussian method against the latest state-of-the-art techniques in human Gaussian splatting and pose estimation, demonstrating its real-time, state-of-the-art performance.
The Effect of Funding on Student Achievement: Evidence from District of Columbia, Virginia, and Maryland
Venue: arXiv cs.CY 2024
Year: 2024

The question of how to best serve the student populations of our country is a complex topic. Since public funding is limited, we must explore the best ways to direct the money to improve student outcomes. Previous research has suggested that socio-economic status is the best predictor of student achievement, while other studies suggest that the amount of money spent on the student is a more significant factor. In this paper, we explore this question and its impacts on Maryland, Virginia, and the District of Columbia schools. We conclude that the graduation rate has a direct relationship with unemployment, suggesting that funding towards improving out-of-school opportunities and quality of life will significantly improve students chances of success. We do not find a significant relationship between per-pupil spending and student achievement.
A Machine Learning Approach for the Efficient Estimation of Ground-Level Air Temperature in Urban Areas
Venue: arXiv cs.LG 2024
Year: 2024

The increasingly populated cities of the 21st Century face the challenge of being sustainable and resilient spaces for their inhabitants. However, climate change, among other problems, makes these objectives difficult to achieve. The Urban Heat Island (UHI) phenomenon that occurs in cities, increasing their thermal stress, is one of the stumbling blocks to achieve a more sustainable city. The ability to estimate temperatures with a high degree of accuracy allows for the identification of the highest priority areas in cities where urban improvements need to be made to reduce thermal discomfort. In this work we explore the usefulness of image-to-image deep neural networks (DNNs) for correlating spatial and meteorological variables of a urban area with street-level air temperature. The air temperature at street-level is estimated both spatially and temporally for a specific use case, and compared with existing, well-established numerical models. Based on the obtained results, deep neural networks are confirmed to be faster and less computationally expensive alternative for ground-level air temperature compared to numerical models.
Distributed Quantum Advantage for Local Problems
Venue: arXiv cs.DC 2024
Year: 2024

We present the first local problem that shows a super-constant separation between the classical randomized LOCAL model of distributed computing and its quantum counterpart. By prior work, such a separation was known only for an artificial graph problem with an inherently global definition [Le Gall et al. 2019]. We present a problem that we call iterated GHZ, which is defined using only local constraints. Formally, it is a family of locally checkable labeling problems [Naor and Stockmeyer 1995]; in particular, solutions can be verified with a constant-round distributed algorithm. We show that in graphs of maximum degree $Δ$, any classical (deterministic or randomized) LOCAL model algorithm will require $Ω(Δ)$ rounds to solve the iterated GHZ problem, while the problem can be solved in $1$ round in quantum-LOCAL. We use the round elimination technique to prove that the iterated GHZ problem requires $Ω(Δ)$ rounds for classical algorithms. This is the first work that shows that round elimination is indeed able to separate the two models, and this also demonstrates that round elimination cannot be used to prove lower bounds for quantum-LOCAL. To apply round elimination, we introduce a new technique that allows us to discover appropriate problem relaxations in a mechanical way; it turns out that this new technique extends beyond the scope of the iterated GHZ problem and can be used to e.g. reproduce prior results on maximal matchings [FOCS 2019, PODC 2020] in a systematic manner.
Exploring Multi-Fidelity Aeroelastic Tailoring: Prospect and Model Assessment
Venue: arXiv cs.CE 2024
Year: 2024

The design and optimisation of aircraft wings are critical tasks in aerospace engineering, requiring a balance between structural integrity, aerostructural performance, and manufacturability. This multifaceted challenge involves the interplay of various disciplines, each with distinct parameters and constraints. Traditional design approaches often fall short, necessitating advanced methodologies like Multidisciplinary Design Optimisation (MDO). MDO integrates aerodynamic, structural, and manufacturability analyses to explore a vast design space and identify optimal solutions that meet performance, safety, and cost criteria. The work highlights the challenge of optimising aircraft designs using multiple models of varying fidelity. Traditional sequential optimisation approaches, which progressively integrate disciplines, may miss potential superior designs due to limited initial information. Instead, concurrent optimisation schemes are explored, utilising both low-fidelity (beam-based) and high-fidelity (shell-based) models. This approach promises structural feasibility, reduces computational costs, and incorporates high-fidelity information early in the design process. The envisioned methodology bridges different design stages, enabling better overall aircraft performance. By aligning and comparing a beam-based and shell-based model, the study explores their use in multi-fidelity optimisation. The results demonstrate the feasibility and benefits of this approach, offering a robust framework for future aircraft design projects.
DiT4Edit: Diffusion Transformer for Image Editing
Venue: arXiv cs.CV 2024
Year: 2024

Despite recent advances in UNet-based image editing, methods for shape-aware object editing in high-resolution images are still lacking. Compared to UNet, Diffusion Transformers (DiT) demonstrate superior capabilities to effectively capture the long-range dependencies among patches, leading to higher-quality image generation. In this paper, we propose DiT4Edit, the first Diffusion Transformer-based image editing framework. Specifically, DiT4Edit uses the DPM-Solver inversion algorithm to obtain the inverted latents, reducing the number of steps compared to the DDIM inversion algorithm commonly used in UNet-based frameworks. Additionally, we design unified attention control and patches merging, tailored for transformer computation streams. This integration allows our framework to generate higher-quality edited images faster. Our design leverages the advantages of DiT, enabling it to surpass UNet structures in image editing, especially in high-resolution and arbitrary-size images. Extensive experiments demonstrate the strong performance of DiT4Edit across various editing scenarios, highlighting the potential of Diffusion Transformers in supporting image editing.
Using Assurance Cases to Guide Verification and Validation of Research Software
Venue: arXiv cs.SE 2024
Year: 2024

Research software engineers can use Assurance Cases (ACs) to guide Verification and Validation (VnV) efforts. An AC is a structured argument that a property like correctness holds. We illustrate how ACs can guide VnV activities via a case study of software for automatically extracting the 3D segmentation of the aorta from medical images of the chest. The AC argument suggests that the following evidence is required: comparison to a pseudo-oracle; traceability between requirements, design, code and tests; review of all artifacts by a domain expert with proper credentials; documentation of input assumptions; and a warning that only qualified people should use the software. The case study highlights that code is not the only artifact of interest for building confidence and that making an explicit distinction between software and user responsibilities is useful.
Geographical and Disciplinary Coverage of Open Access Journals: OpenAlex, Scopus and WoS
Venue: arXiv cs.DL 2024
Year: 2024

This study aims to compare the geographical and disciplinary coverage of OA journals in three databases: OpenAlex, Scopus and the WoS. We used the ROAD database, managed by the ISSN International Centre, as a reference database which indexes 62,701 OA active resources (as of May 2024). Among the 62,701 active resources indexed in the ROAD database, the Web of Science indexes 6,157 journals, while Scopus indexes 7,351, and OpenAlex indexes 34,217. A striking observation is the presence of 25,658 OA journals exclusively in OpenAlex, whereas only 182 journals are exclusively present in WoS and 373 in Scopus. The geographical analysis focusses on two levels: continents and countries. As for disciplinary comparison, we use the ten disciplinary levels of the ROAD database. Moreover, our findings reveal a striking similarity in OA journal coverage between WoS and Scopus. However, while OpenAlex offers better inclusivity and indexing, it is not without biases. WoS and Scopus predictably favor journals from Europe, North America and Oceania. Although OpenAlex presents a much more balanced indexing, certain regions and countries remain relatively underrepresented. Typically, Africa is proportionally as under-represented in OpenAlex as it is in WoS, and some emerging countries are proportionally less represented in OpenAlex than in WoS and Scopus. These results underscore a marked similarity in OA journal indexing between WoS and Scopus, while OpenAlex aligns more closely with the distribution observed in the ROAD database, although it also exhibits some representational biases.
Analysis of Bipartite Networks in Anime Series: Textual Analysis, Topic Clustering, and Modeling
Venue: arXiv cs.SI 2024
Year: 2024

This article analyzes a specific bipartite network that shows the relationships between users and anime, examining how the descriptions of anime influence the formation of user communities. In particular, we introduce a new variable that quantifies the frequency with which words from a description appear in specific word clusters. These clusters are generated from a bigram analysis derived from all descriptions in the database. This approach fully characterizes the dynamics of these communities and shows how textual content affect the cohesion and structure of the social network among anime enthusiasts. Our findings suggest that there may be significant implications for the design of recommendation systems and the enhancement of user experience on anime platforms.
Enhancing Table Representations with LLM-powered Synthetic Data Generation
Venue: arXiv cs.LG 2024
Year: 2024

In the era of data-driven decision-making, accurate table-level representations and efficient table recommendation systems are becoming increasingly crucial for improving table management, discovery, and analysis. However, existing approaches to tabular data representation often face limitations, primarily due to their focus on cell-level tasks and the lack of high-quality training data. To address these challenges, we first formulate a clear definition of table similarity in the context of data transformation activities within data-driven enterprises. This definition serves as the foundation for synthetic data generation, which require a well-defined data generation process. Building on this, we propose a novel synthetic data generation pipeline that harnesses the code generation and data manipulation capabilities of Large Language Models (LLMs) to create a large-scale synthetic dataset tailored for table-level representation learning. Through manual validation and performance comparisons on the table recommendation task, we demonstrate that the synthetic data generated by our pipeline aligns with our proposed definition of table similarity and significantly enhances table representations, leading to improved recommendation performance.
SPINEX_ Symbolic Regression: Similarity-based Symbolic Regression with Explainable Neighbors Exploration
Venue: arXiv cs.LG 2024
Year: 2024

This article introduces a new symbolic regression algorithm based on the SPINEX (Similarity-based Predictions with Explainable Neighbors Exploration) family. This new algorithm (SPINEX_SymbolicRegression) adopts a similarity-based approach to identifying high-merit expressions that satisfy accuracy- and structural similarity metrics. We conducted extensive benchmarking tests comparing SPINEX_SymbolicRegression to over 180 mathematical benchmarking functions from international problem sets that span randomly generated expressions and those based on real physical phenomena. Then, we evaluated the performance of the proposed algorithm in terms of accuracy, expression similarity in terms of presence operators and variables (as compared to the actual expressions), population size, and number of generations at convergence. The results indicate that SPINEX_SymbolicRegression consistently performs well and can, in some instances, outperform leading algorithms. In addition, the algorithm's explainability capabilities are highlighted through in-depth experiments.
Pedestrian Volume Prediction Using a Diffusion Convolutional Gated Recurrent Unit Model
Venue: arXiv cs.LG 2024
Year: 2024

Effective models for analysing and predicting pedestrian flow are important to ensure the safety of both pedestrians and other road users. These tools also play a key role in optimising infrastructure design and geometry and supporting the economic utility of interconnected communities. The implementation of city-wide automatic pedestrian counting systems provides researchers with invaluable data, enabling the development and training of deep learning applications that offer better insights into traffic and crowd flows. Benefiting from real-world data provided by the City of Melbourne pedestrian counting system, this study presents a pedestrian flow prediction model, as an extension of Diffusion Convolutional Grated Recurrent Unit (DCGRU) with dynamic time warping, named DCGRU-DTW. This model captures the spatial dependencies of pedestrian flow through the diffusion process and the temporal dependency captured by Gated Recurrent Unit (GRU). Through extensive numerical experiments, we demonstrate that the proposed model outperforms the classic vector autoregressive model and the original DCGRU across multiple model accuracy metrics.
Towards Interoperability Testing of Smart Energy Systems -- An Overview and Discussion of Possibilities
Venue: arXiv cs.SE 2024
Year: 2024

Interoperability is the key to implementing a wide range of energy systems applications. It involves the seamless cooperation of different methods and components. With smart energy systems, interoperability faces challenges due to integrating differ-ent approaches and technologies. This includes dealing with heterogeneous approaches with various communication proto-cols and data formats. However, it is essential for smart energy systems to carry out thorough interoperability tests. They are usually diverse, and challenging, thus requiring careful consideration of compatibility issues and complex integration scenari-os. Overcoming these challenges requires a systematic approach that includes thorough test planning, rigorous testing, and continuous test monitoring. Although numerous testing approaches exist, most are more developed at the component/device level than at the system level. Consequently, there are few approaches and related facilities to test the interoperability of smart energy approaches and solutions at the system level. This work analyses existing interoperability test concepts, identi-fies enablers and the potential for harmonisation of procedures, and proposes further developments of these approaches.
Blockchain-Based Multi-Path Mobile Access Point Selection for Secure 5G VANETs
Venue: arXiv cs.CR 2024
Year: 2024

This letter presents a blockchain-based multi-path mobile access point (MAP) selection strategy for secure 5G vehicular ad-hoc networks (VANETs). The proposed method leverages blockchain technology for decentralized, transparent, and secure MAP selection, while the multi-path transmission strategy enhances network reliability and reduces communication delays. A trust-based attack detection mechanism is integrated to ensure network security. Simulation results demonstrate that the proposed algorithm reduces both handover frequency and average communication delay by over 80%, and successfully identifies and excludes more than 95% of Sybil nodes, ensuring reliable and secure communication in highly dynamic vehicular environments.
Kernel Approximation using Analog In-Memory Computing
Venue: arXiv cs.LG 2024
Year: 2024

Kernel functions are vital ingredients of several machine learning algorithms, but often incur significant memory and computational costs. We introduce an approach to kernel approximation in machine learning algorithms suitable for mixed-signal Analog In-Memory Computing (AIMC) architectures. Analog In-Memory Kernel Approximation addresses the performance bottlenecks of conventional kernel-based methods by executing most operations in approximate kernel methods directly in memory. The IBM HERMES Project Chip, a state-of-the-art phase-change memory based AIMC chip, is utilized for the hardware demonstration of kernel approximation. Experimental results show that our method maintains high accuracy, with less than a 1% drop in kernel-based ridge classification benchmarks and within 1% accuracy on the Long Range Arena benchmark for kernelized attention in Transformer neural networks. Compared to traditional digital accelerators, our approach is estimated to deliver superior energy efficiency and lower power consumption. These findings highlight the potential of heterogeneous AIMC architectures to enhance the efficiency and scalability of machine learning applications.
SAUCE: Synchronous and Asynchronous User-Customizable Environment for Multi-Agent LLM Interaction
Venue: arXiv cs.CL 2024
Year: 2024

Many human interactions, such as political debates, are carried out in group settings, where there are arbitrarily many participants, each with different views and agendas. To explore such complex social settings, we present SAUCE: a customizable Python platform, allowing researchers to plug-and-play various LLMs participating in discussions on any topic chosen by the user. Our platform takes care of instantiating the models, scheduling their responses, managing the discussion history, and producing a comprehensive output log, all customizable through configuration files, requiring little to no coding skills. A novel feature of SAUCE is our asynchronous communication feature, where models decide when to speak in addition to what to say, thus modeling an important facet of human communication. We show SAUCE's attractiveness in two initial experiments, and invite the community to use it in simulating various group simulations.
Pathway-Guided Optimization of Deep Generative Molecular Design Models for Cancer Therapy
Venue: arXiv cs.LG 2024
Year: 2024

The data-driven drug design problem can be formulated as an optimization task of a potentially expensive black-box objective function over a huge high-dimensional and structured molecular space. The junction tree variational autoencoder (JTVAE) has been shown to be an efficient generative model that can be used for suggesting legitimate novel drug-like small molecules with improved properties. While the performance of the generative molecular design (GMD) scheme strongly depends on the initial training data, one can improve its sampling efficiency for suggesting better molecules with enhanced properties by optimizing the latent space. In this work, we propose how mechanistic models - such as pathway models described by differential equations - can be used for effective latent space optimization(LSO) of JTVAEs and other similar models for GMD. To demonstrate the potential of our proposed approach, we show how a pharmacodynamic model, assessing the therapeutic efficacy of a drug-like small molecule by predicting how it modulates a cancer pathway, can be incorporated for effective LSO of data-driven models for GMD.
Rainfall regression from C-band Synthetic Aperture Radar using Multi-Task Generative Adversarial Networks
Venue: arXiv cs.CV 2024
Year: 2024

This paper introduces a data-driven approach to estimate precipitation rates from Synthetic Aperture Radar (SAR) at a spatial resolution of 200 meters per pixel. It addresses previous challenges related to the collocation of SAR and weather radar data, specifically the misalignment in collocations and the scarcity of rainfall examples under strong wind. To tackle these challenges, the paper proposes a multi-objective formulation, introducing patch-level components and an adversarial component. It exploits the full NEXRAD archive to look for potential co-locations with Sentinel-1 data. With additional enhancements to the training procedure and the incorporation of additional inputs, the resulting model demonstrates improved accuracy in rainfall estimates and the ability to extend its performance to scenarios up to 15 m/s.