OpenRaiser/Intern-Atlas huggingface dataset:
Path-Reporting Distance Oracles with Linear Size
Venue: arXiv cs.DS 2024
Year: 2024

Given an undirected weighted graph, an (approximate) distance oracle is a data structure that can (approximately) answer distance queries. A {\em Path-Reporting Distance Oracle}, or {\em PRDO}, is a distance oracle that must also return a path between the queried vertices. Given a graph on $n$ vertices and an integer parameter $k\ge 1$, Thorup and Zwick \cite{TZ01} showed a PRDO with stretch $2k-1$, size $O(k\cdot n^{1+1/k})$ and query time $O(k)$ (for the query time of PRDOs, we omit the time needed to report the path itself). Subsequent works \cite{MN06,C14,C15} improved the size to $O(n^{1+1/k})$ and the query time to $O(1)$. However, these improvements produce distance oracles which are not path-reporting. Several other works \cite{ENW16,EP15} focused on small size PRDO for general graphs, but all known results on distance oracles with linear size suffer from polynomial stretch, polynomial query time, or not being path-reporting. In this paper we devise the first linear size PRDO with poly-logarithmic stretch and low query time $O(\log\log n)$. More generally, for any integer $k\ge 1$, we obtain a PRDO with stretch at most $O(k^{4.82})$, size $O(n^{1+1/k})$, and query time $O(\log k)$. In addition, we can make the size of our PRDO as small as $n+o(n)$, at the cost of increasing the query time to poly-logarithmic. For unweighted graphs, we improve the stretch to $O(k^2)$. We also consider {\em pairwise PRDO}, which is a PRDO that is only required to answer queries from a given set of pairs ${\cal P}$. An exact PRDO of size $O(n+|{\cal P}|^2)$ and constant query time was provided in \cite{EP15}. In this work we dramatically improve the size, at the cost of slightly increasing the stretch. Specifically, given any $ε>0$, we devise a pairwise PRDO with stretch $1+ε$, constant query time, and near optimal size $n^{o(1)}\cdot (n+|{\cal P}|)$.
An 808 Line Phasor-Based Dehomogenisation Matlab Code For Multi-Scale Topology Optimisation
Venue: arXiv cs.MS 2024
Year: 2024

This work presents an 808-line Matlab educational code for combined multi-scale topology optimisation and phasor-based dehomogenisation titled deHomTop808. The multi-scale formulation utilises homogenisation of optimal microstructures to facilitate efficient coarse-scale optimisation. Dehomogenisation allows for a high-resolution single-scale reconstruction of the optimised multi-scale structure, achieving minor losses in structural performance, at a fraction of the computational cost, compared to its large-scale topology optimisation counterpart. The presented code utilises stiffness optimal Rank-2 microstructures to minimise the compliance of a single-load case problem, subject to a volume fraction constraint. By exploiting the inherent efficiency benefits of the phasor-based dehomogenisation procedure, on-the-fly dehomogenisation to a single-scale structure is obtained. The presented code includes procedures for structural verification of the final dehomogenised structure by comparison to the multi-scale solution. The code is introduced in terms of the underlying theory and its major components, including examples and potential extensions, and can be downloaded from https://github.com/peterdorffler/deHomTop808.git.
stl2vec: Semantic and Interpretable Vector Representation of Temporal Logic
Venue: arXiv cs.AI 2024
Year: 2024

Integrating symbolic knowledge and data-driven learning algorithms is a longstanding challenge in Artificial Intelligence. Despite the recognized importance of this task, a notable gap exists due to the discreteness of symbolic representations and the continuous nature of machine-learning computations. One of the desired bridges between these two worlds would be to define semantically grounded vector representation (feature embedding) of logic formulae, thus enabling to perform continuous learning and optimization in the semantic space of formulae. We tackle this goal for knowledge expressed in Signal Temporal Logic (STL) and devise a method to compute continuous embeddings of formulae with several desirable properties: the embedding (i) is finite-dimensional, (ii) faithfully reflects the semantics of the formulae, (iii) does not require any learning but instead is defined from basic principles, (iv) is interpretable. Another significant contribution lies in demonstrating the efficacy of the approach in two tasks: learning model checking, where we predict the probability of requirements being satisfied in stochastic processes; and integrating the embeddings into a neuro-symbolic framework, to constrain the output of a deep-learning generative model to comply to a given logical specification.
Speculating About Multi-user Conversational Interfaces and LLMs: What If Chatting Wasn't So Lonely?
Venue: arXiv cs.HC 2024
Year: 2024

The advent of LLMs means that CUIs are cool again, but what isn't so cool is that we're doomed to use them alone. The one user, one account, one device paradigm has dominated the design of CUIs and is not going away as new conversational technologies emerge. In this provocation we explore some of the technical, legal, and design difficulties that seem to make multi-user CUIs so difficult to implement. Drawing inspiration from the ways that people manage messy group discussions, such as parliamentary and consensus-based paradigms, we show how LLM-based CUIs might be well suited to bridging the gap. With any luck, this might even result in everyone having to sit through fewer poorly run meetings and agonising group discussions - truly a laudable goal!
BORA: A Personalized Data Display for Large-scale Experiments
Venue: arXiv cs.HC 2024
Year: 2024

Given the rapid improvement of the detectors at high-energy physics experiments, the need for real-time data monitoring systems has become imperative. The significance of these systems lies in their ability to display experiment status, steer software and hardware instrumentation, and provide alarms, thus enabling researchers to manage their experiments better. However, researchers typically build most data monitoring systems as standalone in-house solutions that cannot be reused for other experiments or future upgrades. We present BORA (personalized collaBORAtive data display), a lightweight browser-based monitoring system that supports diverse protocols and is built specifically for customizable visualization of complex data, which we standardize via video streaming. We show how absolute positioning layout and visual overlay background can address the diverse data display design requirements. Using the client-server architecture, we enable support for diverse communication protocols, with the server component responsible for parsing the incoming data. We integrate the Jupyter Notebook as part of our ecosystem to address the limitations of the web-based framework, providing a foundation to leverage scripting capabilities and integrate popular AI frameworks. Since video streaming is a core component of our framework, we evaluate viable approaches to streaming protocols like HLS, WebRTC, and MPEG-Websocket. The study explores the implications for our use case, highlighting its potential to transform data visualization and decision-making processes.
Investigating the Common Authorship of Signatures by Off-Line Automatic Signature Verification Without the Use of Reference Signatures
Venue: arXiv cs.CV 2024
Year: 2024

In automatic signature verification, questioned specimens are usually compared with reference signatures. In writer-dependent schemes, a number of reference signatures are required to build up the individual signer model while a writer-independent system requires a set of reference signatures from several signers to develop the model of the system. This paper addresses the problem of automatic signature verification when no reference signatures are available. The scenario we explore consists of a set of signatures, which could be signed by the same author or by multiple signers. As such, we discuss three methods which estimate automatically the common authorship of a set of off-line signatures. The first method develops a score similarity matrix, worked out with the assistance of duplicated signatures; the second uses a feature-distance matrix for each pair of signatures; and the last method introduces pre-classification based on the complexity of each signature. Publicly available signatures were used in the experiments, which gave encouraging results. As a baseline for the performance obtained by our approaches, we carried out a visual Turing Test where forensic and non-forensic human volunteers, carrying out the same task, performed less well than the automatic schemes.
Leveraging Natural Load Dynamics with Variable Gear-ratio Actuators
Venue: arXiv cs.RO 2024
Year: 2024

This paper presents a robotic system where the gear-ratio of an actuator is dynamically changed to either leverage or attenuate the natural load dynamics. Based on this principle, lightweight robotic systems can be made fast and strong; exploiting the natural load dynamics for moving at higher speeds (small reduction ratio), while also able to bear a large load through the attenuation of the load dynamics (large reduction ratio). A model-based control algorithm to automatically select the optimal gear-ratios that minimize the total actuator torques for an arbitrary dynamic state and expected uncertainty level is proposed. Also, a novel 3-DoF robot arm using custom actuators with two discrete gear-ratios is presented. The advantages of gear-shifting dynamically are demonstrated through experiments and simulations. Results show that actively changing the gear-ratio using the proposed control algorithms can lead to an order-of-magnitude reduction of necessary actuator torque and power, and also increase robustness to disturbances.
JointRF: End-to-End Joint Optimization for Dynamic Neural Radiance Field Representation and Compression
Venue: arXiv cs.CV 2024
Year: 2024

Neural Radiance Field (NeRF) excels in photo-realistically static scenes, inspiring numerous efforts to facilitate volumetric videos. However, rendering dynamic and long-sequence radiance fields remains challenging due to the significant data required to represent volumetric videos. In this paper, we propose a novel end-to-end joint optimization scheme of dynamic NeRF representation and compression, called JointRF, thus achieving significantly improved quality and compression efficiency against the previous methods. Specifically, JointRF employs a compact residual feature grid and a coefficient feature grid to represent the dynamic NeRF. This representation handles large motions without compromising quality while concurrently diminishing temporal redundancy. We also introduce a sequential feature compression subnetwork to further reduce spatial-temporal redundancy. Finally, the representation and compression subnetworks are end-to-end trained combined within the JointRF. Extensive experiments demonstrate that JointRF can achieve superior compression performance across various datasets.
Scalable Visual State Space Model with Fractal Scanning
Venue: arXiv cs.CV 2024
Year: 2024

Foundational models have significantly advanced in natural language processing (NLP) and computer vision (CV), with the Transformer architecture becoming a standard backbone. However, the Transformer's quadratic complexity poses challenges for handling longer sequences and higher resolution images. To address this challenge, State Space Models (SSMs) like Mamba have emerged as efficient alternatives, initially matching Transformer performance in NLP tasks and later surpassing Vision Transformers (ViTs) in various CV tasks. To improve the performance of SSMs, one crucial aspect is effective serialization of image patches. Existing methods, relying on linear scanning curves, often fail to capture complex spatial relationships and produce repetitive patterns, leading to biases. To address these limitations, we propose using fractal scanning curves for patch serialization. Fractal curves maintain high spatial proximity and adapt to different image resolutions, avoiding redundancy and enhancing SSMs' ability to model complex patterns accurately. We validate our method in image classification, detection, and segmentation tasks, and the superior performance validates its effectiveness.
MoGU: A Framework for Enhancing Safety of Open-Sourced LLMs While Preserving Their Usability
Venue: arXiv cs.CL 2024
Year: 2024

Large Language Models (LLMs) are increasingly deployed in various applications. As their usage grows, concerns regarding their safety are rising, especially in maintaining harmless responses when faced with malicious instructions. Many defense strategies have been developed to enhance the safety of LLMs. However, our research finds that existing defense strategies lead LLMs to predominantly adopt a rejection-oriented stance, thereby diminishing the usability of their responses to benign instructions. To solve this problem, we introduce the MoGU framework, designed to enhance LLMs' safety while preserving their usability. Our MoGU framework transforms the base LLM into two variants: the usable LLM and the safe LLM, and further employs dynamic routing to balance their contribution. When encountering malicious instructions, the router will assign a higher weight to the safe LLM to ensure that responses are harmless. Conversely, for benign instructions, the router prioritizes the usable LLM, facilitating usable and helpful responses. On various open-sourced LLMs, we compare multiple defense strategies to verify the superiority of our MoGU framework. Besides, our analysis provides key insights into the effectiveness of MoGU and verifies that our designed routing mechanism can effectively balance the contribution of each variant by assigning weights. Our work released the safer Llama2, Vicuna, Falcon, Dolphin, and Baichuan2.
ArchesWeather: An efficient AI weather forecasting model at 1.5° resolution
Venue: arXiv cs.LG 2024
Year: 2024

One of the guiding principles for designing AI-based weather forecasting systems is to embed physical constraints as inductive priors in the neural network architecture. A popular prior is locality, where the atmospheric data is processed with local neural interactions, like 3D convolutions or 3D local attention windows as in Pangu-Weather. On the other hand, some works have shown great success in weather forecasting without this locality principle, at the cost of a much higher parameter count. In this paper, we show that the 3D local processing in Pangu-Weather is computationally sub-optimal. We design ArchesWeather, a transformer model that combines 2D attention with a column-wise attention-based feature interaction module, and demonstrate that this design improves forecasting skill. ArchesWeather is trained at 1.5° resolution and 24h lead time, with a training budget of a few GPU-days and a lower inference cost than competing methods. An ensemble of four of our models shows better RMSE scores than the IFS HRES and is competitive with the 1.4° 50-members NeuralGCM ensemble for one to three days ahead forecasting. Our code and models are publicly available at https://github.com/gcouairon/ArchesWeather.
EHRMamba: Towards Generalizable and Scalable Foundation Models for Electronic Health Records
Venue: arXiv cs.LG 2024
Year: 2024

Transformers have significantly advanced the modeling of Electronic Health Records (EHR), yet their deployment in real-world healthcare is limited by several key challenges. Firstly, the quadratic computational cost and insufficient context length of these models hinder hospitals' ability in processing the extensive medical histories typical in EHR data. Additionally, existing models employ separate finetuning for each clinical task, complicating maintenance in healthcare environments. Moreover, these models focus exclusively on either clinical prediction or EHR forecasting, lacking proficiency in both tasks. To overcome these limitations, we introduce EHRMamba, a robust foundation model built on the Mamba architecture. EHRMamba can process sequences up to 300% longer than previous models due to its linear computational cost. We also introduce a novel approach to Multitask Prompted Finetuning (MPF) for EHR data, which enables EHRMamba to simultaneously learn multiple clinical tasks in a single finetuning phase, significantly enhancing deployment and cross-task generalization. Furthermore, our model leverages the HL7 FHIR data standard to simplify integration into existing hospital systems. Alongside EHRMamba, we open-source Odyssey, a toolkit designed to support the development and deployment of EHR foundation models, with an emphasis on data standardization and interpretability. Our evaluations on the MIMIC-IV dataset demonstrate that EHRMamba advances state-of-the-art performance across 6 major clinical tasks and excels in EHR forecasting, marking a significant leap forward in the field.
A general method for the development of constrained codes
Venue: arXiv cs.IT 2024
Year: 2024

Nowadays there are several classes of constrained codes intended for different applications. The following two large classes can be distinguished. The first class contains codes with local constraints; for example, the source data must be encoded by binary sequences containing no sub-words 00 and 111. The second class contains codes with global constraints; for example, the code-words must be binary sequences of certain even length with half zeros and half ones. It is important to note that often the necessary codes must fulfill some requirements of both classes. In this paper we propose a general polynomial complexity method for constructing codes for both classes, as well as for combinations thereof. The proposed method uses the enumerative Cover's code, but the main difference between known applications of this code is that the known algorithms require the use of combinatorial formulae when applied, whereas the proposed method calculates all parameters on-the-fly using a polynomial complexity algorithm.
LDM: Large Tensorial SDF Model for Textured Mesh Generation
Venue: arXiv cs.GR 2024
Year: 2024

Previous efforts have managed to generate production-ready 3D assets from text or images. However, these methods primarily employ NeRF or 3D Gaussian representations, which are not adept at producing smooth, high-quality geometries required by modern rendering pipelines. In this paper, we propose LDM, a novel feed-forward framework capable of generating high-fidelity, illumination-decoupled textured mesh from a single image or text prompts. We firstly utilize a multi-view diffusion model to generate sparse multi-view inputs from single images or text prompts, and then a transformer-based model is trained to predict a tensorial SDF field from these sparse multi-view image inputs. Finally, we employ a gradient-based mesh optimization layer to refine this model, enabling it to produce an SDF field from which high-quality textured meshes can be extracted. Extensive experiments demonstrate that our method can generate diverse, high-quality 3D mesh assets with corresponding decomposed RGB textures within seconds.
Data Augmentation Techniques for Process Extraction from Scientific Publications
Venue: arXiv cs.CL 2024
Year: 2024

We present data augmentation techniques for process extraction tasks in scientific publications. We cast the process extraction task as a sequence labeling task where we identify all the entities in a sentence and label them according to their process-specific roles. The proposed method attempts to create meaningful augmented sentences by utilizing (1) process-specific information from the original sentence, (2) role label similarity, and (3) sentence similarity. We demonstrate that the proposed methods substantially improve the performance of the process extraction model trained on chemistry domain datasets, up to 12.3 points improvement in performance accuracy (F-score). The proposed methods could potentially reduce overfitting as well, especially when training on small datasets or in a low-resource setting such as in chemistry and other scientific domains.
Integer Scale: A Free Lunch for Faster Fine-grained Quantization of LLMs
Venue: arXiv cs.LG 2024
Year: 2024

We introduce Integer Scale, a novel post-training quantization scheme for large language models that effectively resolves the inference bottleneck in current fine-grained quantization approaches while maintaining similar accuracies. Integer Scale is a free lunch as it requires no extra calibration or fine-tuning which will otherwise incur additional costs. It can be used plug-and-play for most fine-grained quantization methods. Its integration results in at most 1.85x end-to-end speed boost over the original counterpart with comparable accuracy. Additionally, due to the orchestration of the proposed Integer Scale and fine-grained quantization, we resolved the quantization difficulty for Mixtral-8x7B and LLaMA-3 models with negligible performance degradation, and it comes with an end-to-end speed boost of 2.13x, and 2.31x compared with their FP16 versions respectively.
Adoption of a token-based authentication model for the CMS Submission Infrastructure
Venue: arXiv cs.DC 2024
Year: 2024

The CMS Submission Infrastructure (SI) is the main computing resource provisioning system for CMS workloads. A number of HTCondor pools are employed to manage this infrastructure, which aggregates geographically distributed resources from the WLCG and other providers. Historically, the model of authentication among the diverse components of this infrastructure has relied on the Grid Security Infrastructure (GSI), based on identities and X509 certificates. In contrast, commonly used modern authentication standards are based on capabilities and tokens. The WLCG has identified this trend and aims at a transparent replacement of GSI for all its workload management, data transfer and storage access operations, to be completed during the current LHC Run 3. As part of this effort, and within the context of CMS computing, the Submission Infrastructure group is in the process of phasing out the GSI part of its authentication layers, in favor of IDTokens and Scitokens. The use of tokens is already well integrated into the HTCondor Software Suite, which has allowed us to fully migrate the authentication between internal components of SI. Additionally, recent versions of the HTCondor-CE support tokens as well, enabling CMS resource requests to Grid sites employing this CE technology to be granted by means of token exchange. After a rollout campaign to sites, successfully completed by the third quarter of 2022, the totality of HTCondor CEs in use by CMS are already receiving Scitoken-based pilot jobs. On the ARC CE side, a parallel campaign was launched to foster the adoption of the REST interface at CMS sites (required to enable token-based job submission via HTCondor-G), which is nearing completion as well. In this contribution, the newly adopted authentication model will be described. We will then report on the migration status and final steps towards complete GSI phase out in the CMS SI.