OpenRaiser/Intern-Atlas huggingface dataset:
Fréchet Denoised Distance: Enhancing Plausibility Evaluation for Generated Designs with Denoising Autoencoder
Venue: arXiv cs.CV 2024
Year: 2024

A great interest has arisen in using Deep Generative Models (DGM) for generative design. When assessing the quality of the generated designs, human designers focus more on structural plausibility, e.g., no missing component, rather than visual artifacts, e.g., noises or blurriness. Meanwhile, commonly used metrics such as Fréchet Inception Distance (FID) may not evaluate accurately because they are sensitive to visual artifacts and tolerant to semantic errors. As such, FID might not be suitable to assess the performance of DGMs for a generative design task. In this work, we propose to encode the to-be-evaluated images with a Denoising Autoencoder (DAE) and measure the distribution distance in the resulting latent space. Hereby, we design a novel metric Fréchet Denoised Distance (FDD). We experimentally test our FDD, FID and other state-of-the-art metrics on multiple datasets, e.g., BIKED, Seeing3DChairs, FFHQ and ImageNet. Our FDD can effectively detect implausible structures and is more consistent with structural inspections by human experts. Our source code is publicly available at https://github.com/jiajie96/FDD_pytorch.
Scalable Software as a Service Architecture
Venue: arXiv cs.SE 2024
Year: 2024

This paper explores the architecture of Software as a Service (SaaS) platforms, emphasizing scalability and maintainability. SaaS, a flexible software distribution model suitable for individuals and organizations, has become prevalent with the advent of Cloud services. This paper aims to provide a high-level design reference for establishing a scalable and maintainable SaaS architecture.
Switching the Loss Reduces the Cost in Batch (Offline) Reinforcement Learning
Venue: arXiv cs.LG 2024
Year: 2024

We propose training fitted Q-iteration with log-loss (FQI-log) for batch reinforcement learning (RL). We show that the number of samples needed to learn a near-optimal policy with FQI-log scales with the accumulated cost of the optimal policy, which is zero in problems where acting optimally achieves the goal and incurs no cost. In doing so, we provide a general framework for proving small-cost bounds, i.e. bounds that scale with the optimal achievable cost, in batch RL. Moreover, we empirically verify that FQI-log uses fewer samples than FQI trained with squared loss on problems where the optimal policy reliably achieves the goal.
Sound and Complete Witnesses for Template-based Verification of LTL Properties on Polynomial Programs
Venue: arXiv cs.PL 2024
Year: 2024

We study the classical problem of verifying programs with respect to formal specifications given in the linear temporal logic (LTL). We first present novel sound and complete witnesses for LTL verification over imperative programs. Our witnesses are applicable to both verification (proving) and refutation (finding bugs) settings. We then consider LTL formulas in which atomic propositions can be polynomial constraints and turn our focus to polynomial arithmetic programs, i.e. programs in which every assignment and guard consists only of polynomial expressions. For this setting, we provide an efficient algorithm to automatically synthesize such LTL witnesses. Our synthesis procedure is both sound and semi-complete. Finally, we present experimental results demonstrating the effectiveness of our approach and that it can handle programs which were beyond the reach of previous state-of-the-art tools.
Planning and Inverse Kinematics of Hyper-Redundant Manipulators with VO-FABRIK
Venue: arXiv cs.RO 2024
Year: 2024

Hyper-redundant Robotic Manipulators (HRMs) offer great dexterity and flexibility of operation, but solving Inverse Kinematics (IK) is challenging. In this work, we introduce VO-FABRIK, an algorithm combining Forward and Backward Reaching Inverse Kinematics (FABRIK) for repeatable deterministic IK computation, and an approach inspired from velocity obstacles to perform path planning under collision and joint limits constraints. We show preliminary results on an industrial HRM with 19 actuated joints. Our algorithm achieves good performance where a state-of-the-art IK solver fails.
Safe Spot: Perceived safety of dominant and submissive appearances of quadruped robots in human-robot interactions
Venue: arXiv cs.RO 2024
Year: 2024

Unprecedented possibilities of quadruped robots have driven much research on the technical aspects of these robots. However, the social perception and acceptability of quadruped robots so far remain poorly understood. This work investigates whether the way we design quadruped robots' behaviors can affect people's perception of safety in interactions with these robots. We designed and tested a dominant and submissive personality for the quadruped robot (Boston Dynamics Spot). These were tested in two different walking scenarios (head-on and crossing interactions) in a 2x2 within-subjects study. We collected both behavioral data and subjective reports on participants' perception of the interaction. The results highlight that participants perceived the submissive robot as safer compared to the dominant one. The behavioral dynamics of interactions did not change depending on the robot's appearance. Participants' previous in-person experience with the robot was associated with lower subjective safety ratings but did not correlate with the interaction dynamics. Our findings have implications for the design of quadruped robots and contribute to the body of knowledge on the social perception of non-humanoid robots. We call for a stronger standing of felt experiences in human-robot interaction research.
To Err Is Human, but Llamas Can Learn It Too
Venue: arXiv cs.CL 2024
Year: 2024

This study explores enhancing grammatical error correction (GEC) through artificial error generation (AEG) using language models (LMs). Specifically, we fine-tune Llama 2-based LMs for error generation and find that this approach yields synthetic errors akin to human errors. Next, we train GEC Llama models with the help of these artificial errors and outperform previous state-of-the-art error correction models, with gains ranging between 0.8 and 6 F0.5 points across all tested languages (German, Ukrainian, and Estonian). Moreover, we demonstrate that generating errors by fine-tuning smaller sequence-to-sequence models and prompting large commercial LMs (GPT-3.5 and GPT-4) also results in synthetic errors beneficially affecting error generation models.
Rediscovering the Mullins Effect With Deep Symbolic Regression
Venue: arXiv cs.CE 2024
Year: 2024

The Mullins effect represents a softening phenomenon observed in rubber-like materials and soft biological tissues. It is usually accompanied by many other inelastic effects like for example residual strain and induced anisotropy. In spite of the long term research and many material models proposed in literature, accurate modeling and prediction of this complex phenomenon still remain a challenging task. In this work, we present a novel approach using deep symbolic regression (DSR) to generate material models describing the Mullins effect in the context of nearly incompressible hyperelastic materials. The two step framework first identifies a strain energy function describing the primary loading. Subsequently, a damage function characterizing the softening behavior under cyclic loading is identified. The efficiency of the proposed approach is demonstrated through benchmark tests using the generalized the Mooney-Rivlin and the Ogden-Roxburgh model. The generalizability and robustness of the presented framework are thoroughly studied. In addition, the proposed methodology is extensively validated on a temperature-dependent data set, which demonstrates its versatile and reliable performance.
GEAR: An Efficient KV Cache Compression Recipe for Near-Lossless Generative Inference of LLM
Venue: arXiv cs.LG 2024
Year: 2024

Key-value (KV) caching has become the de-facto to accelerate generation speed for large language models (LLMs) inference. However, the growing cache demand with increasing sequence length has transformed LLM inference to be a memory bound problem, significantly constraining the system throughput. Existing methods rely on dropping unimportant tokens or quantizing all entries uniformly. Such methods, however, often incur high approximation errors to represent the compressed matrices. The autoregressive decoding process further compounds the error of each step, resulting in critical deviation in model generation and deterioration of performance. To tackle this challenge, we propose GEAR, an efficient KV cache compression framework that achieves near-lossless high-ratio compression. GEAR first applies quantization to majority of entries of similar magnitudes to ultra-low precision. It then employs a low rank matrix to approximate the quantization error, and a sparse matrix to remedy individual errors from outlier entries. By adeptly integrating three techniques, GEAR is able to fully exploit their synergistic potentials. Our experiments demonstrate that compared to alternatives, GEAR achieves near-lossless 4-bit KV cache compression with up to 2.38x throughput improvement, while reducing peak-memory size up to 2.29x. Our code is publicly available at https://github.com/HaoKang-Timmy/GEAR.
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Venue: arXiv cs.CL 2024
Year: 2024

In this report, we introduce the Gemini 1.5 family of models, representing the next generation of highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information from millions of tokens of context, including multiple long documents and hours of video and audio. The family includes two new models: (1) an updated Gemini 1.5 Pro, which exceeds the February version on the great majority of capabilities and benchmarks; (2) Gemini 1.5 Flash, a more lightweight variant designed for efficiency with minimal regression in quality. Gemini 1.5 models achieve near-perfect recall on long-context retrieval tasks across modalities, improve the state-of-the-art in long-document QA, long-video QA and long-context ASR, and match or surpass Gemini 1.0 Ultra's state-of-the-art performance across a broad set of benchmarks. Studying the limits of Gemini 1.5's long-context ability, we find continued improvement in next-token prediction and near-perfect retrieval (>99%) up to at least 10M tokens, a generational leap over existing models such as Claude 3.0 (200k) and GPT-4 Turbo (128k). Finally, we highlight real-world use cases, such as Gemini 1.5 collaborating with professionals on completing their tasks achieving 26 to 75% time savings across 10 different job categories, as well as surprising new capabilities of large language models at the frontier; when given a grammar manual for Kalamang, a language with fewer than 200 speakers worldwide, the model learns to translate English to Kalamang at a similar level to a person who learned from the same content.
Extinction Risks from AI: Invisible to Science?
Venue: arXiv cs.CY 2024
Year: 2024

In an effort to inform the discussion surrounding existential risks from AI, we formulate Extinction-level Goodhart's Law as "Virtually any goal specification, pursued to the extreme, will result in the extinction of humanity", and we aim to understand which formal models are suitable for investigating this hypothesis. Note that we remain agnostic as to whether Extinction-level Goodhart's Law holds or not. As our key contribution, we identify a set of conditions that are necessary for a model that aims to be informative for evaluating specific arguments for Extinction-level Goodhart's Law. Since each of the conditions seems to significantly contribute to the complexity of the resulting model, formally evaluating the hypothesis might be exceedingly difficult. This raises the possibility that whether the risk of extinction from artificial intelligence is real or not, the underlying dynamics might be invisible to current scientific methods.
Multi-source and multimodal data fusion for predicting academic performance in blended learning university courses
Venue: arXiv cs.CY 2024
Year: 2024

In this paper we applied data fusion approaches for predicting the final academic performance of university students using multiple-source, multimodal data from blended learning environments. We collected and preprocessed data about first-year university students from different sources: theory classes, practical sessions, on-line Moodle sessions, and a final exam. Our objective was to discover which data fusion approach produced the best results using our data. We carried out experiments by applying four different data fusion approaches and six classification algorithms. The results showed that the best predictions were produced using ensembles and selecting the best attributes approach with discretized data. The best prediction models showed us that the level of attention in theory classes, scores in Moodle quizzes, and the level of activity in Moodle forums were the best set of attributes for predicting students' final performance in our courses.
Revolutionizing Mental Health Care through LangChain: A Journey with a Large Language Model
Venue: arXiv cs.HC 2024
Year: 2024

Mental health challenges are on the rise in our modern society, and the imperative to address mental disorders, especially regarding anxiety, depression, and suicidal thoughts, underscores the need for effective interventions. This paper delves into the application of recent advancements in pretrained contextualized language models to introduce MindGuide, an innovative chatbot serving as a mental health assistant for individuals seeking guidance and support in these critical areas. MindGuide leverages the capabilities of LangChain and its ChatModels, specifically ChatOpenAI, as the bedrock of its reasoning engine. The system incorporates key features such as LangChain's ChatPrompt Template, HumanMessage Prompt Template, ConversationBufferMemory, and LLMChain, creating an advanced solution for early detection and comprehensive support within the field of mental health. Additionally, the paper discusses the implementation of Streamlit to enhance the user experience and interaction with the chatbot. This novel approach holds great promise for proactive mental health intervention and assistance.
Eternal Sunshine of the Mechanical Mind: The Irreconcilability of Machine Learning and the Right to be Forgotten
Venue: arXiv cs.GL 2024
Year: 2024

As we keep rapidly advancing toward an era where artificial intelligence is a constant and normative experience for most of us, we must also be aware of what this vision and this progress entail. By first approximating neural connections and activities in computer circuits and then creating more and more sophisticated versions of this crude approximation, we are now facing an age to come where modern deep learning-based artificial intelligence systems can rightly be called thinking machines, and they are sometimes even lauded for their emergent behavior and black-box approaches. But as we create more powerful electronic brains, with billions of neural connections and parameters, can we guarantee that these mammoths built of artificial neurons will be able to forget the data that we store in them? If they are at some level like a brain, can the right to be forgotten still be protected while dealing with these AIs? The essential gap between machine learning and the RTBF is explored in this article, with a premonition of far-reaching conclusions if the gap is not bridged or reconciled any time soon. The core argument is that deep learning models, due to their structure and size, cannot be expected to forget or delete a data as it would be expected from a tabular database, and they should be treated more like a mechanical brain, albeit still in development.
Privacy Amplification for the Gaussian Mechanism via Bounded Support
Venue: arXiv cs.CR 2024
Year: 2024

Data-dependent privacy accounting frameworks such as per-instance differential privacy (pDP) and Fisher information loss (FIL) confer fine-grained privacy guarantees for individuals in a fixed training dataset. These guarantees can be desirable compared to vanilla DP in real world settings as they tightly upper-bound the privacy leakage for a $\textit{specific}$ individual in an $\textit{actual}$ dataset, rather than considering worst-case datasets. While these frameworks are beginning to gain popularity, to date, there is a lack of private mechanisms that can fully leverage advantages of data-dependent accounting. To bridge this gap, we propose simple modifications of the Gaussian mechanism with bounded support, showing that they amplify privacy guarantees under data-dependent accounting. Experiments on model training with DP-SGD show that using bounded support Gaussian mechanisms can provide a reduction of the pDP bound $ε$ by as much as 30% without negative effects on model utility.
Realizability in Semantics-Guided Synthesis Done Eagerly
Venue: arXiv cs.LO 2024
Year: 2024

We present realizability and realization logic, two program logics that jointly address the problem of finding solutions in semantics-guided synthesis. What is new is that we proceed eagerly and not only analyze a single candidate program but a whole set. Realizability logic computes information about the set of candidate programs in a forward fashion. Realization logic uses this information as guidance to identify a suitable candidate in a backward fashion. Realizability logic is able to analyze a set of programs due to a new form of assertions that tracks synthesis alternatives. Realizability logic then picks alternatives to arrive at a program, and we give the guarantee that this process will not need backtracking. We show how to implement the program logics using verification conditions, and report on experiments with a prototype in the context of safe memory reclamation for lock-free data structures.
AMUSE: Adaptive Multi-Segment Encoding for Dataset Watermarking
Venue: arXiv cs.MM 2024
Year: 2024

Curating high quality datasets that play a key role in the emergence of new AI applications requires considerable time, money, and computational resources. So, effective ownership protection of datasets is becoming critical. Recently, to protect the ownership of an image dataset, imperceptible watermarking techniques are used to store ownership information (i.e., watermark) into the individual image samples. Embedding the entire watermark into all samples leads to significant redundancy in the embedded information which damages the watermarked dataset quality and extraction accuracy. In this paper, a multi-segment encoding-decoding method for dataset watermarking (called AMUSE) is proposed to adaptively map the original watermark into a set of shorter sub-messages and vice versa. Our message encoder is an adaptive method that adjusts the length of the sub-messages according to the protection requirements for the target dataset. Existing image watermarking methods are then employed to embed the sub-messages into the original images in the dataset and also to extract them from the watermarked images. Our decoder is then used to reconstruct the original message from the extracted sub-messages. The proposed encoder and decoder are plug-and-play modules that can easily be added to any watermarking method. To this end, extensive experiments are preformed with multiple watermarking solutions which show that applying AMUSE improves the overall message extraction accuracy upto 28% for the same given dataset quality. Furthermore, the image dataset quality is enhanced by a PSNR of $\approx$2 dB on average, while improving the extraction accuracy for one of the tested image watermarking methods.
Internet Sanctions on Russian Media: Actions and Effects
Venue: arXiv cs.NI 2024
Year: 2024

As a response to the Russian aggression against Ukraine, the European Union (EU), through the notion of "digital sovereignty", imposed sanctions on organizations and individuals affiliated with the Russian Federation that prohibit broadcasting content, including online distribution. In this paper, we interrogate the implementation of these sanctions and interpret them as a means to translate the union of states' governmental edicts into effective technical countermeasures. Through longitudinal traffic analysis, we construct an understanding of how ISPs in different EU countries attempted to enforce these sanctions, and compare these implementations to similar measures in other western countries. We find a wide variation of blocking coverage, both internationally and within individual member states. We draw the conclusion that digital sovereignty through sanctions in the EU has a concrete but distinctly limited impact on information flows.
Feature CAM: Interpretable AI in Image Classification
Venue: arXiv cs.CV 2024
Year: 2024

Deep Neural Networks have often been called the black box because of the complex, deep architecture and non-transparency presented by the inner layers. There is a lack of trust to use Artificial Intelligence in critical and high-precision fields such as security, finance, health, and manufacturing industries. A lot of focused work has been done to provide interpretable models, intending to deliver meaningful insights into the thoughts and behavior of neural networks. In our research, we compare the state-of-the-art methods in the Activation-based methods (ABM) for interpreting predictions of CNN models, specifically in the application of Image Classification. We then extend the same for eight CNN-based architectures to compare the differences in visualization and thus interpretability. We introduced a novel technique Feature CAM, which falls in the perturbation-activation combination, to create fine-grained, class-discriminative visualizations. The resulting saliency maps from our experiments proved to be 3-4 times better human interpretable than the state-of-the-art in ABM. At the same time it reserves machine interpretability, which is the average confidence scores in classification.
How Well Do Multi-modal LLMs Interpret CT Scans? An Auto-Evaluation Framework for Analyses
Venue: arXiv cs.AI 2024
Year: 2024

Automatically interpreting CT scans can ease the workload of radiologists. However, this is challenging mainly due to the scarcity of adequate datasets and reference standards for evaluation. This study aims to bridge this gap by introducing a novel evaluation framework, named ``GPTRadScore''. This framework assesses the capabilities of multi-modal LLMs, such as GPT-4 with Vision (GPT-4V), Gemini Pro Vision, LLaVA-Med, and RadFM, in generating descriptions for prospectively-identified findings. By employing a decomposition technique based on GPT-4, GPTRadScore compares these generated descriptions with gold-standard report sentences, analyzing their accuracy in terms of body part, location, and type of finding. Evaluations demonstrated a high correlation with clinician assessments and highlighted its potential over traditional metrics, such as BLEU, METEOR, and ROUGE. Furthermore, to contribute to future studies, we plan to release a benchmark dataset annotated by clinicians. Using GPTRadScore, we found that while GPT-4V and Gemini Pro Vision fare better, their performance revealed significant areas for improvement, primarily due to limitations in the dataset used for training these models. To demonstrate this potential, RadFM was fine-tuned and it resulted in significant accuracy improvements: location accuracy rose from 3.41\% to 12.8\%, body part accuracy from 29.12\% to 53\%, and type accuracy from 9.24\% to 30\%, thereby validating our hypothesis.