OpenRaiser/Intern-Atlas huggingface dataset:
How Do Microservice API Patterns Impact Understandability? A Controlled Experiment
Venue: arXiv cs.SE 2024
Year: 2024

Microservices expose their functionality via remote Application Programming Interfaces (APIs), e.g., based on HTTP or asynchronous messaging technology. To solve recurring problems in this design space, Microservice API Patterns (MAPs) have emerged to capture the collective experience of the API design community. At present, there is a lack of empirical evidence for the effectiveness of these patterns, e.g., how they impact understandability and API usability. We therefore conducted a controlled experiment with 6 microservice patterns to evaluate their impact on understandability with 65 diverse participants. Additionally, we wanted to study how demographics like years of professional experience or experience with MAPs influence the effects of the patterns. Per pattern, we constructed two API examples, each in a pattern version "P" and a functionally equivalent non-pattern version "N" (24 in total). Based on a crossover design, participants had to answer comprehension questions, while we measured the time. For five of the six patterns, we identified a significant positive impact on understandability, i.e., participants answered faster and / or more correctly for "P". However, effect sizes were mostly small, with one pattern showing a medium effect. The correlations between performance and demographics seem to suggest that certain patterns may introduce additional complexity; people experienced with MAPs will profit more from their effects. This has important implications for training and education around MAPs and other patterns.
Probabilistic Constellation Shaping for Enhancing Spectral Efficiency in NOMA VLC Systems
Venue: arXiv cs.IT 2024
Year: 2024

The limited modulation bandwidth of the light emitting diodes (LEDs) presents a challenge in the development of practical high-data-rate visible light communication (VLC) systems. In this paper, a novel adaptive coded probabilistic shaping (PS)-based nonorthogonal multiple access (NOMA) scheme is proposed to improve spectral efficiency (SE) of VLC systems in multiuser uplink communication scenarios. The proposed scheme adapts its rate to the optical signal-to-noise ratio (OSNR) by utilizing non-uniformly distributed discrete constellation symbols and low complexity channel encoder. Furthermore, an alternate optimization algorithm is proposed to determine the optimal channel coding rate, constellation spacing, and probability mass function (PMF) of each user. The extensive numerical results show that the proposed PS-based NOMA scheme closely approaches the capacity of NOMA with fine granularity. Presented results demonstrate the effectiveness of our scheme in improving the SE of VLC systems in multiuser scenarios. For instance, our scheme exhibits substantial SE gains over existing schemes, namely, the pairwise coded modulation (PCM), geometric shaping (GS), and uniform-distribution schemes. These findings highlight the potential of our approach to significantly enhance VLC systems.
Reasoning Algorithmically in Graph Neural Networks
Venue: arXiv cs.LG 2024
Year: 2024

The development of artificial intelligence systems with advanced reasoning capabilities represents a persistent and long-standing research question. Traditionally, the primary strategy to address this challenge involved the adoption of symbolic approaches, where knowledge was explicitly represented by means of symbols and explicitly programmed rules. However, with the advent of machine learning, there has been a paradigm shift towards systems that can autonomously learn from data, requiring minimal human guidance. In light of this shift, in latest years, there has been increasing interest and efforts at endowing neural networks with the ability to reason, bridging the gap between data-driven learning and logical reasoning. Within this context, Neural Algorithmic Reasoning (NAR) stands out as a promising research field, aiming to integrate the structured and rule-based reasoning of algorithms with the adaptive learning capabilities of neural networks, typically by tasking neural models to mimic classical algorithms. In this dissertation, we provide theoretical and practical contributions to this area of research. We explore the connections between neural networks and tropical algebra, deriving powerful architectures that are aligned with algorithm execution. Furthermore, we discuss and show the ability of such neural reasoners to learn and manipulate complex algorithmic and combinatorial optimization concepts, such as the principle of strong duality. Finally, in our empirical efforts, we validate the real-world utility of NAR networks across different practical scenarios. This includes tasks as diverse as planning problems, large-scale edge classification tasks and the learning of polynomial-time approximate algorithms for NP-hard combinatorial problems. Through this exploration, we aim to showcase the potential integrating algorithmic reasoning in machine learning models.
Adaptive Massively Parallel Coloring in Sparse Graphs
Venue: arXiv cs.DC 2024
Year: 2024

Classic symmetry-breaking problems on graphs have gained a lot of attention in models of modern parallel computation. The Adaptive Massively Parallel Computation (AMPC) is a model that captures the central challenges in data center computations. Chang et al. [PODC'2019] gave an extremely fast, constant time, algorithm for the $(Δ+ 1)$-coloring problem, where $Δ$ is the maximum degree of an input graph of $n$ nodes. The algorithm works in the most restrictive low-space setting, where each machine has $n^δ$ local space for a constant $0 < δ< 1$. In this work, we study the vertex-coloring problem in sparse graphs parameterized by their arboricity $α$, a standard measure for sparsity. We give deterministic algorithms that in constant, or almost constant, time give $\text{poly} ~α$ and $O(α)$-colorings, where $α$ can be arbitrarily smaller than $Δ$. A strong and standard approach to compute arboricity-dependent colorings is through the Nash-Williams forest decomposition, which gives rise to an (acyclic) orientation of the edges such that each node has a small out-degree. Our main technical contribution is giving efficient deterministic algorithms to compute these orientations and showing how to leverage them to find colorings in low-space AMPC. A key technical challenge is that the color of a node may depend on almost all of the other nodes in the graph and these dependencies cannot be stored on a single machine. Nevertheless, our novel and careful exploration technique yields the orientation, and the arboricity-dependent coloring, with a sublinear number of adaptive queries per node.
Weakly supervised localisation of prostate cancer using reinforcement learning for bi-parametric MR images
Venue: arXiv cs.CV 2024
Year: 2024

In this paper we propose a reinforcement learning based weakly supervised system for localisation. We train a controller function to localise regions of interest within an image by introducing a novel reward definition that utilises non-binarised classification probability, generated by a pre-trained binary classifier which classifies object presence in images or image crops. The object-presence classifier may then inform the controller of its localisation quality by quantifying the likelihood of the image containing an object. Such an approach allows us to minimize any potential labelling or human bias propagated via human labelling for fully supervised localisation. We evaluate our proposed approach for a task of cancerous lesion localisation on a large dataset of real clinical bi-parametric MR images of the prostate. Comparisons to the commonly used multiple-instance learning weakly supervised localisation and to a fully supervised baseline show that our proposed method outperforms the multi-instance learning and performs comparably to fully-supervised learning, using only image-level classification labels for training.
MLXP: A Framework for Conducting Replicable Experiments in Python
Venue: arXiv cs.LG 2024
Year: 2024

Replicability in machine learning (ML) research is increasingly concerning due to the utilization of complex non-deterministic algorithms and the dependence on numerous hyper-parameter choices, such as model architecture and training datasets. Ensuring reproducible and replicable results is crucial for advancing the field, yet often requires significant technical effort to conduct systematic and well-organized experiments that yield robust conclusions. Several tools have been developed to facilitate experiment management and enhance reproducibility; however, they often introduce complexity that hinders adoption within the research community, despite being well-handled in industrial settings. To address the challenge of low adoption, we propose MLXP, an open-source, simple, and lightweight experiment management tool based on Python, available at https://github.com/inria-thoth/mlxp . MLXP streamlines the experimental process with minimal practitioner overhead while ensuring a high level of reproducibility.
Kuaiji: the First Chinese Accounting Large Language Model
Venue: arXiv cs.CL 2024
Year: 2024

Large Language Models (LLMs) like ChatGPT and GPT-4 have demonstrated impressive proficiency in comprehending and generating natural language. However, they encounter difficulties when tasked with adapting to specialized domains such as accounting. To address this challenge, we introduce Kuaiji, a tailored Accounting Large Language Model. Kuaiji is meticulously fine-tuned using the Baichuan framework, which encompasses continuous pre-training and supervised fine-tuning processes. Supported by CAtAcctQA, a dataset containing large genuine accountant-client dialogues, Kuaiji exhibits exceptional accuracy and response speed. Our contributions encompass the creation of the first Chinese accounting dataset, the establishment of Kuaiji as a leading open-source Chinese accounting LLM, and the validation of its efficacy through real-world accounting scenarios.
A Uniformly Random Solution to Algorithmic Redistricting
Venue: arXiv cs.DS 2024
Year: 2024

The process of drawing electoral district boundaries is known as political redistricting. Within this context, gerrymandering is the practice of drawing these boundaries such that they unfairly favor a particular political party, often leading to unequal representation and skewed electoral outcomes. One of the few ways to detect gerrymandering is by algorithmically sampling redistricting plans. Previous methods mainly focus on sampling from some neighborhood of ``realistic' districting plans, rather than a uniform sample of the entire space. We present a deterministic subexponential time algorithm to uniformly sample from the space of all possible $ k $-partitions of a bounded degree planar graph, and with this construct a sample of the entire space of redistricting plans. We also give a way to restrict this sample space to plans that match certain compactness and population constraints at the cost of added complexity. The algorithm runs in $ 2^{O(\sqrt{n}\log n)} $ time, although we only give a heuristic implementation. Our method generalizes an algorithm to count self-avoiding walks on a square to count paths that split general planar graphs into $ k $ regions, and uses this to sample from the space of all $ k $-partitions of a planar graph.
Beyond Probabilities: Unveiling the Misalignment in Evaluating Large Language Models
Venue: arXiv cs.CL 2024
Year: 2024

Large Language Models (LLMs) have demonstrated remarkable capabilities across various applications, fundamentally reshaping the landscape of natural language processing (NLP) research. However, recent evaluation frameworks often rely on the output probabilities of LLMs for predictions, primarily due to computational constraints, diverging from real-world LLM usage scenarios. While widely employed, the efficacy of these probability-based evaluation strategies remains an open research question. This study aims to scrutinize the validity of such probability-based evaluation methods within the context of using LLMs for Multiple Choice Questions (MCQs), highlighting their inherent limitations. Our empirical investigation reveals that the prevalent probability-based evaluation method inadequately aligns with generation-based prediction. Furthermore, current evaluation frameworks typically assess LLMs through predictive tasks based on output probabilities rather than directly generating responses, owing to computational limitations. We illustrate that these probability-based approaches do not effectively correspond with generative predictions. The outcomes of our study can enhance the understanding of LLM evaluation methodologies and provide insights for future research in this domain.
Bias correction of wind power forecasts with SCADA data and continuous learning
Venue: arXiv cs.LG 2024
Year: 2024

Wind energy plays a critical role in the transition towards renewable energy sources. However, the uncertainty and variability of wind can impede its full potential and the necessary growth of wind power capacity. To mitigate these challenges, wind power forecasting methods are employed for applications in power management, energy trading, or maintenance scheduling. In this work, we present, evaluate, and compare four machine learning-based wind power forecasting models. Our models correct and improve 48-hour forecasts extracted from a numerical weather prediction (NWP) model. The models are evaluated on datasets from a wind park comprising 65 wind turbines. The best improvement in forecasting error and mean bias was achieved by a convolutional neural network, reducing the average NRMSE down to 22%, coupled with a significant reduction in mean bias, compared to a NRMSE of 35% from the strongly biased baseline model using uncorrected NWP forecasts. Our findings further indicate that changes to neural network architectures play a minor role in affecting the forecasting performance, and that future research should rather investigate changes in the model pipeline. Moreover, we introduce a continuous learning strategy, which is shown to achieve the highest forecasting performance improvements when new data is made available.
BenchCloudVision: A Benchmark Analysis of Deep Learning Approaches for Cloud Detection and Segmentation in Remote Sensing Imagery
Venue: arXiv cs.CV 2024
Year: 2024

Satellites equipped with optical sensors capture high-resolution imagery, providing valuable insights into various environmental phenomena. In recent years, there has been a surge of research focused on addressing some challenges in remote sensing, ranging from water detection in diverse landscapes to the segmentation of mountainous and terrains. Ongoing investigations goals to enhance the precision and efficiency of satellite imagery analysis. Especially, there is a growing emphasis on developing methodologies for accurate water body detection, snow and clouds, important for environmental monitoring, resource management, and disaster response. Within this context, this paper focus on the cloud segmentation from remote sensing imagery. Accurate remote sensing data analysis can be challenging due to the presence of clouds in optical sensor-based applications. The quality of resulting products such as applications and research is directly impacted by cloud detection, which plays a key role in the remote sensing data processing pipeline. This paper examines seven cutting-edge semantic segmentation and detection algorithms applied to clouds identification, conducting a benchmark analysis to evaluate their architectural approaches and identify the most performing ones. To increase the model's adaptability, critical elements including the type of imagery and the amount of spectral bands used during training are analyzed. Additionally, this research tries to produce machine learning algorithms that can perform cloud segmentation using only a few spectral bands, including RGB and RGBN-IR combinations. The model's flexibility for a variety of applications and user scenarios is assessed by using imagery from Sentinel-2 and Landsat-8 as datasets. This benchmark can be reproduced using the material from this github link: https://github.com/toelt-llc/cloud_segmentation_comparative.
Large Language Models are Vulnerable to Bait-and-Switch Attacks for Generating Harmful Content
Venue: arXiv cs.CL 2024
Year: 2024

The risks derived from large language models (LLMs) generating deceptive and damaging content have been the subject of considerable research, but even safe generations can lead to problematic downstream impacts. In our study, we shift the focus to how even safe text coming from LLMs can be easily turned into potentially dangerous content through Bait-and-Switch attacks. In such attacks, the user first prompts LLMs with safe questions and then employs a simple find-and-replace post-hoc technique to manipulate the outputs into harmful narratives. The alarming efficacy of this approach in generating toxic content highlights a significant challenge in developing reliable safety guardrails for LLMs. In particular, we stress that focusing on the safety of the verbatim LLM outputs is insufficient and that we also need to consider post-hoc transformations.
Making Reasoning Matter: Measuring and Improving Faithfulness of Chain-of-Thought Reasoning
Venue: arXiv cs.CL 2024
Year: 2024

Large language models (LLMs) have been shown to perform better when asked to reason step-by-step before answering a question. However, it is unclear to what degree the model's final answer is faithful to the stated reasoning steps. In this paper, we perform a causal mediation analysis on twelve LLMs to examine how intermediate reasoning steps generated by the LLM influence the final outcome and find that LLMs do not reliably use their intermediate reasoning steps when generating an answer. To address this issue, we introduce FRODO, a framework to tailor small-sized LMs to generate correct reasoning steps and robustly reason over these steps. FRODO consists of an inference module that learns to generate correct reasoning steps using an implicit causal reward function and a reasoning module that learns to faithfully reason over these intermediate inferences using a counterfactual and causal preference objective. Our experiments show that FRODO significantly outperforms four competitive baselines. Furthermore, FRODO improves the robustness and generalization ability of the reasoning LM, yielding higher performance on out-of-distribution test sets. Finally, we find that FRODO's rationales are more faithful to its final answer predictions than standard supervised fine-tuning.
BEE-NET: A deep neural network to identify in-the-wild Bodily Expression of Emotions
Venue: arXiv cs.CV 2024
Year: 2024

In this study, we investigate how environmental factors, specifically the scenes and objects involved, can affect the expression of emotions through body language. To this end, we introduce a novel multi-stream deep convolutional neural network named BEE-NET. We also propose a new late fusion strategy that incorporates meta-information on places and objects as prior knowledge in the learning process. Our proposed probabilistic pooling model leverages this information to generate a joint probability distribution of both available and anticipated non-available contextual information in latent space. Importantly, our fusion strategy is differentiable, allowing for end-to-end training and capturing of hidden associations among data points without requiring further post-processing or regularisation. To evaluate our deep model, we use the Body Language Database (BoLD), which is currently the largest available database for the Automatic Identification of the in-the-wild Bodily Expression of Emotions (AIBEE). Our experimental results demonstrate that our proposed approach surpasses the current state-of-the-art in AIBEE by a margin of 2.07%, achieving an Emotional Recognition Score of 66.33%.
Meditating in Live Stream: An Autoethnographic and Interview Study to Investigate Motivations, Interactions and Challenges
Venue: arXiv cs.HC 2024
Year: 2024

Mindfulness practice has many mental and physical well-being benefits. With the increased popularity of live stream technologies and the impact of COVID-19, many people have turned to live stream tools to participate in online meditation sessions. To better understand the practices, challenges, and opportunities in live-stream meditation, we conducted a three-month autoethnographic study, during which two researchers participated in live-stream meditation sessions as the audience. Then we conducted a follow-up semi-structured interview study with 10 experienced live meditation teachers who use different live-stream tools. We found that live meditation, although having a weaker social presence than in-person meditation, facilitates attendees in establishing a practice routine and connecting with other meditators. Teachers use live streams to deliver the meditation practice to the world which also enhances their practice and brand building. We identified the challenges of using live-stream tools for meditation from the perspectives of both audiences and teachers, and provided design recommendations to better utilize live meditation as a resource for mental wellbeing.
Corrective Machine Unlearning
Venue: arXiv cs.LG 2024
Year: 2024

Machine Learning models increasingly face data integrity challenges due to the use of large-scale training datasets drawn from the Internet. We study what model developers can do if they detect that some data was manipulated or incorrect. Such manipulated data can cause adverse effects including vulnerability to backdoored samples, systemic biases, and reduced accuracy on certain input domains. Realistically, all manipulated training samples cannot be identified, and only a small, representative subset of the affected data can be flagged. We formalize Corrective Machine Unlearning as the problem of mitigating the impact of data affected by unknown manipulations on a trained model, only having identified a subset of the corrupted data. We demonstrate that the problem of corrective unlearning has significantly different requirements from traditional privacy-oriented unlearning. We find most existing unlearning methods, including retraining-from-scratch without the deletion set, require most of the manipulated data to be identified for effective corrective unlearning. However, one approach, Selective Synaptic Dampening, achieves limited success, unlearning adverse effects with just a small portion of the manipulated samples in our setting, which shows encouraging signs for future progress. We hope our work spurs research towards developing better methods for corrective unlearning and offers practitioners a new strategy to handle data integrity challenges arising from web-scale training. Code is available at https://github.com/drimpossible/corrective-unlearning-bench.