OpenRaiser/Intern-Atlas huggingface dataset:
Harmful algal bloom forecasting. A comparison between stream and batch learning
Venue: arXiv cs.LG 2024
Year: 2024

Diarrhetic Shellfish Poisoning (DSP) is a global health threat arising from shellfish contaminated with toxins produced by dinoflagellates. The condition, with its widespread incidence, high morbidity rate, and persistent shellfish toxicity, poses risks to public health and the shellfish industry. High biomass of toxin-producing algae such as DSP are known as Harmful Algal Blooms (HABs). Monitoring and forecasting systems are crucial for mitigating HABs impact. Predicting harmful algal blooms involves a time-series-based problem with a strong historical seasonal component, however, recent anomalies due to changes in meteorological and oceanographic events have been observed. Stream Learning stands out as one of the most promising approaches for addressing time-series-based problems with concept drifts. However, its efficacy in predicting HABs remains unproven and needs to be tested in comparison with Batch Learning. Historical data availability is a critical point in developing predictive systems. In oceanography, the available data collection can have some constrains and limitations, which has led to exploring new tools to obtain more exhaustive time series. In this study, a machine learning workflow for predicting the number of cells of a toxic dinoflagellate, Dinophysis acuminata, was developed with several key advancements. Seven machine learning algorithms were compared within two learning paradigms. Notably, the output data from CROCO, the ocean hydrodynamic model, was employed as the primary dataset, palliating the limitation of time-continuous historical data. This study highlights the value of models interpretability, fair models comparison methodology, and the incorporation of Stream Learning models. The model DoME, with an average R2 of 0.77 in the 3-day-ahead prediction, emerged as the most effective and interpretable predictor, outperforming the other algorithms.
Vision System Prototype for Inspection and Monitoring with a Smart Camera
Venue: arXiv cs.CV 2024
Year: 2024

This paper presents the design of an artificial vision system prototype for automatic inspection and monitoring of objects over a conveyor belt and using a Smart camera 2D BOA-INS. The prototype consists of a conveyor belt and an embedded system based on an Arduino Mega card for system control, and it has as main peripherals the smart camera, a direct current motor, a photoelectric sensor, LED illumination and LEDs indicating the status (good or defect) of each evaluated object. The application of the prototype is for educational purposes, so that undergraduate, master and diploma students can simulate a continuous production line, controlled by an embedded system, and perform quality control by monitoring through a visual system and a personal computer. This allows implementing the topics of embedded systems, artificial vision, artificial intelligence, pattern recognition, automatic control, as well as automation of real processes.
Combining unsupervised and supervised learning in microscopy enables defect analysis of a full 4H-SiC wafer
Venue: arXiv cs.CV 2024
Year: 2024

Detecting and analyzing various defect types in semiconductor materials is an important prerequisite for understanding the underlying mechanisms as well as tailoring the production processes. Analysis of microscopy images that reveal defects typically requires image analysis tasks such as segmentation and object detection. With the permanently increasing amount of data that is produced by experiments, handling these tasks manually becomes more and more impossible. In this work, we combine various image analysis and data mining techniques for creating a robust and accurate, automated image analysis pipeline. This allows for extracting the type and position of all defects in a microscopy image of a KOH-etched 4H-SiC wafer that was stitched together from approximately 40,000 individual images.
Investigating the Histogram Loss in Regression
Venue: arXiv cs.LG 2024
Year: 2024

It is becoming increasingly common in regression to train neural networks that model the entire distribution even if only the mean is required for prediction. This additional modeling often comes with performance gain and the reasons behind the improvement are not fully known. This paper investigates a recent approach to regression, the Histogram Loss, which involves learning the conditional distribution of the target variable by minimizing the cross-entropy between a target distribution and a flexible histogram prediction. We design theoretical and empirical analyses to determine why and when this performance gain appears, and how different components of the loss contribute to it. Our results suggest that the benefits of learning distributions in this setup come from improvements in optimization rather than modelling extra information. We then demonstrate the viability of the Histogram Loss in common deep learning applications without a need for costly hyperparameter tuning.
A Neuro-Symbolic Approach to Multi-Agent RL for Interpretability and Probabilistic Decision Making
Venue: arXiv cs.AI 2024
Year: 2024

Multi-agent reinforcement learning (MARL) is well-suited for runtime decision-making in optimizing the performance of systems where multiple agents coexist and compete for shared resources. However, applying common deep learning-based MARL solutions to real-world problems suffers from issues of interpretability, sample efficiency, partial observability, etc. To address these challenges, we present an event-driven formulation, where decision-making is handled by distributed co-operative MARL agents using neuro-symbolic methods. The recently introduced neuro-symbolic Logical Neural Networks (LNN) framework serves as a function approximator for the RL, to train a rules-based policy that is both logical and interpretable by construction. To enable decision-making under uncertainty and partial observability, we developed a novel probabilistic neuro-symbolic framework, Probabilistic Logical Neural Networks (PLNN), which combines the capabilities of logical reasoning with probabilistic graphical models. In PLNN, the upward/downward inference strategy, inherited from LNN, is coupled with belief bounds by setting the activation function for the logical operator associated with each neural network node to a probability-respecting generalization of the Fréchet inequalities. These PLNN nodes form the unifying element that combines probabilistic logic and Bayes Nets, permitting inference for variables with unobserved states. We demonstrate our contributions by addressing key MARL challenges for power sharing in a system-on-chip application.
Potential and Challenges of Model Editing for Social Debiasing
Venue: arXiv cs.CL 2024
Year: 2024

Large language models (LLMs) trained on vast corpora suffer from inevitable stereotype biases. Mitigating these biases with fine-tuning could be both costly and data-hungry. Model editing methods, which focus on modifying LLMs in a post-hoc manner, are of great potential to address debiasing. However, it lacks a comprehensive study that facilitates both internal and external model editing methods, supports various bias types, as well as understands the pros and cons of applying editing methods to stereotypical debiasing. To mitigate this gap, we carefully formulate social debiasing into an editing problem and benchmark seven existing model editing algorithms on stereotypical debiasing, i.e., debias editing. Our findings in three scenarios reveal both the potential and challenges of debias editing: (1) Existing model editing methods can effectively preserve knowledge and mitigate biases, while the generalization of debias effect from edited sentences to semantically equivalent sentences is limited.(2) Sequential editing highlights the robustness of SERAC (Mitchell et al. 2022b), while internal editing methods degenerate with the number of edits. (3) Model editing algorithms achieve generalization towards unseen biases both within the same type and from different types. In light of these findings, we further propose two simple but effective methods to improve debias editing, and experimentally show the effectiveness of the proposed methods.
GradSafe: Detecting Jailbreak Prompts for LLMs via Safety-Critical Gradient Analysis
Venue: arXiv cs.CL 2024
Year: 2024

Large Language Models (LLMs) face threats from jailbreak prompts. Existing methods for detecting jailbreak prompts are primarily online moderation APIs or finetuned LLMs. These strategies, however, often require extensive and resource-intensive data collection and training processes. In this study, we propose GradSafe, which effectively detects jailbreak prompts by scrutinizing the gradients of safety-critical parameters in LLMs. Our method is grounded in a pivotal observation: the gradients of an LLM's loss for jailbreak prompts paired with compliance response exhibit similar patterns on certain safety-critical parameters. In contrast, safe prompts lead to different gradient patterns. Building on this observation, GradSafe analyzes the gradients from prompts (paired with compliance responses) to accurately detect jailbreak prompts. We show that GradSafe, applied to Llama-2 without further training, outperforms Llama Guard, despite its extensive finetuning with a large dataset, in detecting jailbreak prompts. This superior performance is consistent across both zero-shot and adaptation scenarios, as evidenced by our evaluations on ToxicChat and XSTest. The source code is available at https://github.com/xyq7/GradSafe.
Overview of the VLSP 2023 -- ComOM Shared Task: A Data Challenge for Comparative Opinion Mining from Vietnamese Product Reviews
Venue: arXiv cs.CL 2024
Year: 2024

This paper presents a comprehensive overview of the Comparative Opinion Mining from Vietnamese Product Reviews shared task (ComOM), held as part of the 10$^{th}$ International Workshop on Vietnamese Language and Speech Processing (VLSP 2023). The primary objective of this shared task is to advance the field of natural language processing by developing techniques that proficiently extract comparative opinions from Vietnamese product reviews. Participants are challenged to propose models that adeptly extract a comparative "quintuple" from a comparative sentence, encompassing Subject, Object, Aspect, Predicate, and Comparison Type Label. We construct a human-annotated dataset comprising $120$ documents, encompassing $7427$ non-comparative sentences and $2468$ comparisons within $1798$ sentences. Participating models undergo evaluation and ranking based on the Exact match macro-averaged quintuple F1 score.
Unravelling Organisational Rule Systems in Requirements Engineering
Venue: arXiv cs.SE 2024
Year: 2024

Context and motivation: Requirements engineering of complex IT systems needs to manage the many, and often vague and conflicting, organisational rules that exist in the context of a modern enterprise. At the same time, IT systems affect the organisation, essentially setting new rules on how the organisation should work. Question/problem: Gathering requirements for an IT system involves understanding the complex rules that govern an organisation. The research question is: How can the holistic properties of organisational rules be conceptualised? Principal ideas/results: This paper introduces the concept of organisational rule systems that may be used to describe complex organisational rules. The concept and its components are presented as a conceptual framework, which in turn is condensed into a conceptual framework diagram. The framework is grounded in a critical literature review. Contribution: The conceptual framework will, as a first step of a wider research agenda, help requirements engineers understand the influence of organisational rules.
Obstacle crossing strategies for high-speed 4WD small-scale vehicle
Venue: arXiv cs.RO 2024
Year: 2024

Unmanned ground vehicle obstacle crossing generally relies on two strategies: (i) applying a wheel torque for climbing and (ii) modifying the vehicle shape by using a wheel-leg or wheel-paddle to lift the wheel on top of the obstacle. However, most of those strategies sacrifice speed in order to have a longer contact duration between the wheels and the obstacle. This paper investigates the behaviour of a 4WD high-speed vehicle while crossing a step obstacle using a design of experiment (DoE). A 3D multibody vehicle model is equipped with a novel 2-DoF suspension system, which horizontal damping coefficient is modify to dampen wheel motion in longitudinal and vertical directions in relation to the chassis, for a given speed and obstacle height. The DoE results allow to propose a novel high-speed obstacle crossing strategy based on three metrics: (i) the kinetic energy variation of the vehicle, (ii) the contact duration between the wheel and the obstacle, and (iii) the pitch rate at the start of the ballistic phase. Experimental function are proposed to be able modify these metric in real time.
Finding Incompatible Blocks for Reliable JPEG Steganalysis
Venue: arXiv cs.CR 2024
Year: 2024

This article presents a refined notion of incompatible JPEG images for a quality factor of 100. It can be used to detect the presence of steganographic schemes embedding in DCT coefficients. We show that, within the JPEG pipeline, the combination of the DCT transform with the quantization function can map several distinct blocks in the pixel domain to the same block in the DCT domain. However, not every DCT block can be obtained: we call those blocks incompatible. In particular, incompatibility can happen when DCT coefficients are manually modified to embed a message. We show that the problem of distinguishing compatible blocks from incompatible ones is an inverse problem with or without solution and we propose two different methods to solve it. The first one is heuristic-based, fast to find a solution if it exists. The second is formulated as an Integer Linear Programming problem and can detect incompatible blocks only for a specific DCT transform in a reasonable amount of time. We show that the probability for a block to become incompatible only relies on the number of modifications. Finally, using the heuristic algorithm we can derive a Likelihood Ratio Test depending on the number of compatible blocks per image to perform steganalysis. We simulate the result of this test and show that it outperforms a deep learning detector e-SRNet for every payload between 0.001 and 0.01 bpp by using only 10% of the blocks from 256x256 images. A Selection-Channel-Aware version of the test is even more powerful and outperforms e-SRNet while using only 1% of the blocks.
KInIT at SemEval-2024 Task 8: Fine-tuned LLMs for Multilingual Machine-Generated Text Detection
Venue: arXiv cs.CL 2024
Year: 2024

SemEval-2024 Task 8 is focused on multigenerator, multidomain, and multilingual black-box machine-generated text detection. Such a detection is important for preventing a potential misuse of large language models (LLMs), the newest of which are very capable in generating multilingual human-like texts. We have coped with this task in multiple ways, utilizing language identification and parameter-efficient fine-tuning of smaller LLMs for text classification. We have further used the per-language classification-threshold calibration to uniquely combine fine-tuned models predictions with statistical detection metrics to improve generalization of the system detection performance. Our submitted method achieved competitive results, ranking at the fourth place, just under 1 percentage point behind the winner.
$\infty$Bench: Extending Long Context Evaluation Beyond 100K Tokens
Venue: arXiv cs.CL 2024
Year: 2024

Processing and reasoning over long contexts is crucial for many practical applications of Large Language Models (LLMs), such as document comprehension and agent construction. Despite recent strides in making LLMs process contexts with more than 100K tokens, there is currently a lack of a standardized benchmark to evaluate this long-context capability. Existing public benchmarks typically focus on contexts around 10K tokens, limiting the assessment and comparison of LLMs in processing longer contexts. In this paper, we propose $\infty$Bench, the first LLM benchmark featuring an average data length surpassing 100K tokens. $\infty$Bench comprises synthetic and realistic tasks spanning diverse domains, presented in both English and Chinese. The tasks in $\infty$Bench are designed to require well understanding of long dependencies in contexts, and make simply retrieving a limited number of passages from contexts not sufficient for these tasks. In our experiments, based on $\infty$Bench, we evaluate the state-of-the-art proprietary and open-source LLMs tailored for processing long contexts. The results indicate that existing long context LLMs still require significant advancements to effectively process 100K+ context. We further present three intriguing analyses regarding the behavior of LLMs processing long context.
The Effect of Batch Size on Contrastive Self-Supervised Speech Representation Learning
Venue: arXiv cs.SD 2024
Year: 2024

Foundation models in speech are often trained using many GPUs, which implicitly leads to large effective batch sizes. In this paper we study the effect of batch size on pre-training, both in terms of statistics that can be monitored during training, and in the effect on the performance of a downstream fine-tuning task. By using batch sizes varying from 87.5 seconds to 80 minutes of speech we show that, for a fixed amount of iterations, larger batch sizes result in better pre-trained models. However, there is lower limit for stability, and an upper limit for effectiveness. We then show that the quality of the pre-trained model depends mainly on the amount of speech data seen during training, i.e., on the product of batch size and number of iterations. All results are produced with an independent implementation of the wav2vec 2.0 architecture, which to a large extent reproduces the results of the original work (arXiv:2006.11477). Our extensions can help researchers choose effective operating conditions when studying self-supervised learning in speech, and hints towards benchmarking self-supervision with a fixed amount of seen data. Code and model checkpoints are available at https://github.com/nikvaessen/w2v2-batch-size.
Bring Your Own Character: A Holistic Solution for Automatic Facial Animation Generation of Customized Characters
Venue: arXiv cs.HC 2024
Year: 2024

Animating virtual characters has always been a fundamental research problem in virtual reality (VR). Facial animations play a crucial role as they effectively convey emotions and attitudes of virtual humans. However, creating such facial animations can be challenging, as current methods often involve utilization of expensive motion capture devices or significant investments of time and effort from human animators in tuning animation parameters. In this paper, we propose a holistic solution to automatically animate virtual human faces. In our solution, a deep learning model was first trained to retarget the facial expression from input face images to virtual human faces by estimating the blendshape coefficients. This method offers the flexibility of generating animations with characters of different appearances and blendshape topologies. Second, a practical toolkit was developed using Unity 3D, making it compatible with the most popular VR applications. The toolkit accepts both image and video as input to animate the target virtual human faces and enables users to manipulate the animation results. Furthermore, inspired by the spirit of Human-in-the-loop (HITL), we leveraged user feedback to further improve the performance of the model and toolkit, thereby increasing the customization properties to suit user preferences. The whole solution, for which we will make the code public, has the potential to accelerate the generation of facial animations for use in VR applications.
Hidden Gems in the Rough: Computational Notebooks as an Uncharted Oasis for IDEs
Venue: arXiv cs.SE 2024
Year: 2024

In this paper, we outline potential ways for the further development of computational notebooks in Integrated Development Environments (IDEs). We discuss notebooks integration with IDEs, focusing on three main areas: facilitating experimentation, adding collaborative features, and improving code comprehension. We propose that better support of notebooks will not only benefit the notebooks, but also enhance IDEs by supporting new development processes native to notebooks. In conclusion, we suggest that adapting IDEs for more experimentation-oriented notebook processes will prepare them for the future of AI-powered programming.
From Text to CQL: Bridging Natural Language and Corpus Search Engine
Venue: arXiv cs.CL 2024
Year: 2024

Natural Language Processing (NLP) technologies have revolutionized the way we interact with information systems, with a significant focus on converting natural language queries into formal query languages such as SQL. However, less emphasis has been placed on the Corpus Query Language (CQL), a critical tool for linguistic research and detailed analysis within text corpora. The manual construction of CQL queries is a complex and time-intensive task that requires a great deal of expertise, which presents a notable challenge for both researchers and practitioners. This paper presents the first text-to-CQL task that aims to automate the translation of natural language into CQL. We present a comprehensive framework for this task, including a specifically curated large-scale dataset and methodologies leveraging large language models (LLMs) for effective text-to-CQL task. In addition, we established advanced evaluation metrics to assess the syntactic and semantic accuracy of the generated queries. We created innovative LLM-based conversion approaches and detailed experiments. The results demonstrate the efficacy of our methods and provide insights into the complexities of text-to-CQL task.
A Unified Knowledge Graph to Permit Interoperability of Heterogeneous Digital Evidence
Venue: arXiv cs.CR 2024
Year: 2024

The modern digital world is highly heterogeneous, encompassing a wide variety of communications, devices, and services. This interconnectedness generates, synchronises, stores, and presents digital information in multidimensional, complex formats, often fragmented across multiple sources. When linked to misuse, this digital information becomes vital digital evidence. Integrating and harmonising these diverse formats into a unified system is crucial for comprehensively understanding evidence and its relationships. However, existing approaches to date have faced challenges limiting investigators' ability to query heterogeneous evidence across large datasets. This paper presents a novel approach in the form of a modern unified data graph. The proposed approach aims to seamlessly integrate, harmonise, and unify evidence data, enabling cross-platform interoperability, efficient data queries, and improved digital investigation performance. To demonstrate its efficacy, a case study is conducted, highlighting the benefits of the proposed approach and showcasing its effectiveness in enabling the interoperability required for advanced analytics in digital investigations.