Reviewerly/exHarmony huggingface dataset:
TimeSeriesBench: An Industrial-Grade Benchmark for Time Series Anomaly  Detection Models

Authors:
raw_author_name: Si, Haotian
raw_author_name: Pei, Changhua
raw_author_name: Cui, Hang
raw_author_name: Yang, Jingwen
raw_author_name: Sun, Yongqian
raw_author_name: Zhang, Shenglin
raw_author_name: Li, Jingjing
raw_author_name: Zhang, Haiming
raw_author_name: Han, Jing
raw_author_name: Pei, Dan
raw_author_name: Li, Jianhui
raw_author_name: Xie, Gaogang

Abstract: Driven by the proliferation of real-world application scenarios and scales, time series anomaly detection (TSAD) has attracted considerable scholarly and industrial interest. However, existing algorithms exhibit a gap in terms of training paradigm, online detection paradigm, and evaluation criteria when compared to the actual needs of real-world industrial systems. Firstly, current algorithms typically train a specific model for each individual time series. In a large-scale online system with tens of thousands of curves, maintaining such a multitude of models is impractical. The performance of using merely one single unified model to detect anomalies remains unknown. Secondly, most TSAD models are trained on the historical part of a time series and are tested on its future segment. In distributed systems, however, there are frequent system deployments and upgrades, with new, previously unseen time series emerging daily. The performance of testing newly incoming unseen time series on current TSAD algorithms remains unknown. Lastly, although some papers have conducted detailed surveys, the absence of an online evaluation platform prevents answering questions like "Who is the best at anomaly detection at the current stage?" In this paper, we propose TimeSeriesBench, an industrial-grade benchmark that we continuously maintain as a leaderboard. On this leaderboard, we assess the performance of existing algorithms across more than 168 evaluation settings combining different training and testing paradigms, evaluation metrics and datasets. Through our comprehensive analysis of the results, we provide recommendations for the future design of anomaly detection algorithms. To address known issues with existing public datasets, we release an industrial dataset to the public together with TimeSeriesBench. All code, data, and the online leaderboard have been made publicly available.
The Matrix Profile in Seismology: Template Matching of Everything With Everything

Authors:
raw_author_name: Nader Shabikay Senobari
raw_affiliation_string: Department of Computer Science and Engineering University of California  Riverside CA USA
raw_author_name: Peter M. Shearer
raw_affiliation_string: Scripps Institution of Oceanography University of California  San Diego CA USA
raw_author_name: Gareth J. Funning
raw_affiliation_string: Department of Earth and Planetary Sciences University of California  Riverside CA USA
raw_author_name: Yan Zhu
raw_affiliation_string: Department of Computer Science and Engineering University of California  Riverside CA USA; Now at Google  Mountain View CA USA
raw_author_name: Eamonn Keogh
raw_affiliation_string: Department of Computer Science and Engineering University of California  Riverside CA USA

Abstract: Abstract Template matching has proven to be an effective method for seismic event detection, but is biased toward identifying events similar to previously known events, and thus is ineffective at discovering events with non‐matching waveforms (e.g., those dissimilar to existing catalog events). In principle, this limitation can be overcome by cross‐correlating every segment (possible template) of a seismogram with every other segment to identify all similar event pairs, but doing so has been previously considered computationally infeasible for long time series. Here we describe a method, called the ‘Matrix Profile’ (MP), a “correlate everything with everything” calculation that can be efficiently and scalably computed. The MP returns the maximum value of the correlation coefficient of every sub‐window of continuous data with every other sub‐window, as well as the best‐correlated sub‐window location. Here we show how MP methods can obtain valuable results when applied to months and years of continuous seismic data in both local and global case studies. We find that the MP can identify many new events in Parkfield, California seismicity that are not contained in existing event catalogs and that it can efficiently find clusters of similar earthquakes in global seismic data. Either used by itself, or as a starting point for subsequent template matching calculations, the MP is likely to provide a useful new tool for seismology research.
Research on optimization of inventory demand forecast in cloud infrastructure supply chain

Authors:
raw_author_name: Li Yao
raw_affiliation_string: Shanghai Polytechnic Univ. (China)
raw_author_name: Yuwen Wan
raw_affiliation_string: Shanghai Polytechnic Univ. (China)

Abstract: With the development of computer and cloud computing related technologies, cloud products play a more important role in the economic and cultural activities of enterprises, organizations and individuals. In recent years, numerous cloud service providers have emerged globally to offer cloud products to consumers. Cloud infrastructure is the physical computers that cloud service providers rely on in order to provide cloud products, and cloud infrastructure provides resources such as CPU, GPU, memory, hard disk, and network for cloud products. It can be said that cloud infrastructure is a particularly important aspect of cloud computing. When cloud service providers manage the supply chain of cloud infrastructure, they often have too much or too little inventory, which results in a waste of resources or a failure to meet user demand. Therefore, how to accurately predict the inventory demand of cloud infrastructure supply chain has become a problem that cloud service providers need to solve. In this paper, we propose a GA-LightGBM model for predicting the inventory demand of cloud infrastructure supply chain by comparing multiple models. In this paper, GA-LightGBM is experimentally verified and analyzed with the control model, and it is found that the average RMSE of GA-LightGBM is 358.8792, which is significantly higher than that of each model in the control group; in the process of multiple training and validation, the range of the RMSE and the standard deviation of the RMSE of GA-LightGBM are significantly smaller than that of each model in the control group. It can be seen that the GA-LightGBM model has higher prediction accuracy and stability than the models in the control group. It is recommended that cloud service providers adopt this model to optimize the management of cloud infrastructure supply chain inventory demand forecasting, so as to improve enterprise benefits. The GA-LightGBM model proposed in this paper will also complement research in related areas.
Implementing Federated Governance in Data Mesh Architecture

Authors:
raw_author_name: Anton Dolhopolov
raw_author_name: Arnaud Castelltort
raw_author_name: Anne Laurent

Abstract: Analytical data platform have been used for decades for improving the organizational performance. Starting from the data warehouses used primarily for structured data processing, through the data lakes oriented for raw data storage and post-hoc data analyses, to the data lakehouses − a combination of raw storage and business intelligence pre-processing for improving the platform efficacy. But in recent years a new architecture called Data Mesh has emerged. The main promise of this architecture is to remove the barriers between operational and analytical teams in order to boost the overall value extraction from the big data. A number of attempts has been made to formalise and to implement it in existing projects. Although being defined as a socio-technical paradigm, data mesh still lacks the technology support for enabling its widespread adoption. To overcome this limitation, we propose a new view of the platform requirements alongside the formal governance definition that we believe can help in successful adoption of the data mesh. It is based on fundamental aspects such as decentralized data domains and federated computational governance. In addition, we also present a blockchain-based implementation of a mesh platform as a practical validation to our theoretical proposal. Overall, this article demonstrates a novel research direction of the information system decentralization technologies.
Metacognitive Retrieval-Augmented Large Language Models

Authors:
raw_author_name: Zhou, Yujia
raw_author_name: Liu, Zheng
raw_author_name: Jin, Jiajie
raw_author_name: Dou, Zhicheng

Abstract: Retrieval-augmented generation have become central in natural language processing due to their efficacy in generating factual content. While traditional methods employ single-time retrieval, more recent approaches have shifted towards multi-time retrieval for multi-hop reasoning tasks. However, these strategies are bound by predefined reasoning steps, potentially leading to inaccuracies in response generation. This paper introduces MetaRAG, an approach that combines the retrieval-augmented generation process with metacognition. Drawing from cognitive psychology, metacognition allows an entity to self-reflect and critically evaluate its cognitive processes. By integrating this, MetaRAG enables the model to monitor, evaluate, and plan its response strategies, enhancing its introspective reasoning abilities. Through a three-step metacognitive regulation pipeline, the model can identify inadequacies in initial cognitive responses and fixes them. Empirical evaluations show that MetaRAG significantly outperforms existing methods.
ALLaVA: Harnessing GPT4V-synthesized Data for A Lite Vision-Language  Model

Authors:
raw_author_name: Zhang, Ruifei
raw_author_name: Chen, Junying
raw_author_name: Wu, Xiangbo
raw_author_name: Zhang, Zhiyi
raw_author_name: Li, Jianquan
raw_author_name: Wan, Xiang
raw_author_name: Wang, Benyou

Abstract: Recent advancements in Large Vision-Language Models (LVLMs) have enabled processing of multimodal inputs in language models but require significant computational resources for deployment, especially in edge devices. This study aims to bridge the performance gap between traditional-scale LVLMs and resource-friendly lite versions by adopting high-quality training data. To do this, a synthetic dataset is created by leveraging GPT-4V's ability to generate detailed captions, complex reasoning instructions and detailed answers from images. The resulted model trained with our data, ALLaVA, achieves competitive performance on 12 benchmarks up to 3B LVLMs. This work highlights the feasibility of adopting high-quality data in crafting more efficient LVLMs. Our online demo is available at \url{https://allava.freedomai.cn}.
Balanced Data, Imbalanced Spectra: Unveiling Class Disparities with  Spectral Imbalance

Authors:
raw_author_name: Kaushik, Chiraag
raw_author_name: Liu, Ran
raw_author_name: Lin, Chi-Heng
raw_author_name: Khera, Amrit
raw_author_name: Jin, Matthew Y
raw_author_name: Ma, Wenrui
raw_author_name: Muthukumar, Vidya
raw_author_name: Dyer, Eva L

Abstract: Classification models are expected to perform equally well for different classes, yet in practice, there are often large gaps in their performance. This issue of class bias is widely studied in cases of datasets with sample imbalance, but is relatively overlooked in balanced datasets. In this work, we introduce the concept of spectral imbalance in features as a potential source for class disparities and study the connections between spectral imbalance and class bias in both theory and practice. To build the connection between spectral imbalance and class gap, we develop a theoretical framework for studying class disparities and derive exact expressions for the per-class error in a high-dimensional mixture model setting. We then study this phenomenon in 11 different state-of-the-art pretrained encoders and show how our proposed framework can be used to compare the quality of encoders, as well as evaluate and combine data augmentation strategies to mitigate the issue. Our work sheds light on the class-dependent effects of learning, and provides new insights into how state-of-the-art pretrained features may have unknown biases that can be diagnosed through their spectra.
Zero shot VLMs for hate meme detection: Are we there yet?

Authors:
raw_author_name: Das, Mithun
raw_author_name: Saha, Punyajoy
raw_author_name: Mukherjee, Animesh

Abstract: Multimedia content on social media is rapidly evolving, with memes gaining prominence as a distinctive form. Unfortunately, some malicious users exploit memes to target individuals or vulnerable communities, making it imperative to identify and address such instances of hateful memes. Extensive research has been conducted to address this issue by developing hate meme detection models. However, a notable limitation of traditional machine/deep learning models is the requirement for labeled datasets for accurate classification. Recently, the research community has witnessed the emergence of several visual language models that have exhibited outstanding performance across various tasks. In this study, we aim to investigate the efficacy of these visual language models in handling intricate tasks such as hate meme detection. We use various prompt settings to focus on zero-shot classification of hateful/harmful memes. Through our analysis, we observe that large VLMs are still vulnerable for zero-shot hate meme detection.
Kernel KMeans clustering splits for end-to-end unsupervised decision  trees

Authors:
raw_author_name: Ohl, Louis
raw_author_name: Mattei, Pierre-Alexandre
raw_author_name: Leclercq, Mickaël
raw_author_name: Droit, Arnaud
raw_author_name: Precioso, Frédéric

Abstract: Trees are convenient models for obtaining explainable predictions on relatively small datasets. Although there are many proposals for the end-to-end construction of such trees in supervised learning, learning a tree end-to-end for clustering without labels remains an open challenge. As most works focus on interpreting with trees the result of another clustering algorithm, we present here a novel end-to-end trained unsupervised binary tree for clustering: Kauri. This method performs a greedy maximisation of the kernel KMeans objective without requiring the definition of centroids. We compare this model on multiple datasets with recent unsupervised trees and show that Kauri performs identically when using a linear kernel. For other kernels, Kauri often outperforms the concatenation of kernel KMeans and a CART decision tree.
Adaptive Skeleton Graph Decoding

Authors:
raw_author_name: Jin, Shuowei
raw_author_name: Wu, Yongji
raw_author_name: Zheng, Haizhong
raw_author_name: Zhang, Qingzhao
raw_author_name: Prakash, Atul
raw_author_name: Qian, Feng
raw_author_name: Zhuo, Danyang

Abstract: Large language models (LLMs) have seen significant adoption for natural language tasks, owing their success to massive numbers of model parameters (e.g., 70B+); however, LLM inference incurs significant computation and memory costs. Recent approaches propose parallel decoding strategies, such as Skeleton-of-Thought (SoT), to improve performance by breaking prompts down into sub-problems that can be decoded in parallel; however, they often suffer from reduced response quality. Our key insight is that we can request additional information, specifically dependencies and difficulty, when generating the sub-problems to improve both response quality and performance. In this paper, we propose Skeleton Graph Decoding (SGD), which uses dependencies exposed between sub-problems to support information forwarding between dependent sub-problems for improved quality while exposing parallelization opportunities for decoding independent sub-problems. Additionally, we leverage difficulty estimates for each sub-problem to select an appropriately-sized model, improving performance without significantly reducing quality. Compared to standard autoregressive generation and SoT, SGD achieves a 1.69x speedup while improving quality by up to 51%.
Poisoned Forgery Face: Towards Backdoor Attacks on Face Forgery  Detection

Authors:
raw_author_name: Liang, Jiawei
raw_author_name: Liang, Siyuan
raw_author_name: Liu, Aishan
raw_author_name: Jia, Xiaojun
raw_author_name: Cao, Xiaochun

Abstract: The proliferation of face forgery techniques has raised significant concerns within society, thereby motivating the development of face forgery detection methods. These methods aim to distinguish forged faces from genuine ones and have proven effective in practical applications. However, this paper introduces a novel and previously unrecognized threat in face forgery detection scenarios caused by backdoor attack. By embedding backdoors into models and incorporating specific trigger patterns into the input, attackers can deceive detectors into producing erroneous predictions for forged faces. To achieve this goal, this paper proposes \emph{Poisoned Forgery Face} framework, which enables clean-label backdoor attacks on face forgery detectors. Our approach involves constructing a scalable trigger generator and utilizing a novel convolving process to generate translation-sensitive trigger patterns. Moreover, we employ a relative embedding method based on landmark-based regions to enhance the stealthiness of the poisoned samples. Consequently, detectors trained on our poisoned samples are embedded with backdoors. Notably, our approach surpasses SoTA backdoor baselines with a significant improvement in attack success rate (+16.39\% BD-AUC) and reduction in visibility (-12.65\% $L_\infty$). Furthermore, our attack exhibits promising performance against backdoor defenses. We anticipate that this paper will draw greater attention to the potential threats posed by backdoor attacks in face forgery detection scenarios. Our codes will be made available at \url{https://github.com/JWLiang007/PFF}
Decoding News Narratives: A Critical Analysis of Large Language Models  in Framing Bias Detection

Authors:
raw_author_name: Sivakumar, Jasivan A.
raw_author_name: Moosavi, Nafise Sadat

Abstract: This work contributes to the expanding research on the applicability of LLMs in social sciences by examining the performance of GPT-3.5 Turbo, GPT-4, and Flan-T5 models in detecting framing bias in news headlines through zero-shot, few-shot, and explainable prompting methods. A key insight from our evaluation is the notable efficacy of explainable prompting in enhancing the reliability of these models, highlighting the importance of explainable settings for social science research on framing bias. GPT-4, in particular, demonstrated enhanced performance in few-shot scenarios when presented with a range of relevant, in-domain examples. FLAN-T5's poor performance indicates that smaller models may require additional task-specific fine-tuning for identifying framing bias detection. Our study also found that models, particularly GPT-4, often misinterpret emotional language as an indicator of framing bias, underscoring the challenge of distinguishing between reporting genuine emotional expression and intentionally use framing bias in news headlines. We further evaluated the models on two subsets of headlines where the presence or absence of framing bias was either clear-cut or more contested, with the results suggesting that these models' can be useful in flagging potential annotation inaccuracies within existing or new datasets. Finally, the study evaluates the models in real-world conditions ("in the wild"), moving beyond the initial dataset focused on U.S. Gun Violence, assessing the models' performance on framed headlines covering a broad range of topics.
Extensible Embedding: A Flexible Multipler For LLM's Context Length

Authors:
raw_author_name: Xiao, Shitao
raw_author_name: Liu, Zheng
raw_author_name: Zhang, Peitian

Abstract: Large language models (LLMs) call for extension of context to handle many critical applications. However, the existing approaches are prone to expensive costs and inferior quality of context extension. In this work, we propose Extensible Embedding, which realizes high-quality extension of LLM's context with strong flexibility and cost-effectiveness. Extensible embedding stand as an enhancement of typical token embedding, which represents the information for an extensible scope of context instead of a single token. By leveraging such compact input units of higher information density, the LLM can access to a vast scope of context even with a small context window. Extensible embedding is systematically optimized in architecture and training method, which leads to multiple advantages. 1) High flexibility of context extension, which flexibly supports ad-hoc extension of diverse context lengths. 2) Strong sample efficiency of training, which enables the embedding model to be learned in a cost-effective way. 3) Superior compatibility with the existing LLMs, where the extensible embedding can be seamlessly introduced as a plug-in component. Comprehensive evaluations on long-context language modeling and understanding tasks verify extensible embedding as an effective, efficient, flexible, and compatible method to extend the LLM's context.
Momentor: Advancing Video Large Language Model with Fine-Grained  Temporal Reasoning

Authors:
raw_author_name: Qian, Long
raw_author_name: Li, Juncheng
raw_author_name: Chua, Tat-Seng
raw_author_name: Zhuang, Yueting
raw_author_name: Tang, Siliang

Abstract: Large Language Models (LLMs) demonstrate remarkable proficiency in comprehending and handling text-based tasks. Many efforts are being made to transfer these attributes to video modality, which are termed Video-LLMs. However, existing Video-LLMs can only capture the coarse-grained semantics and are unable to effectively handle tasks related to comprehension or localization of specific video segments. In light of these challenges, we propose Momentor, a Video-LLM capable of accomplishing fine-grained temporal understanding tasks. To support the training of Momentor, we design an automatic data generation engine to construct Moment-10M, a large-scale video instruction dataset with segment-level instruction data. We train Momentor on Moment-10M, enabling it to perform segment-level reasoning and localization. Zero-shot evaluations on several tasks demonstrate that Momentor excels in fine-grained temporally grounded comprehension and localization.
Large Language Models Fall Short: Understanding Complex Relationships in  Detective Narratives

Authors:
raw_author_name: Zhao, Runcong
raw_author_name: Zhu, Qinglin
raw_author_name: Li, Jiazheng
raw_author_name: Zhou, Yuxiang
raw_author_name: He, Yulan
raw_author_name: Gui, Lin

Abstract: Existing datasets for narrative understanding often fail to represent the complexity and uncertainty of relationships in real-life social scenarios. To address this gap, we introduce a new benchmark, Conan, designed for extracting and analysing intricate character relation graphs from detective narratives. Specifically, we designed hierarchical relationship categories and manually extracted and annotated role-oriented relationships from the perspectives of various characters, incorporating both public relationships known to most characters and secret ones known to only a few. Our experiments with advanced Large Language Models (LLMs) like GPT-3.5, GPT-4, and Llama2 reveal their limitations in inferencing complex relationships and handling longer narratives. The combination of the Conan dataset and our pipeline strategy is geared towards understanding the ability of LLMs to comprehend nuanced relational dynamics in narrative contexts.