OpenRaiser/Intern-Atlas huggingface dataset:
MIMO Capacity Analysis and Channel Estimation for Electromagnetic Information Theory
Venue: arXiv cs.IT 2024
Year: 2024

Electromagnetic information theory (EIT) is an interdisciplinary subject that serves to integrate deterministic electromagnetic theory with stochastic Shannon's information theory. Existing EIT analysis operates in the continuous space domain, which is not aligned with the practical algorithms working in the discrete space domain. This mismatch leads to a significant difficulty in application of EIT methodologies to practical discrete space systems, which is called as the discrete-continuous gap in this paper. To bridge this gap, we establish the discrete-continuous correspondence with a prolate spheroidal wave function (PSWF)-based ergodic capacity analysis framework. Specifically, we state and prove some discrete-continuous correspondence lemmas to establish a firm theoretical connection between discrete information-theoretic quantities to their continuous counterparts. With these lemmas, we apply the PSWF ergodic capacity bound to advanced MIMO architectures such as continuous-aperture MIMO (CAP-MIMO) and extremely large-scale MIMO (XL-MIMO). From this PSWF capacity bound, we discover the capacity saturation phenomenon both theoretically and empirically. Although the growth of MIMO performance is fundamentally limited in this EIT-based analysis framework, we reveal new opportunities in MIMO channel estimation by exploiting the EIT knowledge about the channel. Inspired by the PSWF capacity bound, we utilize continuous PSWFs to improve the pilot design of discrete MIMO channel estimators, which is called as the PSWF channel estimator (PSWF-CE). Simulation results demonstrate improved performances of the proposed PSWF-CE, compared to traditional minimum mean squared error (MMSE) and compressed sensing-based estimators.
BAMO at SemEval-2024 Task 9: BRAINTEASER: A Novel Task Defying Common Sense
Venue: arXiv cs.CL 2024
Year: 2024

This paper outlines our approach to SemEval 2024 Task 9, BRAINTEASER: A Novel Task Defying Common Sense. The task aims to evaluate the ability of language models to think creatively. The dataset comprises multi-choice questions that challenge models to think "outside of the box". We fine-tune 2 models, BERT and RoBERTa Large. Next, we employ a Chain of Thought (CoT) zero-shot prompting approach with 6 large language models, such as GPT-3.5, Mixtral, and Llama2. Finally, we utilize ReConcile, a technique that employs a "round table conference" approach with multiple agents for zero-shot learning, to generate consensus answers among 3 selected language models. Our best method achieves an overall accuracy of 85 percent on the sentence puzzles subtask.
From cryptomarkets to the surface web: Scouting eBay for counterfeits
Venue: arXiv cs.SI 2024
Year: 2024

Detecting counterfeits on online marketplaces is challenging, and current methods struggle with the volume of sales on platforms like eBay, while cryptomarkets openly sell counterfeits. Leveraging information from 453 cryptomarket counterfeits, we automated a search for corresponding products on eBay, utilizing image and text similarity metrics. We collected data twice over 4-months to analyze changes with an average of 159 eBay products per cryptomarket item, totaling 134k products. We found identical products, which would warrant further investigation as to whether they are counterfeits. Results indicate increasing difficulty finding similar products over time, moderated by product type and origin. Future improved versions of the current system could be used to examine possible connections between cryptomarket and surface web listings more closely and could hold practical value in supporting the detection of counterfeits on the surface web.
GenHeld: Generating and Editing Handheld Objects
Venue: arXiv cs.CV 2024
Year: 2024

Grasping is an important human activity that has long been studied in robotics, computer vision, and cognitive science. Most existing works study grasping from the perspective of synthesizing hand poses conditioned on 3D or 2D object representations. We propose GenHeld to address the inverse problem of synthesizing held objects conditioned on 3D hand model or 2D image. Given a 3D model of hand, GenHeld 3D can select a plausible held object from a large dataset using compact object representations called object codes.The selected object is then positioned and oriented to form a plausible grasp without changing hand pose. If only a 2D hand image is available, GenHeld 2D can edit this image to add or replace a held object. GenHeld 2D operates by combining the abilities of GenHeld 3D with diffusion-based image editing. Results and experiments show that we outperform baselines and can generate plausible held objects in both 2D and 3D. Our experiments demonstrate that our method achieves high quality and plausibility of held object synthesis in both 3D and 2D.
Targeted Mining Precise-positioning Episode Rules
Venue: arXiv cs.DB 2024
Year: 2024

The era characterized by an exponential increase in data has led to the widespread adoption of data intelligence as a crucial task. Within the field of data mining, frequent episode mining has emerged as an effective tool for extracting valuable and essential information from event sequences. Various algorithms have been developed to discover frequent episodes and subsequently derive episode rules using the frequency function and anti-monotonicity principles. However, currently, there is a lack of algorithms specifically designed for mining episode rules that encompass user-specified query episodes. To address this challenge and enable the mining of target episode rules, we introduce the definition of targeted precise-positioning episode rules and formulate the problem of targeted mining precise-positioning episode rules. Most importantly, we develop an algorithm called Targeted Mining Precision Episode Rules (TaMIPER) to address the problem and optimize it using four proposed strategies, leading to significant reductions in both time and space resource requirements. As a result, TaMIPER offers high accuracy and efficiency in mining episode rules of user interest and holds promising potential for prediction tasks in various domains, such as weather observation, network intrusion, and e-commerce. Experimental results on six real datasets demonstrate the exceptional performance of TaMIPER.
SUMIE: A Synthetic Benchmark for Incremental Entity Summarization
Venue: arXiv cs.CL 2024
Year: 2024

No existing dataset adequately tests how well language models can incrementally update entity summaries - a crucial ability as these models rapidly advance. The Incremental Entity Summarization (IES) task is vital for maintaining accurate, up-to-date knowledge. To address this, we introduce SUMIE, a fully synthetic dataset designed to expose real-world IES challenges. This dataset effectively highlights problems like incorrect entity association and incomplete information presentation. Unlike common synthetic datasets, ours captures the complexity and nuances found in real-world data. We generate informative and diverse attributes, summaries, and unstructured paragraphs in sequence, ensuring high quality. The alignment between generated summaries and paragraphs exceeds 96%, confirming the dataset's quality. Extensive experiments demonstrate the dataset's difficulty - state-of-the-art LLMs struggle to update summaries with an F1 higher than 80.4%. We will open source the benchmark and the evaluation metrics to help the community make progress on IES tasks.
On Ambiguity and the Expressive Function of Law: The Role of Pragmatics in Smart Legal Ecosystems
Venue: arXiv cs.CY 2024
Year: 2024

This is a long paper, an essay, on ambiguity, pragmatics, legal ecosystems, and the expressive function of law. It is divided into two parts and fifteen sections. The first part (Pragmatics) addresses ambiguity from the perspective of linguistic and cognitive pragmatics in the legal field. The second part (Computing) deals with this issue from the point of view of human-centered design and artificial intelligence, specifically focusing on the notion and modelling of rules and what it means to comply with the rules. This is necessary for the scaffolding of smart legal ecosystems (SLE). I will develop this subject with the example of the architecture, information flows, and smart ecosystem of OPTIMAI, an EU project of Industry 4.0 for zero-defect manufacturing (Optimizing Manufacturing Processes through Artificial Intelligence and Virtualization).
A Novel Time Series-to-Image Encoding Approach for Weather Phenomena Classification
Venue: arXiv cs.CV 2024
Year: 2024

Rainfall estimation through the analysis of its impact on electromagnetic waves has sparked increasing interest in the research community. Recent studies have delved into its effects on cellular network performance, demonstrating the potential to forecast rainfall levels based on electromagnetic wave attenuation during precipitations. This paper aims to solve the problem of identifying the nature of specific weather phenomena from the received signal level (RSL) in 4G/LTE mobile terminals. Specifically, utilizing time-series data representing RSL, we propose a novel approach to encode time series as images and model the task as an image classification problem, which we finally address using convolutional neural networks (CNNs). The main benefit of the abovementioned procedure is the opportunity to utilize various data augmentation techniques simultaneously. This encompasses applying traditional approaches, such as moving averages, to the time series and enhancing the generated images. We have investigated various image data augmentation methods to identify the most effective combination for this scenario. In the upcoming sections, we will introduce the task of rainfall estimation and conduct a comprehensive analysis of the dataset used. Subsequently, we will formally propose a new approach for converting time series into images. To conclude, the paper's final section will present and discuss the experiments conducted, providing the reader with a brief yet comprehensive overview of the results.
Large Generative Graph Models
Venue: arXiv cs.LG 2024
Year: 2024

Large Generative Models (LGMs) such as GPT, Stable Diffusion, Sora, and Suno are trained on a huge amount of language corpus, images, videos, and audio that are extremely diverse from numerous domains. This training paradigm over diverse well-curated data lies at the heart of generating creative and sensible content. However, all previous graph generative models (e.g., GraphRNN, MDVAE, MoFlow, GDSS, and DiGress) have been trained only on one dataset each time, which cannot replicate the revolutionary success achieved by LGMs in other fields. To remedy this crucial gap, we propose a new class of graph generative model called Large Graph Generative Model (LGGM) that is trained on a large corpus of graphs (over 5000 graphs) from 13 different domains. We empirically demonstrate that the pre-trained LGGM has superior zero-shot generative capability to existing graph generative models. Furthermore, our pre-trained LGGM can be easily fine-tuned with graphs from target domains and demonstrate even better performance than those directly trained from scratch, behaving as a solid starting point for real-world customization. Inspired by Stable Diffusion, we further equip LGGM with the capability to generate graphs given text prompts (Text-to-Graph), such as the description of the network name and domain (i.e., "The power-1138-bus graph represents a network of buses in a power distribution system."), and network statistics (i.e., "The graph has a low average degree, suitable for modeling social media interactions."). This Text-to-Graph capability integrates the extensive world knowledge in the underlying language model, offering users fine-grained control of the generated graphs. We release the code, the model checkpoint, and the datasets at https://lggm-lg.github.io/.
PatchSVD: A Non-uniform SVD-based Image Compression Algorithm
Venue: arXiv cs.CV 2024
Year: 2024

Storing data is particularly a challenge when dealing with image data which often involves large file sizes due to the high resolution and complexity of images. Efficient image compression algorithms are crucial to better manage data storage costs. In this paper, we propose a novel region-based lossy image compression technique, called PatchSVD, based on the Singular Value Decomposition (SVD) algorithm. We show through experiments that PatchSVD outperforms SVD-based image compression with respect to three popular image compression metrics. Moreover, we compare PatchSVD compression artifacts with those of Joint Photographic Experts Group (JPEG) and SVD-based image compression and illustrate some cases where PatchSVD compression artifacts are preferable compared to JPEG and SVD artifacts.
Fight Scene Detection for Movie Highlight Generation System
Venue: arXiv cs.CV 2024
Year: 2024

In this paper of a research based project, using Bidirectional Long Short-Term Memory (BiLSTM) networks, we provide a novel Fight Scene Detection (FSD) model which can be used for Movie Highlight Generation Systems (MHGS) based on deep learning and Neural Networks . Movies usually have Fight Scenes to keep the audience amazed. For trailer generation, or any other application of Highlight generation, it is very tidious to first identify all such scenes manually and then compile them to generate a highlight serving the purpose. Our proposed FSD system utilises temporal characteristics of the movie scenes and thus is capable to automatically identify fight scenes. Thereby helping in the effective production of captivating movie highlights. We observe that the proposed solution features 93.5% accuracy and is higher than 2D CNN with Hough Forests which being 92% accurate and is significantly higher than 3D CNN which features an accuracy of 65%.
Evaluating the Effectiveness of Data Augmentation for Emotion Classification in Low-Resource Settings
Venue: arXiv cs.LG 2024
Year: 2024

Data augmentation has the potential to improve the performance of machine learning models by increasing the amount of training data available. In this study, we evaluated the effectiveness of different data augmentation techniques for a multi-label emotion classification task using a low-resource dataset. Our results showed that Back Translation outperformed autoencoder-based approaches and that generating multiple examples per training instance led to further performance improvement. In addition, we found that Back Translation generated the most diverse set of unigrams and trigrams. These findings demonstrate the utility of Back Translation in enhancing the performance of emotion classification models in resource-limited situations.
AHA! Strategies for Gaining Insights into Software Design
Venue: arXiv cs.SE 2024
Year: 2024

These patterns describe the strategies I use to find novel or unorthodox insights in the area of software design and research. The patterns are driven by inconsistencies between what we say and what we do, and they provide techniques for finding actionable insights to address these inconsistencies. These insights may help to identify research opportunities; they may stimulate critiques of either research or practice; they may suggest new methods.
A Matrix Exponential Generalization of the Laplace Transform of Poisson Shot Noise
Venue: arXiv cs.IT 2024
Year: 2024

We consider a generalization of the Laplace transform of Poisson shot noise defined as an integral transform with respect to a matrix exponential. We denote this as the matrix Laplace transform and establish that it is in general a matrix function extension of the scalar Laplace transform. We show that the matrix Laplace transform of Poisson shot noise admits an expression analogous to that implied by Campbell's theorem. We demonstrate the utility of this generalization of Campbell's theorem in two important applications: the characterization of a Poisson shot noise process and the derivation of the complementary CDF (CCDF) and meta-distribution of signal-to-interference-and-noise (SINR) models in Poisson networks. In the former application, we demonstrate how the higher order moments of Poisson shot noise may be obtained directly from the elements of its matrix Laplace transform. We further show how the CCDF of this object may be bounded using a summation of the first row of its matrix Laplace transform. For the latter application, we show how the CCDF of SINR models with phase-type distributed desired signal power may be obtained via an expectation of the matrix Laplace transform of the interference and noise, analogous to the canonical case of SINR models with Rayleigh fading. Additionally, when the power of the desired signal is exponentially distributed, we establish that the meta-distribution may be obtained in terms of the limit of a sequence expressed in terms of the matrix Laplace transform of a related Poisson shot noise process.
On Subjective Uncertainty Quantification and Calibration in Natural Language Generation
Venue: arXiv cs.CL 2024
Year: 2024

Applications of large language models often involve the generation of free-form responses, in which case uncertainty quantification becomes challenging. This is due to the need to identify task-specific uncertainties (e.g., about the semantics) which appears difficult to define in general cases. This work addresses these challenges from a perspective of Bayesian decision theory, starting from the assumption that our utility is characterized by a similarity measure that compares a generated response with a hypothetical true response. We discuss how this assumption enables principled quantification of the model's subjective uncertainty and its calibration. We further derive a measure for epistemic uncertainty, based on a missing data perspective and its characterization as an excess risk. The proposed methods can be applied to black-box language models. We illustrate the methods on question answering and machine translation tasks. Our experiments provide a principled evaluation of task-specific calibration, and demonstrate that epistemic uncertainty offers a promising deferral strategy for efficient data acquisition in in-context learning.
TLEX: An Efficient Method for Extracting Exact Timelines from TimeML Temporal Graphs
Venue: arXiv cs.CL 2024
Year: 2024

A timeline provides a total ordering of events and times, and is useful for a number of natural language understanding tasks. However, qualitative temporal graphs that can be derived directly from text -- such as TimeML annotations -- usually explicitly reveal only partial orderings of events and times. In this work, we apply prior work on solving point algebra problems to the task of extracting timelines from TimeML annotated texts, and develop an exact, end-to-end solution which we call TLEX (TimeLine EXtraction). TLEX transforms TimeML annotations into a collection of timelines arranged in a trunk-and-branch structure. Like what has been done in prior work, TLEX checks the consistency of the temporal graph and solves it; however, it adds two novel functionalities. First, it identifies specific relations involved in an inconsistency (which could then be manually corrected) and, second, TLEX performs a novel identification of sections of the timelines that have indeterminate order, information critical for downstream tasks such as aligning events from different timelines. We provide detailed descriptions and analysis of the algorithmic components in TLEX, and conduct experimental evaluations by applying TLEX to 385 TimeML annotated texts from four corpora. We show that 123 of the texts are inconsistent, 181 of them have more than one ``real world'' or main timeline, and there are 2,541 indeterminate sections across all four corpora. A sampling evaluation showed that TLEX is 98--100% accurate with 95% confidence along five dimensions: the ordering of time-points, the number of main timelines, the placement of time-points on main versus subordinate timelines, the connecting point of branch timelines, and the location of the indeterminate sections. We provide a reference implementation of TLEX, the extracted timelines for all texts, and the manual corrections of the inconsistent texts.
SuperPos-Prompt: Enhancing Soft Prompt Tuning of Language Models with Superposition of Multi Token Embeddings
Venue: arXiv cs.CL 2024
Year: 2024

Soft prompt tuning techniques have recently gained traction as an effective strategy for the parameter-efficient tuning of pretrained language models, particularly minimizing the required adjustment of model parameters. Despite their growing use, achieving optimal tuning with soft prompts, especially for smaller datasets, remains a substantial challenge. This study makes two contributions in this domain: (i) we introduce SuperPos-Prompt, a new reparameterization technique employing the superposition of multiple pretrained vocabulary embeddings to improve the learning of soft prompts. Our experiments across several GLUE and SuperGLUE benchmarks consistently highlight SuperPos-Prompt's superiority over Residual Prompt tuning, exhibiting an average score increase of $+6.4$ in T5-Small and $+5.0$ in T5-Base along with a faster convergence. Remarkably, SuperPos-Prompt occasionally outperforms even full fine-tuning methods. (ii) Additionally, we demonstrate enhanced performance and rapid convergence by omitting dropouts from the frozen network, yielding consistent improvements across various scenarios and tuning methods.
Beyond Efficiency: Scaling AI Sustainably
Venue: arXiv cs.LG 2024
Year: 2024

Barroso's seminal contributions in energy-proportional warehouse-scale computing launched an era where modern datacenters have become more energy efficient and cost effective than ever before. At the same time, modern AI applications have driven ever-increasing demands in computing, highlighting the importance of optimizing efficiency across the entire deep learning model development cycle. This paper characterizes the carbon impact of AI, including both operational carbon emissions from training and inference as well as embodied carbon emissions from datacenter construction and hardware manufacturing. We highlight key efficiency optimization opportunities for cutting-edge AI technologies, from deep learning recommendation models to multi-modal generative AI tasks. To scale AI sustainably, we must also go beyond efficiency and optimize across the life cycle of computing infrastructures, from hardware manufacturing to datacenter operations and end-of-life processing for the hardware.