pwc-archive/papers-with-abstracts huggingface dataset:
A provable initialization and robust clustering method for general mixture models
Authors: Soham Jana, Jianqing Fan, Sanjeev Kulkarni
Date: 2024-01-10 00:00:00
Tasks: Clustering

Clustering is a fundamental tool in statistical machine learning in the presence of heterogeneous data. Most recent results focus primarily on optimal mislabeling guarantees when data are distributed around centroids with sub-Gaussian errors. Yet, the restrictive sub-Gaussian model is often invalid in practice since various real-world applications exhibit heavy tail distributions around the centroids or suffer from possible adversarial attacks that call for robust clustering with a robust data-driven initialization. In this paper, we present initialization and subsequent clustering methods that provably guarantee near-optimal mislabeling for general mixture models when the number of clusters and data dimensions are finite. We first introduce a hybrid clustering technique with a novel multivariate trimmed mean type centroid estimate to produce mislabeling guarantees under a weak initialization condition for general error distributions around the centroids. A matching lower bound is derived, up to factors depending on the number of clusters. In addition, our approach also produces similar mislabeling guarantees even in the presence of adversarial outliers. Our results reduce to the sub-Gaussian case in finite dimensions when errors follow sub-Gaussian distributions. To solve the problem thoroughly, we also present novel data-driven robust initialization techniques and show that, with probabilities approaching one, these initial centroid estimates are sufficiently good for the subsequent clustering algorithm to achieve the optimal mislabeling rates. Furthermore, we demonstrate that the Lloyd algorithm is suboptimal for more than two clusters even when errors are Gaussian and for two clusters when error distributions have heavy tails. Both simulated data and real data examples further support our robust initialization procedure and clustering algorithm.
What Else Would I Like? A User Simulator using Alternatives for Improved Evaluation of Fashion Conversational Recommendation Systems
Authors: Maria Vlachou, Craig Macdonald
Date: 2024-01-11 00:00:00
Tasks: Conversational Recommendation, Image Captioning, Recommendation Systems

In Conversational Recommendation Systems (CRS), a user can provide feedback on recommended items at each interaction turn, leading the CRS towards more desirable recommendations. Currently, different types of CRS offer various possibilities for feedback, i.e., natural language feedback, or answering clarifying questions. In most cases, a user simulator is employed for training as well as evaluating the CRS. Such user simulators typically critique the current retrieved items based on knowledge of a single target item. Still, evaluating systems in offline settings with simulators suffers from problems, such as focusing entirely on a single target item (not addressing the exploratory nature of a recommender system), and exhibiting extreme patience (consistent feedback over a large number of turns). To overcome these limitations, we obtain extra judgements for a selection of alternative items in common CRS datasets, namely Shoes and Fashion IQ Dresses. Going further, we propose improved user simulators that allow simulated users not only to express their preferences about alternative items to their original target, but also to change their mind and level of patience. In our experiments using the relative image captioning CRS setting and different CRS models, we find that using the knowledge of alternatives by the simulator can have a considerable impact on the evaluation of existing CRS models, specifically that the existing single-target evaluation underestimates their effectiveness, and when simulated users are allowed to instead consider alternatives, the system can rapidly respond to more quickly satisfy the user.
Lifelogging As An Extreme Form of Personal Information Management -- What Lessons To Learn
Authors: Ly-Duyen Tran, Cathal Gurrin, Alan F. Smeaton
Date: 2024-01-11 00:00:00
Tasks: Benchmarking, Form, Management

Personal data includes the digital footprints that we leave behind as part of our everyday activities, both online and offline in the real world. It includes data we collect ourselves, such as from wearables, as well as the data collected by others about our online behaviour and activities. Sometimes we are able to use the personal data we ourselves collect, in order to examine some parts of our lives but for the most part, our personal data is leveraged by third parties including internet companies, for services like targeted advertising and recommendations. Lifelogging is a form of extreme personal data gathering and in this article we present an overview of the tools used to manage access to lifelogs as demonstrated at the most recent of the annual Lifelog Search Challenge benchmarking workshops. Here, experimental systems are showcased in live, real time information seeking tasks by real users. This overview of these systems' capabilities show the range of possibilities for accessing our own personal data which may, in time, become more easily available as consumer-level services.
Large Language Models vs. Search Engines: Evaluating User Preferences Across Varied Information Retrieval Scenarios
Authors: Kevin Matthe Caramancion
Date: 2024-01-11 00:00:00
Tasks: Information Retrieval, Retrieval

This study embarked on a comprehensive exploration of user preferences between Search Engines and Large Language Models (LLMs) in the context of various information retrieval scenarios. Conducted with a sample size of 100 internet users (N=100) from across the United States, the research delved into 20 distinct use cases ranging from factual searches, such as looking up COVID-19 guidelines, to more subjective tasks, like seeking interpretations of complex concepts in layman's terms. Participants were asked to state their preference between using a traditional search engine or an LLM for each scenario. This approach allowed for a nuanced understanding of how users perceive and utilize these two predominant digital tools in differing contexts. The use cases were carefully selected to cover a broad spectrum of typical online queries, thus ensuring a comprehensive analysis of user preferences. The findings reveal intriguing patterns in user choices, highlighting a clear tendency for participants to favor search engines for direct, fact-based queries, while LLMs were more often preferred for tasks requiring nuanced understanding and language processing. These results offer valuable insights into the current state of digital information retrieval and pave the way for future innovations in this field. This study not only sheds light on the specific contexts in which each tool is favored but also hints at the potential for developing hybrid models that leverage the strengths of both search engines and LLMs. The insights gained from this research are pivotal for developers, researchers, and policymakers in understanding the evolving landscape of digital information retrieval and user interaction with these technologies.
Attention Is Not the Only Choice: Counterfactual Reasoning for Path-Based Explainable Recommendation
Authors: Yicong Li, Xiangguo Sun, Hongxu Chen, Sixiao Zhang, Yu Yang, Guandong Xu
Date: 2024-01-11 00:00:00
Tasks: counterfactual, Counterfactual Reasoning, Explainable Recommendation, Learning Theory

Compared with only pursuing recommendation accuracy, the explainability of a recommendation model has drawn more attention in recent years. Many graph-based recommendations resort to informative paths with the attention mechanism for the explanation. Unfortunately, these attention weights are intentionally designed for model accuracy but not explainability. Recently, some researchers have started to question attention-based explainability because the attention weights are unstable for different reproductions, and they may not always align with human intuition. Inspired by the counterfactual reasoning from causality learning theory, we propose a novel explainable framework targeting path-based recommendations, wherein the explainable weights of paths are learned to replace attention weights. Specifically, we design two counterfactual reasoning algorithms from both path representation and path topological structure perspectives. Moreover, unlike traditional case studies, we also propose a package of explainability evaluation solutions with both qualitative and quantitative methods. We conduct extensive experiments on three real-world datasets, the results of which further demonstrate the effectiveness and reliability of our method.
A Toolbox for Modelling Engagement with Educational Videos
Authors: Yuxiang Qiu, Karim Djemili, Denis Elezi, Aaneel Shalman, María Pérez-Ortiz, Emine Yilmaz, John Shawe-Taylor, Sahan Bulathwela
Date: 2023-12-30 00:00:00

With the advancement and utility of Artificial Intelligence (AI), personalising education to a global population could be a cornerstone of new educational systems in the future. This work presents the PEEKC dataset and the TrueLearn Python library, which contains a dataset and a series of online learner state models that are essential to facilitate research on learner engagement modelling.TrueLearn family of models was designed following the "open learner" concept, using humanly-intuitive user representations. This family of scalable, online models also help end-users visualise the learner models, which may in the future facilitate user interaction with their models/recommenders. The extensive documentation and coding examples make the library highly accessible to both machine learning developers and educational data mining and learning analytics practitioners. The experiments show the utility of both the dataset and the library with predictive performance significantly exceeding comparative baseline models. The dataset contains a large amount of AI-related educational videos, which are of interest for building and validating AI-specific educational recommenders.
ANALYTiC: Understanding Decision Boundaries and Dimensionality Reduction in Machine Learning
Authors: Salman Haidri
Date: 2023-12-29 00:00:00
Tasks: Active Learning, Dimensionality Reduction

The advent of compact, handheld devices has given us a pool of tracked movement data that could be used to infer trends and patterns that can be made to use. With this flooding of various trajectory data of animals, humans, vehicles, etc., the idea of ANALYTiC originated, using active learning to infer semantic annotations from the trajectories by learning from sets of labeled data. This study explores the application of dimensionality reduction and decision boundaries in combination with the already present active learning, highlighting patterns and clusters in data. We test these features with three different trajectory datasets with objective of exploiting the the already labeled data and enhance their interpretability. Our experimental analysis exemplifies the potential of these combined methodologies in improving the efficiency and accuracy of trajectory labeling. This study serves as a stepping-stone towards the broader integration of machine learning and visual methods in context of movement data analysis.
Multi-Dimensional Framework for EEG Signal Processing and Denoising Through Tensor-based Architecture
Authors: Aryan Govil, Eric Yao, Christina R. Borao
Date: 2024-01-10 00:00:00
Tasks: Denoising, EEG, Seizure Detection

Electroencephalography (EEG) stands as a crucial tool in neuroscientific research and clinical diagnostics, providing valuable insights into the electrical activities of the brain. Traditional EEG signal processing techniques, predominantly linear and constrained to time-frequency analysis, often fail to capture the intricate, dynamic nature of brain signals. This paper introduces a tensor-based multi-dimensional framework for EEG signal processing and denoising, aimed at overcoming the limitations of current methods. Utilizing the advanced mathematical construct of tensors, this framework allows for a more holistic representation and analysis of EEG data, encompassing multiple dimensions such as time, electrode space, and frequency bands. We propose innovative algorithms for multi-dimensional Fourier transforms and adaptive thresholding, specifically tailored to address the challenges of non-stationary noise and complex signal artifacts in EEG data. The framework is further enriched with a time-slicing algorithm that facilitates real-time analysis, crucial for applications like seizure detection and brain-computer interfacing. Theoretical formulations and simulated scenarios demonstrate the potential of this framework in significantly enhancing the accuracy, efficiency, and speed of EEG signal processing. This approach not only holds promise for advanced EEG analysis but also sets the stage for future integrations with other neuroimaging modalities, paving the way for comprehensive and nuanced understanding of brain function.
Clustering-Guided Class Activation for Weakly Supervised Semantic Segmentation
Authors: Yeong Woo Kim, Wonjun Kim
Date: 2024-01-05 00:00:00
Proceeding: Access 2024 1
Tasks: Clustering, Object, Semantic Segmentation, Weakly supervised Semantic Segmentation, Weakly-Supervised Semantic Segmentation

Weakly-supervised semantic segmentation (WSSS) methods via transformer have been actively studied by leveraging their strong capability to capture the global context. However, since the activation function only highlights a few tokens in the self-attention mechanism of the transformer, these methods still suffer from the sparse attention map, which leads to the generation of incomplete pseudo labels. In this paper, we propose a novel class activation scheme that is able to uniformly highlight the whole object region. The key idea of the proposed method is to activate the object region by following the guide of clusters, which are formed by combining similar image features of the object. Specifically, the clustering-guided class activation map (ClusterCAM) is generated from the proposed clustering-based attention module, and highly responsive regions in this map are then adopted to activate target objects in the encoded feature space. This helps the model to explore the entire region of the target object by using the semantic proximity between patch tokens extracted from the same object. Based on this, we design an end-to-end WSSS framework that can simultaneously train classification and segmentation networks in a single-stage manner. Experimental results on benchmark datasets show that our proposed method significantly outperforms previous WSSS methods, including several multi-stage approaches. The code and model are publicly available at: https://github.com/DCVL-WSSS/ClusterCAM.
EMG subspace alignment and visualization for cross-subject hand gesture classification
Authors: Martin Colot, Cédric Simar, Mathieu Petieau, Ana Maria Cebolla Alvarez, Guy Cheron, Gianluca Bontempi
Date: 2023-12-18 00:00:00
Tasks: Gesture Recognition, Hand Gesture Recognition, Hand-Gesture Recognition

Electromyograms (EMG)-based hand gesture recognition systems are a promising technology for human/machine interfaces. However, one of their main limitations is the long calibration time that is typically required to handle new users. The paper discusses and analyses the challenge of cross-subject generalization thanks to an original dataset containing the EMG signals of 14 human subjects during hand gestures. The experimental results show that, though an accurate generalization based on pooling multiple subjects is hardly achievable, it is possible to improve the cross-subject estimation by identifying a robust low-dimensional subspace for multiple subjects and aligning it to a target subject. A visualization of the subspace enables us to provide insights for the improvement of cross-subject generalization with EMG signals.
Blockchain-enabled fraud discovery through abnormal smart contract detection on Ethereum
Authors: Lin Liu a, ∗, Wei-Tek Tsai a, B, Md. Zakirul Alam Bhuiyan c, Hao Peng d, Mingsheng Liu e
Date: 2023-03-01 00:00:00
Proceeding: https://linkinghub.elsevier.com/retrieve/pii/S0167739X21003319 2023 3
Tasks: Anomaly Detection

Anomaly detection for smart contracts can effectively prevent hidden security risks such as financial
fraud, illegal financing, and money laundering. Ethereum is currently the largest platform for smart
contracts, and anomaly detection is imminent. However, the data related to smart contracts is huge
and contains complex objects and relationships. It is impossible to extract high-order attributes and
low efficiency using traditional methods. The key to reduce fraud is extracting features from complex
smart contracts and effectively identifying abnormal contracts. Therefore, this paper constructs a
Heterogeneous Graph Transformer Networks (S_HGTNs) suitable for smart contract anomaly detection
to detect financial fraud on the Ethereum platform. For feature representation, this paper first extracts
the features to construct a Heterogeneous Information Network (HIN) for smart contract, and uses the
relationship matrix obtained from the meta-path learned in the transformer network as the input of
the convolution network, and finally uses the node embedding for classification tasks. The classification
results show that this model performs better than the traditional model and the standard deviation is
small, which proves the effectiveness and stability of the model.
STR-Cert: Robustness Certification for Deep Text Recognition on Deep Learning Pipelines and Vision Transformers
Authors: Daqian Shao, Lukas Fesser, Marta Kwiatkowska
Date: 2023-11-28 00:00:00
Tasks: Scene Text Recognition

Robustness certification, which aims to formally certify the predictions of neural networks against adversarial inputs, has become an integral part of important tool for safety-critical applications. Despite considerable progress, existing certification methods are limited to elementary architectures, such as convolutional networks, recurrent networks and recently Transformers, on benchmark datasets such as MNIST. In this paper, we focus on the robustness certification of scene text recognition (STR), which is a complex and extensively deployed image-based sequence prediction problem. We tackle three types of STR model architectures, including the standard STR pipelines and the Vision Transformer. We propose STR-Cert, the first certification method for STR models, by significantly extending the DeepPoly polyhedral verification framework via deriving novel polyhedral bounds and algorithms for key STR model components. Finally, we certify and compare STR models on six datasets, demonstrating the efficiency and scalability of robustness certification, particularly for the Vision Transformer.
MicroGlam: Microscopic Skin Image Dataset with Cosmetics
Authors: Toby Chong, Alina Chadwick, I-Chao Shen, Haoran Xie, Takeo Igarashi
Date: 2023-11-29 00:00:00
Tasks: Image-to-Image Translation

In this paper, we present a cosmetic-specific skin image dataset. It consists of skin images from $45$ patches ($5$ skin patches each from $9$ participants) of size $8mm^*8mm$ under three cosmetic products (i.e., foundation, blusher, and highlighter). We designed a novel capturing device inspired by Light Stage. Using the device, we captured over $600$ images of each skin patch under diverse lighting conditions in $30$ seconds. We repeated the process for the same skin patch under three cosmetic products. Finally, we demonstrate the viability of the dataset with an image-to-image translation-based pipeline for cosmetic rendering and compared our data-driven approach to an existing cosmetic rendering method.
DISTWAR: Fast Differentiable Rendering on Raster-based Rendering Pipelines
Authors: Sankeerth Durvasula, Adrian Zhao, Fan Chen, Ruofan Liang, Pawan Kumar Sanjaya, Nandita Vijaykumar
Date: 2023-12-01 00:00:00

Differentiable rendering is a technique used in an important emerging class of visual computing applications that involves representing a 3D scene as a model that is trained from 2D images using gradient descent. Recent works (e.g. 3D Gaussian Splatting) use a rasterization pipeline to enable rendering high quality photo-realistic imagery at high speeds from these learned 3D models. These methods have been demonstrated to be very promising, providing state-of-art quality for many important tasks. However, training a model to represent a scene is still a time-consuming task even when using powerful GPUs. In this work, we observe that the gradient computation phase during training is a significant bottleneck on GPUs due to the large number of atomic operations that need to be processed. These atomic operations overwhelm atomic units in the L2 partitions causing stalls. To address this challenge, we leverage the observations that during the gradient computation: (1) for most warps, all threads atomically update the same memory locations; and (2) warps generate varying amounts of atomic traffic (since some threads may be inactive). We propose DISTWAR, a software-approach to accelerate atomic operations based on two key ideas: First, we enable warp-level reduction of threads at the SM sub-cores using registers to leverage the locality in intra-warp atomic updates. Second, we distribute the atomic computation between the warp-level reduction at the SM and the L2 atomic units to increase the throughput of atomic computation. Warps with many threads performing atomic updates to the same memory locations are scheduled at the SM, and the rest using L2 atomic units. We implement DISTWAR using existing warp-level primitives. We evaluate DISTWAR on widely used raster-based differentiable rendering workloads. We demonstrate significant speedups of 2.44x on average (up to 5.7x).
Generalized Categories Discovery for Long-tailed Recognition
Authors: Ziyun Li, Christoph Meinel, Haojin Yang
Date: 2023-12-04 00:00:00

Generalized Class Discovery (GCD) plays a pivotal role in discerning both known and unknown categories from unlabeled datasets by harnessing the insights derived from a labeled set comprising recognized classes. A significant limitation in prevailing GCD methods is their presumption of an equitably distributed category occurrence in unlabeled data. Contrary to this assumption, visual classes in natural environments typically exhibit a long-tailed distribution, with known or prevalent categories surfacing more frequently than their rarer counterparts. Our research endeavors to bridge this disconnect by focusing on the long-tailed Generalized Category Discovery (Long-tailed GCD) paradigm, which echoes the innate imbalances of real-world unlabeled datasets. In response to the unique challenges posed by Long-tailed GCD, we present a robust methodology anchored in two strategic regularizations: (i) a reweighting mechanism that bolsters the prominence of less-represented, tail-end categories, and (ii) a class prior constraint that aligns with the anticipated class distribution. Comprehensive experiments reveal that our proposed method surpasses previous state-of-the-art GCD methods by achieving an improvement of approximately 6 - 9% on ImageNet100 and competitive performance on CIFAR100.