OpenRaiser/Intern-Atlas huggingface dataset:
Extracting Digital Biomarkers for Unobtrusive Stress State Screening from Multimodal Wearable Data
Venue: arXiv cs.LG 2023
Year: 2023

With the development of wearable technologies, a new kind of healthcare data has become valuable as medical information. These data provide meaningful information regarding an individual's physiological and psychological states, such as activity level, mood, stress, and cognitive health. These biomarkers are named digital since they are collected from digital devices integrated with various sensors. In this study, we explore digital biomarkers related to stress modality by examining data collected from mobile phones and smartwatches. We utilize machine learning techniques on the Tesserae dataset, precisely Random Forest, to extract stress biomarkers. Using feature selection techniques, we utilize weather, activity, heart rate (HR), stress, sleep, and location (work-home) measurements from wearables to determine the most important stress-related biomarkers. We believe we contribute to interpreting stress biomarkers with a high range of features from different devices. In addition, we classify the $5$ different stress levels with the most important features, and our results show that we can achieve $85\%$ overall class accuracy by adjusting class imbalance and adding extra features related to personality characteristics. We perform similar and even better results in recognizing stress states with digital biomarkers in a daily-life scenario targeting a higher number of classes compared to the related studies.
Onsets and Velocities: Affordable Real-Time Piano Transcription Using Convolutional Neural Networks
Venue: arXiv cs.SD 2023
Year: 2023

Polyphonic Piano Transcription has recently experienced substantial progress, driven by the use of sophisticated Deep Learning approaches and the introduction of new subtasks such as note onset, offset, velocity and pedal detection. This progress was coupled with an increased complexity and size of the proposed models, typically relying on non-realtime components and high-resolution data. In this work we focus on onset and velocity detection, showing that a substantially smaller and simpler convolutional approach, using lower temporal resolution (24ms), is still competitive: our proposed ONSETS&VELOCITIES model achieves state-of-the-art performance on the MAESTRO dataset for onset detection (F1=96.78%) and sets a good novel baseline for onset+velocity (F1=94.50%), while having ~3.1M parameters and maintaining real-time capabilities on modest commodity hardware. We provide open-source code to reproduce our results and a real-time demo with a pretrained model.
A robust method for reliability updating with equality information using sequential adaptive importance sampling
Venue: arXiv cs.LG 2023
Year: 2023

Reliability updating refers to a problem that integrates Bayesian updating technique with structural reliability analysis and cannot be directly solved by structural reliability methods (SRMs) when it involves equality information. The state-of-the-art approaches transform equality information into inequality information by introducing an auxiliary standard normal parameter. These methods, however, encounter the loss of computational efficiency due to the difficulty in finding the maximum of the likelihood function, the large coefficient of variation (COV) associated with the posterior failure probability and the inapplicability to dynamic updating problems where new information is constantly available. To overcome these limitations, this paper proposes an innovative method called RU-SAIS (reliability updating using sequential adaptive importance sampling), which combines elements of sequential importance sampling and K-means clustering to construct a series of important sampling densities (ISDs) using Gaussian mixture. The last ISD of the sequence is further adaptively modified through application of the cross entropy method. The performance of RU-SAIS is demonstrated by three examples. Results show that RU-SAIS achieves a more accurate and robust estimator of the posterior failure probability than the existing methods such as subset simulation.
Streaming Kernel PCA Algorithm With Small Space
Venue: arXiv cs.LG 2023
Year: 2023

Principal Component Analysis (PCA) is a widely used technique in machine learning, data analysis and signal processing. With the increase in the size and complexity of datasets, it has become important to develop low-space usage algorithms for PCA. Streaming PCA has gained significant attention in recent years, as it can handle large datasets efficiently. The kernel method, which is commonly used in learning algorithms such as Support Vector Machines (SVMs), has also been applied in PCA algorithms. We propose a streaming algorithm for Kernel PCA problems based on the traditional scheme by Oja. Our algorithm addresses the challenge of reducing the memory usage of PCA while maintaining its accuracy. We analyze the performance of our algorithm by studying the conditions under which it succeeds. Specifically, we show that, when the spectral ratio $R := λ_1/λ_2$ of the target covariance matrix is lower bounded by $C \cdot \log n\cdot \log d$, the streaming PCA can be solved with $O(d)$ space cost. Our proposed algorithm has several advantages over existing methods. First, it is a streaming algorithm that can handle large datasets efficiently. Second, it employs the kernel method, which allows it to capture complex nonlinear relationships among data points. Third, it has a low-space usage, making it suitable for applications where memory is limited.
"How to make them stay?" -- Diverse Counterfactual Explanations of Employee Attrition
Venue: arXiv cs.LG 2023
Year: 2023

Employee attrition is an important and complex problem that can directly affect an organisation's competitiveness and performance. Explaining the reasons why employees leave an organisation is a key human resource management challenge due to the high costs and time required to attract and keep talented employees. Businesses therefore aim to increase employee retention rates to minimise their costs and maximise their performance. Machine learning (ML) has been applied in various aspects of human resource management including attrition prediction to provide businesses with insights on proactive measures on how to prevent talented employees from quitting. Among these ML methods, the best performance has been reported by ensemble or deep neural networks, which by nature constitute black box techniques and thus cannot be easily interpreted. To enable the understanding of these models' reasoning several explainability frameworks have been proposed. Counterfactual explanation methods have attracted considerable attention in recent years since they can be used to explain and recommend actions to be performed to obtain the desired outcome. However current counterfactual explanations methods focus on optimising the changes to be made on individual cases to achieve the desired outcome. In the attrition problem it is important to be able to foresee what would be the effect of an organisation's action to a group of employees where the goal is to prevent them from leaving the company. Therefore, in this paper we propose the use of counterfactual explanations focusing on multiple attrition cases from historical data, to identify the optimum interventions that an organisation needs to make to its practices/policies to prevent or minimise attrition probability for these cases.
Controlled Diversity with Preference : Towards Learning a Diverse Set of Desired Skills
Venue: arXiv cs.LG 2023
Year: 2023

Autonomously learning diverse behaviors without an extrinsic reward signal has been a problem of interest in reinforcement learning. However, the nature of learning in such mechanisms is unconstrained, often resulting in the accumulation of several unusable, unsafe or misaligned skills. In order to avoid such issues and ensure the discovery of safe and human-aligned skills, it is necessary to incorporate humans into the unsupervised training process, which remains a largely unexplored research area. In this work, we propose Controlled Diversity with Preference (CDP), a novel, collaborative human-guided mechanism for an agent to learn a set of skills that is diverse as well as desirable. The key principle is to restrict the discovery of skills to those regions that are deemed to be desirable as per a preference model trained using human preference labels on trajectory pairs. We evaluate our approach on 2D navigation and Mujoco environments and demonstrate the ability to discover diverse, yet desirable skills.
Followback Clusters, Satellite Audiences, and Bridge Nodes: Coengagement Networks for the 2020 US Election
Venue: arXiv cs.SI 2023
Year: 2023

The 2020 United States presidential election was, and has continued to be, the focus of pervasive and persistent mis- and disinformation spreading through our media ecosystems, including social media. This event has driven the collection and analysis of large, directed social network datasets, but such datasets can resist intuitive understanding. In such large datasets, the overwhelming number of nodes and edges present in typical representations create visual artifacts, such as densely overlapping edges and tightly-packed formations of low-degree nodes, which obscure many features of more practical interest. We apply a method, coengagement transformations, to convert such networks of social data into tractable images. Intuitively, this approach allows for parameterized network visualizations that make shared audiences of engaged viewers salient to viewers. Using the interpretative capabilities of this method, we perform an extensive case study of the 2020 United States presidential election on Twitter, contributing an empirical analysis of coengagement. By creating and contrasting different networks at different parameter sets, we define and characterize several structures in this discourse network, including bridging accounts, satellite audiences, and followback communities. We discuss the importance and implications of these empirical network features in this context. In addition, we release open-source code for creating coengagement networks from Twitter and other structured interaction data.
Arion: Arithmetization-Oriented Permutation and Hashing from Generalized Triangular Dynamical Systems
Venue: arXiv cs.CR 2023
Year: 2023

In this paper we propose the (keyed) permutation Arion and the hash function ArionHash over $\mathbb{F}_p$ for odd and particularly large primes. The design of Arion is based on the newly introduced Generalized Triangular Dynamical System (GTDS), which provides a new algebraic framework for constructing (keyed) permutation using polynomials over a finite field. At round level Arion is the first design which is instantiated using the new GTDS. We provide extensive security analysis of our construction including algebraic cryptanalysis (e.g. interpolation and Gröbner basis attacks) that are particularly decisive in assessing the security of permutations and hash functions over $\mathbb{F}_p$. From an application perspective, ArionHash aims for efficient implementation in zkSNARK protocols and Zero-Knowledge proof systems. For this purpose, we exploit that CCZ-equivalence of graphs can lead to a more efficient implementation of Arithmetization-Oriented primitives. We compare the efficiency of ArionHash in R1CS and Plonk settings with other hash functions such as Poseidon, Anemoi and Griffin. For demonstrating the practical efficiency of ArionHash we implemented it with the zkSNARK libraries libsnark and Dusk Network Plonk. Our result shows that ArionHash is significantly faster than Poseidon - a hash function designed for zero-knowledge proof systems. We also found that an aggressive version of ArionHash is considerably faster than Anemoi and Griffin in a practical zkSNARK setting.
Optimizing Utility-Energy Efficiency for the Metaverse over Wireless Networks under Physical Layer Security
Venue: arXiv cs.SI 2023
Year: 2023

The Metaverse, an emerging digital space, is expected to offer various services mirroring the real world. Wireless communications for mobile Metaverse users should be tailored to meet the following user characteristics: 1) emphasizing application-specific perceptual utility instead of simply the transmission rate, 2) concerned with energy efficiency due to the limited device battery and energy intensiveness of some applications, and 3) caring about security as the applications may involve sensitive personal data. To this end, this paper incorporates application-specific utility, energy efficiency, and physical-layer security (PLS) into the studied optimization in a wireless network for the Metaverse. Specifically, after introducing utility-energy efficiency (UEE) to represent each Metaverse user's application-specific objective under PLS, we formulate an optimization to maximize the network's weighted sum-UEE by deciding users' transmission powers and communication bandwidths. The formulated problem belongs to the sum-of-ratios optimization, for which prior studies have demonstrated its difficulty. Nevertheless, our proposed algorithm 1) obtains the global optimum for the weighted sum-UEE optimization, via a transform to parametric convex optimization problems, 2) applies to any utility function which is concave, increasing, and twice differentiable, and 3) achieves a linear time complexity in the number of users (the optimal complexity in the order sense). Simulations confirm the superiority of our algorithm over other approaches. We explain that our technique for solving the sum-of-ratios optimization is applicable to other optimization problems in wireless networks and mobile computing.
Form 10-K Itemization
Venue: arXiv cs.IR 2023
Year: 2023

Form 10-K report is a financial report disclosing the annual financial state of a public company. It is an important evidence to conduct financial analysis, i.e., asset pricing, corporate finance. Practitioners and researchers are constantly designing algorithms to better conduct analysis on information in the Form 10-K report. The vast majority of previous works focus on quantitative data. With recent advancement on natural language processing (NLP), textual data in financial filing attracts more attention. However, to incorporate textual data for analyzing, Form 10-K Itemization is a necessary pre-process step. It aims to segment the whole document into several Item sections, where each Item section focuses on a specific financial aspect of the company. With the segmented Item sections, NLP techniques can directly apply on those Item sections related to downstream tasks. In this paper, we develop a Form 10-K Itemization system which can automatically segment all the Item sections in 10-K documents. The system is both effective and efficient. It reaches a retrieval rate of 93%.
Extending the Pre-Training of BLOOM for Improved Support of Traditional Chinese: Models, Methods and Results
Venue: arXiv cs.CL 2023
Year: 2023

In this paper we present the multilingual language model BLOOM-zh that features enhanced support for Traditional Chinese. BLOOM-zh has its origins in the open-source BLOOM models presented by BigScience in 2022. Starting from released models, we extended the pre-training of BLOOM by additional 7.4 billion tokens in Traditional Chinese and English covering a variety of domains such as news articles, books, encyclopedias, educational materials as well as spoken language. In order to show the properties of BLOOM-zh, both existing and newly created benchmark scenarios are used for evaluating the performance. BLOOM-zh outperforms its predecessor on most Traditional Chinese benchmarks while maintaining its English capability. We release all our models to the research community.
B-Treaps Revised: Write Efficient Randomized Block Search Trees with High Load
Venue: arXiv cs.DS 2023
Year: 2023

Uniquely represented data structures represent each logical state with a unique storage state. We study the problem of maintaining a dynamic set of $n$ keys from a totally ordered universe in this context. We introduce a two-layer data structure called $(α,\varepsilon)$-Randomized Block Search Tree (RBST) that is uniquely represented and suitable for external memory. Though RBSTs naturally generalize the well-known binary Treaps, several new ideas are needed to analyze the {\em expected} search, update, and storage, efficiency in terms of block-reads, block-writes, and blocks stored. We prove that searches have $O(\varepsilon^{-1} + \log_αn)$ block-reads, that $(α, \varepsilon)$-RBSTs have an asymptotic load-factor of at least $(1-\varepsilon)$ for every $\varepsilon \in (0,1/2]$, and that dynamic updates perform $O(\varepsilon^{-1} + \log_α(n)/α)$ block-writes, i.e. $O(1/\varepsilon)$ writes if $α=Ω(\frac{\log n}{\log \log n} )$. Thus $(α, \varepsilon)$-RBSTs provide improved search, storage-, and write-efficiency bounds in regard to the known, uniquely represented B-Treap [Golovin; ICALP'09].
Towards Trust of Explainable AI in Thyroid Nodule Diagnosis
Venue: arXiv cs.CV 2023
Year: 2023

The ability to explain the prediction of deep learning models to end-users is an important feature to leverage the power of artificial intelligence (AI) for the medical decision-making process, which is usually considered non-transparent and challenging to comprehend. In this paper, we apply state-of-the-art eXplainable artificial intelligence (XAI) methods to explain the prediction of the black-box AI models in the thyroid nodule diagnosis application. We propose new statistic-based XAI methods, namely Kernel Density Estimation and Density map, to explain the case of no nodule detected. XAI methods' performances are considered under a qualitative and quantitative comparison as feedback to improve the data quality and the model performance. Finally, we survey to assess doctors' and patients' trust in XAI explanations of the model's decisions on thyroid nodule images.
Xel-FPGAs: An End-to-End Automated Exploration Framework for Approximate Accelerators in FPGA-Based Systems
Venue: arXiv cs.AR 2023
Year: 2023

Generation and exploration of approximate circuits and accelerators has been a prominent research domain to achieve energy-efficiency and/or performance improvements. This research has predominantly focused on ASICs, while not achieving similar gains when deployed for FPGA-based accelerator systems, due to the inherent architectural differences between the two. In this work, we propose a novel framework, Xel-FPGAs, which leverages statistical or machine learning models to effectively explore the architecture-space of state-of-the-art ASIC-based approximate circuits to cater them for FPGA-based systems given a simple RTL description of the target application. We have also evaluated the scalability of our framework on a multi-stage application using a hierarchical search strategy. The Xel-FPGAs framework is capable of reducing the exploration time by up to 95%, when compared to the default synthesis, place, and route approaches, while identifying an improved set of Pareto-optimal designs for a given application, when compared to the state-of-the-art. The complete framework is open-source and available online at https://github.com/ehw-fit/xel-fpgas.
SoftMatch Distance: A Novel Distance for Weakly-Supervised Trend Change Detection in Bi-Temporal Images
Venue: arXiv cs.CV 2023
Year: 2023

General change detection (GCD) and semantic change detection (SCD) are common methods for identifying changes and distinguishing object categories involved in those changes, respectively. However, the binary changes provided by GCD is often not practical enough, while annotating semantic labels for training SCD models is very expensive. Therefore, there is a novel solution that intuitively dividing changes into three trends (``appear'', ``disappear'' and ``transform'') instead of semantic categories, named it trend change detection (TCD) in this paper. It offers more detailed change information than GCD, while requiring less manual annotation cost than SCD. However, there are limited public data sets with specific trend labels to support TCD application. To address this issue, we propose a softmatch distance which is used to construct a weakly-supervised TCD branch in a simple GCD model, using GCD labels instead of TCD label for training. Furthermore, a strategic approach is presented to successfully explore and extract background information, which is crucial for the weakly-supervised TCD task. The experiment results on four public data sets are highly encouraging, which demonstrates the effectiveness of our proposed model.
Error Correcting Codes From General Linear Groups
Venue: arXiv cs.IT 2023
Year: 2023

The parameters of the AG codes on general linear groups are found. The hyperplane sections having the minimum (or maximum) number of rational points are determined.
Comprehensive Event Representations using Event Knowledge Graphs and Natural Language Processing
Venue: arXiv cs.CL 2023
Year: 2023

Recent work has utilised knowledge-aware approaches to natural language understanding, question answering, recommendation systems, and other tasks. These approaches rely on well-constructed and large-scale knowledge graphs that can be useful for many downstream applications and empower knowledge-aware models with commonsense reasoning. Such knowledge graphs are constructed through knowledge acquisition tasks such as relation extraction and knowledge graph completion. This work seeks to utilise and build on the growing body of work that uses findings from the field of natural language processing (NLP) to extract knowledge from text and build knowledge graphs. The focus of this research project is on how we can use transformer-based approaches to extract and contextualise event information, matching it to existing ontologies, to build a comprehensive knowledge of graph-based event representations. Specifically, sub-event extraction is used as a way of creating sub-event-aware event representations. These event representations are then further enriched through fine-grained location extraction and contextualised through the alignment of historically relevant quotes.
The Impact of Coherence Diversity on MIMO Relays
Venue: arXiv cs.IT 2023
Year: 2023

This paper studies MIMO relays with non-identical link coherence times, a frequently occurring condition when, e.g., the nodes in the relay channel do not all have the same mobility, or the scatterers around some nodes have different mobility compared with those around other nodes. Despite its practical relevance, this condition, known as coherence diversity, has not been studied in the relay channel. This paper studies the performance of MIMO relays and proposes efficient transmission strategies under coherence diversity. Since coherence times have a prominent impact on channel training, we do not assume channel state is available to the decoder for free; all channel training resources are accounted for in the calculations. A product superposition technique is employed at the source, which allows a more efficient usage of degrees of freedom when the relay and the destination have different training requirements. Varying configurations of coherence times are studied. The interesting case where the different link coherence intervals are not a multiple of each other, and therefore the coherence intervals do not align, is studied. Relay scheduling is combined with the product superposition to obtain further gains in degrees of freedom. The impact of coherence diversity is further studied in the presence of multiple parallel relays.