OpenRaiser/Intern-Atlas huggingface dataset:
Spot the Bot: Distinguishing Human-Written and Bot-Generated Texts Using Clustering and Information Theory Techniques
Venue: arXiv cs.CL 2023
Year: 2023

With the development of generative models like GPT-3, it is increasingly more challenging to differentiate generated texts from human-written ones. There is a large number of studies that have demonstrated good results in bot identification. However, the majority of such works depend on supervised learning methods that require labelled data and/or prior knowledge about the bot-model architecture. In this work, we propose a bot identification algorithm that is based on unsupervised learning techniques and does not depend on a large amount of labelled data. By combining findings in semantic analysis by clustering (crisp and fuzzy) and information techniques, we construct a robust model that detects a generated text for different types of bot. We find that the generated texts tend to be more chaotic while literary works are more complex. We also demonstrate that the clustering of human texts results in fuzzier clusters in comparison to the more compact and well-separated clusters of bot-generated texts.
Physics-Enhanced TinyML for Real-Time Detection of Ground Magnetic Anomalies
Venue: arXiv cs.LG 2023
Year: 2023

Space weather phenomena like geomagnetic disturbances (GMDs) and geomagnetically induced currents (GICs) pose significant risks to critical technological infrastructure. While traditional predictive models, grounded in simulation, hold theoretical robustness, they grapple with challenges, notably the assimilation of imprecise data and extensive computational complexities. In recent years, Tiny Machine Learning (TinyML) has been adopted to develop Machine Learning (ML)-enabled magnetometer systems for predicting real-time terrestrial magnetic perturbations as a proxy measure for GIC. While TinyML offers efficient, real-time data processing, its intrinsic limitations prevent the utilization of robust methods with high computational needs. This paper developed a physics-guided TinyML framework to address the above challenges. This framework integrates physics-based regularization at the stages of model training and compression, thereby augmenting the reliability of predictions. The developed pruning scheme within the framework harnesses the inherent physical characteristics of the domain, striking a balance between model size and robustness. The study presents empirical results, drawing a comprehensive comparison between the accuracy and reliability of the developed framework and its traditional counterpart. Such a comparative analysis underscores the prospective applicability of the developed framework in conceptualizing robust, ML-enabled magnetometer systems for real-time space weather forecasting.
LLM aided semi-supervision for Extractive Dialog Summarization
Venue: arXiv cs.CL 2023
Year: 2023

Generating high-quality summaries for chat dialogs often requires large labeled datasets. We propose a method to efficiently use unlabeled data for extractive summarization of customer-agent dialogs. In our method, we frame summarization as a question-answering problem and use state-of-the-art large language models (LLMs) to generate pseudo-labels for a dialog. We then use these pseudo-labels to fine-tune a chat summarization model, effectively transferring knowledge from the large LLM into a smaller specialized model. We demonstrate our method on the \tweetsumm dataset, and show that using 10% of the original labelled data set we can achieve 65.9/57.0/61.0 ROUGE-1/-2/-L, whereas the current state-of-the-art trained on the entire training data set obtains 65.16/55.81/64.37 ROUGE-1/-2/-L. In other words, in the worst case (i.e., ROUGE-L) we still effectively retain 94.7% of the performance while using only 10% of the data.
Self-Expression and Sharing around Chronic Illness on TikTok
Venue: arXiv cs.HC 2023
Year: 2023

While prior work has investigated the benefits of online health communities and general purpose social media used for health-related purposes, little work examines the use of TikTok, an emerging social media platform with a substantial user base. The platform's multimodal capabilities foster creative self-expression, while the content-driven network allows users to reach new audiences beyond their personal connections. To investigate users' challenges and motivations, we analyzed 160 TikTok videos that center on users' first hand experiences living with chronic illness. We found that users struggled with a loss of normalcy and stigmatization in daily life. To contend with these challenges, they publicly shared their experiences to raise awareness, seek support from peers, and normalize chronic illness experiences. Based on our findings, we discuss the modalities of TikTok that facilitate self-expression around stigmatized topics and provide implications for the design of online health communities that better support adolescents and young adults.
Perspectives on Privacy in the Post-Roe Era: A Mixed-Methods of Machine Learning and Qualitative Analyses of Tweets
Venue: arXiv cs.HC 2023
Year: 2023

Abortion is a controversial topic that has long been debated in the US. With the recent Supreme Court decision to overturn Roe v. Wade, access to safe and legal reproductive care is once again in the national spotlight. A key issue central to this debate is patient privacy, as in the post-HITECH Act era it has become easier for medical records to be electronically accessed and shared. This study analyzed a large Twitter dataset from May to December 2022 to examine the public's reactions to Roe v. Wade's overruling and its implications for privacy. Using a mixed-methods approach consisting of computational and qualitative content analysis, we found a wide range of concerns voiced from the confidentiality of patient-physician information exchange to medical records being shared without patient consent. These findings may inform policy making and healthcare industry practices concerning medical privacy related to reproductive rights and women's health.
Exploring Prompting Large Language Models as Explainable Metrics
Venue: arXiv cs.CL 2023
Year: 2023

This paper describes the IUST NLP Lab submission to the Prompting Large Language Models as Explainable Metrics Shared Task at the Eval4NLP 2023 Workshop on Evaluation & Comparison of NLP Systems. We have proposed a zero-shot prompt-based strategy for explainable evaluation of the summarization task using Large Language Models (LLMs). The conducted experiments demonstrate the promising potential of LLMs as evaluation metrics in Natural Language Processing (NLP), particularly in the field of summarization. Both few-shot and zero-shot approaches are employed in these experiments. The performance of our best provided prompts achieved a Kendall correlation of 0.477 with human evaluations in the text summarization task on the test data. Code and results are publicly available on GitHub.
Asymptotic CRB Analysis of Random RIS-Assisted Large-Scale Localization Systems
Venue: arXiv cs.IT 2023
Year: 2023

This paper studies the performance of a randomly RIS-assisted multi-target localization system, in which the configurations of the RIS are randomly set to avoid high-complexity optimization. We first focus on the scenario where the number of RIS elements is significantly large, and then obtain the scaling law of Cramér-Rao bound (CRB) under certain conditions, which shows that CRB decreases in the third or fourth order as the RIS dimension increases. Second, we extend our analysis to large systems where both the number of targets and sensors is substantial. Under this setting, we explore two common RIS models: the constant module model and the discrete amplitude model, and illustrate how the random RIS configuration impacts the value of CRB. Numerical results demonstrate that asymptotic formulas provide a good approximation to the exact CRB in the proposed randomly configured RIS systems.
Semantic-Preserved Point-based Human Avatar
Venue: arXiv cs.CV 2023
Year: 2023

To enable realistic experience in AR/VR and digital entertainment, we present the first point-based human avatar model that embodies the entirety expressive range of digital humans. We employ two MLPs to model pose-dependent deformation and linear skinning (LBS) weights. The representation of appearance relies on a decoder and the features that attached to each point. In contrast to alternative implicit approaches, the oriented points representation not only provides a more intuitive way to model human avatar animation but also significantly reduces both training and inference time. Moreover, we propose a novel method to transfer semantic information from the SMPL-X model to the points, which enables to better understand human body movements. By leveraging the semantic information of points, we can facilitate virtual try-on and human avatar composition through exchanging the points of same category across different subjects. Experimental results demonstrate the efficacy of our presented method.
Incorporating LLM Priors into Tabular Learners
Venue: arXiv cs.LG 2023
Year: 2023

We present a method to integrate Large Language Models (LLMs) and traditional tabular data classification techniques, addressing LLMs challenges like data serialization sensitivity and biases. We introduce two strategies utilizing LLMs for ranking categorical variables and generating priors on correlations between continuous variables and targets, enhancing performance in few-shot scenarios. We focus on Logistic Regression, introducing MonotonicLR that employs a non-linear monotonic function for mapping ordinals to cardinals while preserving LLM-determined orders. Validation against baseline models reveals the superior performance of our approach, especially in low-data scenarios, while remaining interpretable.
Cancer-Net PCa-Data: An Open-Source Benchmark Dataset for Prostate Cancer Clinical Decision Support using Synthetic Correlated Diffusion Imaging Data
Venue: arXiv cs.CV 2023
Year: 2023

The recent introduction of synthetic correlated diffusion (CDI$^s$) imaging has demonstrated significant potential in the realm of clinical decision support for prostate cancer (PCa). CDI$^s$ is a new form of magnetic resonance imaging (MRI) designed to characterize tissue characteristics through the joint correlation of diffusion signal attenuation across different Brownian motion sensitivities. Despite the performance improvement, the CDI$^s$ data for PCa has not been previously made publicly available. In our commitment to advance research efforts for PCa, we introduce Cancer-Net PCa-Data, an open-source benchmark dataset of volumetric CDI$^s$ imaging data of PCa patients. Cancer-Net PCa-Data consists of CDI$^s$ volumetric images from a patient cohort of 200 patient cases, along with full annotations (gland masks, tumor masks, and PCa diagnosis for each tumor). We also analyze the demographic and label region diversity of Cancer-Net PCa-Data for potential biases. Cancer-Net PCa-Data is the first-ever public dataset of CDI$^s$ imaging data for PCa, and is a part of the global open-source initiative dedicated to advancement in machine learning and imaging research to aid clinicians in the global fight against cancer.
Control in Hybrid Chatbots
Venue: arXiv cs.IR 2023
Year: 2023

Customer data typically is held in database systems, which can be seen as rule-based knowledge base, whereas businesses increasingly want to benefit from the capabilities of large, pre-trained language models. In this technical report, we describe a case study of how a commercial rule engine and an integrated neural chatbot may be integrated, and what level of control that particular integration mode leads to. We also discuss alternative ways (including past ways realized in other systems) how researchers strive to maintain control and avoid what has recently been called model "hallucination".
Configuring an heterogeneous smartgrid network: complexity and approximations for tree topologies
Venue: arXiv cs.CC 2023
Year: 2023

We address the problem of configuring a power distribution network with reliability and resilience objectives by satisfying the demands of the consumers and saturating each production source as little as possible. We consider power distribution networks containing source nodes producing electricity, nodes representing electricity consumers and switches between them. Configuring this network consists in deciding the orientation of the links between the nodes of the network. The electric flow is a direct consequence of the chosen configuration and can be computed in polynomial time. It is valid if it satisfies the demand of each consumer and capacity constraints on the network. In such a case, we study the problem of determining a feasible solution that balances the loads of the sources, that is their production rates. We use three metrics to measure the quality of a solution: minimizing the maximum load, maximizing the minimum load and minimizing the difference of the maximum and the minimum loads. This defines optimization problems called respectively min-M, max-m and min-R. In the case where the graph of the network is a tree, it is known that the problem of building a valid configuration is polynomial. We show the three optimization variants have distinct properties regarding the theoretical complexity and the approximability. Particularly, we show that min-M is polynomial, that max-m is NP-Hard but belongs to the class FPTAS and that min-R is NP-Hard, cannot 1 be approximated to within any exponential relative ratio but, for any $ε$ > 0, there exists an algorithm for which the value of the returned solution equals the value of an optimal solution shifted by at most $ε$.
Can we infer the presence of Differential Privacy in Deep Learning models' weights? Towards more secure Deep Learning
Venue: arXiv cs.LG 2023
Year: 2023

Differential Privacy (DP) is a key property to protect data and models from integrity attacks. In the Deep Learning (DL) field, it is commonly implemented through the Differentially Private Stochastic Gradient Descent (DP-SGD). However, when a model is shared or released, there is no way to check whether it is differentially private, that is, it required to trust the model provider. This situation poses a problem when data privacy is mandatory, specially with current data regulations, as the presence of DP can not be certificated consistently by any third party. Thus, we face the challenge of determining whether a DL model has been trained with DP, according to the title question: Can we infer the presence of Differential Privacy in Deep Learning models' weights? Since the DP-SGD significantly changes the training process of a DL model, we hypothesize that DP leaves an imprint in the weights of a DL model, which can be used to predict whether a model has been trained with DP regardless of its architecture and the training dataset. In this paper, we propose to employ the imprint in model weights of using DP to infer the presence of DP training in a DL model. To substantiate our hypothesis, we developed an experimental methodology based on two datasets of weights of DL models, each with models with and without DP training and a meta-classifier to infer whether DP was used in the training process of a DL model, by accessing its weights. We accomplish both, the removal of the requirement of a trusted model provider and a strong foundation for this interesting line of research. Thus, our contribution is an additional layer of security on top of the strict private requirements of DP training in DL models, towards to DL models.
Large Language Models and Explainable Law: a Hybrid Methodology
Venue: arXiv cs.AI 2023
Year: 2023

The paper advocates for LLMs to enhance the accessibility, usage and explainability of rule-based legal systems, contributing to a democratic and stakeholder-oriented view of legal technology. A methodology is developed to explore the potential use of LLMs for translating the explanations produced by rule-based systems, from high-level programming languages to natural language, allowing all users a fast, clear, and accessible interaction with such technologies. The study continues by building upon these explanations to empower laypeople with the ability to execute complex juridical tasks on their own, using a Chain of Prompts for the autonomous legal comparison of different rule-based inferences, applied to the same factual case.
Zero redundancy distributed learning with differential privacy
Venue: arXiv cs.LG 2023
Year: 2023

Deep learning using large models have achieved great success in a wide range of domains. However, training these models on billions of parameters is very challenging in terms of the training speed, memory cost, and communication efficiency, especially under the privacy-preserving regime with differential privacy (DP). On the one hand, DP optimization has comparable efficiency to the standard non-private optimization on a single GPU, but on multiple GPUs, existing DP distributed learning (such as pipeline parallel) has suffered from significantly worse efficiency. On the other hand, the Zero Redundancy Optimizer (ZeRO) is a state-of-the-art solution to the standard distributed learning, exhibiting excellent training efficiency on large models, but to work compatibly with DP is technically complicated. In this work, we develop a new systematic solution, DP-ZeRO, (I) to scale up the trainable DP model size, e.g. to GPT-100B, (II) to obtain the same computation and communication efficiency as the standard ZeRO, and (III) to enable mixed-precision DP training. Our DP-ZeRO, like the standard ZeRO, has the potential to train models with arbitrary size and is evaluated on the world's largest DP models in terms of the number of trainable parameters.
System 2 Attention (is something you might need too)
Venue: arXiv cs.CL 2023
Year: 2023

Soft attention in Transformer-based Large Language Models (LLMs) is susceptible to incorporating irrelevant information from the context into its latent representations, which adversely affects next token generations. To help rectify these issues, we introduce System 2 Attention (S2A), which leverages the ability of LLMs to reason in natural language and follow instructions in order to decide what to attend to. S2A regenerates the input context to only include the relevant portions, before attending to the regenerated context to elicit the final response. In experiments, S2A outperforms standard attention-based LLMs on three tasks containing opinion or irrelevant information, QA, math word problems and longform generation, where S2A increases factuality and objectivity, and decreases sycophancy.
Towards Exploratory Reformulation of Constraint Models
Venue: arXiv cs.AI 2023
Year: 2023

It is well established that formulating an effective constraint model of a problem of interest is crucial to the efficiency with which it can subsequently be solved. Following from the observation that it is difficult, if not impossible, to know a priori which of a set of candidate models will perform best in practice, we envisage a system that explores the space of models through a process of reformulation from an initial model, guided by performance on a set of training instances from the problem class under consideration. We plan to situate this system in a refinement-based approach, where a user writes a constraint specification describing a problem above the level of abstraction at which many modelling decisions are made. In this position paper we set out our plan for an exploratory reformulation system, and discuss progress made so far.
Multimodal Characterization of Emotion within Multimedia Space
Venue: arXiv cs.MM 2023
Year: 2023

Technological advancement and its omnipresent connection have pushed humans past the boundaries and limitations of a computer screen, physical state, or geographical location. It has provided a depth of avenues that facilitate human-computer interaction that was once inconceivable such as audio and body language detection. Given the complex modularities of emotions, it becomes vital to study human-computer interaction, as it is the commencement of a thorough understanding of the emotional state of users and, in the context of social networks, the producers of multimodal information. This study first acknowledges the accuracy of classification found within multimodal emotion detection systems compared to unimodal solutions. Second, it explores the characterization of multimedia content produced based on their emotions and the coherence of emotion in different modalities by utilizing deep learning models to classify emotion across different modalities.
Generalization of Fitness Exercise Recognition from Doppler Measurements by Domain-adaption and Few-Shot Learning
Venue: arXiv cs.AI 2023
Year: 2023

In previous works, a mobile application was developed using an unmodified commercial off-the-shelf smartphone to recognize whole-body exercises. The working principle was based on the ultrasound Doppler sensing with the device built-in hardware. Applying such a lab-environment trained model on realistic application variations causes a significant drop in performance, and thus decimate its applicability. The reason of the reduced performance can be manifold. It could be induced by the user, environment, and device variations in realistic scenarios. Such scenarios are often more complex and diverse, which can be challenging to anticipate in the initial training data. To study and overcome this issue, this paper presents a database with controlled and uncontrolled subsets of fitness exercises. We propose two concepts to utilize small adaption data to successfully improve model generalization in an uncontrolled environment, increasing the recognition accuracy by two to six folds compared to the baseline for different users.
Deep Calibration of Market Simulations using Neural Density Estimators and Embedding Networks
Venue: arXiv cs.LG 2023
Year: 2023

The ability to construct a realistic simulator of financial exchanges, including reproducing the dynamics of the limit order book, can give insight into many counterfactual scenarios, such as a flash crash, a margin call, or changes in macroeconomic outlook. In recent years, agent-based models have been developed that reproduce many features of an exchange, as summarised by a set of stylised facts and statistics. However, the ability to calibrate simulators to a specific period of trading remains an open challenge. In this work, we develop a novel approach to the calibration of market simulators by leveraging recent advances in deep learning, specifically using neural density estimators and embedding networks. We demonstrate that our approach is able to correctly identify high probability parameter sets, both when applied to synthetic and historical data, and without reliance on manually selected or weighted ensembles of stylised facts.