Reviewerly/exHarmony huggingface dataset:
End-to-end Adversarial Sample Generation for Data Augmentation

Adversarial samples pose a significant challenge to neural inference models. In this paper, we propose a novel enhancing approach A3 for the robustness of the neural NLP models, which combines the adversarial training and data augmentation. We propose an adversarial sample generator that consists of a conditioned paraphrasing model and a condition generator. The latter aims to generate conditions which guides the paraphrasing model to generate adversarial samples. A pretrained discriminator is introduced to help the adversarial sample generator adapt to the data characteristics for different tasks. We adopt a weighted loss to incorporate the generated adversarial samples with the original samples for augmented training. Compared to existing methods, our approach is much efficient since the generation process is independent to the target model and the generated samples are reusable for different models. Experimental results on several tasks show that our approach improves the overall performance of the trained model. Specially, the enhanced model is robust for various attacking techniques.
UDAPDR: Unsupervised Domain Adaptation via LLM Prompting and Distillation of Rerankers

Jon Saad-Falcon, Omar Khattab, Keshav Santhanam, Radu Florian, Martin Franz, Salim Roukos, Avirup Sil, Md Sultan, Christopher Potts. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.
SOUL: Towards Sentiment and Opinion Understanding of Language

Sentiment analysis is a well-established natural language processing task, with sentiment polarity classification being one of its most popular and representative tasks. However, despite the success of pre-trained language models in this area, they often fall short of capturing the broader complexities of sentiment analysis. To address this issue, we propose a new task called Sentiment and Opinion Understanding of Language (SOUL). SOUL aims to evaluate sentiment understanding through two subtasks: Review Comprehension (RC) and Justification Generation (JG). RC seeks to validate statements that focus on subjective information based on a review text, while JG requires models to provide explanations for their sentiment predictions. To enable comprehensive evaluation, we annotate a new dataset comprising 15,028 statements from 3,638 reviews. Experimental results indicate that SOUL is a challenging task for both small and large language models, with a performance gap of up to 27% when compared to human performance. Furthermore, evaluations conducted with both human experts and GPT-4 highlight the limitations of the small language model in generating reasoning-based justifications. These findings underscore the challenging nature of the SOUL task for existing models, emphasizing the need for further advancements in sentiment analysis to address its complexities. The new dataset and code are available at https://github.com/DAMO-NLP-SG/SOUL.
Look-back Decoding for Open-Ended Text Generation

Given a prefix (context), open-ended generation aims to decode texts that are coherent, which do not abruptly drift from previous topics, and informative, which do not suffer from undesired repetitions. In this paper, we propose Look-back, an improved decoding algorithm that leverages the Kullback–Leibler divergence to track the distribution distance between current and historical decoding steps. Thus Look-back can automatically predict potential repetitive phrase and topic drift, and remove tokens that may cause the failure modes, restricting the next token probability distribution within a plausible distance to the history. We perform decoding experiments on document continuation and story generation, and demonstrate that Look-back is able to generate more fluent and coherent text, outperforming other strong decoding methods significantly in both automatic and human evaluations.
HW-TSC 2023 Submission for the Quality Estimation Shared Task

Quality estimation (QE) is an essential technique to assess machine translation quality without reference translations. In this paper, we focus on Huawei Translation Services Center’s (HW-TSC’s) submission to the sentence-level QE shared task, named Ensemble-CrossQE. Our system uses CrossQE, the same model architecture as our last year’s submission, which consists of a multilingual base model and a task-specific downstream layer. The input is the concatenation of the source and the translated sentences. To enhance the performance, we finetuned and ensembled multiple base models such as XLM-R, InfoXLM, RemBERT and CometKiwi. Moreover, we introduce a new corruption-based data augmentation method, which generates deletion, substitution and insertion errors in the original translation and uses a reference-based QE model to obtain pseudo scores. Results show that our system achieves impressive performance on sentence-level QE test sets and ranked the first place for three language pairs: English-Hindi, English-Tamil and English-Telegu. In addition, we participated in the error span detection task. The submitted model outperforms the baseline on Chinese-English and Hebrew-English language pairs.
Crossing the Aisle: Unveiling Partisan and Counter-Partisan Events in News Reporting

News media is expected to uphold unbiased reporting. Yet they may still affect public opinion by selectively including or omitting events that support or contradict their ideological positions. Prior work in NLP has only studied media bias via linguistic style and word usage. In this paper, we study to which degree media balances news reporting and affects consumers through event inclusion or omission. We first introduce the task of detecting both partisan and counter-partisan events: events that support or oppose the author’s political ideology. To conduct our study, we annotate a high-quality dataset, PAC, containing 8,511 (counter-)partisan event annotations in 304 news articles from ideologically diverse media outlets. We benchmark PAC to highlight the challenges of this task. Our findings highlight both the ways in which the news subtly shapes opinion and the need for large language models that better understand events within a broader context. Our dataset can be found at https://github.com/launchnlp/Partisan-Event-Dataset.
Latent Feature-based Data Splits to Improve Generalisation Evaluation: A Hate Speech Detection Case Study

With the ever-growing presence of social media platforms comes the increased spread of harmful content and the need for robust hate speech detection systems. Such systems easily overfit to specific targets and keywords, and evaluating them without considering distribution shifts that might occur between train and test data overestimates their benefit. We challenge hate speech models via new train-test splits of existing datasets that rely on the clustering of models’ hidden representations. We present two split variants (Subset-Sum-Split and Closest-Split) that, when applied to two datasets using four pretrained models, reveal how models catastrophically fail on blind spots in the latent space. This result generalises when developing a split with one model and evaluating it on another. Our analysis suggests that there is no clear surface-level property of the data split that correlates with the decreased performance, which underscores that task difficulty is not always humanly interpretable. We recommend incorporating latent feature-based splits in model development and release two splits via the GenBench benchmark.
EPS$^{2}$: Privacy Preserving Set-Valued Data Analysis in the Shuffle Model

Collecting and analyzing users' set-valued data with privacy-preserving is a common scenario in real life. However, the existing solutions in LDP are not efficient enough, where users perturbing their data locally introduces a large amount of noise. The shuffle model, which adds a shuffler in LDP to shuffle all perturbed values, can amplify privacy, then improve utility. Inspired by this, we study the frequency estimation and top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> frequent item estimation of set-valued data in the shuffle model. To solve the challenges of different item quantities of users and further improve the utility, we combine sampling and shuffling together, and propose the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Encoding, Padding, Sampling, and Shuffling</i> framework, i.e., EPS <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$^{2}$</tex-math></inline-formula> . Based on this framework, we propose three protocols for frequency estimation in different application scenarios, then assemble them into multi-phase protocols for the top- <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> frequent item estimation. Theoretically, we identify all three protocols gain dual privacy amplification from sampling and shuffling. And by setting the size of users' set to 1, we can extend this amplified bound to the single-valued frequency estimation scenario, producing a tighter privacy bound than existing works. Finally, we perform experiments on both synthetic and real-world datasets to demonstrate the effectiveness of our protocols.
Photo-detection wait time distributions for light from degenerate parametric oscillator

Analytic expressions for conditional and unconditional wait-time distributions for the n -th photo-detection are derived for the light from a degenerate parametric oscillator and subthreshold laser, in the small photon number regime where quantum effects dominate.
Clustering Tweets Data on Twitter Social Media Using K-Means Method

Twitter, as a popular social media with millions to billions of global users, stores a wide variety of information. This study focuses on the use of Text Mining to analyze tweet content through the application of clustering techniques, specifically using the K-Means algorithm. The implementation process involves several stages of text processing, including casefolding, tokenizing, stopword removal, and stemming. Feature extraction is performed to provide input for the K-Means algorithm. The clustering evaluation uses the Silhouette coefficient method. The test results show that different K values result in a variation of the silhouette value. In a particular test scenario, a value of K=2 resulted in a silhouette of 0.5000421, K=5 had a value of 0.0501051, and K=9 had a value of 0.501893. From these values, the data structure of the dataset taken can be categorized as medium structure, because the silhouette value is in the range of 0.5 to 0.7. These results show that cluster quality is influenced by the K value, with the silhouette value being the main determinant.
Hybrid approach combining deep learning and a rule based expert system for concept extraction from prescriptions

Concept extraction from prescriptions is a very important task that provides a foundation for many of the downstream healthcare applications in decision making across the areas of pharmacovigilance, medication adherence, inventory management, and other matters of value-based care. Although short, these directions can sometimes be complex. With the increase in complexity of direction, it becomes harder to extract various concepts by only rule based expert system. It identifies major concepts like frequency, dosage, duration, etc. from the natural text direction using a combination of rules and deep learning (DL) based methods on a large real world data of a pharmacy chain. The DL module includes a fine-tuned BERT transformer and Gram CNN (Convolutional Neural Network) based NER (Named Entity Recognition) architecture. The proposed method utilizes the domain heuristics along with intelligent labelling and bootstrapping to help DL models extract concepts with high evaluation scores and thus provides a way for carrying out concept extraction using targeted methods instead of one single method. To the best of our knowledge, this is the best performance reported in the literature for concept extraction from doctor's prescription.
Assessing Mode-Switching Strategies for Assistive Robotic Manipulators Using a Preliminary Version of the Novel Non-invasive Tongue-Computer Interface

The inductive tongue-computer interface allows individuals with tetraplegia to control assistive devices. However, controlling assistive robotic arms often requires more than 14 different commands, which cannot always fit into a single control layout. Previous studies have separated the commands into modes, but few have investigated strategies to switch between them. In this feasibility study, we compare the efficiency of switching modes using buttons, swipe gestures and double taps using a preliminary version of a new non-invasive mouthpiece unit (nMPU), which includes an integrated activation unit and a single sensor board. Three participants controlled a JACO assistive robot to pick up a bottle using different mode-switching strategies. Compared with switching modes with buttons, switching modes with swipes and double taps increased the task completion time by 21% and 58% respectively. Therefore, we recommend that configurations with multiple modes for the non-invasive tongue-computer interface include buttons for mode-switching.Clinical relevance- Cumbersome mode-switching strategies can lower a control interface's responsiveness and contribute to end-user abandonment of assistive technologies. This study showed that using buttons to switch modes is more reliable. Moreover, this study will inform the development of future control layouts with improved usability.
DEPAS: De-novo Pathology Semantic Masks using a Generative Model

The integration of artificial intelligence (AI) into digital pathology has the potential to automate and improve various tasks, such as image analysis and diagnostic decision-making. Yet, the inherent variability of tissues, together with the need for image labeling, lead to biased datasets that limit the generalizability of algorithms trained on them. One of the emerging solutions for this challenge is synthetic histological images. Debiasing real datasets require not only generating photorealistic images but also the ability to control the cellular features within them. A common approach is to use generative methods that perform image translation between semantic masks that reflect prior knowledge of the tissue and a histological image. However, unlike other image domains, the complex structure of the tissue prevents a simple creation of histology semantic masks that are required as input to the image translation model, while semantic masks extracted from real images reduce the process’s scalability. In this work, we introduce a scalable generative model, coined as DEPAS (De-novo Pathology Semantic Masks), that captures tissue structure and generates high-resolution semantic masks with state-of-the-art quality. We demonstrate the ability of DEPAS to generate realistic semantic maps of tissue for three types of organs: skin, prostate, and lung. Moreover, we show that these masks can be processed using a generative image translation model to produce photorealistic histology images of two types of cancer with two different types of staining techniques. Finally, we harness DEPAS to generate multi-label semantic masks that capture different cell types distributions and use them to produce histological images with on-demand cellular features. Overall, our work provides a state-of-the-art solution for the challenging task of generating synthetic histological images while controlling their semantic information in a scalable way.
A Novel Magnetometer Array-based wearable system for ASL gesture recognition

Hand gesture classification is of high importance in any sign language recognition (SLR) system, which is expected to assist individuals suffering from hearing and speech impairment. American sign language (ASL) comprises of static and dynamic gestures representing many alphabets, phrases, and words. ASL recognition system allows us to digitize communication and use it effectively within or outside the hearing-deprived community. Developing an ASL recognition system has been a challenge since some of the involved hand gestures closely resemble each other, and thereby it demands high discriminability features to classify these gestures. SLR through surface-based electromyography (sEMG) signals is computationally intensive to process and using inertial measurement units (IMUs) or flex sensors for SLR occupies too much space on the patient’s hand. Video-based recognition systems place restrictions on the users by requiring them to make gestures or motions within the camera’s field of view. A novel approach with a precision preserved static gesture classification system is proposed to fulfill the much-needed gap. The paper proposes an array of magnetometers-enabled static hand gesture classification system that offers an average accuracy of 98.60% for classifying alphabets and 94.07% for digits using the KNN classification model. The magnetometer array-based wearable system is devised to minimize the electronics coverage around the hand, and yet establish robust classification results that are useful for ASL recognition. The paper discusses the design of the proposed SLR system and also looks into optimizations that can be made to reduce the cost of the system.Clinical relevance — The proposed novel magnetometer array-based wearable system is cost-effective and works well across different hand sizes. It occupies a negligible amount of space on the user’s hand and thus does not interfere with the user’s everyday tasks. It is reliable, robust, and error-free for easy adoption towards building ASL recognition system.
Single Image based Super Resolution Ultrasound Imaging Using Residual Learning of Wavelet Features

The generation of super resolution ultrasound images from the low-resolution (LR) brightness mode (B-mode) images acquired by the portable point of care ultrasound systems has been of sufficient interest in the recent past. With the advancements in deep learning, there have been numerous attempts in this direction. However, all the approaches have been concentrated on employing the direct image as the input to the neural network. In this work, a stationary wavelet (SWT) decomposition is employed to extract the features from the input LR image which is passed through a modified residual network and the learned features are combined using the inverse SWT to reconstruct the high resolution (HR) image at a 4× scale factor. The proposed approach when compared to the state-of-the art approaches, results in an improved high resolution reconstruction.Clinical relevance— The proposed approach will enable the generation of high-resolution images from portable ultrasound systems, allowing for easier interpretation and faster diagnostics in primary care settings.
Inferring Personalized Treatment Effect of Antihypertensives on Alzheimer’s Disease Using Deep Learning

Alzheimer’s disease (AD) is one of the leading causes of death in the United States, especially among the elderly. Recent studies have shown how hypertension is related to cognitive decline in elderly patients, which in turn leads to increased mortality as well as morbidity. There have been various studies that have looked at the effect of antihypertensive drugs in reducing cognitive decline, and their results have proved inconclusive. However, most of these studies assume the treatment effect is similar for all patients, thus considering only the average treatment effects of antihypertensive drugs. In this paper, we assume that the effect of antihypertensives on the onset of AD depends on patient characteristics. We develop a deep learning method called LASSO-Dragonnet to estimate the individualized treatment effects of each patient. We considered six antihypertensive drugs, and each of the six models considered one of the drugs as the treatment and the remaining as control. Our studies showed that although many antihypertensives have a positive impact in delaying AD onset on average, the impact varies from individual to individual, depending on their various characteristics. We also analyzed the importance of various covariates in such an estimation. Our results showed that the individualized treatment effects of each patient could be estimated accurately using a deep learning method, and that the importance of various covariates could be determined.
Formation of activity scenarios based on generative artificial intelligence services

The work is dedicated to describing a methodology for generating activity scenarios based on causal networks formed using generative artificial intelligence. The methodology is based on the use of a bidirectional algorithm for generating causal networks. According to this algorithm, two networks are formed and then combined – the first network starts from a node corresponding to the initial state of the problem (the root cause), and the second network corresponds to the goal that needs to be achieved. The article demonstrates the possibility of constructing such causal networks based on the use of a generative transformer like ChatGPT, and provides an example of scenario generation in the subject field of mobile communication. The methodology combines tools for text analysis and the formation of causal networks, followed by the selection and ranking of narrative chains based on the analysis of these networks
The nature of the psyche and consciousness, quantum computers and information models in psychoinformatics (socionics)

Modeling the psyche, consciousness and artificial intelligence have encountered conceptual difficulties in understanding real mental processes. Numerous modern studies and experiments presented in the review indicate the probable quantum nature of the psyche and consciousness. The analysis shows that a number of quantum processes are associated with known classical processes and structures, from a single cell to the brain. Therefore, adequate modeling of mental processes, including consciousness and intelligence, is possible only on multiprocessor quantum computers with a special architecture corresponding to the real structure of the psyche. This possible architecture is proposed by the theory of information metabolism - socionics (psychoinformatics), in which detailed holistic information models of the psyche have been developed, which have been used in practice for more than 30 years and have proven their effectiveness. These models represent a system of mental functions of information metabolism, which can be simulated by special quantum processors interacting with each other according to certain laws
Using retrieval-augmented generation to elevate low-code developer skills

This article proposes applying retrieval-augmented generation (RAG) to improve the skills of low-code developers by augmenting large language models with up-to-date domain-specific knowledge. As low-code development requires combining multiple systems into a final product, developers must consult several sources of documentation and various articles, videos, and forum threads. Such a process may be time-consuming, prompting the use of an LLM for the authoritative answer. However, LLMs often lack knowledge of low-code platforms, leading to hallucinations and superficial responses. RAG utilizes the benefits of LLMs on relevant information, suggesting a presumption that it may be effectively applied in low-code development. Heterogeneous data sources concerning low-code systems are converted to a text representation, split into logical chunks, and stored in a vector database. During the exploitation of the model, cosine similarity is used to retrieve top-K documents and concatenate them with user query, using the produced text as a prompt to an LLM. The results support the hypothesis that RAG models outperform standard LLMs in knowledge retrieval in this domain