Reviewerly/exHarmony huggingface dataset:
Domain Knowledge Transferring for Pre-trained Language Model via Calibrated Activation Boundary Distillation

Since the development and wide use of pretrained language models (PLMs), several approaches have been applied to boost their performance on downstream tasks in specific domains, such as biomedical or scientific domains.Additional pre-training with in-domain texts is the most common approach for providing domain-specific knowledge to PLMs.However, these pre-training methods require considerable in-domain data and training resources and a longer training time.Moreover, the training must be re-performed whenever a new PLM emerges.In this study, we propose a domain knowledge transferring (DoKTra) framework for PLMs without additional in-domain pretraining.Specifically, we extract the domain knowledge from an existing in-domain pretrained language model and transfer it to other PLMs by applying knowledge distillation.In particular, we employ activation boundary distillation, which focuses on the activation of hidden neurons.We also apply an entropy regularization term in both teacher training and distillation to encourage the model to generate reliable output probabilities, and thus aid the distillation.By applying the proposed DoKTra framework to downstream tasks in the biomedical, clinical, and financial domains, our student models can retain a high percentage of teacher performance and even outperform the teachers in certain tasks.Our code is available at https://github.com/DMCB-GIST/DoKTra.
Photonic decision making for solving competitive multi-armed bandit problem using semiconductor laser networks

Multi-armed bandit problems concern decision making when selecting a slot machine among many slot machines with initially uncertain hit probabilities to maximize the total reward; this is a fundamental problem of reinforcement learning. Furthermore, competitive multi-armed bandit problems involve multiple agents in play, manifesting fundamental concerns regarding social figures, not just individual rewards. A representative issue is selection conflict, in which multiple players select the same slot machine and may miss the total reward as a whole. This study proposes a scheme for solving the competitive multi-armed bandit problem using semiconductor laser networks by introducing an exclusive selection mechanism. We numerically implement our method and compare it with conventional algorithms. We show that our method outperforms conventional algorithms in solving the competitive multi-armed bandit problem.
Evaluating zero-shot transfers and multilingual models for dependency parsing and POS tagging within the low-resource language family Tupían

This work presents two experiments with the goal of replicating the transferability of dependency parsers and POS taggers trained on closely related languages within the lowresource language family Tupían.The experiments include both zero-shot settings as well as multilingual models.Previous studies have found that even a comparably small treebank from a closely related language will improve sequence labelling considerably in such cases.Results from both POS tagging and dependency parsing confirm previous evidence that the closer the phylogenetic relation between two languages, the better the predictions for sequence labelling tasks get.In many cases, the results are improved if multiple languages from the same family are combined.This suggests that in addition to leveraging similarity between two related languages, the incorporation of multiple languages of the same family might lead to better results in transfer learning for NLP applications.
A Swapping Target Q-Value Technique for Data Augmentation in Offline Reinforcement Learning

Offline reinforcement learning (RL) is applied to fixed datasets of logged interactions pertaining to actual applications in healthcare, autonomous vehicles, and robotics. In limited and fixed dataset settings, data augmentation can be beneficial in developing better policies. Several online RL methods for data augmentation have recently been utilized to enhance sampling efficiency and generalization. Here, a novel, simple data-augmentation technique referred to as Swapping Target Q-Value (SQV) is introduced to enhance offline RL algorithms and enable robust pixel-based learning without auxiliary loss. Our method matches the current Q-value of a transformed image to the target Q-value of the next original image, whereby the current Q-value of the original image is matched to the target Q-value of the next transformed image. The proposed method considers similar states as the same and different states as more distinct. Furthermore, the approach ties unseen data (lacking in the dataset) to similar states in the seen data. After training, these effects were observed to increase the performance of the offline RL algorithm. The method was tested on 23 games in the Atari 2600 game domain. As a result, the performance of our method improved in 18 out of 23 games, with an average performance improvement of 144% compared with batch-constrained deep Q-learning (BCQ), which is the latest offline RL method. The implementation can be found at https://github.com/hotaekjoo/SQV.
Intrinsic Gradient Compression for Scalable and Efficient Federated Learning

Federated learning is a rapidly growing area of research, holding the promise of privacypreserving distributed training on edge devices.The largest barrier to wider adoption of federated learning is the communication cost of model updates, which is accentuated by the fact that many edge devices are bandwidthconstrained.At the same time, within the machine learning theory community, a separate line of research has emerged around optimizing networks within a subspace of the full space of all parameters.The dimension of the smallest subspace for which these methods still yield strong results is called the intrinsic dimension.In this work, we prove a general correspondence between the notions of intrinsic dimension and gradient compressibility, and we show that a family of low-bandwidth federated learning algorithms, which we call intrinsic gradient compression algorithms, naturally emerges from this correspondence.Finally, we conduct large-scale NLP experiments using transformer models with over 100M parameters (GPT-2 and BERT), and show that our method outperforms the state-of-the-art in gradient compression.
A hidden Markov optimization model for processing and recognition of English speech feature signals

Abstract Speech recognition plays an important role in human–computer interaction. The higher the accuracy and efficiency of speech recognition are, the larger the improvement of human–computer interaction performance. This article briefly introduced the hidden Markov model (HMM)-based English speech recognition algorithm and combined it with a back-propagation neural network (BPNN) to further improve the recognition accuracy and reduce the recognition time of English speech. Then, the BPNN-combined HMM algorithm was simulated and compared with the HMM algorithm and the BPNN algorithm. The results showed that increasing the number of test samples increased the word error rate and recognition time of the three speech recognition algorithms, among which the word error rate and recognition time of the BPNN-combined HMM algorithm were the lowest. In conclusion, the BPNN-combined HMM can effectively recognize English speeches, which provides a valid reference for intelligent recognition of English speeches by computers.
Understanding Gender Bias in Knowledge Base Embeddings

Knowledge base (KB) embeddings have been shown to contain gender biases (Fisher et al., 2020b).In this paper, we study two questions regarding these biases: how to quantify them, and how to trace their origins in KB? Specifically, first, we develop two novel bias measures respectively for a group of person entities and an individual person entity.Evidence of their validity is observed by comparison with real-world census data.Second, we use influence function to inspect the contribution of each triple in KB to the overall group bias.To exemplify the potential applications of our study, we also present two strategies (by adding and removing KB triples) to mitigate gender biases in KB embeddings.
PDDL Planning and Ontologies, a Tool for Automatic Composition of Intentional-Contextual Web Services

Among the motivations of the related work on semantic Web services is to find optimal solutions for the composition of Web services in an automatic and adaptive way, whose main goal is to solve a given problem. To this end, several works have been implemented to provide solutions for service composition, and among these solutions, we find the use of artificial intelligence (AI) planning systems. The idea introduced in this paper is based on the presentation of our service composition approach guided by the user-intention and the context awareness, in addition to a Context-Intentional Service Composition Architecture (CISCA) for implementing this approach. In the same way, we will present an AI planning technique to solve a composition problem, namely, the Planning Domain Description Language (PDDL) and the basis for reciprocal transformation with the Web Ontology Language (OWL).
MUCS@DravidianLangTech@ACL2022: Ensemble of Logistic Regression Penalties to Identify Emotions in Tamil Text

Emotion Analysis (EA) is the process of automatically analyzing and categorizing the input text into one of the predefined sets of emotions.In recent years, people have turned to social media to express their emotions, opinions or feelings about news, movies, products, services, and so on.These users' emotions may help the public, governments, business organizations, film producers, and others in devising strategies, making decisions, and so on.The increasing number of social media users and the increasing amount of user generated text containing emotions on social media demands automated tools for the analysis of such data as handling this data manually is labor intensive and error prone.Further, the characteristics of social media data makes the EA challenging.Most of the EA research works have focused on English language leaving several Indian languages including Tamil unexplored for this task.To address the challenges of EA in Tamil texts, in this paper, we -team MUCS, describe the model submitted to the shared task on Emotion Analysis in Tamil at DravidianLangTech@ACL 2022.Out of the two subtasks in this shared task, our team submitted the model only for Task a.The proposed model comprises of an Ensemble of Logistic Regression (LR) classifiers with three penalties, namely: L1, L2, and Elasticnet.This Ensemble model trained with Term Frequency -Inverse Document Frequency (TF-IDF) of character bigrams and trigrams secured 4 th rank in Task a with a macro averaged F1-score of 0.04.The code to reproduce the proposed models is available in github 1 .
Vehicle Re-Identification Based on Quadratic Split Architecture and Auxiliary Information Embedding

Convolutional neural network (CNN) based vehicle re-identificatioin (ReID) inevitably has many disadvantages, such as information loss caused by downsampling operation. Therefore we propose a vision transformer (Vit) based vehicle ReID method to solve this problem. To improve the feature representation of vision transformer and make full use of additional vehicle information, the following methods are presented. (I) We propose a Quadratic Split Architecture (QSA) to learn both global and local features. More precisely, we split an image into many patches as “global part” and further split them into smaller sub-patches as “local part”. Features of both global and local part will be aggregated to enhance the representation ability. (II) The Auxiliary Information Embedding (AIE) is proposed to improve the robustness of the model by plugging a learnable camera/viewpoint embedding into Vit. Experimental results on several benchmarks indicate that our method is superior to many advanced vehicle ReID methods.
Implementation of Electronic Health Record and Health Insurance Management System using Blockchain Technology

Electronic health records (EHR) play an important role in digital health transition.EHRs contain medical information such as demographics, laboratory test results, radiological images, vaccination status, insurance policy, and claims.EHR is essential for doctors and healthcare organizations to analyze a patient's profile and provide appropriate therapy.Despite this, current electronic health record (EHR) systems lag with difficulties such as Interoperability and security.Better and faster care may be provided with an integrated and secure health record for each patient that can be transmitted easily in real-time across countries.People having health insurance policies are often confronted by insurance jargon and the insurer's cumbersome requirements while filing a claim for treatment.There are times when the claims processing takes longer than expected.The insurer, Third-Party Administrators (TPAs), and network provider hospitals examine, approve, and initiate the sum claimed.The use of blockchain in the process allows for more efficient information sharing at a lower cost and with more security.Only authorized individuals have access to the shared ledger on a blockchain, making it more confidential and secure.All parties engaged in a health insurance policy, including the insurer, the insured, the TPA, and the network provider hospital, may be members of the blockchain network and have access to the same set of policy data.In our proposed work we implemented a Blockchain-based EHR and Health insurance management system using Ethereum and deployed smart contracts using solidity and created a web application with web3js and React Framework.
The Wiener Filter-Based Adaptive Denoising for Pseudo Analogy Video Transmission

With the popularity of video conferences, video calls and other activities, video transmission has been widely used. To meet a huge number of subscribers’ requirements, the mobile video transmission scheme needs to overcome some disadvantages, such as resources limitation and noise interference. The knowledge-enhanced mobile video broadcasting (KMV-Cast) is a scheme utilizing joint source-channel coding and correlated information in clouds. However, there is an item of noise that cannot be removed in the originalKMV-Cast scheme. In this paper, an adaptiveWiener filtering denoising algorithm is proposed to reduce such noise at the receiver in order to maximize the signal-to-noise ratio (SNR) of the reconstructed video frame. The simulation results show that the proposed Wiener filter algorithm is superior to other schemes without the Wiener filter under different sources and channel qualities. At lower-SNR channels (i.e., -5dB), the proposed algorithm achieves 2dB gains in terms of peak signal-to-noise ratio (PSNR), while at higher-SNR channels (i.e., 10dB), the proposed algorithm achieves 3dB gains in terms of PSNR.
DEFTri: A Few-Shot Label Fused Contextual Representation Learning For Product Defect Triage in e-Commerce

Defect Triage is a time-sensitive and critical process in a large-scale agile software development lifecycle for e-commerce.Inefficiencies arising from human and process dependencies in this domain have motivated research in automated approaches using machine learning to accurately assign defects to qualified teams.This work proposes a novel framework for automated defect triage (DEFTri) using finetuned state-of-the-art pre-trained BERT on labels fused text embeddings to improve contextual representations from human-generated product defects.For our multi-label text classification defect triage task, we also introduce a Walmart proprietary dataset of product defects using weak supervision and adversarial learning, in a few-shot setting.
Entanglement Quantification and Classification: A Systematic Literature Review

Quantum entanglement is one of the essences of quantum mechanics and quantum information theory.It is a physical phenomenon in which entangled particles remain correlated with each other regardless of the distance between them.Quantum entanglement plays a significant role in areas such as quantum computing, quantum cryptography, and quantum teleportation.Quantifying entanglement is important for determining the depth of the entanglement level and has an impact on quantum information tasks performance.Entanglement classification is critical in quantum information theory for determining the class of states in a quantum system.The entanglement classification of two qubits as separable or entangled has been established.The classification of multiqubit entanglement is more challenging, especially in higher-qubit systems.The goal of this study is to identify different established measurements for entanglement quantification and entanglement classification methods through a systematic literature review.Indexed articles between 2017 and 2021 were selected as secondary resources from several sources based on specific keywords.This study presents a conceptual framework of entanglement quantification and classification based on previous studies.
Input-specific Attention Subnetworks for Adversarial Detection

Self-attention heads are characteristic of Transformer models and have been well studied for interpretability and pruning.In this work, we demonstrate an altogether different utility of attention heads, namely for adversarial detection.Specifically, we propose a method to construct input-specific attention subnetworks (IAS) from which we extract three features to discriminate between authentic and adversarial inputs.The resultant detector significantly improves (by over 7.5%) the state-of-the-art adversarial detection accuracy for the BERT encoder on 10 NLU datasets with 11 different adversarial attack types.We also demonstrate that our method (a) is more accurate for larger models which are likely to have more spurious correlations and thus vulnerable to adversarial attack, and (b) performs well even with modest training sets of adversarial examples.
Voice Biometrics for Indonesian Language Users using Algorithm of Deep Learning CNN Residual and Hybrid of DWT-MFCC Extraction Features

This research develops a Voice Biometrics model for the Indonesian language users by using deep learning algorithm of CNN Residual and Hybrid of DWT-MFCC Feature Extraction.The voice dataset of Indonesian speakers were created with a duration of 5, 10, 15, 20, and 25 minutes.The testing phase of speaker recognition and speech recognition were carried out by comparing the model of CNN Residual with CNN Standard.In the phase of speaker recognition, CNN Residual model has obtained the best results with the highest precision percentage of 99.91% and the highest accuracy of 99.47% at 25 minutes voice samples, compared to the CNN Standard obtaining precision of 96.83% and accuracy of 99.00%.In the phase of speech recognition, CNN Residual model has reached the best performance at 100% accuracy during 20 trials, while CNN Standard only gave 95% accuracy.CNN Residual Model provides a better performance for its accuracy and precision, but it is slightly slower than the CNN Standard, with a time difference of 0.03 -1.28 seconds.
Bayesian Inference-Based Gaussian Mixture Models With Optimal Components Estimation Towards Large-Scale Synthetic Data Generation for <i>In Silico</i> Clinical Trials

Goal: To develop a computationally efficient and unbiased synthetic data generator for large-scale in silico clinical trials (CTs). Methods: We propose the BGMM-OCE, an extension of the conventional BGMM (Bayesian Gaussian Mixture Models) algorithm to provide unbiased estimations regarding the optimal number of Gaussian components and yield high-quality, large-scale synthetic data at reduced computational complexity. Spectral clustering with efficient eigenvalue decomposition is applied to estimate the hyperparameters of the generator. A case study is conducted to compare the performance of BGMM-OCE against four straightforward synthetic data generators for in silico CTs in hypertrophic cardiomyopathy (HCM). Results: The BGMM-OCE generated 30000 virtual patient profiles having the lowest coefficient-of-variation (0.046), inter- and intra-correlation differences (0.017, and 0.016, respectively) with the real ones in reduced execution time. Conclusions: BGMM-OCE overcomes the lack of population size in HCM which obscures the development of targeted therapies and robust risk stratification models.
Disentangled Knowledge Transfer for OOD Intent Discovery with Unified Contrastive Learning

Discovering Out-of-Domain(OOD) intents is essential for developing new skills in a taskoriented dialogue system.The key challenge is how to transfer prior IND knowledge to OOD clustering.Different from existing work based on shared intent representation, we propose a novel disentangled knowledge transfer method via a unified multi-head contrastive learning framework.We aim to bridge the gap between IND pre-training and OOD clustering.Experiments and analysis on two benchmark datasets show the effectiveness of our method.
IAGA: Interference Aware Genetic Algorithm based VM Allocation Policy for Cloud Systems

Diversified systems hosted on cloud infrastructure have to work increasingly on physical servers.Cloud applications running on physical machines require diverse resources.The resource requirements of cloud applications are fluctuating based on the resource intensity of the applications.The multitenancy of Cloud servers can be achieved based on effective resource utilization.The optimum resource utilization, maximum service level agreement, and minimization of interference are the major objectives to be achieved.Using live Virtual Machine (VM) migration techniques cloud resources can be utilized efficiently.But the migrated VMs can interfere with the ongoing applications on the targeted server which may lead to the service level agreement violation (SLAV) and performance degradation.To resolve this issue, understanding the current state of cloud hosts before the allocation of newly migrated VM is necessary.This paper presents Interference Attentive Genetic Algorithm (IAGA) based VM allocation strategy to achieve the aforementioned objectives.The proposed IAGA policy has outperformed existing policies for quantifiable performance metrics such as energy consumed by cloud systems, count of hosts shut down, average SLAV, and count of VM migrations.
Team ÚFAL at CMCL 2022 Shared Task: Figuring out the correct recipe for predicting Eye-Tracking features using Pretrained Language Models

Eye-Tracking data is a very useful source of information to study cognition and especially language comprehension in humans.In this paper, we describe our systems for the CMCL 2022 shared task on predicting eye-tracking information.We describe our experiments with pretrained models like BERT and XLM and the different ways in which we used those representations to predict four eye-tracking features.Along with analysing the effect of using two different kinds of pretrained multilingual language models and different ways of pooling the tokenlevel representations, we also explore how contextual information affects the performance of the systems.Finally, we also explore if factors like augmenting linguistic information affect the predictions.Our submissions achieved an average MAE of 5.72 and ranked 5 th in the shared task.The average MAE showed further reduction to 5.25 in post task evaluation.