Reviewerly/exHarmony huggingface dataset:
Automatic underwater fish species classification with limited data using few-shot learning

Underwater cameras are widely used to monitor marine biodiversity, and the trend is increasing due to the availability of cheap action cameras. The main bottleneck of video methods now resides in the manual processing of images, a time-consuming task requiring trained experts. Recently, several solutions based on Deep Learning (DL) have been proposed to automatically process underwater videos. The main limitation of such algorithms is that they require thousands of annotated images in order to learn to discriminate classes (here species). This limitation implies two issues: 1) the annotation of hundreds of common species requires a lot of efforts 2) many species are too rare to gather enough data to train a classic DL algorithm. Here, we propose to explore how few-shot learning (FSL), an emerging research field, could overcome DL limitations. Few-shot learning is based on the principle of training a Deep Learning algorithm on “how to learn a new classification problem with only few images”. In our case-study, we assess the robustness of FSL to discriminate 20 coral reef fish species with a range of training databases from 1 image per class to 30 images per class, and compare FSL to a classic DL approach with thousands of images per class. We found that FSL outperform classic DL approach in situations where annotated images are limited, yet still providing good classification accuracy.
Adaptive Federated Learning for Digital Twin Driven Industrial Internet of Things

Industrial Internet of Things (IoT) enables distributed intelligent services varying with the complex industrial environment to achieve the benefits of Industry 4.0. In this paper, we consider a new architecture of digital twin empowered Industrial IoT, in which digital twins capture characteristics of industrial devices to assist federated learning tasks of industrial scenarios. A trust-based aggregation is proposed in federated learning to alleviate the effects of digital twins deviation and emphasize the contribution of high-performance clients. Based on Lyapunov dynamic deficit queue and deep reinforcement learning, we propose a federated learning framework that adaptively adjusts the aggregation frequency to improve the learning performance under resource constraints. Numerical results show that the proposed framework outperforms the benchmark in terms of learning accuracy, convergence, and energy saving.
SIEA: Secure Image Encryption Algorithm Based on Chaotic Systems Optimization Algorithms and PUFs

One of the general problems in modern digital society is undoubtedly the information security topic. It is critical to ensure the security of information transferred, processed, and stored throughout digital channels. Among this information, digital images draw attention in terms of frequency of use in digital channels. In this study, a new image encryption algorithm is proposed to address the security problems of digital images. The aspect that differentiates the proposed algorithm from thousands of image encryption algorithms in the literature is that it is designed within the framework of the provable security design principle. The provable security design approach has ensured that the proposed algorithm is theoretically secure with mathematical proof techniques. In addition to addressing the proposed architecture security concerns, the hybrid random number generator used as the key generator constitutes another unique aspect. This generator, which was designed using chaotic systems, physical unclonable functions, and optimization algorithms, stands out as the innovative aspect of the study. The statistical randomness properties of the proposed random number generator were tested using the NIST SP 800-22 Statistical Test Suite. Successful results were obtained for 15 tests in the test package. In addition, the success of these outputs was tested on a new image encryption algorithm. The security of the proposed algorithm was tested from different angles using various experimental analyzes and a 12-step provable security analysis roadmap. Successful analysis results and performance measurements indicate that the proposed cryptographic components can be used in many information security applications and many future designs.
Sentiment Analysis on Covid19 Vaccines in Indonesia: From The Perspective of Sinovac and Pfizer

The Covid-19 pandemic that hit the world, including in Indonesia, had a significant impact. Casualties, economic downturn, extreme poverty, and major changes in education are still happening today. The presence of the Covid19 vaccine is new hope for mankind to end this pandemic situation. The emergence of two types of vaccines in Indonesia, Sinovac, and Pfizer, lead to different Indonesian society reactions. This study aims to do a sentiment analysis of the two types of vaccines on the Twitter platform. Data from October until November 2020 has been crawled and processed to see the citizen opinion. The dataset was split into two types: Sinovac and Pfizer dataset. Both datasets were labeled manually into three classes: positive, negative, and neutral. The results show that 77% of Tweets indicate the positive segments, while 19% represent negative, and 4% seem to be neutral for Sinovac. From the standpoint of Pfizer, the results were 81%, 17%, and 3% for positive, negative, and neutral, respectively. In terms of model performance evaluation, with 10-fold cross-validation, the highest average accuracy in the Sinovac dataset is Support Vector Machine with 85% accuracy. Furthermore, the Support Vector Machine classifier has a superior accuracy value of 78% in the Pfizer dataset compared to other classifiers.
Internal Language Model Training for Domain-Adaptive End-To-End Speech Recognition

The efficacy of external language model (LM) integration with existing end-to-end (E2E) automatic speech recognition (ASR) systems can be improved significantly using the internal language model estimation (ILME) method [1]. In this method, the internal LM score is subtracted from the score obtained by interpolating the E2E score with the external LM score, during inference. To improve the ILME-based inference, we propose an internal LM training (ILMT) method to minimize an additional internal LM loss by updating only the E2E model components that affect the internal LM estimation. ILMT encourages the E2E model to form a standalone LM inside its existing components, without sacrificing ASR accuracy. After ILMT, the more modular E2E model with matched training and inference criteria enables a more thorough elimination of the source-domain internal LM, and therefore leads to a more effective integration of the target-domain external LM. Experimented with 30K-hour trained recurrent neural network transducer and attention-based encoder- decoder models, ILMT with ILME-based inference achieves up to 31.5% and 11.4% relative word error rate reductions from standard E2E training with Shallow Fusion on out-of-domain LibriSpeech and in-domain Microsoft production test sets, respectively.
DocNLI: A Large-scale Dataset for Document-level Natural Language Inference

Natural language inference (NLI) is formulated as a unified framework for solving various NLP problems such as relation extraction, question answering, summarization, etc.It has been studied intensively in the past few years thanks to the availability of large-scale labeled datasets.However, most existing studies focus on merely sentence-level inference, which limits the scope of NLI's application in downstream NLP problems.This work presents DOCNLI -a newly-constructed large-scale dataset for document-level NLI.DOCNLI is transformed from a broad range of NLP problems and covers multiple genres of text.The premises always stay in the document granularity, whereas the hypotheses vary in length from single sentences to passages with hundreds of words.Additionally, DOCNLI has pretty limited artifacts 1 which unfortunately widely exist in some popular sentence-level NLI datasets.Our experiments demonstrate that, even without fine-tuning, a model pretrained on DOCNLI shows promising performance on popular sentence-level benchmarks, and generalizes well to out-of-domain NLP tasks that rely on inference at document granularity.Task-specific fine-tuning can bring further improvements.Data, code and pretrained models can be found at https://github.com/salesforce/DocNLI.
CoMAE: A Multi-factor Hierarchical Framework for Empathetic Response Generation

The capacity of empathy is crucial to the success of open-domain dialog systems.Due to its nature of multi-dimensionality, there are various factors that relate to empathy expression, such as communication mechanism, dialog act and emotion.However, existing methods for empathetic response generation usually either consider only one empathy factor or ignore the hierarchical relationships between different factors, leading to a weak ability of empathy modeling.In this paper, we propose a multi-factor hierarchical framework, CoMAE, for empathetic response generation, which models the above three key factors of empathy expression in a hierarchical way.We show experimentally that our CoMAEbased model can generate more empathetic responses than previous methods.We also highlight the importance of hierarchical modeling of different factors through both the empirical analysis on a real-life corpus and the extensive experiments.
End-to-End Training of Neural Retrievers for Open-Domain Question Answering

Recent work on training neural retrievers foropen-domain question answering (OpenQA) has employed both supervised and unsupervised approaches.However, it remains unclear how unsupervised and supervised methods can be used most effectively for neural retrievers.In this work, we systematically study retriever pre-training.We first propose an approach of unsupervised pre-training with the Inverse Cloze Task and masked salient spans, followed by supervised finetuning using question-context pairs.This approach leads to absolute gains of 2+ points over the previous best result in the top-20 retrieval accuracy on Natural Questions and TriviaQA datasets.We next explore two approaches for end-toend training of the reader and retriever components in OpenQA models, which differ in the manner the reader ingests the retrieved documents.Our experiments demonstrate the effectiveness of these approaches as we obtain state-of-the-art results.On the Natural Questions dataset, we obtain a top-20 retrieval accuracy of 84%, an improvement of 5 points over the recent DPR model.We also showcase good results on answer extraction, outperforming recent models such as REALM and RAG by 3+ points.
Improved Programmable Bootstrapping with Larger Precision and Efficient Arithmetic Circuits for TFHE

Fully Homomorphic Encryption (FHE) schemes enable to compute over encrypted data. Among them, TFHE [] has the great advantage of offering an efficient method for bootstrapping noisy ciphertexts, i.e., reduce the noise. Indeed, homomorphic computation increases the noise in ciphertexts and might compromise the encrypted message. TFHE bootstrapping, in addition to reducing the noise, also evaluates (for free) univariate functions expressed as look-up tables. It however requires to have the most significant bit of the plaintext to be known a priori, resulting in the loss of one bit of space to store messages. Furthermore it represents a non negligible overhead in terms of computation in many use cases.
COVID-Fact: Fact Extraction and Verification of Real-World Claims on COVID-19 Pandemic

We introduce a FEVER-like dataset COVID-Fact of 4, 086 claims concerning the COVID-19 pandemic.The dataset contains claims, evidence for the claims, and contradictory claims refuted by the evidence.Unlike previous approaches, we automatically detect true claims and their source articles and then generate counter-claims using automatic methods rather than employing human annotators.Along with our constructed resource, we formally present the task of identifying relevant evidence for the claims and verifying whether the evidence refutes or supports a given claim.In addition to scientific claims, our data contains simplified general claims from media sources, making it better suited for detecting general misinformation regarding COVID-19.Our experiments indicate that COVID-Fact will provide a challenging testbed for the development of new systems and our approach will reduce the costs of building domainspecific datasets for detecting misinformation. Original ClaimClosed environments facilitate secondary transmission of coronavirus disease 2019 Counter-Claim Closed environments prevent secondary transmission of coronavirus disease 2019 Gold Document https://www.medrxiv.org/content/10.1101/2020.02.28.20029272v2 Gold EvidenceIt is plausible that closed environments contribute to secondary transmission of COVID-19 and promote superspreading events.Original Claim Oxford vaccine triggers immune response Counter-Claim Oxford vaccine inhibits immune response Gold Document https://www.bbc.com/news/uk-53469839 Gold EvidenceThey are injecting coronavirus RNA (its genetic code), which then starts making viral proteins in order to trigger an immune response.
What the Role is vs. What Plays the Role: Semi-Supervised Event Argument Extraction via Dual Question Answering

Event argument extraction is an essential task in event extraction, and become particularly challenging in the case of low-resource scenarios. We solve the issues in existing studies under low-resource situations from two sides. From the perspective of the model, the existing methods always suffer from the concern of insufficient parameter sharing and do not consider the semantics of roles, which is not conducive to dealing with sparse data. And from the perspective of the data, most existing methods focus on data generation and data augmentation. However, these methods rely heavily on external resources, which is more laborious to create than obtain unlabeled data. In this paper, we propose DualQA, a novel framework, which models the event argument extraction task as question answering to alleviate the problem of data sparseness and leverage the duality of event argument recognition which is to ask "What plays the role", as well as event role recognition which is to ask "What the role is", to mutually improve each other.Experimental results on two datasets prove the effectiveness of our approach, especially in extremely low-resource situations.
Few-Shot Event Detection with Prototypical Amortized Conditional Random Field

Event detection tends to struggle when it needs to recognize novel event types with a few samples.The previous work attempts to solve this problem in the identify-then-classify manner but ignores the trigger discrepancy between event types, thus suffering from the error propagation.In this paper, we present a novel unified model which converts the task to a few-shot tagging problem with a double-part tagging scheme.To this end, we first propose the Prototypical Amortized Conditional Random Field (PA-CRF) to model the label dependency in the few-shot scenario, which approximates the transition scores between labels based on the label prototypes.Then Gaussian distribution is introduced for modeling of the transition scores to alleviate the uncertain estimation resulting from insufficient data.Experimental results show that the unified models work better than existing identifythen-classify models and our PA-CRF further achieves the best results on the benchmark dataset FewEvent.
Out-of-Distribution Detection Using Union of 1-Dimensional Subspaces

The goal of out-of-distribution (OOD) detection is to handle the situations where the test samples are drawn from a different distribution than the training data. In this paper, we argue that OOD samples can be detected more easily if the training data is embedded into a low-dimensional space, such that the embedded training samples lie on a union of 1-dimensional subspaces. We show that such embedding of the in-distribution (ID) samples provides us with two main advantages. First, due to compact representation in the feature space, OOD samples are less likely to occupy the same region as the known classes. Second, the first singular vector of ID samples belonging to a 1-dimensional subspace can be used as their robust representative. Motivated by these observations, we train a deep neural network such that the ID samples are embedded onto a union of 1-dimensional subspaces. At the test time, employing sampling techniques used for approximate Bayesian inference in deep learning, input samples are detected as OOD if they occupy the region corresponding to the ID samples with probability 0. Spectral components of the ID samples are used as robust representative of this region. Our method does not have any hyperparameter to be tuned using extra information and it can be applied on different modalities with minimal change. The effectiveness of the proposed method is demonstrated on different benchmark datasets, both in the image and video classification domains.
HybridCTrm: Bridging CNN and Transformer for Multimodal Brain Image Segmentation

Multimodal medical image segmentation is always a critical problem in medical image segmentation. Traditional deep learning methods utilize fully CNNs for encoding given images, thus leading to deficiency of long-range dependencies and bad generalization performance. Recently, a sequence of Transformer-based methodologies emerges in the field of image processing, which brings great generalization and performance in various tasks. On the other hand, traditional CNNs have their own advantages, such as rapid convergence and local representations. Therefore, we analyze a hybrid multimodal segmentation method based on Transformers and CNNs and propose a novel architecture, HybridCTrm network. We conduct experiments using HybridCTrm on two benchmark datasets and compare with HyperDenseNet, a network based on fully CNNs. Results show that our HybridCTrm outperforms HyperDenseNet on most of the evaluation metrics. Furthermore, we analyze the influence of the depth of Transformer on the performance. Besides, we visualize the results and carefully explore how our hybrid methods improve on segmentations.
Semantic Reinforced Attention Learning for Visual Place Recognition

Large-scale visual place recognition (VPR) is inherently challenging because not all visual cues in the image are beneficial to the task. In order to highlight the task-relevant visual cues in the feature embedding, the existing attention mechanisms are either based on artificial rules or trained in a thorough data-driven manner. To fill the gap between the two types, we propose a novel Semantic Reinforced Attention Learning Network (SRALNet), in which the inferred attention can benefit from both semantic priors and data-driven fine-tuning. The contribution lies in two-folds. (1) To suppress misleading local features, an interpretable local weighting scheme is proposed based on hierarchical feature distribution. (2) By exploiting the interpretability of the local weighting scheme, a semantic constrained initialization is proposed so that the local attention can be reinforced by semantic priors. Experiments demonstrate that our method outperforms state-of-the-art techniques on city-scale VPR benchmark datasets.
Timely Diagnosis of Acute Lymphoblastic Leukemia Using Artificial Intelligence-Oriented Deep Learning Methods

Background. Leukemia is fatal cancer in both children and adults and is divided into acute and chronic. Acute lymphoblastic leukemia (ALL) is a subtype of this cancer. Early diagnosis of this disease can have a significant impact on the treatment of this disease. Computational intelligence-oriented techniques can be used to help physicians identify and classify ALL rapidly. Materials and Method. In this study, the utilized dataset was collected from a CodaLab competition to classify leukemic cells from normal cells in microscopic images. Two famous deep learning networks, including residual neural network (ResNet-50) and VGG-16 were employed. These two networks are already trained by our assigned parameters, meaning we did not use the stored weights; we adjusted the weights and learning parameters too. Also, a convolutional network with ten convolutional layers and <math xmlns="http://www.w3.org/1998/Math/MathML" id="M1"> <mn>2</mn> <mi>∗</mi> <mn>2</mn> </math> max-pooling layers—with strides 2—was proposed, and six common machine learning techniques were developed to classify acute lymphoblastic leukemia into two classes. Results. The validation accuracies (the mean accuracy of training and test networks for 100 training cycles) of the ResNet-50, VGG-16, and the proposed convolutional network were found to be 81.63%, 84.62%, and 82.10%, respectively. Among applied machine learning methods, the lowest obtained accuracy was related to multilayer perceptron (27.33%) and highest for random forest (81.72%). Conclusion. This study showed that the proposed convolutional neural network has optimal accuracy in the diagnosis of ALL. By comparing various convolutional neural networks and machine learning methods in diagnosing this disease, the convolutional neural network achieved good performance and optimal execution time without latency. This proposed network is less complex than the two pretrained networks and can be employed by pathologists and physicians in clinical systems for leukemia diagnosis.
A RISC-V Post Quantum Cryptography Instruction Set Extension for Number Theoretic Transform to Speed-Up CRYSTALS Algorithms

In recent years, public-key cryptography and digital signature have become fundamental components of digital infrastructures. Such a scenario has to face a new and increasing threat, represented by quantum computers. It is well known that quantum computers in the next years will be able to run algorithms capable of breaking the security of currently widespread cryptographic schemes used for public-key encryption and digital signatures. Post-quantum cryptography aims to defining and executing algorithms on classical computer architectures, capable to withstand attacks from quantum computers. The National Institute of Standards and Technology is currently running a selection process to define one or more quantum-resistant public-key algorithms and lattice-based cryptographic constructions are considered one of the leading candidates. However, such algorithms require non-negligible computational resources to be executed. One viable solution is to accelerate them totally or partially in hardware, to alleviate the workload of the main processing unit. In this paper, we investigate a solution trading-off performances and complexity to execute the lattice-based algorithms CRYSTALS-Kyber and -Dilithium: we introduce a dedicated Post- Quantum Arithmetic Logic Unit, embedded directly in the pipeline of a RISC-V processor. This results in an almost negligible area overhead with a large impact on the algorithm speed-up and a consistent reduction in the energy required per single operation.