Reviewerly/exHarmony huggingface dataset:
Chinese named entity recognition for apple diseases and pests based on character augmentation

Aiming at the problems of Chinese named entity recognition in the field of apple diseases and pests, including various entities categories, entities with aliases or abbreviations, and the difficulty of identifying rare entities, we propose a novel Chinese named entity recognition model APD-CA based on character augmentation. Specifically, we incorporate dictionaries and similar words into the character-based BiLSTM-CRF model to augment character representation. To verify the validity of the model, experiments were performed on ApdCNER, a manually derived Chinese apple disease and pest corpus containing 21 entity categories. The experimental results show that the precision, recall, and F1-score of the APD-CA model based on ApdCNER are 92.29%, 91.99%, and 92.14%, respectively, which are improved compared with those for the baseline model and four other state-of-the-art models. The improvement verifies that the proposed model in this paper has performance advantages in named entity recognition in the field of apple diseases and pests. Other experimental results also prove that this model has efficiency advantages and certain generalization advantages.
Algorithm-hardware Co-design of Attention Mechanism on FPGA Devices

Multi-head self-attention (attention mechanism) has been employed in a variety of fields such as machine translation, language modeling, and image processing due to its superiority in feature extraction and sequential data analysis. This is benefited from a large number of parameters and sophisticated model architecture behind the attention mechanism. To efficiently deploy attention mechanism on resource-constrained devices, existing works propose to reduce the model size by building a customized smaller model or compressing a big standard model. A customized smaller model is usually optimized for the specific task and needs effort in model parameters exploration. Model compression reduces model size without hurting the model architecture robustness, which can be efficiently applied to different tasks. The compressed weights in the model are usually regularly shaped (e.g. rectangle) but the dimension sizes vary (e.g. differs in rectangle height and width). Such compressed attention mechanism can be efficiently deployed on CPU/GPU platforms as their memory and computing resources can be flexibly assigned with demand. However, for Field Programmable Gate Arrays (FPGAs), the data buffer allocation and computing kernel are fixed at run time to achieve maximum energy efficiency. After compression, weights are much smaller and different in size, which leads to inefficient utilization of FPGA on-chip buffer. Moreover, the different weight heights and widths may lead to inefficient FPGA computing kernel execution. Due to the large number of weights in the attention mechanism, building a unique buffer and computing kernel for each compressed weight on FPGA is not feasible. In this work, we jointly consider the compression impact on buffer allocation and the required computing kernel during the attention mechanism compressing. A novel structural pruning method with memory footprint awareness is proposed and the associated accelerator on FPGA is designed. The experimental results show that our work can compress Transformer (an attention mechanism based model) by 95x. The developed accelerator can fully utilize the FPGA resource, processing the sparse attention mechanism with the run-time throughput performance of 1.87 Tops in ZCU102 FPGA.
Knowledge distillation via instance-level sequence learning

Recently, distillation approaches for extracting general knowledge from a teacher network to guide a student network have been suggested. Most existing methods transfer knowledge from the teacher to the student network by feeding a sequence of random minibatches sampled uniformly from the data. We argue that, instead, a compact student network should be guided gradually using samples ordered in a meaningful sequence. Thus, the gap in feature representation between the teacher and student network can be bridged step by step. In this paper, we provide a curriculum learning knowledge distillation framework via instance-level sequence learning. It employs the student network of the early epoch as a snapshot to create a curriculum for the student network’s next training phase. We performed extensive experiments using the CIFAR-10, CIFAR-100, SVHN, and CINIC-10 datasets. When compared with several state-of-the-art methods, our framework achieved the best performance with fewer iterations.
Defending against GAN-based DeepFake Attacks via Transformation-aware Adversarial Faces

DeepFake represents a category of face-swapping attacks that leverage machine learning models such as autoen-coders or generative adversarial networks. Although the concept of the face-swapping is not new, its recent technical advances make fake content (e.g., images, videos) imperceptible to Humans. Various detection techniques for DeepFake attacks have been explored. These methods, however, are passive measures against DeepFakes as they are mitigation strategies after the high-quality fake content is generated. This work aims to take an offensive measure to impede the generation of high-quality fake images or videos. Specifically, we propose to use novel transformation-aware adversarially perturbed faces as a defense against GAN-based DeepFake attacks, which leverages differentiable random image transformations during the generation. We also propose an ensemble-based approach to enhance the defense robustness against GAN-based DeepFake variants under the black-box setting. We show that training a DeepFake model with adversarial faces can lead to a significant degradation in the quality of synthesized faces.
Bootstrapping more QM systems

Abstract We test the bootstrap approach for determining the spectrum of one dimensional Hamiltonians. In this paper we focus on problems that have a two parameter search space in the bootstrap approach: the double well and a periodic potential associated with the Mathieu equation. For the double well, we compare the bootstrapped energies with contributions from perturbative and non-perturbative results, finding good agreement. For the periodic potentials, we notice that the bootstrap approach gives the band structure of the periodic potential, but it has trouble finding the quasi-momentum of the system. To make further progress on determining the dispersion relation of the bands, new techniques are needed.
Deep learning approaches for workout repetition counting and validation

The study of the human body motion throughout different activities is one of the most challenging and long-standing problems in Computer Vision. With the recent advances in Deep Learning algorithms, the information acquired from conventional frame sensors can be used to infer the human body pose for further analysis. Specifically, one can contemplate the performance of body movements throughout physical exercises to provide feedback to the individual. Therefore, we propose a system for workout repetition counting and validation based on a set of skeleton-based and deep semantic features that are obtained from a 2D human pose estimation network. To this end, we have acquired over 130 participants performing five popular Cross-Fit exercises in order to train a Convolutional Neural Network to predict the exercises’ moments at a frame level. Hence, this underlying idea of inferring the moment of the exercise is two-fold: (i) to provide information about the exercise execution with a fine-level of detail; (ii) the ability to detect invalid repetitions promptly. Finally, a repetition counting and validation module receives the predicted moment and outputs the current number of valid repetitions that one has been performing with over 92% precision scores for 4 out of 5 considered exercises.
Accelerating recommendation system training by leveraging popular choices

Recommender models are commonly used to suggest relevant items to a user for e-commerce and online advertisement-based applications. These models use massive embedding tables to store numerical representation of items' and users' categorical variables (memory intensive) and employ neural networks (compute intensive) to generate final recommendations. Training these large-scale recommendation models is evolving to require increasing data and compute resources. The highly parallel neural networks portion of these models can benefit from GPU acceleration however, large embedding tables often cannot fit in the limited-capacity GPU device memory. Hence, this paper deep dives into the semantics of training data and obtains insights about the feature access, transfer, and usage patterns of these models. We observe that, due to the popularity of certain inputs, the accesses to the embeddings are highly skewed with a few embedding entries being accessed up to 10000X more. This paper leverages this asymmetrical access pattern to offer a framework, called FAE, and proposes a hot-embedding aware data layout for training recommender models. This layout utilizes the scarce GPU memory for storing the highly accessed embeddings, thus reduces the data transfers from CPU to GPU. At the same time, FAE engages the GPU to accelerate the executions of these hot embedding entries. Experiments on production-scale recommendation models with real datasets show that FAE reduces the overall training time by 2.3X and 1.52X in comparison to XDL CPU-only and XDL CPU-GPU execution while maintaining baseline accuracy.
Biophysical matter in a marine estuary identified by the Sentinel-3B OLCI satellite and the presence of terrestrial iron (Fe) nanoparticles

The analysis of marine matter using the Sentinel-3B OLCI (Ocean Land Color Instrument) satellite is the most advanced technique for evaluating: the absorption of colored detrital and dissolved material (ADG_443_NN), total suspended matter concentration (TSM_NN) and of chlorophyll-a (CHL_NN) on a global scale. The objective is to analyze ADG_443_NN, TSM_NN and CHL_NN using the Sentinel-3B OLCI satellite and the presence of Fe-nanoparticles (NPs) + hazardous elements (HEs) in suspended sediments (SSs) in the maritime estuary of the Colombian city of Barranquilla. The study used the unpublished image of the Sentinel-3B OLCI satellite in the evaluation of ADG_443_NN, TSM_NN and CHL_NN in 72 sampled points. Subsequently, 36 samples of SSs were carried out in the Magdalena River, in the identification of Fe-NPs by advanced electron microscopies. The Sentinel-3B satellite revealed particulate accumulations in OCE1 through the intensity of OLCI in ocean. There was also a high Fe-NPs intensity of SSs in the Magdalena channel, spreading contamination to large regions.
Few-Shot Object Detection by Attending to Per-Sample-Prototype

Few-shot object detection aims to detect instances of specific categories in a query image with only a handful of support samples. Although this takes less effort than obtaining enough annotated images for supervised object detection, it results in a far inferior performance compared to the conventional object detection methods. In this paper, we propose a meta-learning-based approach that con-siders the unique characteristics of each support sample. Rather than simply averaging the information of the support samples to generate a single prototype per category, our method can better utilize the information of each support sample by treating each support sample as an individual prototype. Specifically, we introduce two types of attention mechanisms for aggregating the query and support feature maps. The first is to refine the information of few-shot samples by extracting shared information between the support samples through attention. Second, each support sample is used as a class code to leverage the information by comparing similarities between each support feature and query features. Our proposed method is complementary to the previous methods, making it easy to plug and play for further improvement. We have evaluated our method on PASCAL VOC and COCO benchmarks, and the results verify the effectiveness of our method. In particular, the advantages of our method are maximized when there is more diversity among support data.
Automating Seccomp Filter Generation for Linux Applications

Software vulnerabilities undermine the security of applications. By blocking unused functionality, the impact of potential exploits can be reduced. While seccomp provides a solution for filtering syscalls, it requires manual implementation of filter rules for each individual application. Recent work has investigated approaches to automate this task. However, as we show, these approaches make assumptions that are not necessary or require overly time-consuming analysis.
Complemental Attention Multi-Feature Fusion Network for Fine-Grained Classification

Transformer-based architecture network has shown excellent performance in the coarse-grained image classification. However, it remains a challenge for the fine-grained image classification task, which needs more significant regional information. As one of the attention mechanisms, transformer pays attention to the most significant region while neglecting other sub-significant regions. To use more regional information, in this letter, we propose a complemental attention multi-feature fusion network (CAMF), which extracts multiple attention features to obtain more effective features. In CAMF, we propose two novel modules: (i) a complemental attention module (CAM) that extracts the most salient attention feature and the complemental attention feature. (ii) a multi-feature fusion module (MFM) that uses different branches to extract multiple regional discriminative features. Furthermore, a new feature similarity loss is proposed to measure the diversity of inter-class features. Experiments were conducted on four public fine-grained classification datasets. Our CAMF achieves 91.2%, 92.8%, 93.3%, 95.3% on CUB-200-2011, Stanford Dogs, FGVC-Aircraft, and Stanford Cars. The ablation study verified that CAM and MFM can focus on more local discriminative regions and improve fine-grained classification performance.
Parameter-Free Loss for Class-Imbalanced Deep Learning in Image Classification

Current state-of-the-art class-imbalanced loss functions for deep models require exhaustive tuning on hyperparameters for high model performance, resulting in low training efficiency and impracticality for nonexpert users. To tackle this issue, a parameter-free loss (PF-loss) function is proposed, which works for both binary and multiclass-imbalanced deep learning for image classification tasks. PF-loss provides three advantages: 1) training time is significantly reduced due to NO tuning on hyperparameter(s); 2) it dynamically pays more attention on minority classes (rather than outliers compared to the existing loss functions) with NO hyperparameters in the loss function; and 3) higher accuracy can be achieved since it adapts to the changes of data distribution in each mini-batch instead of the fixed hyperparameters in the existing methods during training, especially when the data are highly skewed. Experimental results on some classical image datasets with different imbalance ratios (IR, up to 200) show that PF-loss reduces the training time down to 1/148 of that spent by compared state-of-the-art losses and simultaneously achieves comparable or even higher accuracy in terms of both G-mean and area under receiver operating characteristic (ROC) curve (AUC) metrics, especially when the data are highly skewed.
Improved Manta Ray Foraging Optimization for Parameters Identification of Magnetorheological Dampers

Magnetorheological (MR) dampers play a crucial role in various engineering systems, and how to identify the control parameters of MR damper models without any prior knowledge has become a burning problem. In this study, to identify the control parameters of MR damper models more accurately, an improved manta ray foraging optimization (IMRFO) is proposed. The new algorithm designs a searching control factor according to a weak exploration ability of MRFO, which can effectively increase the global exploration of the algorithm. To prevent the premature convergence of the local optima, an adaptive weight coefficient based on the Levy flight is designed. Moreover, by introducing the Morlet wavelet mutation strategy to the algorithm, the mutation space is adaptively adjusted to enhance the ability of the algorithm to step out of stagnation and the convergence rate. The performance of the IMRFO is evaluated on two sets of benchmark functions and the results confirm the competitiveness of the proposed algorithm. Additionally, the IMRFO is applied in identifying the control parameters of MR dampers, the simulation results reveal the effectiveness and practicality of the IMRFO in the engineering applications.
Do cryptocurrencies really have (no) intrinsic value?

Depending on the chosen perspective, cryptocurrencies either constitute a unique opportunity to end national patronizing built on debt-based fiat money (currency) or a menace to a well-established financial order that ensures economic stability. A central issue in the heated debate surrounding cryptocurrencies is whether they have any intrinsic value at all. In this conceptual position paper, we briefly summarize existing standpoints and suggest three alternative propositions: (1) to avoid using the term "intrinsic value" for the valuation of cryptocurrencies, (2) to refer to the sum total of all properties that could potentially qualify them as money, and (3) to consider the amount of capital and energy that is needed to create them. These suggestions bear substantial implications for the economic classification of cryptocurrencies.
Demystifying Pythia: A Survey of ChainLink Oracles Usage on Ethereum

Smart contracts are dependent on oracle systems for their adoption and usability. We perform an empirical study of oracle systems' usage trends and adoption metrics to provide better insight into the health of the smart contract ecosystem. We collect ChainLink usage data on the Ethereum network using a modified Ethereum client and running a full node. We analyze the collected data and present our findings and insights surrounding the usage trends, adoption metrics, oracle pricing and service quality associated with ChainLink on the Ethereum network. We infer that ChainLink's usage and growth are dominated by the DeFi ecosystem and for its demand for decentralized price feeds.
Non-Greedy Algorithms for Decision Tree Optimization: An Experimental Comparison

Learning decision trees is a difficult optimization problem: nonconvex, nondifferentiable and over a huge number of tree structures. The dominant paradigm in practice, established in the 1980s, are axis-aligned trees trained with a greedy recursive partitioning algorithm such as CART or C5.0. Several non-greedy optimization algorithms have been proposed recently, which optimize all the nodes' parameters jointly, and we compare experimentally some of them in a range of classification and regression datasets, in terms of accuracy, training time and tree size. The non-greedy algorithms do not improve over CART significantly with one exception, tree alternating optimization (TAO). TAO scales to large datasets and produces axis-aligned and especially oblique trees that consistently outperform all other algorithms, often by a large margin. TAO makes oblique trees preferable to axis-aligned ones in many cases, since they are much more accurate while remaining small and interpretable. This suggests a change in paradigm in practical applications of decision trees.
Improving low-resource machine transliteration by using 3-way transfer learning

Transfer learning brings improvement to machine translation by using a resource-rich language pair to pretrain the model and then adapting it to the desired language pair. However, to date, there have been few attempts to tackle machine transliteration with transfer learning. In this article, we propose a method of using source–pivot and pivot–target datasets to improve source–target machine transliteration. Our approach first bridges the source–pivot and pivot–target datasets by reducing the distance between source and pivot embeddings. Then, our model learns to translate from the pivot language to the target language. Finally, the source–target dataset is used to fine tune the model. Our experiments show that our method is superior to the transfer learning method. When implemented with a state-of-the-art source–target translation model from NEWS’18, our transfer learning method can improve the accuracy by 1.1%. • Unifying the representation of source and pivot language can improve performance. • Romanization increases performance of transliteration for non-phonogram language. • Parameter sharing is more effective than data sharing. • Performance can be increased by adding more auxiliary datasets.
Self-testing of a single quantum device under computational assumptions

Self-testing is a method to characterise an arbitrary quantum system based only on its classical input-output correlations, and plays an important role in device-independent quantum information processing as well as quantum complexity theory. Prior works on self-testing require the assumption that the system's state is shared among multiple parties that only perform local measurements and cannot communicate. Here, we replace the setting of <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow class="MJX-TeXAtom-ORD"><mml:mtext class="MJX-tex-mathit" mathvariant="italic">multiple non-communicating</mml:mtext></mml:mrow></mml:math> parties, which is difficult to enforce in practice, by a <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"><mml:mrow class="MJX-TeXAtom-ORD"><mml:mtext class="MJX-tex-mathit" mathvariant="italic">single computationally bounded</mml:mtext></mml:mrow></mml:math> party. Specifically, we construct a protocol that allows a classical verifier to robustly certify that a single computationally bounded quantum device must have prepared a Bell pair and performed single-qubit measurements on it, up to a change of basis applied to both the device's state and measurements. This means that under computational assumptions, the verifier is able to certify the presence of entanglement, a property usually closely associated with two separated subsystems, inside a single quantum device. To achieve this, we build on techniques first introduced by Brakerski et al. (2018) and Mahadev (2018) which allow a classical verifier to constrain the actions of a quantum device assuming the device does not break post-quantum cryptography.