Reviewerly/exHarmony huggingface dataset:
A Low-Cost Reconfigurable Nonlinear Core for Embedded DNN Applications

Nonlinear layers (NLs) are indispensable parts of deep neural networks (DNNs). Targeting different inference tasks, NLs vary with different DNN models. Based on input range reduction and piecewise polynomial approximation, this paper proposes a low-cost reconfigurable nonlinear core to accelerate diverse NLs. Three types of NLs are supported: pooling operations like max-pooling and average-pooling, classifiers like softmax and log-softmax, and element-wise functions like GELU, sigmoid and swish. By configuring the data paths and contents of the lookup tables (LUTs), a function can be easily implemented by the approximation-based method. The proposed core contains only two low bit-width multipliers, two adders, and one subtractor as computing units. Their full utilization has been achieved for all operations except pooling. A lossless bit-width reduction scheme and a coefficient rearrangement scheme are proposed to reduce the multiplier bit widths and the LUT sizes. When implemented on a Xilinx Zynq-7000 ZC706 FPGA board, our design can utilize a high-speed DSP block as two independent multipliers. Compared with the state-of-the-art nonlinear cores, the proposed core achieves higher precision for functions like sigmoid and tanh, and higher throughput for functions like softmax and log-softmax. Besides, the proposed core can support more nonlinear operations. Experiments on the latest DNNs such as BERT and EfficientNet show that our design can still achieve high accuracies with less than 0.2% loss.
Simplified Timed Attack Trees

This paper considers attack trees, a graphical security model that can be used to visualised varying ways an asset may be compromised. We proposed an extension of this model, termed Simplified Timed Attack Trees (STAT). The primary aim here is to use STAT for the analysis of CPS assets. STAT extend attack trees gate refinements with time parameters. In order to reach a parent node, an attacker has to achieve the child nodes within a specified time interval. This adds a level of security to the gates refinement. We propose how to translate STAT to a parallel composition of timed automata (TA). We reduce the root reachability of STAT to place reachability in the TA. This reachability can be checked; using a formal verification tool UPPAAL.
Research on Detection Algorithm of Foreign Object Debris and Small Targets in Airport Runway Based on SSD

The SSD algorithm is currently a widely used target detection algorithm. In view of the small size of common foreign objects on airport runways and the low accuracy of SSD processing small target objects, this paper applies feature fusion method in SSD model to construct a new small target object detection algorithm model. The accuracy of the model is detected on the self-built airport runway foreign body data set, and the detection result mAP is 5% higher than the original SSD network.
Mobile Service Traffic Classification Based on Joint Deep Learning With Attention Mechanism

With the rapid development of mobile devices, smartphones have become the chief access to Internet and generated huge mobile service traffic. Mobile service traffic classification (MSTC) has been an important task that contributes to providing personalized services for end-users. With the excellent ability of automatic feature learning, deep learning has better performance than traditional machine learning methods. Giving more attention to a local focus, the attention mechanism can reduce computational complexity by filtering out useless information. Therefore, deep learning with attention mechanism can effectively realize automatic feature learning and reduce computational complexity. In this paper, a novel method for MSTC with a two-step strategy is proposed, which reduces the computational complexity of the deep learning model by attention mechanism. In the first step, a joint deep learning model is designed as a basic classifier, which learns features of mobile service traffic from multiple time scales. In the second step, the attention mechanism is adopted to aggregates the basic predictions generated in the first step. To verify this methodology, an experiment is performed to classify seven mobile services. The results show that we get the mean F1-score of 92.7% with 3.1 seconds time-delay, where the pure deep learning model gets the highest mean F1-score of 90.4% with 6.7 seconds time-delay.
Color-encoded single-shot computer-generated Moiré profilometry

Abstract A color-encoded single-shot computer-generated Moiré profilometry (CSCGMP) is proposed. Two sinusoidal gratings with a π phase difference are encoded in red and blue channels respectively to combine a composite color grating. While this composite color grating is projected onto the measured object, the corresponding color deformed pattern can be captured. So two deformed patterns with a π phase difference are separated from its red and blue components respectively. After normalization and subtraction, the AC component of both separated deformed patterns can be extracted. If this AC component respectively multiplied by the two AC components of fringe patterns of reference plane with a π /2 phase difference prepared and saved on the computer in advance, two computer-generated Moiré fringes just respectively standing for sine and cosine of phase which is modulated by the height of the object relative to the reference plane are figured out. So the 3D shape of the measured object can be reconstructed with normal computer-generated Moiré profilometry. Both simulation and experimental results show the feasibility and validity of the proposed method. It has potential in real-time 3D measurement due to its single-shot feature.
Real-Time Turkish Sign Language Recognition Using Cascade Voting Approach with Handcrafted Features

Abstract In this study, a machine learning-based system, which recognises the Turkish sign language person-independent in real-time, was developed. A leap motion sensor was used to obtain raw data from individuals. Then, handcraft features were extracted by using Euclidean distance on the raw data. Handcraft features include finger-to-finger, finger -to-palm, finger -to-wrist bone, palm-to-palm and wrist-to-wrist distances. LR, k -NN, RF, DNN, ANN single classifiers were trained using the handcraft features. Cascade voting approach was applied with two-step voting. The first voting was applied for each classifier’s final prediction. Then, the second voting, which voted the prediction of all classifiers at the final decision stage, was applied to improve the performance of the proposed system. The proposed system was tested in real-time by an individual whose hand data were not involved in the training dataset. According to the results, the proposed system presents 100 % value of accuracy in the classification of one hand letters. Besides, the recognition accuracy ratio of the system is 100 % on the two hands letters, except “J” and “H” letters. The recognition accuracy rates were 80 % and 90 %, respectively for “J” and “H” letters. Overall, the cascade voting approach presented a high average classification performance with 98.97 % value of accuracy. The proposed system enables Turkish sign language recognition with high accuracy rates in real time.
WeNet: Production oriented Streaming and Non-streaming End-to-End Speech Recognition Toolkit

In this paper, we propose an open source, production first, and production ready speech recognition toolkit called WeNet in which a new two-pass approach is implemented to unify streaming and non-streaming end-to-end (E2E) speech recognition in a single model. The main motivation of WeNet is to close the gap between the research and the production of E2E speechrecognition models. WeNet provides an efficient way to ship ASR applications in several real-world scenarios, which is the main difference and advantage to other open source E2E speech recognition toolkits. In our toolkit, a new two-pass method is implemented. Our method propose a dynamic chunk-based attention strategy of the the transformer layers to allow arbitrary right context length modifies in hybrid CTC/attention architecture. The inference latency could be easily controlled by only changing the chunk size. The CTC hypotheses are then rescored by the attention decoder to get the final result. Our experiments on the AISHELL-1 dataset using WeNet show that, our model achieves 5.03\% relative character error rate (CER) reduction in non-streaming ASR compared to a standard non-streaming transformer. After model quantification, our model perform reasonable RTF and latency.
Deep neural networks for automatic grain-matrix segmentation in plane and cross-polarized sandstone photomicrographs

Grain segmentation of sandstone that is partitioning the grain from its surrounding matrix/cement in the thin section is the primary step for computer-aided mineral identification and sandstone classification. The photomicrograph of sandstone contain many mineral grains and their surrounding matrix/cement. The distinction between adjacent grains and the matrix is often ambiguous, making grain segmentation difficult. Various solutions exist in literature to handle these problems; however, they are not robust against sandstone petrography's varied pattern. In this paper, we formulate grain segmentation as a pixel-wise two-class (i.e., grain and background) semantic segmentation task. We develop a deep learning-based end-to-end trainable framework named Deep Semantic Grain Segmentation network (dsgsn), a data-driven method, and provide a generic solution. As per the authors' knowledge, this is the first work where the deep neural network is explored to solve the grain segmentation problem. Extensive experiments on the images highlight that our method obtains better segmentation accuracy than various segmentation architectures with more parameters.
Product Identification System Using Block chain

In recent years, block chain has received increasing attention and various applications have emerged from this technology. A renowned Blockchain application is that the cryptocurrency Bitcoin, that has not only been effectively solving the double-spending problem but also it can confirm the legitimacy of transactional records without counting on a centralized system to try and do so. Therefore, any application using Blockchain technology because the base architecture ensures that the contents of its data are tamper-proof. This paper uses the decentralized Blockchain technology approach to confirm that buyers don't fully depend on the merchants to work out if products are genuine. We describe a decentralized Blockchain system with products anti-counterfeiting; therein way manufacturers can use this method to supply genuine products without having to manage direct-operated stores, which may significantly reduce the value of product quality assurance.
Stereo Matching Algorithm Based on Hybrid Convolutional Neural Network and Directional Intensity Difference

Fundamentally, a stereo matching algorithm produces a disparity map or depth map. This map contains valuable information for many applications, such as range estimation, autonomous vehicle navigation and 3D surface reconstruction. The stereo matching process faces various challenges to get an accurate result for example low texture area, repetitive pattern and discontinuity regions. The proposed algorithm must be robust and viable with all of these challenges and is capable to deliver good accuracy. Hence, this article proposes a new stereo matching algorithm based on a hybrid Convolutional Neural Network (CNN) combined with directional intensity differences at the matching cost stage. The proposed algorithm contains a deep learning-based method and a handcrafted method. Then, the bilateral filter is used to aggregate the matching cost volume while preserving the object edges. The Winner-Take-All (WTA) is utilized at the optimization stage which the WTA normalizes the disparity values. At the last stage, a series of refinement processes will be applied to enhance the final disparity map. A standard benchmarking evaluation system from the Middlebury Stereo dataset is used to measure the algorithm performance. This dataset provides images with the characteristics of low texture area, repetitive pattern and discontinuity regions. The average error produced for all pixel regions is 8.51%, while the nonoccluded region is 5.77%. Based on the experimental results, the proposed algorithm produces good accuracy and robustness against the stereo matching challenges. It is also competitive with other published methods and can be used as a complete algorithm
Enhancing trajectory prediction using sparse outputs: application to team sports

Sophisticated trajectory prediction models that effectively mimic team dynamics have many potential uses for sports coaches, broadcasters and spectators. However, through experiments on soccer data we found that it can be surprisingly challenging to train a deep learning model for player trajectory prediction which outperforms linear extrapolation on average distance between predicted and true future trajectories. We propose and test a novel method for improving training by predicting a sparse trajectory and interpolating using constant acceleration, which improves performance for several models. This interpolation can also be used on models that aren't trained with sparse outputs, and we find that this consistently improves performance for all tested models. Additionally, we find that the accuracy of predicted trajectories for a subset of players can be improved by conditioning on the full trajectories of the other players, and that this is further improved when combined with sparse predictions. We also propose a novel architecture using graph networks and multi-head attention (GraN-MA) which achieves better performance than other tested state-of-the-art models on our dataset and is trivially adapted for both sparse trajectories and full-trajectory conditioned trajectory prediction.
LSTM Network based Sentiment Analysis for Customer Reviews

Continuously increasing data bring new problems and problems usually reveal new research areas. One of the new areas is Sentiment Analysis. This field has some difficulties. The fact that people have complex sentiments is the main cause of the difficulty, but this has not prevented the progress of the studies in this field. Sentiment analysis is generally used to obtain information about persons by collecting their texts or expressions. Sentiment analysis can sometimes bring serious benefits. In this study, with singular tag-plural class approach, a binary classification was performed. An LSTM network and several machine learning models were tested. The dataset collected in Turkish, and Stanford Large Movie Reviews datasets were used in this study. Due to the noise in the dataset, the Zemberek NLP Library for Turkic Languages and Regular Expression techniques were used to normalize and clean texts, later, the data were transformed into vector sequences. The preprocessing process made 2% increase to the model performance on the Turkish Customer Reviews dataset. The model was established using an LSTM network. Our model showed better performance than Machine Learning techniques and achieved an accuracy of 90.59% on the Turkish dataset and an accuracy of 89.02% on the IMDB dataset.
Graph-signal Reconstruction and Blind Deconvolution for Structured Inputs

Key to successfully deal with complex contemporary datasets is the development of tractable models that account for the irregular structure of the information at hand. This paper provides a comprehensive and unifying view of several sampling, reconstruction, and recovery problems for signals defined on irregular domains that can be accurately represented by a graph. The workhorse assumption is that the (partially) observed signals can be modeled as the output of a graph filter to a structured (parsimonious) input graph signal. When either the input or the filter coefficients are known, this is tantamount to assuming that the signals of interest live on a subspace defined by the supporting graph. When neither is known, the model becomes bilinear. Upon imposing different priors and additional structure on either the input or the filter coefficients, a broad range of relevant problem formulations arise. The goal is then to leverage those priors, the shift operator of the supporting graph, and the samples of the signal of interest to recover: the signal at the non-sampled nodes (graph-signal interpolation), the input (deconvolution), the filter coefficients (system identification), or any combination thereof (blind deconvolution).
Improving Arabic Diacritization with Regularized Decoding and Adversarial Training

Arabic diacritization is a fundamental task for Arabic language processing.Previous studies have demonstrated that automatically generated knowledge can be helpful to this task.However, these studies regard the autogenerated knowledge instances as gold references, which limits their effectiveness since such knowledge is not always accurate and inferior instances can lead to incorrect predictions.In this paper, we propose to use regularized decoding and adversarial training to appropriately learn from such noisy knowledge for diacritization.Experimental results on two benchmark datasets show that, even with quite flawed auto-generated knowledge, our model can still learn adequate diacritics and outperform all previous studies, on both datasets.
Bayesian inference for continuous-time hidden Markov models with an unknown number of states

We consider the modeling of data generated by a latent continuous-time Markov jump process with a state space of finite but unknown dimensions. Typically in such models, the number of states has to be pre-specified, and Bayesian inference for a fixed number of states has not been studied until recently. In addition, although approaches to address the problem for discrete-time models have been developed, no method has been successfully implemented for the continuous-time case. We focus on reversible jump Markov chain Monte Carlo which allows the trans-dimensional move among different numbers of states in order to perform Bayesian inference for the unknown number of states. Specifically, we propose an efficient split-combine move which can facilitate the exploration of the parameter space, and demonstrate that it can be implemented effectively at scale. Subsequently, we extend this algorithm to the context of model-based clustering, allowing numbers of states and clusters both determined during the analysis. The model formulation, inference methodology, and associated algorithm are illustrated by simulation studies. Finally, We apply this method to real data from a Canadian healthcare system in Quebec.
Diagnosing Transformers in Task-Oriented Semantic Parsing

Modern task-oriented semantic parsing approaches typically use seq2seq transformers to map textual utterances to semantic frames comprised of intents and slots. While these models are empirically strong, their specific strengths and weaknesses have largely remained unexplored. In this work, we study BART and XLM-R, two state-of-the-art parsers, across both monolingual and multilingual settings. Our experiments yield several key results: transformer-based parsers struggle not only with disambiguating intents/slots, but surprisingly also with producing syntactically-valid frames. Though pre-training imbues transformers with syntactic inductive biases, we find the ambiguity of copying utterance spans into frames often leads to tree invalidity, indicating span extraction is a major bottleneck for current parsers. However, as a silver lining, we show transformer-based parsers give sufficient indicators for whether a frame is likely to be correct or incorrect, making them easier to deploy in production settings.
Towards Faithfulness in Open Domain Table-to-text Generation from an Entity-centric View

In open domain table-to-text generation, we notice the unfaithful generation usually contains hallucinated entities which can not be aligned to any input table record. We thus try to evaluate the generation faithfulness with two entity-centric metrics: table record coverage and the ratio of hallucinated entities in text, both of which are shown to have strong agreement with human judgements. Then based on these metrics, we quantitatively analyze the correlation between training data quality and generation fidelity which indicates the potential usage of entity information in faithful generation. Motivated by these findings, we propose two methods for faithful generation: 1) augmented training by incorporating the auxiliary entity information, including both an augmented plan-based model and an unsupervised model and 2) training instance selection based on faithfulness ranking. We show these approaches improve generation fidelity in both full dataset setting and few shot setting by both automatic and human evaluations.
On Sample Based Explanation Methods for NLP: Faithfulness, Efficiency and Semantic Evaluation

In the recent advances of natural language processing, the scale of the state-of-the-art models and datasets is usually extensive, which challenges the application of sample-based explanation methods in many aspects, such as explanation interpretability, efficiency, and faithfulness.In this work, for the first time, we can improve the interpretability of explanations by allowing arbitrary text sequences as the explanation unit.On top of this, we implement a hessian-free method with a model faithfulness guarantee.Finally, to compare our method with the others, we propose a semantic-based evaluation metric that can better align with humans' judgment of explanations than the widely adopted diagnostic or retraining measures.The empirical results on multiple real data sets demonstrate the proposed method's superior performance to popular explanation techniques such as Influence Function or TracIn on semantic evaluation.
Text-based RL Agents with Commonsense Knowledge: New Challenges, Environments and Baselines

Text-based games have emerged as an important test-bed for Reinforcement Learning (RL) research, requiring RL agents to combine grounded language understanding with sequential decision making.In this paper, we examine the problem of infusing RL agents with commonsense knowledge. Such knowledge would allow agents to efficiently act in the world by pruning out implausible actions,and to perform look-ahead planning to determine how current actions might affect future world states. We design a new text-based gaming environment called TextWorld Commonsense (TWC) for training and evaluating RL agents with a specific kind of commonsense knowledge about objects, their attributes, and affordances. We also introduce several baseline RL agents which track the sequential context and dynamically retrieve the relevant commonsense knowledge from ConceptNet. We show that agents which incorporate commonsense knowledge in TWC perform better, while acting more efficiently. We conduct user-studies to estimate human performance on TWC and show that there is ample room for future improvement.
Towards Self-Explaining Ambient Applications

In the past decade, pervasive environments have emerged from promising research concepts to available products present in our everyday lives. By connecting multiple smart objects, device ensembles can be formed to assist users in performing tasks. Furthermore, smart objects can be used to control applications, that, in turn, can be used to control other smart objects. While dynamically connecting these components allows for solutions tailored to the needs and respective tasks of a user, it obfuscates the handling and ultimately may decrease usability.