Reviewerly/exHarmony huggingface dataset:
Augmented Lagrangian Method for Instantaneously Constrained Reinforcement Learning Problems

In this paper, we study the Instantaneously Constrained Reinforcement Learning (ICRL) problem, in which we are tasked to find a reward-maximizing policy while satisfying certain constraints at each time step. We first extend a result on the strong duality of Constrained Markov Decision Process (CMDP) in the literature and propose a sufficient condition for strong duality of the ICRL problem. Inspired by the Augmented Lagrangian Method in constrained optimization, we propose a new surrogate objective function for ICRL, which could be efficiently optimized by common policy-gradient based RL algorithms. We show theoretically that a feasible and optimal policy could be obtained by optimizing this surrogate function, under certain conditions related to the feasible policy set. Our empirical results on a tabular Markov Decision Process and two nonlinear optimal control problems, a constrained pendulum and a constrained half-cheetah, justify our analysis, and suggest that our method could promote safety during learning and converge in a smaller number of iterations compared to the existing algorithms.
TraceMe: Real-Time Contact Tracing and Early Prevention of COVID-19 based on Online Social Networks

With the outbreak of COVID-19, and its terrible and fast spread among communities, contact tracing methods have become crucial to protect people. However, conventional mechanisms, for instance based on the manual search of close contacts, lack of high efficiency due to the high time consumption. The research community is therefore exploring new ways to track contagious diseases by exploiting modern communications paradigms and technologies. In this paper, we propose a new contact tracing method based on Online Social Network (OSN) platforms. In our design, contact detection occurs in real-time by means of traditional proximity approaches. Then, a likely future contact forecast is notified through OSN communities.
Towards Understanding and Answering Comparative Questions

In this paper, we analyze comparative questions and answers. At least 3%~of the questions submitted to search engines are comparative; ranging from simple facts like "Did Messi or Ronaldo score more goals in 2021?'' to life-changing and probably highly subjective questions like "Is it better to move abroad or stay?''. Ideally, answers to subjective comparative questions would reflect diverse opinions so that the asker can come to a well-informed decision. To better understand the information needs behind comparative questions, we develop approaches to extract the mentioned comparison objects and aspects. As a first step to answer comparative questions, we develop an approach that detects the stances of potential result nuggets (i.e., text passages containing the comparison objects). Our approaches are trained and evaluated on a set of 31,000~English questions from existing datasets that we label as comparative or not. In the 3,500~comparative questions, we label the comparison objects, aspects, and predicates. For 950~questions, we collect answers from online forums and label the stance towards the comparison objects. In the experiments, our approaches recall~71% of the comparative questions with a perfect precision of~1.0, recall~92% of subjective comparative questions with a precision of~0.98, and identify the comparison objects and aspects with an F1 of~0.93 and~0.80, respectively. The stance detector fine-tuned on pairs of objects and answers achieves an accuracy of~0.63.
Channel Pruning via Lookahead Search Guided Reinforcement Learning

Channel pruning has become an effective yet still challenging approach to achieve compact neural networks. It aims to prune the optimal set of filters whose removal results in minimal performance degradation of the slimmed network. Due to the prohibitively vast search space of filter combinations, existing approaches usually use various criteria to estimate the filter importance while sacrificing some precision. Here we present a new approach to optimizing the filter selection in channel pruning with lookahead search guided reinforcement learning (RL). A neural network that takes as input filterrelated features is trained with RL to prune the optimal sequence of filters and maximize the performance of the remaining network. In addition, we employ Monte Carlo tree search (MCTS) to provide a lookahead search for filter selection, which increases the sample efficiency for the RL training. Experiments on MNIST, CIFAR-10, and ILSVRC-2012 validate the effectiveness of our approach compared to both traditional and automated existing channel pruning approaches.
A Hybrid Strategy Improved Whale Optimization Algorithm for Web Service Composition

Abstract With the rapid growth of the number of web services on the Internet, various service providers provide many similar services with the same function but different quality of service (QoS) attributes. It is a key problem to be solved urgently to select the service composition quickly, meeting the users’ QoS requirements from many candidate services. Optimization of web service composition is an NP-hard issue and intelligent optimization algorithms have become the mainstream method to solve this complex problem. This paper proposed a hybrid strategy improved whale optimization algorithm, which is based on the concepts of chaos initialization, nonlinear convergence factor and mutation. By maintaining a balance between exploration and exploitation, the problem of slow or early convergence is overcome to a certain extent. To evaluate its performance more accurately, the proposed algorithm was first tested on a set of standard benchmarks. After, simulations were performed using the real quality of web service dataset. Experimental results show that the proposed algorithm is better than the original version and other meta-heuristic algorithms on average, as well as verifies the feasibility and stability of web service composition optimization.
Blockchain Technology

Blockchain is an exciting new technology that is being widely used nowadays. Blockchain uses programs in a low-key and secure way and thus ensures a certain level of confidence. This is one of the main reasons for the widespread acceptance of Blockchain in an unscrupulous society. However, exciting new technologies often create disruption in the marketplace; disruption breeds misunderstanding, and Blockchain technology is no exception. For example, the first and most important application of Blockchain was Bitcoin. After the development of Bitcoin, no new application was developed, and for this reason various myths arose, such as: Blockchain is only Bitcoin, is secure, cannot be manipulated, etc. To overcome the issue, there is a pressing need to explore the myths and realities and sort myths from reality. The objective is to unravel some of the commonly assumed misconceptions and remove the misperceptions, thereby providing users with a better understanding of Blockchain. The chapter collects and reviews the contents of various research papers and online sources so as to achieve this. The chapter discusses the Blockchain concepts and presents the various myths and reality associated with Blockchain. The chapter attempts to sort the myths from the reality, highlighting the fact that using Blockchain is not restricted to only some domains. It is hoped that the chapter will greatly assist users in understanding the concepts, knowing the myths and reality associated with Blockchain, and gaining insight into a few of the common misperceptions. It is also hoped that the chapter will provide a strong foundation to the researchers who wish to pursue research in this domain.
FastLink: a machine learning and GPU-based fast phase association method and its application to Yangbi <i>M</i>s 6.4 aftershock sequences

SUMMARY The development of sensitive automatic phase picking method can produce P/S phase picks for lots of seismic stations quickly. This raises the requirement for fast phase association method to group phase picks into candidate events. Here we present FastLink, a machine learning and GPU based phase associator, to group phase picks efficiently. FastLink is designed to utilize parallel computing power of GPU, it uses a 3-layer neural network to determine valid picks and the non-maximum suppression (NMS) algorithm to distinguish the events simultaneously occurred in different places. With the aid of GPU, FastLink can perform faster grid search than traditional methods. We apply FastLink to aftershock sequences identification of the Yangbi Ms 6.4 earthquake occurred on 21 May 2021, in Yunnan Province, China. Over 17 000 events are successfully associated using about 2 million picks from 1-week records of 79 stations surrounding the main shock. Over 93.5 per cent of the ML ≥ 0 events in the manual verified bulletin ML are successfully associated by FastLink. Compared with the widely used REAL method, FastLink produces comparable results and is over 12 times faster with GPU acceleration. This shows its potential for processing a large number of phase picks quickly.
Online learning on tiny micro-controllers for anomaly detection in water distribution systems

Researchers have developed novel approaches and algorithms to aid in the planning, design, and administration of water distribution systems (WDS) since the 1960s. While early research focused on simulating the hydraulic of such systems, the 1980s saw a shift in focus to water quality subjects. Recent study has delved deeper into system reliability and resilience. This paper introduces a resource constrained anomaly detector for WDS to monitor its reliability. The proposed solution uses Reservoir Computing (RC), in particular Deep Echo State Network (DeepESN), to achieve high accuracy with low complexity. Furthermore, thanks to the special properties of DeepESN, online learning capabilities of this model is implemented in the micro-controller (MCU) to ensure the continuous adaptability over time of the model to adapt to the time varying data distribution. In most cases, researchers have relied on either hypothetical water distribution systems or a handful of actual systems for use as benchmark test systems. Skoltech Anomaly Benchmark (SKAB) dataset was used to measure the performance of those models. It consists of several anomalies in a sensorized WDS. The online learning feature is proposed in two different approaches: single iteration and batch decomposition Which give options for different implementations depending on the application needs and the MCU memory constraints. The accuracy, complexity and robustness of the proposed models are comprehensively validated through comparative analysis of many sets of hyper-parameters.
A comprehensive study on the application of firefly algorithm in prediction of energy dissipation on block ramps

In this study novel integrative machine learning models embedded with the firefly algorithm (FA) were developed and employed to predict energy dissipation on block ramps. The used models include multi-layer perceptron neural network (MLPNN), adaptive neuro-fuzzy inference system (ANFIS), group method of data handling (GMDH), support vector regression (SVR), linear equation (LE), and nonlinear regression equation (NE). The investigation focused on the evaluation of the performance of standard and integrative models in different runs. The performances of machine learning models and the nonlinear equation are higher than the linear equation. The results also show that FA increases the performance of all ap�plied models. Moreover, the results indicate that the ANFIS-FA is the most stable integrative model in comparison to the other embedded methods and reveal that GMDH and SVR are the most stable technique among all applied models. The results also show that the accuracy of the LE-FA technique is relatively low, RMSE=0.091. The most accurate results provide SVR-FA, RMSE=0.034 &lt;b&gt;Results&lt;/b&gt; &lt;b&gt;Conclusions&lt;/b&gt; &lt;b&gt;&lt;/b&gt; &lt;b&gt;&lt;/b&gt;
Identifying key features in reactive flows: A tutorial on combining dimensionality reduction, unsupervised clustering, and feature correlation

This study examines the capabilities of a data-driven workflow for automated key feature identification in reactive flows. The proposed approach aims at expediting the analysis of chemical engineering datasets by generating an automatic and explainable classification of regions showcasing distinct physics. The three main steps of this process, i.e., dimensionality reduction, unsupervised clustering, and feature correlation are discussed. A previously published framework based on these steps is used to compare against our proposed workflow, which employs different and more modern algorithms. The theoretical and practical differences between the previous and current algorithms are demonstrated in full. Overall, the key feature identification capability of the updated workflow is shown to be faster, more accurate, more robust, and closer to human intuition than previous methods. Throughout this study no substantial knowledge of machine learning is required from the reader. This makes this work also double up as a tutorial for researchers aiming at applying these algorithms.
Classification of Urban Pollution Levels Based on Clustering and Spatial Statistics

In recent years, the occurrence and frequency of haze are constantly increasing, severely threatening people’s daily lives and health and bringing enormous losses to the economy. To this end, we used cluster analysis and spatial autocorrelation methods to discuss the spatial and temporal distribution characteristics of severe haze in China and to classify regions of China. Furthermore, we analyzed the interaction between haze pollution and the influence of economy and energy structure in 31 provinces in China, providing references for the prevention and treatment of haze pollution. The processed data mainly include API, meteorological station data, and PM 2.5 concentration distribution vector graph. The results show the yearly haze pattern from 2008 to 2012, and present a strong pattern of pollution concentrated around Beijing–Tianjin, the Yangtze River Delta, southwest China, and central China. The overall spatial pattern of decreasing from north to south is relatively constant over the study period.
Cooperative Transportation With Mobile Manipulator: A Capability Map-Based Framework for Physical Human–Robot Collaboration

In the cooperative transportation task with a mobile manipulator (MM), the mobile robot and the manipulator must move simultaneously to adapt to the human motion. In addition, the motion of the MM is underconstrained due to redundancy, which makes MM real-time motion planning challenging. In this article, a capability map-based framework was proposed to enable real-time motion planning of the MM. In the motion planner, the dexterity of the MM, the formation of the human–robot system, and obstacle avoidance of the mobile robot are considered to get safe and human-like robot motion. Moreover, to make optimization faster and avoid local minimum, capability map, which represents the manipulability distribution of the MM in its workspace, is queried to determine the seed of the optimization algorithm. The proposed motion planner is self-contained and can be extended to the transportation task with multiple MMs. The effectiveness of this proposed framework is validated both in simulations and real-world experiments.
Exploring reliable visual tracking via target embedding network

Most prevalent trackers have the aid of more powerful appearance models, aiming to learn more discriminative deep representations for a reliable response map. Nevertheless, it is not a piece of cake to obtain an accurate response map on account of diverse challenges. What ismore, discriminative appearance model based trackers with an online update component require dynamic samples to update the target classifier, which inevitably learns the imprecise tracking results into the model, thus reducing its discriminant ability. To alleviate this matter, we propose a brand-new verification mechanism via Target Embedding Network, the intention of which is to learn general target embedding features offline for improving the similarity of the same target while dissimilarity of the different targets. In particular, we devise a simpler select strategy for negative samples and adopt the multiple triple loss to effectively train the network. Furthermore, we adopt a valid cosine similarity method to metric the target embedding features between the initial frame and current frame targets. According to the comparison results between the similarity score and piecewise thresholds, this method can retain the discriminant ability of the tracker by controlling the update of the sample memory and the learning rate. We replace the hard sample mining strategy used by recent SuperDiMP and conduct comprehensive experimental tests and analyses of our approach on six public datasets. Extensive experiments demonstrate that the discriminative ability of the model can be maintained with effect by using the proposed method, acquiring a superior performance against state-of-the-art trackers. The code, raw tracking results and trained models will be released at https://github.com/hexdjx/VisTrack.
Implementation of specialised attention mechanisms: ICD-10 classification of Gastrointestinal discharge summaries in English, Spanish and Swedish

Multi-label classification according to the International Classification of Diseases (ICD) is an Extreme Multi-label Classification task aiming to categorise health records according to a set of relevant ICD codes. We implemented PlaBERT, a new multi-label text classification head with per-label attention, on top of a BERT model. The model assessment is conducted on Electronic Health Records, conveying Discharge Summaries in three languages - English, Spanish, and Swedish. The study focuses on 157 diagnostic codes from the ICD. We additionally measure the labelling noise to estimate the consistency of the gold standard. Our specialised attention mechanism computes attention weights for each input token and label pair, obtaining the specific relevance of every word concerning each ICD code. The PlaBERT model outputs the computed attention importance for each token and label, allowing for visualisation. Our best results are 40.65, 38.36, and 41.13 F1-Score points on the English, Spanish and Swedish datasets, respectively, for the 157 gastrointestinal codes. Besides, Precision is the metric that most significantly improves owing to the attention mechanism of PlaBERT, with an increase of 44.63, 40.93, and 12.92 points, respectively, for the Spanish, Swedish and English datasets.
Customizable FPGA-Based Hardware Accelerator for Standard Convolution Processes Empowered with Quantization Applied to LiDAR Data

In recent years there has been an increase in the number of research and developments in deep learning solutions for object detection applied to driverless vehicles. This application benefited from the growing trend felt in innovative perception solutions, such as LiDAR sensors. Currently, this is the preferred device to accomplish those tasks in autonomous vehicles. There is a broad variety of research works on models based on point clouds, standing out for being efficient and robust in their intended tasks, but they are also characterized by requiring point cloud processing times greater than the minimum required, given the risky nature of the application. This research work aims to provide a design and implementation of a hardware IP optimized for computing convolutions, rectified linear unit (ReLU), padding, and max pooling. This engine was designed to enable the configuration of features such as varying the size of the feature map, filter size, stride, number of inputs, number of filters, and the number of hardware resources required for a specific convolution. Performance results show that by resorting to parallelism and quantization approach, the proposed solution could reduce the amount of logical FPGA resources by 40 to 50%, enhancing the processing time by 50% while maintaining the deep learning operation accuracy.
User Satisfaction Estimation with Sequential Dialogue Act Modeling in Goal-oriented Conversational Systems

User Satisfaction Estimation (USE) is an important yet challenging task in goal-oriented conversational systems. Whether the user is satisfied with the system largely depends on the fulfillment of the user’s needs, which can be implicitly reflected by users’ dialogue acts. However, existing studies often neglect the sequential transitions of dialogue act or rely heavily on annotated dialogue act labels when utilizing dialogue acts to facilitate USE. In this paper, we propose a novel framework, namely USDA, to incorporate the sequential dynamics of dialogue acts for predicting user satisfaction, by jointly learning User Satisfaction Estimation and Dialogue Act Recognition tasks. In specific, we first employ a Hierarchical Transformer to encode the whole dialogue context, with two task-adaptive pre-training strategies to be a second-phase in-domain pre-training for enhancing the dialogue modeling ability. In terms of the availability of dialogue act labels, we further develop two variants of USDA to capture the dialogue act information in either supervised or unsupervised manners. Finally, USDA leverages the sequential transitions of both content and act features in the dialogue to predict the user satisfaction. Experimental results on four benchmark goal-oriented dialogue datasets across different applications show that the proposed method substantially and consistently outperforms existing methods on USE, and validate the important role of dialogue act sequences in USE.
Efficient quantum private comparison protocol utilizing single photons and rotational encryption

As a branch of quantum secure multiparty computation, quantum private comparison is applied frequently in many fields, such as secret elections, private voting, and identification. A quantum private comparison protocol with higher efficiency and easier implementation is proposed in this paper. The private secrets are encoded as single polarized photons and then encrypted with a homomorphic rotational encryption method. Relying on this method and the circular transmission mode, we implement the multiplexing of photons, raising the efficiency of our protocol to 100%. Our protocol is easy to realize since only single photons, unitary operation, and single-particle measurement are introduced. Meanwhile, the analysis shows that our protocol is also correct and secure.
Sign Language Avatars: A Question of Representation

Given the achievements in automatically translating text from one language to another, one would expect to see similar advancements in translating between signed and spoken languages. However, progress in this effort has lagged in comparison. Typically, machine translation consists of processing text from one language to produce text in another. Because signed languages have no generally-accepted written form, translating spoken to signed language requires the additional step of displaying the language visually as animation through the use of a three-dimensional (3D) virtual human commonly known as an avatar. Researchers have been grappling with this problem for over twenty years, and it is still an open question. With the goal of developing a deeper understanding of the challenges posed by this question, this article gives a summary overview of the unique aspects of signed languages, briefly surveys the technology underlying avatars and performs an in-depth analysis of the features in a textual representation for avatar display. It concludes with a comparison of these features and makes observations about future research directions.