Reviewerly/exHarmony huggingface dataset:
Comparison of Deep Reinforcement Learning Approaches for Intelligent Game Playing

In Reinforcement Learning, a category of machine learning, learning is based on evaluative feedbacks without any supervised signals. The paper presents work aimed to understand the deep reinforcement learning approaches to creating such intelligent agents, by reproducing existing research and comparing their results. The project uses the Atari 2600 game called Breakout, in which the agent will learn control policies using deep reinforcement learning approaches to achieve a high score. The project explores two deep reinforcement learning approaches, Asynchronous Advantage actor-critic and Deep Q-Learning, both proposed by the DeepMind team, to train intelligent agents that can interact with an environment with automatic feature engineering thus requiring minimal domain knowledge.
Direct Answers in Google Search Results

This study aims to explore Google's new ''featured snippet'' (a direct answer that appears in web search results) to better understand the method of extracting and displaying such answers above regular ranked results.The motivation for the study was to investigate form, structure, and relation to query, content, and domain of the direct answer.This study utilized a dataset of 743,798 keywords and the displayed direct answers.It also included web search data for every direct answer, e.g.resulting domain, full URL address and its original ranking, content of the direct answer, and competition (measured as the number of monthly searches for every query).The main finding concerns the form of construction of keywords used in the website content.Keywords should be built in the form of short, two-to-four-word sentences comprising the subject and its attribute.Using relative pronouns, articles, and prepositions, as well as using questions as queries, can help to properly define a query and display the best direct answer.The dataset is relatively small compared to the volume of searches made daily.No other factors were extracted from the URL, and all data concerned the Google search engine only.Implications for webmasters include: keywords used in website content should be close to grammar forms used in queries; use keywords in URLs; write a comprehensive introduction section on the webpage; and use tables and lists HTML markup.The main objective of this study was to examine the structure of the user's queries which caused in appearing of direct answers.It is comprehensive and based on real data used by web search engine users.
S-MDP: Streaming With Markov Decision Processes

In recent years, finding adaptive players competing for network resources at a single bottleneck link has become common. This competition generally occurs at routers in household local area networks. Such competition severely reduces viewers' quality of experience, such as fairness and stability. Researchers harness Markov decision process (MDP) models to optimize the adaptive video streaming process. Typically, players follow a policy based on numerous parameters, such as buffer occupancy or average bandwidth. In this study, we defied this traditional decentralized client-side MDP approach by allowing players to share a network state among themselves, which we called a streaming MDP (S-MDP). This state includes a discrete data rate measurement (DRM) value. The DRM value is a normalized value of a player's incoming bitrate and is an example of an interval measurement scale. Players use video bitrates to produce unique state transition matrices. The S-MDP reward matrix penalizes excessive switching along the DRM interval scale and thus encourages stability. At intervals during streaming, players create unique policies. The near-real-time update of policies enables players DRM values to converge. S-MDP shows relatively good performance in emulation experiments compared with four streaming methods, namely, k-chunk MDP, stochastic dynamic programming for adaptive streaming over hypertext transfer protocol (sdpDASH), MDP-based DASH, and RTRA_S. In Internet experiments, we compare the performance of streaming methods with a Roku, Amazon Fire TV, and Apple TV. We also compared it against users who play online games, download files, and/or chat online, and S-MDP outperforms the other methods in terms of both objective and subjective visual quality, except in the presence of transmission control protocol long-lived flows, such as Skype.
Diversity Adversarial Training against Adversarial Attack on Deep Neural Networks

This paper presents research focusing on visualization and pattern recognition based on computer science. Although deep neural networks demonstrate satisfactory performance regarding image and voice recognition, as well as pattern analysis and intrusion detection, they exhibit inferior performance towards adversarial examples. Noise introduction, to some degree, to the original data could lead adversarial examples to be misclassified by deep neural networks, even though they can still be deemed as normal by humans. In this paper, a robust diversity adversarial training method against adversarial attacks was demonstrated. In this approach, the target model is more robust to unknown adversarial examples, as it trains various adversarial samples. During the experiment, Tensorflow was employed as our deep learning framework, while MNIST and Fashion-MNIST were used as experimental datasets. Results revealed that the diversity training method has lowered the attack success rate by an average of 27.2 and 24.3% for various adversarial examples, while maintaining the 98.7 and 91.5% accuracy rates regarding the original data of MNIST and Fashion-MNIST.
A Collective Intelligence Based Differential Evolution Algorithm for Optimizing the Structure and Parameters of a Neural Network

In this paper, a Self-learning Collective Intelligence Differential Evolution (SLCIDE) algorithm was proposed to optimize both the architecture and parameters of a Feedforward Neural Network (FNN).In order to improve the exploration-exploitation capability, a new Collective Intelligence (CI) based mutation operator was proposed by mixing some promising donor vectors in the current population.Besides, a self-learning mechanism which was designed to adaptively select m top ranked donor vectors was developed by using a widely used unsupervised learning method, k-means.As a result, the proposed approach can be more adaptive and statistically powerful on versatile problems.Then, we evaluated the performances of the proposed SLCIDE approach by studying some numerical optimization problems of CEC 2013 with D = 30 and D = 50.Further, the proposed SLCIDE method was applied to train a FNN on four most popular datasets, resulting in very competitive performances.The comprehensive experimental results have demonstrated that the presented SLCIDE method obtain better results compared with other state-of-the-art algorithms.
MR-MVPP: A map-reduce-based approach for creating MVPP in data warehouses for big data applications

Materialized view selection (MVS) is the problem of selecting an appropriate set of views to be materialized to speed up analytical query processing of data warehouses. Online analytical processing (OLAP) of queries is an essential application of the MVS problem, in which, the response times of the queries are reduced by storing the selected views. Views are intermediate results of query processing and are selected in the MVS problem to be stored and will then be exploited in answering process of several queries. Views are usually organized as a view representation structure in the MVS problem. Multiple Views Processing Plan (MVPP) is a standard structure used for view representation in the MVS problem. Due to the tremendous amount of data, constructing the MVPP is a challenge in the big data applications. The MR-MVPP (Map-Reduce-based construction of the MVPP) is the proposed method of this paper to address this problem. The MR-MVPP performs a set similarity join (similarity-based join) on the base relations and views using the map-reduce model and the hashing technique. The MVPP construction time in the proposed method is reduced by avoiding redundant calculations in the process of creating the MVPP. The performance of the proposed method is empirically evaluated. According to the results of the experiments, the execution time of the MR-MVPP method is better than the other methods. The average time improvement is about 26.5 units. This improvement is better than the other similar researches in this area and is significant due to the high volume of data in real applications. Moreover, the proposed method works well in terms of the effectiveness of the created MVPP and has about a 50% coverage rate for view selection methods. Deterministic methods are more accurate than hashing methods and can be utilized for set similarity join as future work to probably improve the effectiveness of the constructed MVPP.
Scalability vs. Utility: Do We Have to Sacrifice One for the Other in Data Importance Quantification?

Quantifying the importance of each training point to a learning task is a fundamental problem in machine learning and the estimated importance scores have been leveraged to guide a range of data workflows such as data summarization and domain adaption. One simple idea is to use the leave-one-out error of each training point to indicate its importance. Recent work has also proposed to use the Shapley value, as it defines a unique value distribution scheme that satisfies a set of appealing properties. However, calculating Shapley values is often expensive, which limits its applicability in real-world applications at scale. Multiple heuristics to improve the scalability of calculating Shapley values have been proposed recently, with the potential risk of compromising their utility in real-world applications.How well do existing data quantification methods perform on existing workflows? How do these methods compare with each other, empirically and theoretically? Must we sacrifice scalability for the utility in these workflows when using these methods? In this paper, we conduct a novel theoretical analysis comparing the utility of different importance quantification methods, and report extensive experimental studies on existing and proposed workflows such as noisy label detection, watermark removal, data summarization, data acquisition, and domain adaptation. We show that Shapley value approximation based on a KNN surrogate over pretrained feature embeddings obtains comparable utility with existing algorithms while achieving significant scalability improvement, often by orders of magnitude. Our theoretical analysis also justifies its advantage over the leave-one-out error.The code is available at https://github.com/AIsecure/Shapley-Study.
Unsupervised Hierarchical Semantic Segmentation with Multiview Cosegmentation and Clustering Transformers

Unsupervised semantic segmentation aims to discover groupings within and across images that capture object-and view-invariance of a category without external supervision. Grouping naturally has levels of granularity, creating ambiguity in unsupervised segmentation. Existing methods avoid this ambiguity and treat it as a factor outside modeling, whereas we embrace it and desire hierarchical grouping consistency for unsupervised segmentation. We approach unsupervised segmentation as a pixel-wise feature learning problem. Our idea is that a good representation shall reveal not just a particular level of grouping, but any level of grouping in a consistent and predictable manner. We enforce spatial consistency of grouping and bootstrap feature learning with co-segmentation among multiple views of the same image, and enforce semantic consistency across the grouping hierarchy with clustering transformers between coarse- and fine-grained features. We deliver the first data-driven unsupervised hierarchical semantic segmentation method called Hierarchical Segment Grouping (HSG). Capturing visual similarity and statistical co-occurrences, HSG also outperforms existing un-supervised segmentation methods by a large margin on five major object- and scene-centric benchmarks.
Multi-object Bayesian filter for jump Markov system under glint noise

The probability hypothesis density filter under glint noise is efficient for multi-object tracking. However, this filter is inapplicable to tracking the multiple maneuvering objects in case of low detection probability. In order to track the multiple maneuvering objects under glint noise more efficiently, we propose a novel multi-object Bayesian (MOB) filter for jump Markov system under glint noise by applying jump Markov system models and variational Bayesian approach to the MOB filter and develop an implementation of this filter. The developed implementation uses the Student's t-distribution to depict the glint noise, uses the Gaussian-Gamma distribution to represent the predicted state distribution and updated state distribution of each target, and uses the variational Bayesian method to acquire the approximate state distribution of each target. A comparison of the proposed MOB filter with the existing filters demonstrates that the proposed MOB filter is better at tracking the maneuvering objects under glint noise than the existing filters.
Radial Distortion Triangulation

This paper presents the first optimal, maximal likelihood, solution to the triangulation problem for radially distorted cameras. The proposed solution to the two-view triangulation problem minimizes the L2-norm of the reprojection error in the distorted image space. We cast the problem as the search for corrected distorted image points, and we use a Lagrange multiplier formulation to impose the epipolar constraint for undistorted points. For the one-parameter division model, this formulation leads to a system of five quartic polynomial equations in five unknowns, which can be exactly solved using the Groebner basis method. While the proposed Groebner basis solution is provably optimal; it is too slow for practical applications. Therefore, we developed a fast iterative solver to this problem. Extensive empirical tests show that the iterative algorithm delivers the optimal solution virtually every time, thus making it an L2-optimal algorithm de facto. It is iterative in nature, yet in practice, it converges in no more than five iterations. We thoroughly evaluate the proposed method on both synthetic and real-world data, and we show the benefits of performing the triangulation in the distorted space in the presence of radial distortion.
Transformers analyzing poetry: multilingual metrical pattern prediction with transfomer-based language models

Abstract The splitting of words into stressed and unstressed syllables is the foundation for the scansion of poetry, a process that aims at determining the metrical pattern of a line of verse within a poem. Intricate language rules and their exceptions, as well as poetic licenses exerted by the authors, make calculating these patterns a nontrivial task. Some rhetorical devices shrink the metrical length, while others might extend it. This opens the door for interpretation and further complicates the creation of automated scansion algorithms useful for automatically analyzing corpora on a distant reading fashion. In this paper, we compare the automated metrical pattern identification systems available for Spanish, English, and German, against fine-tuned monolingual and multilingual language models trained on the same task. Despite being initially conceived as models suitable for semantic tasks, our results suggest that transformers-based models retain enough structural information to perform reasonably well for Spanish on a monolingual setting, and outperforms both for English and German when using a model trained on the three languages, showing evidence of the benefits of cross-lingual transfer between the languages.
Structural correlation filters combined with a Gaussian particle filter for hierarchical visual tracking

Visual tracking is a key problem for many computer vision applications such as human-computer interaction, intelligent medical diagnosis, navigation and traffic control management. Most of the existing tracking methods are mainly based on correlation filters. However, boundary effect, scale estimation and template updating have not been fully resolved. Herein, this paper presents a new hierarchical tracking method combining structural correlation filters with a Gaussian Particle Filter (GPF), named KCF-GPF. Weak KCF classifiers are constructed via a Lukas-Kanade (LK) method and the preliminary target location is presented as a weighted sum of these classifiers. Specially, a facile weight strategy is implemented to estimate the reliability of each weak classifier. On the basis of the preliminary target location, the GPF using features from a Convolutional Neural Network (CNN) is employed to predict the location and scale of a target. Extensive experiments with the OTB-2013 and the OTB-2015 databases demonstrate that the proposed algorithm performs favourably against state-of-the-art trackers.
Deep Reinforcement Learning-based Continuous Control for Multicopter Systems

In this paper we apply deep reinforcement learning techniques on a multicopter for learning a stable hovering task in a continuous state action environment. We present a framework based on OpenAI GYM, Gazebo, Robotic Operating System and RotorS MAV simulator, used for successfully training different agents to perform various tasks. The deep reinforcement learning method used for the training is a model-free, on-policy, actor-critic based algorithm called Trust Region Policy Optimization (TRPO). Two neural networks have been used as nonlinear function approximators. Our experiments show that such learning approach achieves successful results, and facilitates the process of controller design.
A Multi-scale Convolutional Attention Based GRU Network for Text Classification

Neural network models have been widely used in natural language processing (NLP). Recurrent neural network (RNNs) has proved to be a powerful sequence model. Gated recurrent unit (GRU) is one kind of RNNs which has achieved excellent performance in NLP. Nevertheless, because of the sparsity and high dimensionality of text data, there are some difficulties in complex semantic representations. To solve these problems, a novel and efficient method is proposed in this paper for text classification. The proposed model is called multi-scale convolutional attention based GRU network (MCA-GRU). In MCA-GRU, one-dimension convolutions with dense connections extract attention signals from text sequences. Then the attention signals are combined with features of GRU network. MCA-GRU is able to capture the local feature of phrases and sequence information. Experimental verifications are conducted on five text classification datasets. The results clearly show that the proposed model MCA-GRU approach achieves equivalent or even superior performance than other state-of-the-art text classification methods.
DEPOSafe: Demystifying the Fake Deposit Vulnerability in Ethereum Smart  Contracts

Cryptocurrency has seen an explosive growth in recent years, thanks to the evolvement of blockchain technology and its economic ecosystem. Besides Bitcoin, thousands of cryptocurrencies have been distributed on blockchains, while hundreds of cryptocurrency exchanges are emerging to facilitate the trading of digital assets. At the same time, it also attracts the attentions of attackers. Fake deposit, as one of the most representative attacks (vulnerabilities) related to exchanges and tokens, has been frequently observed in the blockchain ecosystem, causing large financial losses. However, besides a few security reports, our community lacks of the understanding of this vulnerability, for example its scale and the impacts. In this paper, we take the first step to demystify the fake deposit vulnerability. Based on the essential patterns we have summarized, we implement DEPOSafe, an automated tool to detect and verify (exploit) the fake deposit vulnerability in ERC-20 smart contracts. DEPOSafe incorporates several key techniques including symbolic execution based static analysis and behavior modeling based dynamic verification. By applying DEPOSafe to 176,000 ERC-20 smart contracts, we have identified over 7,000 vulnerable contracts that may suffer from two types of attacks. Our findings demonstrate the urgency to identify and prevent the fake deposit vulnerability.
Loss Prediction: End-to-End Active Learning Approach For Speech Recognition

End-to-end speech recognition systems usually require huge amounts of labeling resource, while annotating the speech data is complicated and expensive. Active learning is the solution by selecting the most valuable samples for annotation. In this paper, we proposed to use a predicted loss that estimates the uncertainty of the sample. The CTC (Connectionist Temporal Classification) and attention loss are informative for speech recognition since they are computed based on all decoding paths and alignments. We defined an end-to-end active learning pipeline, training an ASR/LP (Automatic Speech Recognition/Loss Prediction) joint model. The proposed approach was validated on an English and a Chinese speech recognition task. The experiments show that our approach achieves competitive results, outperforming random selection, least confidence, and estimated loss method.
InsCLR: Improving Instance Retrieval with Self-Supervision

This work aims at improving instance retrieval with self-supervision. We find that fine-tuning using the recently developed self-supervised learning (SSL) methods, such as SimCLR and MoCo, fails to improve the performance of instance retrieval. In this work, we identify that the learnt representations for instance retrieval should be invariant to large variations in viewpoint and background etc., whereas self-augmented positives applied by the current SSL methods can not provide strong enough signals for learning robust instance-level representations. To overcome this problem, we propose InsCLR, a new SSL method that builds on the instance-level contrast, to learn the intra-class invariance by dynamically mining meaningful pseudo positive samples from both mini-batches and a memory bank during training. Extensive experiments demonstrate that InsCLR achieves similar or even better performance than the state-of-the-art SSL methods on instance retrieval. Code is available at https://github.com/zeludeng/insclr.
“It Feels Like Being Locked in A Cage”: Understanding Blind or Low Vision Streamers’ Perceptions of Content Curation Algorithms

Blind or low vision (BLV) people were recently reported to be live streamers on the online platforms that employed content curation algorithms. Recent research uncovered perceived algorithmic biases suppressing the content created by marginalized populations (e.g., people of color, the LGBT+ community, and content creators of lower socioeconomic status). However, little is known about how BLV streamers, as a marginalized population, perceive the effects of the algorithms adopted by live streaming platforms. We interviewed BLV streamers (N=19) of Douyin — a popular live stream platform in China — to understand their perceptions of algorithms, perceived challenges, and mitigation strategies. Our findings show the perceived factors contributing to disadvantages under algorithmic evaluation of BLV streamers’ content (e.g., issues with filming and timely interaction with viewers) and perceived algorithmic suppression (e.g., content not amplified to sighted users but suppressed within the BLV community). Their mitigation strategies (e.g., not watching other BLV streamers’ shows) tended to be passive. We discuss design considerations to design a more inclusive and fair live streaming platform.