Reviewerly/exHarmony huggingface dataset:
The Advance of Reinforcement Learning and Deep Reinforcement Learning

Reinforcement learning is one of the leading research fields of artificial intelligence. Unlike other machine learning methods, reinforcement learning is learning from the environment to action mappings. Thus, the chosen action could maximize the accumulated reward value from the environment and develop an optimal strategy via trial-and-error. In recent years, the achievements of deep reinforcement learning represented by AlphaGo have attracted wide attention from researchers. This paper first introduces the development of reinforcement learning, including classic reinforcement learning methods and deep reinforcement learning methods. Then, this paper discusses the advanced reinforcement learning work at present, including distributed deep reinforcement learning algorithms, deep reinforcement learning methods based on fuzzy theory, Large-Scale Study of Curiosity-Driven Learning, and so on. Finally, this essay discusses the challenges faced by reinforcement learning.
Interacting Attention Graph for Single Image Two-Hand Reconstruction

Graph convolutional network (GCN) has achieved great success in single hand reconstruction task, while interacting two-hand reconstruction by GCN remains unexplored. In this paper, we present Interacting Attention Graph Hand (IntagHand), the first graph convolution based network that reconstructs two interacting hands from a single RGB image. To solve occlusion and interaction challenges of two-hand reconstruction, we introduce two novel attention based modules in each upsampling step of the original GCN. The first module is the pyramid image feature attention (PIFA) module, which utilizes multiresolution features to implicitly obtain vertex-to-image alignment. The second module is the cross hand attention (CHA) module that encodes the coherence of interacting hands by building dense cross-attention between two hand vertices. As a result, our model outperforms all existing two-hand re-construction methods by a large margin on InterHand2.6M benchmark. Moreover, ablation studies verify the effectiveness of both PIFA and CHA modules for improving the reconstruction accuracy. Results on in-the-wild images and live video streams further demonstrate the generalization ability of our network. Our code is available at https://github.com/Dw1010/IntagHand.
If and how do 360° videos fit into education settings? Results from a scoping review of empirical research

Abstract Background Over recent years, interest towards the use of immersive technologies has grown in parallel with the lowering of equipment costs: educators have started embedding them into their teaching practices at different levels of the educational system, from primary school to higher education. In general, immersive technologies are seen as suitable tools to live vicarious experiences, when the reality is hardly accessible, for instance due to safety issues. Objectives In this paper, we focus on 360° videos with the aim of developing a more systematic and evidence informed approach to the understanding of the educational uses of 360° in different educational contexts. Methods A scoping literature review approach has been adopted in order to obtain a comprehensive insight based on objective, replicable and transparent procedures. Results and conclusions The review includes studies published from 2016 onwards. According to the findings, 360° videos are more suitable for disciplines requiring careful observation of how knowledge is transferred into practice and where practical scenarios are involved: 360° videos have positive effects on learning, especially referring to motivation, attentiveness, information retention and transfer of knowledge. However, as for learners' reactions, physical discomfort is reported, mainly associated with the use of HMD. In addition, challenges, and barriers to the educational use of 360° video should be further analysed, since up to now logistical constraints have limited the studies to a very small sample population, without the possibility of providing suggestions for a wider adoption at organisation and institution level. Implications As 360° videos are reported to be promising in fostering learning processes, students' engagement, and information retention, further studies are recommend to cover the current gaps in the literature, particularly focusing on drawbacks and organizational challenges.
Dependency Grammar

Dependency grammar is a descriptive and theoretical tradition in linguistics that can be traced back to antiquity. It has long been influential in the European linguistics tradition and has more recently become a mainstream approach to representing syntactic and semantic structure in natural language processing. In this review, we introduce the basic theoretical assumptions of dependency grammar and review some key aspects in which different dependency frameworks agree or disagree. We also discuss advantages and disadvantages of dependency representations and introduce Universal Dependencies, a framework for multilingual dependency-based morphosyntactic annotation that has been applied to more than 60 languages.
Classification

In classification or class prediction, it’s best to try to use the information from the predictors or independent variables to sort a data sample into two or more distinct classes or buckets. Classification is the most widely used data science task in business. There are several ways to build classification models. In this chapter, six of the most commonly used classification algorithms will be discussed and demonstrated: decision trees, rule induction, k-nearest neighbors (k-NNs), naïve Bayesian, artificial neural networks, and support vector machines. This chapter is concluded by building ensemble classification models and a discussion on bagging, boosting, and random forests.
ChatGPT and Bard exhibit spontaneous citation fabrication during psychiatry literature search

ChatGPT (Generative Pre-Trained Transformer) is a large language model (LLM), which comprises a neural network that has learned information and patterns of language use from large amounts of text on the internet. ChatGPT, introduced by OpenAI, responds to human queries in a conversational manner. Here, we aimed to assess whether ChatGPT could reliably produce accurate references to supplement the literature search process. We describe our March 2023 exchange with ChatGPT, which generated thirty-five citations, two of which were real. 12 citations were similar to actual manuscripts (e.g., titles with incorrect author lists, journals, or publication years) and the remaining 21, while plausible, were in fact a pastiche of multiple existent manuscripts. In June 2023, we re-tested ChatGPT's performance and compared it to that of Google's GPT counterpart, Bard 2.0. We investigated performance in English, as well as in Spanish and Italian. Fabrications made by LLMs, including erroneous citations, have been called “hallucinations”; we discuss reasons for which this is a misnomer. Furthermore, we describe potential explanations for citation fabrication by GPTs, as well as measures being taken to remedy this issue, including reinforcement learning. Our results underscore that output from conversational LLMs should be verified.
Collaborative Representation Cascade for Single-Image Super-Resolution

Most recent learning-based single-image superresolution methods first interpolate the low-resolution (LR) input, from which overlapped LR features are then extracted to reconstruct their high-resolution (HR) counterparts and the final HR image. However, most of them neglect to take advantage of the intermediate recovered HR image to enhance image quality further. We conduct principal component analysis (PCA) to reduce LR feature dimension. Then we find that the number of principal components after conducting PCA in the LR feature space from the reconstructed images is larger than that from the interpolated images by using bicubic interpolation. Based on this observation, we present an unsophisticated yet effective framework named collaborative representation cascade (CRC) that learns multilayer mapping models between LR and HR feature pairs. In particular, we extract the features from the intermediate recovered image to upscale and enhance LR input progressively. In the learning phase, for each cascade layer, we use the intermediate recovered results and their original HR counterparts to learn single-layer mapping model. Then, we use this single-layer mapping model to super-resolve the original LR inputs. And the intermediate HR outputs are regarded as training inputs for the next cascade layer, until we obtain multilayer mapping models. In the reconstruction phase, we extract multiple sets of LR features from the LR image and intermediate recovered. Then, in each cascade layer, mapping model is utilized to pursue HR image. Our experiments on several commonly used image SR testing datasets show that our proposed CRC method achieves state-of-the-art image SR results.
The cloud application modelling and execution language

Abstract Cloud computing offers a flexible pay-as-you-go model for provisioning application resources, which enables applications to scale on-demand based on the current workload. In many cases, though, users face the single vendor lock-in effect, missing opportunities for optimal and adaptive application deployment across multiple clouds. Several cloud modelling languages have been developed to support multi-cloud resource management, but still they lack holistic cloud management of all aspects and phases. This work defines the Cloud Application Modelling and Execution Language (CAMEL), which (i) allows users to specify the full set of design time aspects for multi-cloud applications, and (ii) supports the models@runtime paradigm that enables capturing an application’s current state facilitating its adaptive provisioning. CAMEL has been already used in many projects, domains and use cases due to its wide coverage of cloud management features. Finally, CAMEL has been positively evaluated in this work in terms of its usability and applicability in several domains (e.g., data farming, flight scheduling, financial services) based on the technology acceptance model (TAM).
Cost effective deadline aware scheduling strategy for workflow applications on virtual machines in cloud computing

In Cloud paradigm, resources are provisioned on demand and used for the leased time interval as pay per use basis. Generally, Cloud users use these resources to execute their applications. These applications may be the batch of independent or workflow tasks. The existing researches focus mainly on optimizing the various quality of service (QoS) parameters. In this paper, a Cost Effective Deadline Aware (CEDA) scheduling strategy has been proposed for the workflow tasks/applications in order to optimize the total execution time and economic cost while meeting the deadline and precedence constraints in the workflow. CEDA selects the task which has the highest upward rank and dispatches the selected task to the cheapest instance of virtual machines (VMs) considering the VMs acquisition delay. The performance evaluation of proposed strategy has been carried out by comparing it with ICPCP and ICPCPD2 on standard scientific workflows like Montage, CyberShake, LIGO and SIPHT. The experimental results show that CEDA outperforms among all considered algorithms on almost all parameters under study.
I Sense You by Breath: Speaker Recognition via Breath Biometrics

Over last two decades, Speaker Recognition has primarily been focused on source, system, and prosodic features of the speech. The breath, however, has either been treated as a trivial part of the speech, or considered a noise entity. Our observation reveals that breath is a unique fingerprint of human respiratory system which offers overwhelming results for Speaker Recognition. Moreover, its passive nature, short-duration, fewer occurrences and simple processing results to a light-weight, text-independent and transparent system, which we articulate as BreathID. The breath features are extracted and classified by Mel Frequency Cepstral Coefficients, MFCC, based template matching technique. The verification is performed by a similarity based scheme, whose efficiency competes with classification algorithms. We process a data set collected from 50 users. Our system offers a 0.04 percent False Identification Rate, FIR, for Speaker Identification, and 0.12 percent False Acceptance Rate, FAR, and 0.15 percent False Rejection Rate, FRR, for Speaker Verification. We further evaluate our scheme under various practical modalities, like text in-dependence, replay scenario, users' motion status (sitting and walking), recording equipment (03 smartphones and 02 microphones), recording period (08 months), and bilingual contents (English and Chinese). Though we use Matlab to formulate a fine-grained approach, we foresee breath biometric as a viable security measure for practical realizations.
Secure Verifiable Database Supporting Efficient Dynamic Operations in Cloud Computing

Storage is one of the main services that the cloud provides users. Utilizing the cloud to store large amounts of data can alleviate the cost of storage and the hardware investment on the local side, which is very important for resource-restricted clients. An emerging issue for clients is how to verify the database in the could after outsourcing their data to the cloud. Many verifiable database (VDB) schemes have been proposed by researchers that can solve this issue. However, some of the existing schemes cannot satisfy the practical requirements of database verifiability. In this paper, we propose a secure verifiable database scheme that is based on the polynomial commitment for cloud computing, which can realize the verifiability of database records in the cloud. Moreover, the proposed scheme can support public verifiability in that all clients in the system can verify the database. In addition, we use the BLS signature and the index-hash table to construct dynamic operations for the database. Security analysis shows that our scheme can achieve real-world security requirements. The simulation results show that our scheme is more efficient than similar schemes.
A Secure and Efficient Location-based Service Scheme for Smart Transportation

With the advance of wireless communication techniques and the popularity of embedded devices, Location-Based Service (LBS) has gained a great attention in the Internet-of-Things (IoT) recently. For instance, in smart transportation, reporting all the drivers’ location in a regular frequency is conducive to sense urban traffic congestion. However, it also posts a serious threat to drivers’ privacy because of the exposure to the real-time location. To alleviate this problem, we propose a Secure and Efficient Location-based Service (SELS) scheme for smart transportation in this paper. In the SELS scheme, drivers first utilize smart phones to encrypt their location-based data and then outsource them to a cloud center incessantly. Then they have the ability to call the cloud center to compute the ciphertext of LBS queries, by using a secure homomorphic encryption scheme and group signature scheme. After decrypting the ciphertext, they obtain accurate results of the queries without sacrificing the location privacy. Furthermore, the SELS scheme also supports multi-dimension data by employing a weighted distance algorithm. Finally, the security of the SELS scheme is proved and the cost of computation and communication is analyzed, in order to prove the security and efficiency of the scheme.
MOO-MDP: An Object-Oriented Representation for Cooperative Multiagent Reinforcement Learning

Reinforcement learning (RL) is a widely known technique to enable autonomous learning. Even though RL methods achieved successes in increasingly large and complex problems, scaling solutions remains a challenge. One way to simplify (and consequently accelerate) learning is to exploit regularities in a domain, which allows generalization and reduction of the learning space. While object-oriented Markov decision processes (OO-MDPs) provide such generalization opportunities, we argue that the learning process may be further simplified by dividing the workload of tasks amongst multiple agents, solving problems as multiagent systems (MAS). In this paper, we propose a novel combination of OO-MDP and MAS, called multiagent OO-MDP (MOO-MDP). Our proposal accrues the benefits of both OO-MDP and MAS, better addressing scalability issues. We formalize the general model MOO-MDP and present an algorithm to solve deterministic cooperative MOO-MDPs. We show that our algorithm learns optimal policies while reducing the learning space by exploiting state abstractions. We experimentally compare our results with earlier approaches in three domains and evaluate the advantages of our approach in sample efficiency and memory requirements.
An improved differential evolution with information intercrossing and sharing mechanism for numerical optimization

This paper presents a novel differential evolution algorithm by designing a stochastic mixed mutation strategy and an information intercrossing and sharing mechanism. To effectively avoid the premature convergence and enhance the information dissemination between subpopulations under the work specialization, a stochastic mixed mutation strategy is first proposed by incorporating a cosine perturbation into the probability parameter setting and using two mutation strategies with this probability to balance the exploration and exploitation. Then, an information intercrossing and sharing mechanism is developed to make good use of the information of individuals by dividing the population into superior and inferior subpopulations according to their fitness values and exchanging or sharing their information. Furthermore, a simple and efficient approach is applied to adjust control parameters. Finally, the proposed algorithm is compared with twelve typical algorithms by numerical experiments on 55 benchmark functions from both CEC2005 and CEC2014, and is applied to solve the Parameter Estimation for Frequency-Modulated Sound Waves. Experimental results show that the proposed algorithm is very competitive.
Strong error analysis for stochastic gradient descent optimization algorithms

Stochastic gradient descent (SGD) optimization algorithms are key ingredients in a series of machine learning applications. In this article we perform a rigorous strong error analysis for SGD optimization algorithms. In particular, we prove for every arbitrarily small $\varepsilon \in (0,\infty)$ and every arbitrarily large $p\in (0,\infty)$ that the considered SGD optimization algorithm converges in the strong $L^p$-sense with order $\frac{1}{2}-\varepsilon$ to the global minimum of the objective function of the considered stochastic approximation problem under standard convexity-type assumptions on the objective function and relaxed assumptions on the moments of the stochastic errors appearing in the employed SGD optimization algorithm. The key ideas in our convergence proof are, first, to employ techniques from the theory of Lyapunov-type functions for dynamical systems to develop a general convergence machinery for SGD optimization algorithms based on such functions, then, to apply this general machinery to concrete Lyapunov-type functions with polynomial structures, and, thereafter, to perform an induction argument along the powers appearing in the Lyapunov-type functions in order to achieve for every arbitrarily large $ p \in (0,\infty) $ strong $ L^p $-convergence rates. This article also contains an extensive review of results on SGD optimization algorithms in the scientific literature.
Deep motion and appearance cues for visual tracking

Generic visual tracking is a challenging computer vision problem, with numerous applications. Most existing approaches rely on appearance information by employing either hand-crafted features or deep RGB features extracted from convolutional neural networks. Despite their success, these approaches struggle in case of ambiguous appearance information, leading to tracking failure. In such cases, we argue that motion cue provides discriminative and complementary information that can improve tracking performance. Contrary to visual tracking, deep motion features have been successfully applied for action recognition and video classification tasks. Typically, the motion features are learned by training a CNN on optical flow images extracted from large amounts of labeled videos. In this paper, we investigate the impact of deep motion features in a tracking-by-detection framework. We also evaluate the fusion of hand-crafted, deep RGB, and deep motion features and show that they contain complementary information. To the best of our knowledge, we are the first to propose fusing appearance information with deep motion features for visual tracking. Comprehensive experiments clearly demonstrate that our fusion approach with deep motion features outperforms standard methods relying on appearance information alone.
A Dynamic and Failure-Aware Task Scheduling Framework for Hadoop

Hadoop has become a popular framework for processing data-intensive applications in cloud environments. A core constituent of Hadoop is the scheduler, which is responsible for scheduling and monitoring the jobs and tasks, and rescheduling them in case of failures. Although fault-tolerance mechanisms have been proposed for Hadoop, the performance of Hadoop can be significantly impacted by unforeseen events in the cloud environment. In this paper, we introduce a dynamic and failure-aware framework that can be integrated within Hadoop scheduler and adjust the scheduling decisions based on collected information about the cloud environment. Our framework relies on predictions made by machine learning algorithms and scheduling policies generated by a Markovian Decision Process (MDP), to adjust its scheduling decisions on the fly. Instead of the fixed heartbeat-based failure detection commonly used in Hadoop to track active TaskTrackers (i.e., nodes that process the scheduled tasks), our proposed framework implements an adaptive algorithm that can dynamically detect the failures of the TaskTracker. To deploy our proposed framework, we have built, ATLAS+, an AdapTive Failure-Aware Scheduler for Hadoop. To assess the performance of ATLAS+, we conduct a large empirical study on a 100-node Hadoop cluster deployed on Amazon Elastic MapReduce (EMR), comparing the performance of ATLAS+ with those of three Hadoop schedulers (FIFO, Fair, and Capacity). Results show that ATLAS+ outperforms FIFO, Fair, and Capacity schedulers. ATLAS+ can reduce the number of failed jobs by up to 43 percent and the number of failed tasks by up to 59 percent. On average, ATLAS+ could reduce the total execution time of jobs by 10 minutes, which represents 40 percent of the job execution times, and by up to 3 minutes for tasks, which represents 47 percent of the task execution time. ATLAS+ also reduced CPU and memory usage by 22 and 20 percent, respectively.
Detecting multi-oriented text with corner-based region proposals

Previous approaches for scene text detection usually rely on manually defined sliding windows. This work presents an intuitive two-stage region-based method to detect multi-oriented text without any prior knowledge regarding the textual shape. In the first stage, we estimate the possible locations of text instances by detecting and linking corners instead of shifting a set of default anchors. The quadrilateral proposals are geometry adaptive, which allows our method to cope with various text aspect ratios and orientations. In the second stage, we design a new pooling layer named Dual-RoI Pooling which embeds data augmentation inside the region-wise subnetwork for more robust classification and regression over these proposals. Experimental results on public benchmarks confirm that the proposed method is capable of achieving comparable performance with state-of-the-art methods. The code is publicly available at https://github.com/xhzdeng/crpn