Miracle-dz/UniFAR-FTUs huggingface dataset:
Introspective Q-learning and learning from demonstration

Abstract One challenge faced by reinforcement learning (RL) agents is that in many environments the reward signal is sparse, leading to slow improvement of the agent’s performance in early learning episodes. Potential-based reward shaping can help to resolve the aforementioned issue of sparse reward by incorporating an expert’s domain knowledge into the learning through a potential function. Past work on reinforcement learning from demonstration (RLfD) directly mapped (sub-optimal) human expert demonstration to a potential function, which can speed up RL. In this paper we propose an introspective RL agent that significantly further speeds up the learning. An introspective RL agent records its state–action decisions and experience during learning in a priority queue. Good quality decisions, according to a Monte Carlo estimation, will be kept in the queue, while poorer decisions will be rejected. The queue is then used as demonstration to speed up RL via reward shaping. A human expert’s demonstration can be used to initialize the priority queue before the learning process starts. Experimental validation in the 4-dimensional CartPole domain and the 27-dimensional Super Mario AI domain shows that our approach significantly outperforms non-introspective RL and state-of-the-art approaches in RLfD in both domains.
Task Scheduling in Data Stream Processing Systems

In the era of big data, with streaming applications such as social media, surveillance monitoring and real-time search generating large volumes of data, efficient Data Stream Processing Systems (DSPSs) have become essential. When designing an efficient DSPS, a number of challenges need to be considered including task allocation, scalability, fault tolerance, QoS, parallelism degree, and state management, among others. In our research, we focus on task allocation as it has a significant impact on performance metrics such as data processing latency and system throughput. An application processed by DSPSs is represented as a Directed Acyclic Graph (DAG), where each vertex represents a task and the edges show the dataflow between the tasks. Task allocation can be defined as the assignment of the vertices in the DAG to the physical compute nodes such that the data movement between the nodes is minimised. Finding an optimal task placement for stream processing systems is NP-hard. Thus, approximate scheduling approaches are required to improve the performance of DSPSs. In this thesis, we present our three proposed schedulers, each having a different heuristic partitioning approach to minimise inter-node communication for either homogeneous or heterogeneous clusters. We demonstrate how each scheduler can efficiently assign groups of highly communicating tasks to compute nodes. Our schedulers are able to outperform two state-of-theart schedulers for three micro-benchmarks and two real-world applications, increasing throughput and reducing data processing latency as a result of a better task placement.
Vegetable Pest Image Recognition Method Based on Improved VGG Convolution Neural Network

Vegetables are one of the main crops in China, and pests are one of the important factors affecting the quality of vegetables. In order to improve the recognition accuracy of vegetable pest images, a vegetable pest image recognition method based on improved VGG convolution neural network is proposed. Based on the VGG16 and VGG19 models, the method optimizes the number of full connection layers, replaces the original SoftMax classifier in VGGNet with the three-label SoftMax classifier, optimizes the structure and parameters of the model, and uses the weight parameters of convolution layer and pooling layer in the pre-training model in transfer learning. Experiments were carried out on the self-expanding data set of vegetable pest images, and the performance of the method was tested. Tensorflow was used to train the network model. The experimental results showed that the pre-trained models (VGG16, VGG19, Inception V3, ResNet50) were trained on the vegetable pest image data set to adapt to the recognition of vegetable pest images. The experimental results also showed that compared with Inception V3 and ResNet50, the recognition accuracy of the pre-trained models using VGG16 and VGG19 were higher, and the test accuracy of the two models were 99.90% and 99.99% respectively. Finally, the methods were compared with the traditional VGG method in self-expanding data sets. The results showed that the accuracy of VGG16 model and VGG19 model were improved from 85.90% and 86.21% to 100% and 100% respectively; the classification accuracy of VGG16 model was improved from 64.02% to 99.90%, and the classification accuracy of VGG19 model was improved from 85.83% to 99.99%, which effectively improved the recognition accuracy.
Digital narrative conventions in heritage trail mobile apps

ABSTRACT Galleries, libraries, archives and museums (GLAMs) are increasingly using digital technologies for storytelling and creating mobile applications (apps) for cultural heritage content, but how apps are used in practice to communicate information to users has not been widely studied. A team of people from a heritage organisation, a university, and mobile app development group plan to create a bespoke heritage trail app for Ireland, but to date design conventions/recommendations for this genre are lacking. This article applies a systematic approach to digital narrative content analysis to better understand how apps are being used specifically for heritage trails with the aim of identifying what the common features are, which modalities and narrative techniques are used. The selected corpus included 55 apps downloaded from the Google Play Store. The results of this content analysis—based on the App Walkthrough Method (Light, B., Burgess, J., & Duguay, S. (2018). The walkthrough method: An approach to the study of apps. New Media & Society, 20(3), 881–900)—show that there is a gap between academic research themes/trends and how digital narrative is actually being communicated in the current market, and it aims to inform the future development of heritage trail apps by including a list of design and content features common to this genre.
Impact of Pre-and Post-Equalization on the Performance of PNC

The work presented considers a Physical Layer Network Coded (PNC) communication which is affected by Intersymbol Interference (ISI) due to Rayleigh fading channel. A pre-equalizer filter configured to combat the ISI is considered at the end nodes. The relay uses the Amplify and Forward (AF) strategy. The signal in the Multiple Access (MAC) phase is amplified at the relay and forwarded to the receiving nodes. The signal received by the end nodes in the broadcast (BC) phase is subjected to post-equalization by a receiver filter configured to overcome ISI. The performance of the PNC with only pre-equalizer applied, only post-equalization applied and the combined effect of pre-equalizer and post-equalizer is simulated . Simulation results indicate that the performance of PNC when only pre-equalizer applied , only post-equalization is applied are not much varying but better than the no equalization case. But the combined effect of pre-and post-equalization is much improved.
Utilizing word embeddings for part-of-speech tagging

In this paper, we illustrate the power of distributed word representations for the part-of-speech tagging of Hungarian texts. We trained CRF models for POS-tagging that made use of features derived from the sparse coding of the word embeddings of Hungarian words as signals. We show that relying on such a representation, it is possible to avoid the creation of language specific features for achieving reliable performance. We evaluated our models on all the subsections of the Szeged Treebank both using MSD and universal morphology tag sets. Furthermore, we also report results for inter-subcorpora experiments.
Lifetime Expectancy in Dementia with Lewy Bodies: Effects of Donepezil Administration and Special Nursing Home Replacement. A Retrospective Analysis in the Tajiri Project

Objective: Cholinesterase inhibitors (ChEIs) can delay the progression of Alzheimer disease (AD). We previously demonstrated a positive effect of donepezil (DNP) administration and a Special Nursing Home (SNH) replacement on lifetime expectancy after the onset of AD. Recently DNP has been indicated for use in the treatment of dementia with Lewy Bodies (DLB); however, the effect on lifetime expectancy remains unclear. Herein, we analyzed the effects of DNP on DLB. Methods: All outpatients at the Tajiri Clinic with available medical records and death certificates from 1999- 2012 were included in this retrospective analysis. The entry criteria were a diagnosis of dementia based on DSM-IV criteria and diagnosis of DLB using the international consensus criteria; medical treatment for more than 3 months and follow up to less than 1 year before death. Results: We identified 510 subjects based upon medical records and death certificates, of which 360 had a diagnosis of dementia that met the entry criteria. Of 51 patients diagnosed with DLB, 23 had taken DNP and 28 patients had not undergone drug treatment due to treatment prior to the introduction of DNP in 1999 in Japan. The lifetime expectancies after onset were 6.4 years in the DNP group and 3.6 years in the non-DNP group; with a significant drug effect. However, in contrast with the previous AD data, no significant effect of SNH residency was noted. Conclusion: Although this report has the limitation that all analyses were retrospective and lacked randomization, we found a positive effect of DNP on lifetime expectancy after the onset of DLB. The lower life expectancy compared with that of AD and the lack of an effect of SNH residency suggest the cholinergic deficiency in DLB is greater than that in AD.
Indexes of patent value: a systematic literature review and classification

ABSTRACT A proliferation of studies has been carried out on patent value indexes encompassing the most different viewpoints and considering different scale. However, the existing literature is fragmented and not properly integrated, demanding for an adequate organization in order to clarify the matter in a systematic manner. This article summarizes the results of a systematic review of the literature on patent value indexes, offering an organized synthesis according to the intrinsic and extrinsic properties of patents, which are the two extremely informatory components in every process of value recognition. This study improves on state patent value measurement by providing a thorough array of the indexes proposed in literature over a 20-year period and by empirically grouping them into core categories. Based on this classification, the authors offer a well-grounded path to achieve the intrinsic and extrinsic value of patents. The review concludes by suggesting promising directions for future investigation.
Target Search Control of AUV in Underwater Environment With Deep Reinforcement Learning

The autonomous underwater vehicle (AUV) is widely used to search for unknown targets in the complex underwater environment. Due to the unpredictability of the underwater environment, this paper combines the traditional frontier exploration method with deep reinforcement learning (DRL) to enable the AUV to explore the unknown underwater environment autonomously. In this paper, a grid map of the search environment is built by the grid method. The designed asynchronous advantage actor-critic (A3C) network structure is used in the traditional frontier exploration method for target search tasks. This network structure enables the AUV to learn from its own experience and generate search strategies for the various unknown environment. At the same time, DRL and dual-stream Q-learning algorithms are used for AUV navigation to further optimize the search path. The simulations and experiments in an unknown underwater environment with different layouts show that the proposed algorithm can accomplish target search tasks with a high success rate, and it can adapt to different environments. In addition, compared to other search methods, the frontier exploration algorithm based on DRL can search a wider environment faster, which results in a higher search efficiency and reduced search time.
Energy-Efficient Hybrid Precoding With Low Complexity for mmWave Massive MIMO Systems

Millimeter-wave (mmWave) massive multiple-input multiple-output (MIMO) utilizes large antenna arrays and is considered a promising technology for fifth-generation (5G) and beyond wireless communication systems. However, the high-power consumption of the radio-frequency (RF) chains makes it infeasible. To solve this problem, hybrid precoding is proposed, which is a combination of analog and digital precoding. The fully connected architecture hybrid precoding still requires a large number of phase shifters (PSs). The sub-connected architecture can greatly reduce the required power consumption, and however, it cannot obtain a satisfactory achievable rate. To avoid the high energy consumption and obtain a high resolution, we propose a novel partly connected architecture in this paper. In addition, we propose an energy-efficient successive interference cancelation (SIC) hybrid precoding based on the partly connected architecture, which transforms the problem of maximizing the total achievable rate with non-convex constraints into a series of sub-rate optimization problems. Furthermore, a low-complexity energy-efficient SIC hybrid precoding based on the partly connected architecture is developed, which uses the partial singular value decomposition (SVD) to realize the sub-rate optimization and significantly reduce the complexity. Theoretical analysis demonstrates the superiority of the proposed hybrid precoding in terms of complexity. The simulation results indicate that the proposed hybrid precoding algorithms enjoy better energy efficiency and achievable rate performance than some recently proposed hybrid precoding algorithms.
Metric Learning-Based Multi-Instance Multi-Label Classification With Label Correlation

In multi-instance multi-label learning (MIML) problems, predicting the labels of unseen bags becomes difficult when the labels of their instances are not provided directly. Therefore, it is necessary to exploit the label correlations to enhance the accuracy of the MIML classification. This paper presents the metric learning-based MIML-<inline-formula> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula>NN (MI(ML)<inline-formula> <tex-math notation="LaTeX">$^{\mathrm{ 2}}k$ </tex-math></inline-formula>NN) method, which is composed of three parts. First, the label Laplacian matrix can be learned to obtain the label correlations by minimizing the label manifold regularizer. Then, based on label correlations, a novel objective function for the MIML is proposed where Mahalanobis distances between positively correlated labels and bags are minimized. Moreover, this objective function can be optimized by employing the Rayleigh-Ritz theorem and gradient descent (GD) alternately. Finally, the average Hausdorff distances of bag-bag pairs and bag-label pairs are calculated to construct the MIML-<inline-formula> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula>NN classifier. Multiple classification experiments on three image and text benchmarks show the practicability and validity of our proposed method by comparing with the baseline methods.
Assessment of MVAR Injection in Power Optimization for a Hydrocarbon Industrial Plant Using a Genetic Algorithm

In this paper, a genetic algorithm (GA) is considered for assessing the effect of Million Volt-Ampere Reactive (MVAR) power injection via shunt capacitors in optimizing the electrical power loss for a real hydrocarbon industrial plant. The subject plant electrical system consists of 275 buses, two gas turbine generators, two steam turbine generators, large synchronous motors, and other rotational and static loads. The minimization of power losses objective is used to guide the optimization process, and, consequently, the injected power in the grid is monitored. First, the optimal locations of MVAR injection will be identified using a voltage stability index. The potential of power loss optimization with and without MVAR injection versus the base case will be discussed in the results. The results obtained demonstrate the potential and effectiveness of the proposed approach to optimize the power consumption in both scenarios (with and without MVAR injection). Also, in this paper a cost appraisal for the potential daily, monthly and annual cost saving in both scenarios will be addressed. Keywords-genetic algorithm, power loss optimization, electrical submersible pump, hydrocarbon facility, British thermal unit (BTU), millions of standard cubical feet of gas (MMscf).
Large Margin Nearest Neighbor Classification With Privileged Information for Biometric Applications

In this paper, a new metric learning algorithm is proposed to improve face verification and person re-identification in RGB images by learning from RGB and Depth (RGB-D) training images. We address this problem by formulating it as a Learning Using Privileged Information problem, in which the additional depth images associated with the RGB training images are not available for the testing process. Based on the large margin nearest neighbor (LMNN) classification framework, we propose an effective metric learning method called large margin nearest neighbor classification with privileged information (LMNN+) by incorporating depth information to improve decision function learning in the training process. Specifically, two distance metrics based on visual features as well as depth features are jointly learned by minimizing the triplet loss in which the within-class difference is minimized, while the between-class difference is maximized. The distances in the depth feature space can be utilized to guide the training process in the visual feature space. In addition, we propose an efficient optimization method which can handle billions of constraints in the optimization problem of LMNN+. The comprehensive experiments on the EUROCOM data set, the CurtinFaces data set as well as the BIWI RGBD-ID data set demonstrate the effectiveness of our algorithm for face verification and person re-identification by leveraging privileged information.
Maturity Models Architecture: A large systematic mapping

Maturity models are widespread in research and in particular, IT practitioner communities. However, theoretically sound, methodologically rigorous and empirically validated maturity models are quite rare. This systematic mapping paper focuses on the challenges faced during the development of maturity models. More specifically, it explores the literature on maturity models and standard guidelines to develop maturity models, the challenges identified and solutions proposed. Our systematic mapping revealed over six hundred articles on maturity models. Extant literature reveals that researchers have primarily focused on developing new maturity models pertaining to domain-specific problems and/or new enterprise technologies. We find rampant re-use of the design structure of widely adopted models such as Nolan’s Stage of Growth Model, Crosby’s Grid, and Capability Maturity Model (CMM). We also identify three dominant views of maturity models and provide guidelines for various approaches to constructing maturity models with a standard vocabulary. We finally propose using process theories and configurational approaches to address the main theoretical criticisms with regard to maturity models and conclude with some recommendations for maturity model developers.
An Improved Power Factor Correction for Interleaved Flyback Switched Mode Power Supply

Nowadays the use of electronic equipment finds a progressive development in the modern world. Hence it becomes a mandate to check whether the harmonic content of line current of any electronic device which is connected to the ac supply meets the appropriate standards. This demand is satisfied by implementing the Power Factor Correction (PFC) circuit in order to make the input current to be in sinusoidal in nature and in-phase with the input voltage. Numerous solutions are available to make the line current almost sinusoidal. This paper deals with the inclusion of passive PFC in the interleaved Fly back SMPS to improve the power factor. The proposed work also examines the reduction of current ripple at the output using interleaved converter.
Passive Copy-Move Forgery Detection Using Halftoning-based Block Truncation Coding Feature

This paper presents a new method on passive copy-move forgery detection by exploiting the effectiveness and usability of Halftoning-based Block Truncation Coding (HBTC) image feature. Copy-move forgery detection precisely locates the large size or flat tampered regions of an image. On our method, the tampered input image is firstly divided into several overlapping image blocks to construct the image feature descriptors. Each image block is further divided into several non-overlapping image blocks for processing HBTC. Two image feature descriptors, namely Color Feature (CF) and Bit Pattern Feature (BF) are computed from the HBTC compressed data-stream of each image block. Lexicography sorting rearranges the image feature descriptors in ascending manner for whole image. The similarity between some tampered image regions is measured based on their CF and BF under specific shift frequency threshold. As documented in the experimental results, the proposed method yields a promising result for detecting the tampered or copy-move forgery regions. It has proved that the HBTC is not only suitable for image compression, but it can also be used in the copy-move forgery detection.
Developing a massive real-time crowd simulation framework on the GPU

Crowd simulations are used to imitate the behaviour of a large group of people. Such simulations are used in industries ranging from video-games to public security. In recent years, research has turned to the parallel nature of GPUs to simulate the behaviour of individuals in a crowd in parallel. This allows for real time visualisation and interaction with massive crowds and/or their environment. There are, however, no tool that could facilitate the integration of GPU-accelerated crowd simulations in an existing program like a game. This paper proposes an OpenCL port of the PedSim library. It is a tool that allows CPU-run crowd simulations to be used in another application. It uses the social forces algorithm which is a crowd simulation method using a set of forces to compute each agents motion. Our implementation is compared in performance and functionality with the original PedSim. Finally, we discuss the functionalities missing from our implementation to provide a more complete crowd simulation. Acknowledgements I would like to thank Dr. Mukundan Ramakrishnan for all his supervision, support, feedback and patience throughout this year. I would also like to thank Prof. Andrew Cockburn and Dr. for their feedback and advice which might not seem much to them but were much appreciated. Lastly, I would like to thank all my friends and family who kept faith in me, supported me and cheered me on when I faltered.