Reviewerly/exHarmony huggingface dataset:
Clustering experience replay for the effective exploitation in reinforcement learning

• The limitation of the exploitation efficiency in existing reinforcement learning methods is analyzed in detail. • Clustering is combined into the experience replay by a divide-and-conquer framework to improve the exploitation efficiency. • Our experience replay can sufficiently replay all kinds of transitions in the current training with low time consumption. • A new reinforcement learning method is proposed to implement our experience replay. Reinforcement learning is a useful tool for training an agent to effectively achieve the desired goal in the sequential decision-making problem. It trains the agent to make decision by exploiting the experience in the transitions resulting from the different decisions. To exploit this experience, most reinforcement learning methods replay the explored transitions by uniform sampling. But in this way, it is easy to ignore the last explored transitions. Another way to exploit this experience defines the priority of each transition by the estimation error in training and then replays the transitions according to their priorities. But it only updates the priorities of the transitions replayed at the current training time step, thus the transitions with low priorities will be ignored. In this paper, we propose a clustering experience replay, called CER, to effectively exploit the experience hidden in all explored transitions in the current training. CER clusters and replays the transitions by a divide-and-conquer framework based on time division as follows. Firstly, it divides the whole training process into several periods. Secondly, at the end of each period, it uses k -means to cluster the transitions explored in this period. Finally, it constructs a conditional probability density function to ensure that all kinds of transitions will be sufficiently replayed in the current training. We construct a new method, TD3 _ CER, to implement our clustering experience replay on TD3. Through the theoretical analysis and experiments, we illustrate that our TD3 _ CER is more effective than the existing reinforcement learning methods. The source code can be downloaded from https://github.com/grcai/CER-Master .
Explainable Artificial Intelligence: An Updated Perspective

Artificial intelligence has become mainstream and its applications will only proliferate. Specific measures must be done to integrate such systems into society for the general benefit. One of the tools for improving that is explainability which boosts trust and understanding of decisions between humans and machines. This research offers an update on the current state of explainable AI (XAI). Recent XAI surveys in supervised learning show convergence of main conceptual ideas. We list the applications of XAI in the real world with concrete impact. The list is short and we call to action - to validate all the hard work done in the field with applications that go beyond experiments on datasets, but drive decisions and changes. We identify new frontiers of research, explainability of reinforcement learning and graph neural networks. For the latter, we give a detailed overview of the field.
Performance of Graph and Relational Databases in Complex Queries

In developing NoSQL databases, a major motivation is to achieve better efficient query performance compared with relational databases. The graph database is a NoSQL paradigm where navigation is based on links instead of joining tables. Links can be implemented as pointers, and following a pointer is a constant time operation, whereas joining tables is more complicated and slower, even in the presence of foreign keys. Therefore, link-based navigation has been seen as a more efficient query approach than using join operations on tables. Existing studies strongly support this assumption. However, query complexity has received less attention. For example, in enterprise information systems, queries are usually complex so data need to be collected from several tables or by traversing paths of graph nodes of different types. In the present study, we compared the query performance of a graph-based database system (Neo4j) and relational database systems (MySQL and MariaDB). The effect of different efficiency issues (e.g., indexing and optimization) were included in the comparison in order to investigate the most efficient solutions for different query types. The outcome is that although Neo4j is more efficient for simple queries, MariaDB is essentially more efficient when the complexity of queries increases. The study also highlighted how dramatically the efficiency of relational database has grown during the last decade.
Effects of Restorative Environment and Presence on Anxiety and Depression Based on Interactive Virtual Reality Scenarios

Anxiety and depression have been growing global mental health problems. The following studies explored the effect of interactive VR scenarios to find a low-cost and high-efficiency solution. Study 1 designed a 2 (anxiety and depression state) × 4 (interactive VR scenarios) experiment, the results of 20 participants showed that the designed scenarios had good restoration and presence, assisting to improve depression mood for people with mild to moderate anxiety and depression. Study 2 further investigated the intervention effects of two environment types (urban and park) and four interactive activities (automatic viewing, free-roaming, fishing, and watering plants in the park environment), based on data from a 10-minute experiment conducted by 195 participants with mild to moderate anxiety and depression. The subjective scales, EEG and EMG, and scenario experience were analyzed and the results showed that: (1) the restorative and present VR scenarios were beneficial in alleviating state anxiety and depression; (2) the restorative environment and presence were significantly and positively related to the reduction of anxiety and depression respectively, moreover, presence mediated the restorative environment on the recovery from anxiety and depression; (3) the environmental settings, the complexity of interaction, human factors, and maturity of VR devices and technology were also key factors that influenced the effects of interactive VR scenario experience and intervention. These studies revealed VR psychological intervention scenarios could be designed with comprehensive factors. Moreover, they might help pave the way for future study in exploring the physiology and psychology mode in virtual and real spaces, enhancing intervention effectiveness.
Blockchain: The Next Direction of Digital Payment in Drug Purchase

Several years ago, an unknown person / group proposed the coined term Bitcoin, described how the innovation of the blockchain, a peer-to-peer correlated structure, could be used to shed light on the issue of maintaining order of exchanges and maintaining a strategic distance from the issue of double spending. Bitcoin orders are exchanged and combined into a structure of limited size called squares that share the same timestamp. Arrangement axes (miners) are keen to connect the pieces to each other in chronological order, as each block has a hash of the previous square to form the blockchain. In this way, the blockchain architecture oversees the containment of a robust and auditable registry for all exchanges. The objective of this paper is to investigate how blockchain innovation can be utilized in digital payment of drug purchase supply chains. we chose this zone to center on, since it is exceptionally dependent on believe, contracts, arrangements, overseeing, human interaction and installments through a third party. Item forging, generation and dissemination issues, robberies and fraudulent drugs cause multi-billion-dollar income misfortunes within the world and posture a serious threat to open wellbeing Blockchain innovation can tremendously move forward execution in all these zones and diminish the chance of the issues.
Storing and analyzing a genome on a blockchain

There are major efforts underway to make genome sequencing a routine part of clinical practice. A critical barrier to these is achieving practical solutions for data ownership and integrity. Blockchain provides solutions to these challenges in other realms, such as finance. However, its use in genomics is stymied due to the difficulty in storing large-scale data on-chain, slow transaction speeds, and limitations on querying. To overcome these roadblocks, we developed a private blockchain network to store genomic variants and reference-aligned reads on-chain. It uses nested database indexing with an accompanying tool suite to rapidly access and analyze the data.
Immersive Auditory-Visual Real-Time Avatar System of ANA Avatar XPRIZE Finalist Avatar-Hubo

The need for real-time high-quality audio-visual systems skyrocketed with the advent of the ANA Avatar XPRIZE, a worldwide Avatar Robotic System competition. Avatar-Hubo scored 6 <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">th</sup> amongst teams worldwide; due to its humanoid characteristics, an operator is able to embody the robot. Therefore through vision, audio, and motion, one may transport their presence to another location. However, many telepresence setups suffer from high latency and poor depth perception. In this paper, both the latency and visual acuity of LiDAR and dual-camera stereoscopic vision systems were analyzed for Avatar-Hubo’s ANA Avatar XPRIZE semifinals setup. This analysis found that the LiDAR camera significantly underperformed in terms of Mean Data Transmission Rate when compared to the humanoid’s newly proposed vision-audio system. Additionally, trials with 17 participants during several System Usability Scale (SUS) testing found that Avatar-Hubo’s real-time auditory-visual system produced effective depth perception, allowing users to determine object proximity.
BAR: Behavior-aware recommendation for sequential heterogeneous one-class collaborative filtering

In our daily life, we are often greatly assisted with recommendation engines in finding the required information efficiently and accurately. In this paper, we focus on an emerging and important recommendation problem in enormous real-world applications, i.e., sequential heterogeneous one-class collaborative filtering (SHOCCF). In the studied problem, we have some users' sequential and heterogeneous one-class feedback, i.e., sequences of (item, behavior) pairs, where the behaviors can be of different types such as examinations and purchases. We propose a generic solution called behavior-aware recommendation (BAR), which is able to adapt an existing RNN-, CNN-, attention- or GNN-based sequential recommendation method to SHOCCF. The main idea of our BAR is to provide the behavior information to the input and output of a representation module that models the item sequence. Specifically, we design a behavior attention layer that uses the behavior at the next timestamp in order to digest the behaviors at different positions in the sequence and obtain more accurate attention scores that will be fed to the input of the representation module. Moreover, we further design a task-specific layer to fuse the real behavior at the next timestamp with the sequential feature generated by the representation module to distinguish different prediction tasks w.r.t. the behavior types. We then conduct extensive empirical studies on four public datasets and find that our BAR is able to significantly improve the performance of a certain sequential recommendation method when it is adapted to SHOCCF.
Automatic Text Summarization for Hindi Using Real Coded Genetic Algorithm

In the present scenario, Automatic Text Summarization (ATS) is in great demand to address the ever-growing volume of text data available online to discover relevant information faster. In this research, the ATS methodology is proposed for the Hindi language using Real Coded Genetic Algorithm (RCGA) over the health corpus, available in the Kaggle dataset. The methodology comprises five phases: preprocessing, feature extraction, processing, sentence ranking, and summary generation. Rigorous experimentation on varied feature sets is performed where distinguishing features, namely- sentence similarity and named entity features are combined with others for computing the evaluation metrics. The top 14 feature combinations are evaluated through Recall-Oriented Understudy for Gisting Evaluation (ROUGE) measure. RCGA computes appropriate feature weights through strings of features, chromosomes selection, and reproduction operators: Simulating Binary Crossover and Polynomial Mutation. To extract the highest scored sentences as the corpus summary, different compression rates are tested. In comparison with existing summarization tools, the ATS extractive method gives a summary reduction of 65%.
A Blockchain-based Data Governance with Privacy and Provenance: a case study for e-Prescription

Real-world applications in healthcare and supply chain domains produce, exchange, and share data in a multi-stakeholder environment. Data owners want to control their data and privacy in such settings. On the other hand, data consumers demand methods to understand when, how, and who produced the data. These requirements necessitate data governance frameworks that guarantee data provenance, privacy protection, and consent management. We introduce a decentralized data governance framework based on blockchain technology and proxy re-encryption to let data owners control and track their data through privacy-enhancing and consent management mechanisms. Besides, our framework allows the data consumers to understand data lineage through a blockchain-based provenance mechanism. We have used Digital e-prescription as the use case since it has multiple stakeholders and sensitive data while enabling the medical fraternity to manage patients’ prescription data, involving patients as data owners, doctors, and pharmacists as data consumers. Our proof-of-concept implementation and evaluation results based on CosmWasm and pyUmbral PRE show that the proposed decentralized system guarantees transparency, privacy, and trust with minimal overhead.
Robust outlier detection based on the changing rate of directed density ratio

The task of outlier detection aims at mining abnormal objects that deviate from normal distribution. Traditional unsupervised outlier detection methods can detect most global outliers, but only perform well under relatively single data distribution. Although the methods based on k-nearest neighbors can fit more complex data distribution, they also have the problem of hardly detecting local outliers or the performance easily influenced by data manifolds. At the same time, the outlier detection performance of most methods based on k-nearest neighbors is greatly affected by parameter k. We proposed a robust outlier detection method based on the changing rate of directed density ratio. The local density of samples is calculated by combining kernel density estimation and extended neighbor set which contains k-nearest neighbors and reverse k-nearest neighbors. Then we define the directed density ratio of a sample based on the density ratio and the vector between the sample and its neighbors. The local information can be better estimated by directed density ratio under different local densities and data manifolds. Then, by increasing the size of neighbors, the change of directed density ratio of a sample was calculated and finally summed up as the outlier score. Experiments are carried out on 12 synthetic datasets that simulate different data distributions and 22 public datasets. The experimental results show that compared with several state-of-the-art methods, the proposed method can achieve better outlier detection performance under different data distributions. In addition, the proposed method shows better robustness when parameter k changes in experimental results.
Toward Cranfield-inspired reusability assessment in interactive information retrieval evaluation

Re-using research resources is essential for advancing knowledge and developing repeatable, empirically solid experiments in scientific fields, including interactive information retrieval (IIR). Despite recent efforts on standardizing research re-use and documentation, how to quantitatively measure the reusability of IIR resources still remains an open challenge. Inspired by the reusability evaluations on Cranfield experiments, our work proactively explores the problem of measuring IIR test collection reusability and makes threefold contributions: (1) constructing a novel usefulness-oriented framework with specific analytical methods for evaluating the reusability of IIR test collections consisting of query sets, document/page sets, and sets of task-document usefulness ( tuse ); (2) explaining the potential impacts of varying IIR-specific factors (e.g. search tasks, sessions, user characteristics) on test collection reusability; (3) proposing actionable methods for building reusable test collections in IIR and thereby amortizing the true cost of user-oriented evaluations. The Cranfield-inspired reusability assessment framework serves as an initial step towards accurately evaluating the reusability of IIR research resources and measuring the reproducibility of IIR evaluation results. It also demonstrates an innovative approach to integrating the insights from individual heterogeneous user studies with the evaluation techniques developed in standardized ad hoc retrieval experiments, which will facilitate the maturation of IIR fields and eventually benefits both sides of research.
FAPN: Feature Augmented Pyramid Network for polyp segmentation

Accurate polyp segmentation during colonoscopy examinations can help the clinicians accurately locate polyp areas for further diagnosis or surgeries and thereby decrease the chances of polyps growing into cancer. Although existing approaches can achieve significant improvement by multi-scale feature learning, attention/contextual augmentation, and deep supervision, polyp segmentation is still far from being solved. Actually, enhancing the capability of feature representation may be an excellent way to improve the polyp segmentation performance. From this perspective, we propose a simple but strong framework over feature pyramid network (FPN), called Feature Augmented Pyramid Networks (FAPN), for accurate polyp segmentation with augmented feature representation. Specifically, FAPN consists of three components: Cross-Embedding Module (CEM), Predictive Calibration Module (PCM), and Hierarchical Feature Fusion Module (HFFM). CEM is a two-stage fusion approach that first performs an interactive embedding of multi-level features followed by a second fusion, thus enhancing the fused feature representation. After fusing, PCM leverages the predicted probability maps of each stage (after supervised optimization) to calibrate the fused feature representations, which effectively highlights the regions of interest while avoiding the interference of irrelevant information. Finally, HFFM sequentially combines features from each stage in the top-down pathway, yielding a more robust multi-scale feature representation that allows the framework to segment polyps more accurately. Extensive experiments demonstrate that the proposed network performs favorably against more than a dozen of state-of-the-art methods on five popular polyp segmentation benchmarks.
A Unified End-to-End Retriever-Reader Framework for Knowledge-based VQA

Knowledge-based Visual Question Answering (VQA) expects models to rely on external knowledge for robust answer prediction. Though significant it is, this paper discovers several leading factors impeding the advancement of current state-of-the-art methods. On the one hand, methods which exploit the explicit knowledge take the knowledge as a complement for the coarsely trained VQA model. Despite their effectiveness, these approaches often suffer from noise incorporation and error propagation. On the other hand, pertaining to the implicit knowledge, the multi-modal implicit knowledge for knowledge-based VQA still remains largely unexplored. This work presents a unified end-to-end retriever-reader framework towards knowledge-based VQA. In particular, we shed light on the multi-modal implicit knowledge from vision-language pre-training models to mine its potential in knowledge reasoning. As for the noise problem encountered by the retrieval operation on explicit knowledge, we design a novel scheme to create pseudo labels for effective knowledge supervision. This scheme is able to not only provide guidance for knowledge retrieval, but also drop these instances potentially error-prone towards question answering. To validate the effectiveness of the proposed method, we conduct extensive experiments on the benchmark dataset. The experimental results reveal that our method outperforms existing baselines by a noticeable margin. Beyond the reported numbers, this paper further spawns several insights on knowledge utilization for future research with some empirical findings.
Two-dimensional phase unwrapping by a high-resolution deep learning network

Phase measurements are of considerable interest in the field of optical metrology. Retrieving unambiguous phases from a single ambiguous wrapped phase map is a tricky problem. Conventional spatial phase unwrapping approaches are vulnerable to noise, the large variation range of phase surfaces, and phase discontinuity. In this study, the task of phase unwrapping is formulated as a pixel-wise semantic segmentation problem, and a high-resolution deep learning network named the HRNet is employed to infer an integer multiple of 2π for every pixel aiming to remove 2π discontinuities. The effectiveness and superiority of the proposed method are verified on both simulated and real data and through qualitative and quantitative comparisons with two deep learning-based methods and five classical spatial phase unwrapping methods.
Semantic driven attention network with attribute learning for unsupervised person re-identification

Unsupervised domain adaptation (UDA) person re-identification (re-ID) aims to transfer knowledge from a labeled source domain to guide the task proposed on the unlabeled target domain, in which people share different identifications and cross multiple camera views within two different domains. Consequently, traditional UDA re-ID techniques generally suffer due to the negative transfer caused by the inevitable noise generated by variant backgrounds, while the foregrounds also lack sufficient reliable identification knowledge to guarantee the qualified cross-domain re-ID. To remedy the raised negative transfer caused by variant backgrounds, we propose a novel body structure estimation (BSE) mechanism enforced semantic driven attention network (SDA), which enables the designed model with semantic effectiveness to distinguish the foreground and background. In searching for the reliable feature representations as in the foreground areas, we propose a novel label refinery mechanism to dynamically optimize the traditional attribute learning techniques for the strengthened personal attribute features and thus resulting the qualified UDA-re-ID. Extensive experiments demonstrate the effectiveness of our method in solving unsupervised domain adaptation person re-ID task on three large-scale datasets including Market-1501, DukeMTMC-reID and MSMT17.