Reviewerly/exHarmony huggingface dataset:
A survey on cross-media search based on user intention understanding in social networks

With the increasing popularity of online social networks, more and more people are posting information, updating their statuses, and searching for topics there. Massive cross-media big data has been gathered by online social networks, with high dynamics, context-sparsity, and cross-media semantic gaps. In addition, it can be challenging to understand users' search intentions in the setting of social networks. The above problems have brought severe challenges and obstacles to cross-media searches, which also attracted more and more attention on social networks. As a relatively new research topic and interest, the concept, methodology, and overall research idea of cross-media search based on user search intention understanding are not evident in the literature. The research also lacks a unified paradigm and relatively complete research ideas on social networks. To solve these problems, we reviewed more than 100 references based on our preliminary exploration and research experience in this field from the whole process involved. We also detailed methodology, datasets, evaluation indicators, experiment evaluation, and research trends and analyzed the challenges. These works will help beginners quickly establish research ideas and processes in this field, and enable them to focus on algorithm design without paying too much attention to datasets, evaluation metrics, and research frameworks. We believe this review will attract more researchers to focus on social network cross-media search based on user search intention understanding and benefit their work.
Semantic matching in machine reading comprehension: An empirical study

Machine reading comprehension (MRC) is a challenging task in the field of artificial intelligence. Most existing MRC works contain a semantic matching module, either explicitly or intrinsically, to determine whether a piece of context answers a question. However, there is scant work which systematically evaluates different paradigms using semantic matching in MRC. In this paper, we conduct a systematic empirical study on semantic matching. We formulate a two-stage framework which consists of a semantic matching model and a reading model, based on pre-trained language models. We compare and analyze the effectiveness and efficiency of using semantic matching modules with different setups on four types of MRC datasets. We verify that using semantic matching before a reading model improves both the effectiveness and efficiency of MRC. Compared with answering questions by extracting information from concise context, we observe that semantic matching yields more improvements for answering questions with noisy and adversarial context. Matching coarse-grained context to questions, e.g., paragraphs, is more effective than matching fine-grained context, e.g., sentences and spans. We also find that semantic matching is helpful for answering who/where/when/what/how/which questions, whereas it decreases the MRC performance on why questions. This may imply that semantic matching helps to answer a question whose necessary information can be retrieved from a single sentence. The above observations demonstrate the advantages and disadvantages of using semantic matching in different scenarios.
MFXSS: An effective XSS vulnerability detection method in JavaScript based on multi-feature model

The widespread use of web applications has also made them more vulnerable to hackers, resulting in the leakage of large amounts of application and personal privacy data. Cross-site scripting (XSS) attacks are one of the most significant threats to web applications. Attackers can inject scripts to control the victim’s browser to send data or execute commands, leading to the theft of privacy or the hijacking of login tokens. Therefore, we proposed a multi-feature fusion-based neural network vulnerability detection model for detecting XSS vulnerabilities in the JavaScript source code of websites (We termed our implementation of this approach MFXSS). We combine abstract syntax tree (AST) and code control flow graph (CFG) to convert the generalized sample data into graph structure and code string structure. Then, through the graph convolutional neural network, weighted aggregation, and the bidirectional recurrent neural network, the logical call features and the context execution relationship features of the source code are extracted and fused respectively. Finally, the fused feature vectors are used to detect and predict XSS vulnerabilities in JavaScript. In the experiment, we designed multiple control experiments to verify that the model construction is optimal, and the accuracy rates in the standard and variant datasets are 0.997 and 0.986. Moreover, in comparing similar detection schemes, MFXSS also performs better. Applying the model to an actual web environment, we successfully detected the presence of XSS vulnerabilities in websites.
Multidimensional Bose quantum error correction based on neural network decoder

Abstract Boson quantum error correction is an important means to realize quantum error correction information processing. In this paper, we consider the connection of a single-mode Gottesman-Kitaev-Preskill (GKP) code with a two-dimensional (2D) surface (surface-GKP code) on a triangular quadrilateral lattice. On the one hand, we use a Steane-type scheme with maximum likelihood estimation for surface-GKP code error correction. On the other hand, the minimum-weight perfect matching (MWPM) algorithm is used to decode surface-GKP codes. In the case where only the data GKP qubits are noisy, the threshold reaches σ ≈ 0.5 ( $$\bar{p}\approx 12.3 \%$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mover> <mml:mrow> <mml:mi>p</mml:mi> </mml:mrow> <mml:mo>¯</mml:mo> </mml:mover> <mml:mo>≈</mml:mo> <mml:mn>12.3</mml:mn> <mml:mi>%</mml:mi> </mml:mrow> </mml:math> ). If the measurement is also noisy, the threshold is reached σ ≈ 0.25 ( $$\bar{p}\approx 10.02 \%$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mover> <mml:mrow> <mml:mi>p</mml:mi> </mml:mrow> <mml:mo>¯</mml:mo> </mml:mover> <mml:mo>≈</mml:mo> <mml:mn>10.02</mml:mn> <mml:mi>%</mml:mi> </mml:mrow> </mml:math> ). More importantly, we introduce a neural network decoder. When the measurements in GKP error correction are noise-free, the threshold reaches σ ≈ 0.78 ( $$\bar{p}\approx 15.12 \%$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mover> <mml:mrow> <mml:mi>p</mml:mi> </mml:mrow> <mml:mo>¯</mml:mo> </mml:mover> <mml:mo>≈</mml:mo> <mml:mn>15.12</mml:mn> <mml:mi>%</mml:mi> </mml:mrow> </mml:math> ). The threshold reaches σ ≈ 0.34 ( $$\bar{p}\approx 11.37 \%$$ <mml:math xmlns:mml="http://www.w3.org/1998/Math/MathML"> <mml:mrow> <mml:mover> <mml:mrow> <mml:mi>p</mml:mi> </mml:mrow> <mml:mo>¯</mml:mo> </mml:mover> <mml:mo>≈</mml:mo> <mml:mn>11.37</mml:mn> <mml:mi>%</mml:mi> </mml:mrow> </mml:math> ) when all measurements are noisy. Through the above optimization method, multi-party quantum error correction will achieve a better guarantee effect in fault-tolerant quantum computing.
Integrating safety constraints into adversarial training for robust deep reinforcement learning

The ability to resist interference is the key to the widespread application of reinforcement learning. Although adversarial training is a promising method for robust promotion, standard adversarial training leads to unstable results or performance deterioration due to the presence of perturbation. To address the problem, a robust reinforcement learning method which integrates safety constraints that are modelled by environment termination conditions into adversarial training is proposed, where safety constraints are adopted to restrict agent’s actions and guide the training process. For better modelling the robust reinforcement learning problem, a modified constrained Markov Decision Process (MDP) that considers perturbation for robust reinforcement learning, named Constrained Markov Decision Process (CMDP) with Perturbation (CMDPP) is also introduced. The proposed safe robust reinforcement learning method based on CMDPP utilizes the penalty function to solve CMDP and generates perturbation from the gradient of state for adversarial training. Tests on the robustness of the proposed method under several attack methods and evaluation of generalization through changing environment dynamics were carried out on the OpenAI gym and Roboschool environments. The results demonstrate that our method not only has a better performance confronting the attack but also has a higher generalization capability with reference to the changing environment dynamics.
IWT and RSA based asymmetric image encryption algorithm

An asymmetric image encryption algorithm based on integer wavelet transformation (IWT) and Rivest-Shamir-Adleman (RSA) algorithm is proposed. Firstly, two plain characteristic parameters (PCP) of the plain image are extracted and two random numbers are chosen. Then, a new parameter transformation model (PTM) is constructed to do nonlinear processing for them, and three cipher characteristic parameters (CCP) are got. After applying RSA operation for CCP (seen as plain messages), three cipher messages are obtained. Secondly, a new initial value obtaining model (IOM) for all plain messages and cipher messages is established, by which initial values of 3D chaotic system are produced. Then, three chaotic sequences can be generated. Thirdly, chaotic sequences are used to confuse the plain image by a way of row-column cycle. Then, IWT operation is carried out and the above chaotic sequences are employed to confuse again the wavelet coefficients. Thereafter, inverse IWT is applied to get the confused image, realizing double confusion operations on both spatial domain and frequency domain. Finally, the confused image is diffused as a whole to get the cipher image. Experiment results explain that the proposed algorithm can realize the encryption in short time, and resist effectively against brute-force attack and noise attack.
Auto-GNN: Neural architecture search of graph neural networks

Graph neural networks (GNNs) have been widely used in various graph analysis tasks. As the graph characteristics vary significantly in real-world systems, given a specific scenario, the architecture parameters need to be tuned carefully to identify a suitable GNN. Neural architecture search (NAS) has shown its potential in discovering the effective architectures for the learning tasks in image and language modeling. However, the existing NAS algorithms cannot be applied efficiently to GNN search problem because of two facts. First, the large-step exploration in the traditional controller fails to learn the sensitive performance variations with slight architecture modifications in GNNs. Second, the search space is composed of heterogeneous GNNs, which prevents the direct adoption of parameter sharing among them to accelerate the search progress. To tackle the challenges, we propose an automated graph neural networks (AGNN) framework, which aims to find the optimal GNN architecture efficiently. Specifically, a reinforced conservative controller is designed to explore the architecture space with small steps. To accelerate the validation, a novel constrained parameter sharing strategy is presented to regularize the weight transferring among GNNs. It avoids training from scratch and saves the computation time. Experimental results on the benchmark datasets demonstrate that the architecture identified by AGNN achieves the best performance and search efficiency, comparing with existing human-invented models and the traditional search methods.
15 Years of (Who)man Robot Interaction: Reviewing the H in Human-Robot Interaction

Recent work identified a concerning trend of disproportional gender representation in research participants in Human–Computer Interaction (HCI). Motivated by the fact that Human–Robot Interaction (HRI) shares many participant practices with HCI, we explored whether this trend is mirrored in our field. By producing a dataset covering participant gender representation in all 684 full papers published at the HRI conference from 2006–2021, we identify current trends in HRI research participation. We find an over-representation of men in research participants to date, as well as inconsistent and/or incomplete gender reporting, which typically engages in a binary treatment of gender at odds with published best practice guidelines. We further examine if and how participant gender has been considered in user studies to date, in-line with current discourse surrounding the importance and/or potential risks of gender based analyses. Finally, we complement this with a survey of HRI researchers to examine correlations between who is doing with the who is taking part, to further reflect on factors which seemingly influence gender bias in research participation across different sub-fields of HRI. Through our analysis, we identify areas for improvement, but also reason for optimism, and derive some practical suggestions for HRI researchers going forward.
End-User Audits: A System Empowering Communities to Lead Large-Scale Investigations of Harmful Algorithmic Behavior

Because algorithm audits are conducted by technical experts, audits are necessarily limited to the hypotheses that experts think to test. End users hold the promise to expand this purview, as they inhabit spaces and witness algorithmic impacts that auditors do not. In pursuit of this goal, we propose end-user audits-system-scale audits led by non-technical users-and present an approach that scaffolds end users in hypothesis generation, evidence identification, and results communication. Today, performing a system-scale audit requires substantial user effort to label thousands of system outputs, so we introduce a collaborative filtering technique that leverages the algorithmic system's own disaggregated training data to project from a small number of end user labels onto the full test set. Our end-user auditing tool, IndieLabel, employs these predicted labels so that users can rapidly explore where their opinions diverge from the algorithmic system's outputs. By highlighting topic areas where the system is under-performing for the user and surfacing sets of likely error cases, the tool guides the user in authoring an audit report. In an evaluation of end-user audits on a popular comment toxicity model with 17 non-technical participants, participants both replicated issues that formal audits had previously identified and also raised previously underreported issues such as under-flagging on veiled forms of hate that perpetuate stigma and over-flagging of slurs that have been reclaimed by marginalized communities.
An examination of the effectiveness of problem‐based learning method supported by augmented reality in science education

Abstract Background Augmented reality (AR) is a widely used technology in science education today. Problem‐based learning (PBL) is one of the teaching methods employed in science education for a long time. Studies where AR and PBL are used together are new and rare. PBL and AR technology are used together in the study because it is believed that integrating AR technology with PBL increases the effectiveness of PBL. Objectives In the present study, the effect of PBL supported by AR on academic achievement, reflective thinking skills towards problem‐solving, decision‐making abilities, and the permanence levels of the 7th‐grade students were researched. Methods The embedded design from mixed methods was used in the study, which had two experimental groups and one control group. For the Experiment‐1, Experiment‐2, and Control Groups, respectively, PBL supported by AR, PBL alone, and the current teaching method were used. The sample consists of 92 students from the seventh grade. Results and Conclusions In terms of academic achievement, reflective thinking skills towards problem‐solving, and decision‐making abilities, the Experiment‐1 Group outperformed the Experiment‐2 and Control Groups, and the Experiment‐2 Group outperformed the Control Group. It has been discovered that PBL supported by AR is more effective than solely PBL and the currently used teaching method in terms of improving reflective thinking skills towards problem‐solving and decision‐making abilities, as well as the persistence of academic achievement in a science course.
ColBERT-PRF: Semantic Pseudo-Relevance Feedback for Dense Passage and Document Retrieval

Pseudo-relevance feedback mechanisms, from Rocchio to the relevance models, have shown the usefulness of expanding and reweighting the users’ initial queries using information occurring in an initial set of retrieved documents, known as the pseudo-relevant set. Recently, dense retrieval – through the use of neural contextual language models such as BERT for analysing the documents’ and queries’ contents and computing their relevance scores – has shown a promising performance on several information retrieval tasks still relying on the traditional inverted index for identifying documents relevant to a query. Two different dense retrieval families have emerged: the use of single embedded representations for each passage and query, e.g., using BERT’s [CLS] token, or via multiple representations, e.g., using an embedding for each token of the query and document (exemplified by ColBERT). In this work, we conduct the first study into the potential for multiple representation dense retrieval to be enhanced using pseudo-relevance feedback and present our proposed approach ColBERT-PRF. In particular, based on the pseudo-relevant set of documents identified using a first-pass dense retrieval, ColBERT-PRF extracts the representative feedback embeddings from the document embeddings of the pseudo-relevant set. Among the representative feedback embeddings, the embeddings that most highly discriminate among documents are employed as the expansion embeddings, which are then added to the original query representation. We show that these additional expansion embeddings both enhance the effectiveness of a reranking of the initial query results as well as an additional dense retrieval operation. Indeed, experiments on the MSMARCO passage ranking dataset show that MAP can be improved by up to 26% on the TREC 2019 query set and 10% on the TREC 2020 query set by the application of our proposed ColBERT-PRF method on a ColBERT dense retrieval approach.We further validate the effectiveness of our proposed pseudo-relevance feedback technique for a dense retrieval model on MSMARCO document ranking and TREC Robust04 document ranking tasks. For instance, ColBERT-PRF exhibits up to 21% and 14% improvement in MAP over the ColBERT E2E model on the MSMARCO document ranking TREC 2019 and TREC 2020 query sets, respectively. Additionally, we study the effectiveness of variants of the ColBERT-PRF model with different weighting methods. Finally, we show that ColBERT-PRF can be made more efficient, attaining up to 4.54× speedup over the default ColBERT-PRF model, and with little impact on effectiveness, through the application of approximate scoring and different clustering methods.
ICD: A new interpretable cognitive diagnosis model for intelligent tutor systems

Numerous models have been proposed for cognitive diagnosis in intelligent tutoring systems. However, the existing models still have room for improvement: (1) they ignore the interaction among knowledge concepts and (2) they ignore the quantitative relation between exercises and concepts. Here, we propose a cognitive diagnostic model comprising three layers of novel neural networks called ICD to solve the above two problems. Specifically, the first layer fits the influence of exercises on concepts, the second layer fits the interaction between concepts, and the third layer fits the influence of concepts on exercises. The three layers allow ICD to effectively distinguish learners with different cognitive levels, that is, ICD has good interpretability. The experimental results show that both the performance and interpretability of ICD are better than those of the latest state-of-the-art CDMs such as RCD, NCDM, and CDGK, and classical CDMs such as DINA and MIRT.
Recurrent prediction model for partially observable MDPs

Partially observable Markov decision process (POMDP) is a key challenging problem in the application of reinforcement learning since it comprehensively describes real agent-environment interactions. Recent works mainly utilize conventional reward signals to train a representation that converts POMDPs to MDPs. However, rewards alone are not enough for a good representation without temporal information. In this paper, we first introduce a novel Recurrent Prediction Model to integrate temporal information into the representation that solves POMDP problems by training three additional unsupervised prediction models, named transition model, reward recovery model, and observation recovery model. This paper secondly makes a modification of the data structure of vanilla replay buffer to reduce the memory usage and thirdly proposes an off-policy correction algorithm to decrease the policy lag in POMDPs. The experiments show that our model achieves better performance in partially observable environments on both stand-alone and distributed training systems.
Skew information correlations and local quantum Fisher information in two gravitational cat states

Simulating quantum features of quantum systems is an important path to approaching quantum technologies and understanding the physical world. This paper inspects the dynamics of quantum correlations within two cat states carrying equal gravitational charges. We study the influence of both the thermal coupling and the gravitational interaction on the behavior of nonclassical correlations. In such a scenario, we assume neglecting all the environmental effects except gravitational interaction to explore how gravity impacts the quantum features of the considered system. The density operator is constructed in each instance, and the quantum correlations dynamics is investigated in detail as a function of all characteristics of the system leveraging local quantum uncertainty, uncertainty-induced nonlocality, and local quantum Fisher information. The findings reveal that quantum correlations depend upon the characteristics of the gravitational cat states and gravitational force intensity. Such investigation offers a compelling framework for assessing gravity effects at a low scale, leading to a better understanding of its impacts on the quantum aspects for prospective applications in developing quantum technologies.
Design and implementation of a metaverse platform for traditional culture: the chime bells of Marquis Yi of Zeng

Abstract Metaverse platforms have become increasingly prevalent for collaboration in virtual environments. Metaverse platforms, as opposed to virtual reality, augmented reality, and mixed reality, expand with enhanced social meanings within virtual worlds. The research object in this study is the chime bells of Marquis Yi of Zeng, one of China's most treasured cultural heritage sites. We aimed to create a metaverse platform for the chime bells of Marquis Yi of Zeng to provide visitors with a highly immersive and interactive experience. First, we collected the materials and data of the chime bells and other exhibits, as well as historical information. Then, the data were processed and integrated for 3D model reconstruction. In addition, we designed the virtual roaming system through which visitors could interact with the exhibits to obtain multimedia information and even knock to ring the chime bells. Finally, we built our system to connect multiple visitors in different geographic locations and encourage them to collaborate and communicate within the virtual space. This platform helps users visualize cultural heritage, simulates real-life tour experiences with intuitive manners of interaction, and motivates visitors’ interest in traditional culture. This research also reveals the potential use of metaverse-related techniques in cultural heritage sectors.