Reviewerly/exHarmony huggingface dataset:
An Exploratory Study on Sustaining Cyber Security Protection through SETA Implementation

It is undeniable that most business organizations rely on the Internet to conduct their highly competitive businesses nowadays. Cyber security is one of the important elements for companies to guarantee the normal operation of their business activities. However, there is no panacea in cyber security protection. Common security practices used are to deploy hardware and software security protection tools to combat the known security threats which may become more and more powerful later. In fact, the attackers and security practitioners are at war from time to time. As a result, such a tools-based security protection strategy cannot be sustained. On the other hand, the related awareness training for employees is ignored in a number of companies, which has made biased the decisions made by staff when facing cyber security breaches. In this study, in order to find ways to sustain such protection, we conduct a quantitative analysis to explore the key elements contributing to the SETA implementation of the companies and organizations. We evaluate the performances of eight supervised learning models in a dataset collected from cyber security breach surveys on UK businesses to perform a fundamental analysis. The detailed analysis is performed via the feature importance of features generated in the model with better performance in the task of detecting the companies and organizations with SETA implementation. The experiment result shows that the awareness related factors play the most significant role in the SETA implementation decision-making for the businesses, and most of the businesses are lacking the awareness to prevent the potential cyber security risks in the stuff using externally-hosted web services and products as well as services depending on online services.
A New Approach to the Allocation of Multidimensional Resources in Production Processes

Modern technologies in the field of automation, robotics and IT have significantly changed the face of modern production systems. In particular, the use of AVG, PLC, mobile robots, RFID, IoT, etc. results in modern production processes being characterized by, among others, shortened production cycles and supply chains, reduced production costs, increased product quality and reliability, etc. Moreover, the application of these technologies requires a new definition and methods of using production resources. Most often these are resources that are characterized by many functionalities, the so-called multidimensional resources, which can be configured, remotely controlled, updated, etc., and their use in many cases enables the self-optimization and self-organization of the production system. The article presents the problem of allocation and control of multidimensional resources in production processes. The proprietary formal model of the problem is proposed, as well as how to use it in both proactive and reactive modes. A procedure for reducing the size of the modeled problem is also proposed, the use of which enables a two-fold reduction in the number of constraints and even a fifty-fold reduction in the number of decision variables of the proposed model. This results in an almost hundredfold reduction in computation time for the considered data instances. An original hybrid approach is used to implement the model, which enables the integration of mathematical programming (MP) and programming in constrained logic (CLP). Model data and parameters have been saved as facts.
T2L: A traceable and trustable consortium blockchain for logistics

Traceability and trustiness are two critical issues in the logistics sector. Blockchain provides a potential way for logistics tracking systems due to its traits of tamper resistance. However, it is non-trivial to apply blockchain on logistics because of firstly, the binding relationship between virtue data and physical location cannot be guaranteed so that frauds may exist. Secondly, it is neither practical to upload complete data on the blockchain due to the limited storage resources nor convincing to trust the digest of the data. This paper proposes a traceable and trustable consortium blockchain for logistics T2L to provide an efficient solution to the mentioned problems. Specifically, the authenticated geocoding data from telecom operators' base stations are adopted to ensure the location credibility of the data before being uploaded to the blockchain for the purpose of reliable traceability of the logistics. Moreover, we propose a scheme based on Zero Knowledge Proof of Retrievability (ZK BLS-PoR) to ensure the trustiness of the data digest and the proofs to the blockchain. Any user in the system can check the data completeness by verifying the proofs instead of downloading and examining the whole data based on the proposed ZK BLS- PoR scheme, which can provide solid theoretical verification. In all, the proposed T2L framework is a traceable and trustable logistics system with a high level of security.
Classification of Sciences and Systematization of Knowledge for the General and Professional Integrative Education Content Design

The article highlights the need for a new classification of scientific knowledge in order to reform the educational doctrine in modern Russian society. Several reasons explain this position. First, the constantly increasing volume of information leads to the “lag effect” of educational programs in comparison to scientific achievements; the effect of “redundancy” of knowledge cannot be envisioned and learned within the school year. Second, the post-non-classical stage of science is marked by the interdisciplinary nature of any knowledge and a focus on its object, which limits its representation within the framework of classical scientific differentiation and results in the loss of a holistic view of the world, which is unproductive in the context modern man’s cognitive styles. Third, modern education has evolved to a practice-oriented system that helps students adapt to world situations, but it has not altered considerably. The article examines prior classifications of sciences and puts forward a new conceptual model of systematization of knowledge that takes into account society’s educational demands. The anthropic principle underpins the proposed model of classification of sciences and knowledge systematization. This model can be used to create new educational programs based on the logic of integrative education, define constant academic disciplines for general and professional education, the order in which they should be studied, and the possibility of constant modernization of knowledge without expanding the number of academic disciplines. The provided methodology of scientific classification and knowledge systematization is thought to be useful in addressing the pressing issues in current education.
Heuristic Attention Representation Learning for Self-Supervised Pretraining

Recently, self-supervised learning methods have been shown to be very powerful and efficient for yielding robust representation learning by maximizing the similarity across different augmented views in embedding vector space. However, the main challenge is generating different views with random cropping; the semantic feature might exist differently across different views leading to inappropriately maximizing similarity objective. We tackle this problem by introducing Heuristic Attention Representation Learning (HARL). This self-supervised framework relies on the joint embedding architecture in which the two neural networks are trained to produce similar embedding for different augmented views of the same image. HARL framework adopts prior visual object-level attention by generating a heuristic mask proposal for each training image and maximizes the abstract object-level embedding on vector space instead of whole image representation from previous works. As a result, HARL extracts the quality semantic representation from each training sample and outperforms existing self-supervised baselines on several downstream tasks. In addition, we provide efficient techniques based on conventional computer vision and deep learning methods for generating heuristic mask proposals on natural image datasets. Our HARL achieves +1.3% advancement in the ImageNet semi-supervised learning benchmark and +0.9% improvement in AP50 of the COCO object detection task over the previous state-of-the-art method BYOL. Our code implementation is available for both TensorFlow and PyTorch frameworks.
Meeting of the Section of special scientific, scientific-technical and technical libraries. XXVI Annual session of the RBA conference

.
Word2Box: Capturing Set-Theoretic Semantics of Words using Box Embeddings

Learning representations of words in a continuous space is perhaps the most fundamental task in NLP, however words interact in ways much richer than vector dot product similarity can provide.Many relationships between words can be expressed settheoretically, for example adjective-noun compounds (eg."red cars"⊆"cars") and homographs (eg."tongue"∩"body" should be similar to "mouth", while "tongue"∩"language" should be similar to "dialect") have natural set-theoretic interpretations.Box embeddings are a novel region-based representation which provide the capability to perform these settheoretic operations.In this work, we provide a fuzzy-set interpretation of box embeddings, and learn box representations of words using a set-theoretic training objective.We demonstrate improved performance on various word similarity tasks, particularly on less common words, and perform a quantitative and qualitative analysis exploring the additional unique expressivity provided by WORD2BOX.
Digital intelligent virtual assistant (DIVA) with natural speech and accent recognition

A Virtual Assistant (VA) system is a computer program that recognizes designated language voice commands and executes tasks on the user&apos;s behalf. This paper aims to conduct a state-of-the-art review for Digital Intelligent Virtual Assistant (DIVA) and their extended use for diverse users. Furthermore, the paper proposes a workflow scheme which takes into account accent recognition, voice registration and gives more manual control to the users. The proposed design considers the user ease and their struggles with technology; hence the proposed system is destined to serve as a mobile application. The VA encompasses techniques such as Natural Language Processing, Speech Pattern Recognition, Machine Learning and Language Data Statistical Analysis to accomplish the aforementioned user goals. Subsequently, this research will indulge in data analysis using questionnaires filled out anonymously and the results of this survey will allow us to validate our hypothesis. The analysis results are expected to demonstrate how users view their Digital Virtual Assistants and address their concerns regarding security, accent support, control and ease of use. Finally the paper summarizes the findings, methodology and results obtained through questionnaires and highlighting the potential for future works.
A Systematic Review of Deep Knowledge Graph-Based Recommender Systems, with Focus on Explainable Embeddings

Recommender systems (RS) have been developed to make personalized suggestions and enrich users’ preferences in various online applications to address the information explosion problems. However, traditional recommender-based systems act as black boxes, not presenting the user with insights into the system logic or reasons for recommendations. Recently, generating explainable recommendations with deep knowledge graphs (DKG) has attracted significant attention. DKG is a subset of explainable artificial intelligence (XAI) that utilizes the strengths of deep learning (DL) algorithms to learn, provide high-quality predictions, and complement the weaknesses of knowledge graphs (KGs) in the explainability of recommendations. DKG-based models can provide more meaningful, insightful, and trustworthy justifications for recommended items and alleviate the information explosion problems. Although several studies have been carried out on RS, only a few papers have been published on DKG-based methodologies, and a review in this new research direction is still insufficiently explored. To fill this literature gap, this paper uses a systematic literature review framework to survey the recently published papers from 2018 to 2022 in the landscape of DKG and XAI. We analyze how the methods produced in these papers extract essential information from graph-based representations to improve recommendations’ accuracy, explainability, and reliability. From the perspective of the leveraged knowledge-graph related information and how the knowledge-graph or path embeddings are learned and integrated with the DL methods, we carefully select and classify these published works into four main categories: the Two-stage explainable learning methods, the Joint-stage explainable learning methods, the Path-embedding explainable learning methods, and the Propagation explainable learning methods. We further summarize these works according to the characteristics of the approaches and the recommendation scenarios to facilitate the ease of checking the literature. We finally conclude by discussing some open challenges left for future research in this vibrant field.
Constructing Open Cloze Tests Using Generation and Discrimination Capabilities of Transformers

This paper presents the first multi-objective transformer model for constructing open cloze tests that exploits generation and discrimination capabilities to improve performance.Our model is further enhanced by tweaking its loss function and applying a post-processing reranking algorithm that improves overall test structure.Experiments using automatic and human evaluation show that our approach can achieve up to 82% accuracy according to experts, outperforming previous work and baselines.We also release a collection of highquality open cloze tests along with sample system output and human annotations that can serve as a future benchmark.
Rényi State Entropy Maximization for Exploration Acceleration in Reinforcement Learning

One of the most critical challenges in deep reinforcement learning is to maintain the long-term exploration capability of the agent. To tackle this problem, it has been recently proposed to provide intrinsic rewards for the agent to encourage exploration. However, most existing intrinsic reward-based methods proposed in the literature fail to provide sustainable exploration incentives, a problem known as vanishing rewards. In addition, these conventional methods incur complex models and additional memory in their learning procedures, resulting in high computational complexity and low robustness. In this work, a novel intrinsic reward module based on the Rényi entropy is proposed to provide high-quality intrinsic rewards. It is shown that the proposed method actually generalizes the existing state entropy maximization methods. In particular, a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -nearest neighbor estimator is introduced for entropy estimation while a <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"><tex-math notation="LaTeX">$k$</tex-math></inline-formula> -value search method is designed to guarantee the estimation accuracy. Extensive simulation results demonstrate that the proposed Rényi entropy-based method can achieve higher performance as compared to existing schemes.
Image Inpainting Detection Based on High-Pass Filter Attention Network

Image inpainting based on deep learning has been greatly improved.The original purpose of image inpainting was to repair some broken photos, such as inpainting artifacts.However, it may also be used for malicious operations, such as destroying evidence.Therefore, detection and localization of image inpainting operations are essential.Recent research shows that high-pass filtering full convolutional network (HPFCN) is applied to image inpainting detection and achieves good results.However, those methods did not consider the spatial location and channel information of the feature map.To solve these shortcomings, we introduce the squeezed excitation blocks (SE) and propose a high-pass filter attention full convolutional network (HPACN).In feature extraction, we apply concurrent spatial and channel attention (scSE) to enhance feature extraction and obtain more information.Channel attention (cSE) is introduced in upsampling to enhance detection and localization.The experimental results show that the proposed method can achieve improvement on ImageNet.
Sentence-level Privacy for Document Embeddings

User language data can contain highly sensitive personal content.As such, it is imperative to offer users a strong and interpretable privacy guarantee when learning from their data.In this work, we propose SentDP: pure local differential privacy at the sentence level for a single user document.We propose a novel technique, DeepCandidate, that combines concepts from robust statistics and language modeling to produce high-dimensional, general-purpose ϵ-SentDP document embeddings.This guarantees that any single sentence in a document can be substituted with any other sentence while keeping the embedding ϵ-indistinguishable.Our experiments indicate that these private document embeddings are useful for downstream tasks like sentiment analysis and topic classification and even outperform baseline methods with weaker guarantees like word-level Metric DP.
Good Night at 4 pm?! Time Expressions in Different Cultures

We propose the task of culture-specific time expression grounding, i.e. mapping from expressions such as "morning" in English or "manhã" in Portuguese to specific hours in the day.We propose 3 language-agnostic methods, one of which achieves promising results on gold standard annotations that we collected for a small number of languages.We then apply this method to 27 languages and analyze the similarities across languages in the grounding of time expressions.
Students’ Learning Experience in a Mixed Reality Environment: Drivers and Barriers

Mixed reality technologies have the potential to provide students with an effective learning experience by bridging the gap between the digital and physical world. Mixed reality enables the user to see their real-world surroundings with virtual features overlayed in order to create a digitally enhanced experience. As we strategically move towards large-scale integration of mixed reality technologies to maximize the learning affordances in the future, there is a need to understand the drivers of and barriers to students’ learning in a mixed reality environment. We present in this paper a multi-site case study to evaluate the feedback from 223 students on their learning experiences after using Microsoft HoloLens, a mixed reality device, in their classroom. While we found that interactivity, immersion, and presence are the main drivers, the key barriers identified were difficulty using the device, accessibility issues, and the lack of flexibility. Affordances and fatigue are the two themes that evolved in this study. We recommend three design principles that will contribute toward designing mixed reality technologies to effectively build new embodied learning environments.
End-to-End Speech Translation for Code Switched Speech

Code switching (CS) refers to the phenomenon of interchangeably using words and phrases from different languages.CS can pose significant accuracy challenges to NLP, due to the often monolingual nature of the underlying systems.In this work, we focus on CS in the context of English/Spanish conversations for the task of speech translation (ST), generating and evaluating both transcript and translation.To evaluate model performance on this task, we create a novel ST corpus derived from existing public data sets. 1 We explore various ST architectures across two dimensions: cascaded (transcribe then translate) vs end-toend (jointly transcribe and translate) and unidirectional (source → target) vs bidirectional (source ↔ target).We show that our ST architectures, and especially our bidirectional end-to-end architecture, perform well on CS speech, even when no CS training data is used.
Challenges to Open-Domain Constituency Parsing

Neural constituency parsers have reached practical performance on news-domain benchmarks.However, their generalization ability to other domains remains weak.Existing findings on cross-domain constituency parsing are only made on a limited number of domains.Tracking this, we manually annotate a high-quality constituency treebank containing five domains.We analyze challenges to open-domain constituency parsing using a set of linguistic features on various strong constituency parsers.Primarily, we find that 1) BERT significantly increases parsers' cross-domain performance by reducing their sensitivity on the domain-variant features.2) Compared with single metrics such as unigram distribution and OOV rate, challenges to open-domain constituency parsing arise from combinations of factors, including cross-domain lexical and constituent structure variations.
Right for the Right Reason: Evidence Extraction for Trustworthy Tabular Reasoning

When pre-trained contextualized embeddingbased models developed for unstructured data are adapted for structured tabular data, they perform admirably.However, recent probing studies show that these models use spurious correlations, and often predict inference labels by focusing on false evidence or ignoring it altogether.To study this issue, we introduce the task of Trustworthy Tabular Reasoning, where a model needs to extract evidence to be used for reasoning, in addition to predicting the label.As a case study, we propose a twostage sequential prediction approach, which includes an evidence extraction and an inference stage.First, we crowdsource evidence row labels and develop several unsupervised and supervised evidence extraction strategies for INFOTABS, a tabular NLI benchmark.Our evidence extraction strategy outperforms earlier baselines.On the downstream tabular inference task, using only the automatically extracted evidence as the premise, our approach outperforms prior benchmarks.
Visualizing and Explaining Language Models

During the last decade, Natural Language Processing has become, after Computer Vision, the second field of Artificial Intelligence that was massively changed by the advent of Deep Learning. Regardless of the architecture, the language models of the day need to be able to process or generate text, as well as predict missing words, sentences or relations depending on the task. Due to their black-box nature, such models are difficult to interpret and explain to third parties. Visualization is often the bridge that language model designers use to explain their work, as the coloring of the salient words and phrases, clustering or neuron activations can be used to quickly understand the underlying models. This paper showcases the techniques used in some of the most popular Deep Learning for NLP visualizations, with a special focus on interpretability and explainability.