Reviewerly/exHarmony huggingface dataset:
Development of Theory, Scope and Tools for Entropy Signals and Data Processing

There are presented theoretical bases of entropy calculation for random messages. The analytical expressions of known entropy estimates are classified. A comparative analysis of the information characteristics for the estimates of the probability and correlation measures of entropy is performed. There is also reviewed the scope of entropy analysis, software and hardware tools for entropy calculation. The characteristics of hardware and time complexity of circuit-technical solutions of special entropy calculation processors are developed. There is proposed an extension of Shannon's entropy estimation functionalities and development of special processors for the forming and receiving of crypto-protected entropy-manipulated signals in the environment of cyber-physical systems.
Adversarial Named Entity Recognition with POS label embedding

Named Entity Recognition (NER) is dedicated to recognizing different types of named entity. Previous works have shown that part-of-speech, as an important feature, provides complementary syntactical information to NER systems. However, these studies suffer from two limitations: (i) the previous models do not consider the noise from part-of-speech; (ii) the previous models need to re-extract features from token representations. In this paper, we propose a novel approach that can alleviate the above issues as well as make full use of part-of-speech features via attention mechanism and adversarial training. We evaluate our model on three NER datasets, and the experimental results demonstrate that our model achieves a state-of-the-art F1-score of Twitter dataset while matching a state-of-the-art performance on the CoNLL-2003 and Weibo datasets.
Empirical Study of Focus-Plus-Context and Aggregation Techniques for the Visualization of Streaming Data

Analysis of streaming data often involves both real-time monitoring of incoming data as well as contextual awareness of data history. A focus-plus-context approach can support both goals, with variable levels of visual aggregation making it possible to provide a high level of detail for incoming and recent data while providing contextual information about recent history. Visual aggregation reduces data resolution in order to show the context of data over large periods of time within a limited display space. With a controlled experiment, we evaluated the effectiveness of different types of aggregation for four types of stream-analysis tasks. Overall, the results show that a focus-plus-context design has little negative impact on the ability to successfully monitor and analyze streaming data, making it possible to show longer periods of time than other approaches. However, visual aggregation can be problematic for trend recognition tasks. This research demonstrates how the effectiveness of the visualization depends on the specifics of the analysis task.
U-Net with Dense Encoder, Residual Decoder and Depth-wise Skip Connections

For tasks like medical image segmentation and understanding, U-Net is one of the most prominent convolutional neural networks (CNNs) in recent years. Most of the models for image segmentation today are the variants of the classical U-Net. By applying some improvements on the convolution blocks and skip connections in U-Net, this paper proposes a dense-residual depth-wise U-Net (DR-DW U-Net). The DR-DW U-Net aims at extracting more useful features and alleviating the pressure during gradient descent. It adopts dense blocks in the left encoding path while using residual blocks in the right decoding path. In addition, the skip connections of DR-DW U-Net are injected with additional convolution blocks in the form of depth-wise convolution. The experimental results on two medical segmentation datasets indicate that the DR-DW U-Net achieves competitive performances and notable improvements over the classical U-Net as well as some well-designed variants of U-Net.
Learning Embodied Agents with Policy Gradients to Navigate in Realistic Environments

Indoor navigation is one of the main tasks in robotic systems. Most decisions in this area rely on ideal agent coordinates and a pre-known room map. However, the high accuracy of indoor localization cannot be achieved in realistic scenarios. For example, the GPS has low accuracy in the room; odometry often gives much noise for accurate positioning, etc. In this paper, we conducted a study of the navigation problem in the realistic Habitat simulator. We proposed a method based on the neural network approach and reinforcement learning that takes into account these factors. The most promising recent approaches were DDPPO and ANM for agent control and DF-VO for localization, during the analysis of which a new approach was developed. This method takes into account the non-determinism of the robot’s actions and the noise level of data from its sensors.
Grounding of Modal Responses in Question Answering System Equipped with Hierarchical Categorisation

Intelligent and dialogue systems highly utilise natural language interfaces. Such systems do not only process linguistic questions, but also formulate proper linguistic responses. Often neglected and important aspect of such responses lies in the ability to express and communicate systems internal beliefs. The proposed model fills in the current research gap in the grounding theory by enriching empirical experiences with hierarchical semantic structures in establishing agents internal belief stance. Such an extension significantly influences the process of grounding and allows for a dedicated computational mechanism of moving the conversations focus, which is incorporated into grounding mechanisms and fully presented.
Novelty-Guided Reinforcement Learning via Encoded Behaviors

Despite the successful application of Deep Reinforcement Learning (DRL) in a wide range of complex tasks, agents either often learn sub-optimal behavior due to the sparse/deceptive nature of rewards or require a lot of interactions with the environment. Recent methods combine a class of algorithms known as Novelty Search (NS), which circumvents this problem by encouraging exploration towards novel behaviors. Even without exploiting any environment rewards, they are capable of learning skills that yield competitive results in several tasks. However, to assign novelty scores to policies, these methods rely on neighborhood models that store behaviors in an archive set. Hence they do not scale and generalize to complex tasks requiring too many policy evaluations. Addressing these challenges, we propose a function approximation paradigm to instead learn sparse representations of agent behaviors using auto-encoders, which are later used to assign novelty scores to policies. Experimental results on benchmark tasks suggest that this way of novelty-guided exploration is a viable alternative to classic novelty search methods.
Sequential Analysis with Specified Confidence Level and Adaptive Convolutional Neural Networks in Image Recognition

In this paper the problem of high computational complexity of deep convolutional nets in image recognition is considered. An existing framework of adaptive neural networks is extended by appending the separate classifier to intermediate layers. The hierarchical representations of the input image are sequentially analyzed. If the first classifier returns rather high confidence score, the inference process will be terminated. Otherwise, the inference to the next intermediate layer with attached classifier is continued until the reliable solution is obtained or the penultimate layer is reached. The thresholds for classifier scores at each layer are automatically chosen based on the Benjamini-Hochberg multiple comparisons for a specified confidence level. Experimental study for both pre-trained and fine-tuned deep convolutional neural networks demonstrates that the proposed approach reduces the running time by up to 1.7 times without significant accuracy degradation. Moreover, the larger is the training sample, the more noticeable is the gain in performance.
Task-Aware Novelty Detection for Visual-based Deep Learning in Autonomous Systems

Deep-learning driven safety-critical autonomous systems, such as self-driving cars, must be able to detect situations where its trained model is not able to make a trustworthy prediction. This ability to determine the novelty of a new input with respect to a trained model is critical for such systems because novel inputs due to changes in the environment, adversarial attacks, or even unintentional noise can potentially lead to erroneous, perhaps life-threatening decisions. This paper proposes a learning framework that leverages information learned by the prediction model in a task-aware manner to detect novel scenarios. We use network saliency to provide the learning architecture with knowledge of the input areas that are most relevant to the decision-making and learn an association between the saliency map and the predicted output to determine the novelty of the input. We demonstrate the efficacy of this method through experiments on real-world driving datasets as well as through driving scenarios in our in-house indoor driving environment where the novel image can be sampled from another similar driving dataset with similar features or from adversarial attacked images from the training dataset. We find that our method is able to systematically detect novel inputs and quantify the deviation from the target prediction through this task-aware approach.
Super Gaussian Priors for Blind Color Deconvolution of Histological Images

Color deconvolution aims at separating multi-stained images into single stained ones. In digital histopathological images, true stain color vectors vary between images and need to be estimated to obtain stain concentrations and separate stain bands. These band images can be used for image analysis purposes and, once normalized, utilized with other multi-stained images (from different laboratories and obtained using different scanners) for classification purposes. In this paper we propose the use of Super Gaussian (SG) priors for each stain concentration together with the similarity to a given reference matrix for the color vectors. Variational inference and an evidence lower bound are utilized to automatically estimate all the latent variables. The proposed methodology is tested on real images and compared to classical and state-of-the-art methods for histopathological blind image color deconvolution.
Spatiotemporal Representation Learning with GAN Trained LSTM-LSTM Networks

Learning robot behaviors in unstructured environments often requires handcrafting the features for a given task. In this paper, we present and evaluate an unsupervised representation learning architecture, Layered Spatiotemporal Memory Long Short-Term Memory (LSTM-LSTM), that learns the underlying representation without knowledge of the task. The goal of this architecture is to learn the dynamics of the environment from high-dimensional raw video inputs. Using a Generative Adversarial Network (GAN) framework with the proposed network, this architecture is able to learn a spatiotemporal representation in its lower-dimensional latent space directly from raw input sequences. We show that our approach learns the spatial and temporal information simultaneously as opposed to a two-stage learning approach of alternating between training a Convolutional Neural Network (ConvNet) and a Long Short-Term Network (LSTM). Furthermore, by using LSTM-LSTM cells that shrink in size with the increase in the number of layers, the network learns a hierarchical representation with a low-dimensional representation at the top layer. We show that this architecture achieves state-of-the-art results with a substantially lower-dimensional representation than existing methods. We evaluate our approach on a video prediction task with standard benchmark datasets like Moving MNIST and KTH Action, as well as a simulated robot dataset.
Multimodal Prediction of Breast Cancer Relapse Prior to Neoadjuvant Chemotherapy Treatment

Neoadjuvant chemotherapy (NAC) is one of the treatment options for women diagnosed with breast cancer, in which chemotherapy is administered prior to surgery. In current clinical practice, it is not possible to predict whether the patient is likely to encounter a relapse after treatment and have the breast cancer reoccur in the same place. If this outcome could be predicted prior to the start of NAC, it could inform therapeutic options. We explore the use of multimodal imaging and clinical features to predict the risk of relapse following NAC treatment. We performed a retrospective study on a cohort of 1738 patients who were administered with NAC. Of these patients, 567 patients also had magnetic resonance imaging (MRI) taken before the treatment started. We analyzed the data using deep learning and traditional machine learning algorithms to increase the set of discriminating features and create effective models. Our results demonstrate the ability to predict relapse prior to NAC treatment initiation, using each modality alone. We then show the possible improvement achieved by combining MRI and clinical data, as measured by the AUC, sensitivity, and specificity. When evaluated on holdout data, the overall combined model achieved 0.735 AUC and 0.438 specificity at a sensitivity operation point of 0.95. This means that almost every patient encountering relapse will also be correctly classified by our model, enabling the reassessment of this treatment prior to its start. Additionally, the same model was able to correctly predict in advance 44% of the patients that would not encounter relapse.
How to Build Optimally Secure PRFs Using Block Ciphers

In EUROCRYPT '96, Aiello and Venkatesan proposed two candidates for 2n-bit to 2n-bit pseudorandom functions (PRFs), called Benes and modified Benes (or mBenes), based on n-bit to n-bit PRFs. While Benes is known to be secure up to $$ 2^n $$ queries (Patarin, AFRICACRYPT '08), the security of mBenes has only been proved up to $$ 2^{n(1-\epsilon )} $$ queries for all $$ \epsilon > 0 $$ by Patarin and Montreuil in ICISC '05. In this work, we show that the composition of a 2n-bit hash function with mBenes is a secure variable input length (VIL) PRF up to $$ 2^{n-2} $$ queries (given appropriate hash function bounds). We extend our analysis with block ciphers as the underlying primitive and obtain two optimally secure VIL PRFs using block ciphers. The first of these candidates requires 6 calls to the block cipher. The second candidate requires just 4 calls to the block cipher, but here the proof is based on Patarin's mirror theory. Further, we instantiate the hash function with a PMAC+/LightMAC+ like hash, to get six candidates for deterministic message authentication codes with optimal security.
Fast Switch Naïve Bayes to Avoid Redundant Update for Concept Drift Learning

In data stream mining, concept drift may cause the predictions given by machine learning models become less accurate as time passes. Existing concept drift detection and adaptation methods are built based on a framework that is buffering new samples if a drift-warming level is triggered and retraining a new model if a drift-alarm level is triggered. However, these methods neglected the problem that the performance of a learning model could be more sensitive to the amount of training data rather than the concept drift. In other words, a retrained model built on very few data instances could be even worse than the old model trained before the drift. To elaborate and address this problem, we propose a fast switch Naïve Bayes model (fsNB) for concept drift detection and adaptation. The intuition is to apply the idea of following the leader in online learning. We manipulate a sliding and an incremental Naïve Bayes classifier, if the sliding one overwhelms the incremental one, the model reports a drift. The experimental evaluation shows the advantages of fsNB and demonstrates that retraining may not be the best options for a marginal drift.
Meta-Path Generation Online for Heterogeneous Network Embedding

Graph neural networks (GNNs), powerful deep representation learning methods for graph data, have been widely used in various tasks, such as recommendation systems and link prediction. Most existing GNNs are designed to learn node embeddings on homogeneous graphs. Heterogeneous information network (HIN) with various types of nodes and edges still faces great challenges for the heterogeneity and rich semantic information. To make full use of the heterogeneous information, many works try to manually design meta-paths, which are paths connected with two objects. They utilize meta-paths to capture more semantic information in heterogeneous graphs. However, manually designed meta-paths require domain knowledge and meta-path-based heterogeneous graph embedding methods only utilize the information of nodes with the same type, ignoring the impacts of the different types of nodes. We propose meta-path generation online for heterogeneous network embedding for all types of nodes, which can generate meta-paths and learn node embeddings simultaneously. Firstly, we exhaust all meta-paths within k-hop for specific nodes and apply a meta-path guided nodes aggregation. Secondly, we adopt an attention mechanism to select Top-N meta-paths with the largest attention coefficients for the semantic aggregation. The above two stages constitute one layer of our approach. Through stacking multi-layers, we can generate longer and more complex meta-paths. Without domain-specific preprocessing, extensive experiments on two datasets demonstrate that our proposed approach achieves better performance compared with other recent methods that require predefined meta-paths from domain knowledge.
Recognizing English Cursive Using Generative Adversarial Networks

English cursive is present in many historical documents, and there is a need to recognize such writing and convert them to electronic documents. However, automated English cursive recognition is still considered difficult, mainly because of the segmentation difficulty presented by connected characters in a word. To resolve this issue, we propose a method to recognize whole words using GAN (Generative Adversarial Networks). This is done by using trained GAN to generate a recognizable font from cursive words, and then using conventional OCR (Optical Character Recognition) method on this font. The performance of this method is evaluated using English cursive datasets.
PolSentiLex: Sentiment Detection in Socio-Political Discussions on Russian Social Media

We present a freely available Russian language sentiment lexicon PolSentiLex designed to detect sentiment in user-generated content related to social and political issues. The lexicon was generated from a database of posts and comments of the top 2,000 LiveJournal bloggers posted during one year (\(\sim \)1.5 million posts and 20 million comments). Following a topic modeling approach, we extracted 85,898 documents that were used to retrieve domain-specific terms. This term list was then merged with several external sources. Together, they formed a lexicon (16,399 units) marked-up using a crowdsourcing strategy. A sample of Russian native speakers (n = 105) was asked to assess words’ sentiment given the context of their use (randomly paired) as well as the prevailing sentiment of the respective texts. In total, we received 59,208 complete annotations for both texts and words. Several versions of the marked-up lexicon were experimented with, and the final version was tested for quality against the only other freely available Russian language lexicon and against three machine learning algorithms. All experiments were run on two different collections. They have shown that, in terms of \(\text {F}_{\text {macro}}\), lexicon-based approaches outperform machine learning by 11%, and our lexicon outperforms the alternative one by 11% on the first collection, and by 7% on the negative scale of the second collection while showing similar quality on the positive scale and being three times smaller. Our lexicon also outperforms or is similar to the best existing sentiment analysis results for other types of Russian-language texts .
DHT-PDP: A Distributed Hash Table based Provable Data Possession Mechanism in Cloud Storage

The popularity of cloud storage among data users is due to easy maintenance, and no initial infrastructure setup cost as compared to local storage. However, although the data users outsource their data to cloud storage (a third party) still, they concern about their physical data. To check whether the data stored in the cloud storage has been modified or not, public auditing of the data is required before its utilization. To audit over vast outsourced data, the availability of the auditor is an essential requirement as nowadays, data owners are using mobile devices. But unfortunately, a single auditor leads to a single point of failure and inefficient to preserve the security and correctness of outsourced data. So, we introduce a distributed public auditing scheme which is based on peer-to-peer (P2P) architecture. In this work, the auditors are organized using a distributed hash table (DHT) mechanism and audit the outsourced data with the help of a published hashed key of the data. The computation and communication overhead of our proposed scheme is compared with the existing schemes, and it found to be an effective solution for public auditing on outsourced data with no single point of failure.
The Development of an Information System for Technical University Students’ Teaching and Control in English

The main idea of the chosen topic is that the development of an information system will allow to unify the methodological and technological approach to the learning process, simplify and facilitate the educational process of students, increase their number, thereby increasing the intensity of education and student interest, as well as reduce the burden on teachers of this discipline. The basic definitions such as distance educational technologies, e-learning, distance learning are considered in the article, their advantages are determined. The paper presents the development and creation of an information system for teaching and controlling students in English in the educational program "Information science and computer engineering", based on the modern technologies for developing web applications, the purpose and main problems are defined. To build an information system, we selected a network client — server model that has two-tier architecture. Charts of the use variants have been developed regarding the roles of Administrator-Teacher and User-Student. A.s a result of the subject area analysis. The main functions of the application and the navigation map, which are designed to display a holistic script for a web application, which allows you to see the general logic of the application and understand its basic structure are given. The paper describes the results of work with this information system in details.