Reviewerly/exHarmony huggingface dataset:
Reducing network size and improving prediction stability of reservoir computing

Reservoir computing is a very promising approach for the prediction of complex nonlinear dynamical systems. Besides capturing the exact short-term trajectories of nonlinear systems, it has also proved to reproduce its characteristic long-term properties very accurately. However, predictions do not always work equivalently well. It has been shown that both short- and long-term predictions vary significantly among different random realizations of the reservoir. In order to gain an understanding on when reservoir computing works best, we investigate some differential properties of the respective realization of the reservoir in a systematic way. We find that removing nodes that correspond to the largest weights in the output regression matrix reduces outliers and improves overall prediction quality. Moreover, this allows to effectively reduce the network size and, therefore, increase computational efficiency. In addition, we use a nonlinear scaling factor in the hyperbolic tangent of the activation function. This adjusts the response of the activation function to the range of values of the input variables of the nodes. As a consequence, this reduces the number of outliers significantly and increases both the short- and long-term prediction quality for the nonlinear systems investigated in this study. Our results demonstrate that a large optimization potential lies in the systematical refinement of the differential reservoir properties for a given dataset.
AutoDNNchip: An Automated DNN Chip Predictor and Builder for Both FPGAs and ASICs

Recent breakthroughs in Deep Neural Networks (DNNs) have fueled a growing demand for domain-specific hardware accelerators (i.e., DNN chips). However, designing DNN chips is non-trivial because: (1) mainstream DNNs have millions of parameters and billions of operations; (2) the design space is large due to numerous design choices of dataflows, processing elements, memory hierarchy, etc.; and (3) there is an algorithm/hardware co-design need for the same DNN functionality to have a different decomposition that would require different hardware IPs and thus correspond to dramatically different performance/energy/area tradeoffs. Therefore, DNN chips often take months to years to design and require a large team of cross-disciplinary experts. To enable fast and effective DNN chip design, we propose AutoDNNchip - a DNN chip generator that can automatically produce both FPGA- and ASIC-based DNN chip implementation (i.e., synthesizable RTL code with optimized algorithm-to-hardware mapping) from DNNs developed by machine learning frameworks (e.g., PyTorch) for a designated application and dataset without humans in the loop. Specifically, AutoDNNchip consists of 2 integrated enablers: (1) a Chip Predictor, which can accurately and efficiently predict a DNN accelerator's energy, throughput, latency, and area based on the DNN model parameters, hardware configurations, technology-based IPs, and platform constraints; and (2) a Chip Builder, which can automatically explore the design space of DNN chips (including IP selections, block configurations, resource balancing, etc.), optimize chip designs via the Chip Predictor, and then generate synthesizable RTL code with optimized dataflows to achieve the target design metrics. Experimental results show that our Chip Predictor's predicted performance differs from real-measured ones by
Multi-turn Response Selection using Dialogue Dependency Relations

Multi-turn response selection is a task designed for developing dialogue agents.The performance on this task has a remarkable improvement with pre-trained language models.However, these models simply concatenate the turns in dialogue history as the input and largely ignore the dependencies between the turns.In this paper, we propose a dialogue extraction algorithm to transform a dialogue history into threads based on their dependency relations.Each thread can be regarded as a self-contained sub-dialogue.We also propose Thread-Encoder model to encode threads and candidates into compact representations by pre-trained Transformers and finally get the matching score through an attention layer.The experiments show that dependency relations are helpful for dialogue context understanding, and our model outperforms the state-of-the-art baselines on both DSTC7 and DSTC8*, with competitive results on UbuntuV2.
HSCNN: A Hybrid-Siamese Convolutional Neural Network for Extremely Imbalanced Multi-label Text Classification

The data imbalance problem is a crucial issue for the multi-label text classification.Some existing works tackle it by proposing imbalanced loss objectives instead of the vanilla cross-entropy loss, but their performances remain limited in the cases of extremely imbalanced data.We propose a hybrid solution which adapts general networks for the head categories, and few-shot techniques for the tail categories.We propose a Hybrid-Siamese Convolutional Neural Network (HSCNN) with additional technical attributes, i.e., a multi-task architecture based on Single and Siamese networks; a category-specific similarity in the Siamese structure; a specific sampling method for training HSCNN.The results using two benchmark datasets and three loss objectives show that our method can improve the performance of Single networks with diverse loss objectives on the tail or entire categories.
Linguistically-Informed Transformations (LIT): A Method for Automatically Generating Contrast Sets

Although large-scale pretrained language models, such as BERT and RoBERTa, have achieved superhuman performance on indistribution test sets, their performance suffers on out-of-distribution test sets (e.g., on contrast sets).Building contrast sets often requires human-expert annotation, which is expensive and hard to create on a large scale.In this work, we propose a Linguistically-Informed Transformation (LIT) method to automatically generate contrast sets, which enables practitioners to explore linguistic phenomena of interests as well as compose different phenomena.Experimenting with our method on SNLI and MNLI shows that current pretrained language models, although being claimed to contain sufficient linguistic knowledge, struggle on our automatically generated contrast sets.Furthermore, we improve models' performance on the contrast sets by applying LIT to augment the training data, without affecting performance on the original data.
De-Identification of Facial Features in Magnetic Resonance Images: Software Development Using Deep Learning Technology

Background High-resolution medical images that include facial regions can be used to recognize the subject’s face when reconstructing 3-dimensional (3D)-rendered images from 2-dimensional (2D) sequential images, which might constitute a risk of infringement of personal information when sharing data. According to the Health Insurance Portability and Accountability Act (HIPAA) privacy rules, full-face photographic images and any comparable image are direct identifiers and considered as protected health information. Moreover, the General Data Protection Regulation (GDPR) categorizes facial images as biometric data and stipulates that special restrictions should be placed on the processing of biometric data. Objective This study aimed to develop software that can remove the header information from Digital Imaging and Communications in Medicine (DICOM) format files and facial features (eyes, nose, and ears) at the 2D sliced-image level to anonymize personal information in medical images. Methods A total of 240 cranial magnetic resonance (MR) images were used to train the deep learning model (144, 48, and 48 for the training, validation, and test sets, respectively, from the Alzheimer's Disease Neuroimaging Initiative [ADNI] database). To overcome the small sample size problem, we used a data augmentation technique to create 576 images per epoch. We used attention-gated U-net for the basic structure of our deep learning model. To validate the performance of the software, we adapted an external test set comprising 100 cranial MR images from the Open Access Series of Imaging Studies (OASIS) database. Results The facial features (eyes, nose, and ears) were successfully detected and anonymized in both test sets (48 from ADNI and 100 from OASIS). Each result was manually validated in both the 2D image plane and the 3D-rendered images. Furthermore, the ADNI test set was verified using Microsoft Azure's face recognition artificial intelligence service. By adding a user interface, we developed and distributed (via GitHub) software named “Deface program” for medical images as an open-source project. Conclusions We developed deep learning–based software for the anonymization of MR images that distorts the eyes, nose, and ears to prevent facial identification of the subject in reconstructed 3D images. It could be used to share medical big data for secondary research while making both data providers and recipients compliant with the relevant privacy regulations.
Incremental Text-to-Speech Synthesis with Prefix-to-Prefix Framework

Text-to-speech synthesis (TTS) has witnessed rapid progress in recent years, where neural methods became capable of producing audios with high naturalness.However, these efforts still suffer from two types of latencies: (a) the computational latency (synthesizing time), which grows linearly with the sentence length, and (b) the input latency in scenarios where the input text is incrementally available (such as in simultaneous translation, dialog generation, and assistive technologies).To reduce these latencies, we propose a neural incremental TTS approach using the prefix-to-prefix framework from simultaneous translation.We synthesize speech in an online fashion, playing a segment of audio while generating the next, resulting in an O(1) rather than O(n) latency.Experiments on English and Chinese TTS show that our approach achieves similar speech naturalness compared to full sentence TTS, but only with a constant (1-2 words) latency.* M. M. and B. Z. contributed equally; M. M. co-directed the project (with L. H.), and was responsible for the majority of ideas and implementations; B. Z. improved the speech quality significantly and implemented the ideas on different TTS systems.
RNNs can generate bounded hierarchical languages with optimal memory

Recurrent neural networks empirically generate natural language with high syntactic fidelity.However, their success is not wellunderstood theoretically.We provide theoretical insight into this success, proving in a finiteprecision setting that RNNs can efficiently generate bounded hierarchical languages that reflect the scaffolding of natural language syntax.We introduce Dyck-(k,m), the language of well-nested brackets (of k types) and mbounded nesting depth, reflecting the bounded memory needs and long-distance dependencies of natural language syntax.The best known results use O(k m 2 ) memory (hidden units) to generate these languages.We prove that an RNN with O(m log k) hidden units suffices, an exponential reduction in memory, by an explicit construction.Finally, we show that no algorithm, even with unbounded computation, can suffice with o(m log k) hidden units.
RDF 1.1: Knowledge Representation and Data Integration Language for the Web

Resource Description Framework (RDF) can seen as a solution in today’s landscape of knowledge representation research. An RDF language has symmetrical features because subjects and objects in triples can be interchangeably used. Moreover, the regularity and symmetry of the RDF language allow knowledge representation that is easily processed by machines, and because its structure is similar to natural languages, it is reasonably readable for people. RDF provides some useful features for generalized knowledge representation. Its distributed nature, due to its identifier grounding in IRIs, naturally scales to the size of the Web. However, its use is often hidden from view and is, therefore, one of the less well-known of the knowledge representation frameworks. Therefore, we summarise RDF v1.0 and v1.1 to broaden its audience within the knowledge representation community. This article reviews current approaches, tools, and applications for mapping from relational databases to RDF and from XML to RDF. We discuss RDF serializations, including formats with support for multiple graphs and we analyze RDF compression proposals. Finally, we present a summarized formal definition of RDF 1.1 that provides additional insights into the modeling of reification, blank nodes, and entailments.
Deterministic generation of a four-component optical cat state

The four-component cat state represents a particularly useful quantum state for realizing fault-tolerant continuous variable quantum computing. While such encoding has been experimentally generated and employed in the microwave regime, the states have not yet been produced in the optical regime. Here we propose a simple linear optical circuit combined with photon counters for the generation of such optical four-component cat states. This work might pave the way for the first experimental generation of fault-tolerant optical continuous variable quantum codes.
Minimax Estimation of Divergences Between Discrete Distributions

We study the minimax estimation of α-divergences between discrete distributions for integer α ≥ 1, which include the Kullback-Leibler divergence and the χ <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> -divergences as special examples. Dropping the usual theoretical tricks to acquire independence, we construct the first minimax rate-optimal estimator which does not require any Poissonization, sample splitting, or explicit construction of approximating polynomials. The estimator uses a hybrid approach which solves a problemindependent linear program based on moment matching in the non-smooth regime, and applies a problem-dependent biascorrected plug-in estimator in the smooth regime, with a soft decision boundary between these regimes.
Learning Strategies at a Higher Taxonomic Level in Primary Education Students in the Digital Age

The education system must respond to the demands and needs of today’s society. To meet this demand, teachers must identify the degree of cognitive skills that our students possess through digital learning strategies that encourage the development of these superior skills because of the impact of technological resources on student motivation. To carry out this assessment, two types of strategy tests were applied to evaluate cognitive skills, one determining the student’s ability to create an opinion of their own based on ideas and visions of other authors (perspective analysis) and the other strategy test to distinguish the main elements of an information, identifying a general pattern (abstraction), which were reviewed with a task performance instrument (rubrics). A mixed methodology was used through a case study, with a selected and intentional sample of 34 students of the Primary Education Grade. The results show that elaborating questions that go deeper into a topic and labelling contents is not an easy task and that abstraction is one of the most complex skills to develop in students. It should be noted that, in practically all areas, men’s averages exceed those of women. Finally, the use of digital strategies must be incorporated as permanent routines and the relationship between technology and pedagogy allows for the management of learning and therefore increases higher level cognitive abilities.
Modeling Large Sparse Data for Feature Selection: Hospital Admission Predictions of the Dementia Patients Using Primary Care Electronic Health Records

A growing elderly population suffering from incurable, chronic conditions such as dementia present a continual strain on medical services due to mental impairment paired with high comorbidity resulting in increased hospitalization risk. The identification of at risk individuals allows for preventative measures to alleviate said strain. Electronic health records provide opportunity for big data analysis to address such applications. Such data however, provides a challenging problem space for traditional statistics and machine learning due to high dimensionality and sparse data elements. This article proposes a novel machine learning methodology: entropy regularization with ensemble deep neural networks (ECNN), which simultaneously provides high predictive performance of hospitalization of patients with dementia whilst enabling an interpretable heuristic analysis of the model architecture, able to identify individual features of importance within a large feature domain space. Experimental results on health records containing 54,647 features were able to identify 10 event indicators within a patient timeline: a collection of diagnostic events, medication prescriptions and procedural events, the highest ranked being essential hypertension. The resulting subset was still able to provide a highly competitive hospitalization prediction (Accuracy: 0.759) as compared to the full feature domain (Accuracy: 0.755) or traditional feature selection techniques (Accuracy: 0.737), a significant reduction in feature size. The discovery and heuristic evidence of correlation provide evidence for further clinical study of said medical events as potential novel indicators. There also remains great potential for adaption of ECNN within other medical big data domains as a data mining tool for novel risk factor identification.
Constellation Loss: Improving the Efficiency of Deep Metric Learning Loss Functions for the Optimal Embedding of histopathological images

Deep learning diagnostic algorithms are proving comparable results with human experts in a wide variety of tasks, and they still require a huge amount of well-annotated data for training, which is often non affordable. Metric learning techniques have allowed a reduction in the required annotated data allowing few-shot learning over deep learning architectures.In this work, we analyze the state-of-the-art loss functions such as triplet loss, contrastive loss, and multi-class N-pair loss for the visual embedding extraction of hematoxylin and eosin (H&E) microscopy images and we propose a novel constellation loss function that takes advantage of the visual distances of the embeddings of the negative samples and thus, performing a regularization that increases the quality of the extracted embeddings.To this end, we employed the public H&E imaging dataset from the University Medical Center Mannheim (Germany) that contains tissue samples from low-grade and high-grade primary tumors of digitalized colorectal cancer tissue slides. These samples are divided into eight different textures (1. tumour epithelium, 2. simple stroma, 3. complex stroma, 4. immune cells, 5. debris and mucus, 6. mucosal glands, 7. adipose tissue and 8. background,). The dataset was divided randomly into train and test splits and the training split was used to train a classifier to distinguish among the different textures with just 20 training images. The process was repeated 10 times for each loss function. Performance was compared both for cluster compactness and for classification accuracy on separating the aforementioned textures.Our results show that the proposed loss function outperforms the other methods by obtaining more compact clusters (Davis-Boulding: 1.41 ± 0.08, Silhouette: 0.37 ± 0.02) and better classification capabilities (accuracy: 85.0 ± 0.6) over H and E microscopy images. We demonstrate that the proposed constellation loss can be successfully used in the medical domain in situations of data scarcity.
Quantum simulations employing connected moments expansions

Further advancement of quantum computing (QC) is contingent on enabling many-body models that avoid deep circuits and excessive use of CNOT gates. To this end, we develop a QC approach employing finite-order connected moment expansions (CMX) and affordable procedures for initial state preparation. We demonstrate the performance of our approach employing several quantum variants of CMX through the classical emulations on the H2 molecule potential energy surface and the Anderson model with a broad range of correlation strength. The results show that our approach is robust and flexible. Good agreement with exact solutions can be maintained even at the dissociation and strong correlation limits.
The effect of artificial intelligence technologies on audit evidence

Technologies of Artificial Intelligence (AI) are critical for future of the auditing profession. These technologies are actually vital tools that provide the auditing professionals with the means necessary for increasing the effectiveness and efficiency of their jobs. The aim of this study was to examine the effect of artificial intelligence technologies on audit evidence, from the point of view of certified auditors in information technology (IT) companies in Jordan. Descriptive research design was adopted in the study among 314 auditors. Structured questionnaire was used to obtain the information needed for the study. The findings of the study showed that expert system had a significant effect on the audit evidence. Neural network technology did not provide any significant effect on the audit evidence. The study recommended increased interest in artificial intelligence technologies by audit offices operating in Jordan because of its scientific importance in improving the collection of audit evidence.
Nanomagnetic Self-Organizing Logic Gates

The end of Moore's law for CMOS technology has prompted the search for low-power computing alternatives, resulting in promising approaches such as nanomagnetic logic. However, nanomagnetic logic is unable to solve a class of interesting problems efficiently, as it only allows for forward computing, due to the need for clocking and/or thermal annealing. Here, we introduce nanomagnetic self-organizing logic gates that can dynamically satisfy their logical proposition, irrespective of whether the signal is applied to the traditional input or output terminals, thus allowing for reversible computing. We present a design of a self-organizing nand gate, the logically correct states of which are occupied equally in thermodynamical equilibrium, and illustrate its capabilities by implementing reversible Boolean circuitry to solve a two-bit factorization problem via numerical modeling. Our approach offers an alternative path to explore memcomputing, an unconventional computing paradigm the usefulness of which has already been demonstrated by solving a variety of hard combinatorial optimization problems.
Imagechain—Application of Blockchain Technology for Images

Imagechain is a cryptographic structure that chain digital images with hash links. The most important feature, which differentiates it from blockchain, is that the pictures are not stored inside the blocks. Instead, the block and the image are combined together in the embedding process. Therefore, the imagechain is built from standard graphic files that may be used in the same way as any other image, but additionally, each of them contains a data block that links it to a previous element of the chain. The presented solution does not require any additional files except the images themselves. It supports multiple file formats and embedding methods, which makes it portable and user-friendly. At the same time, the scheme provides a high level of security and resistance to forgery. This is achieved by hashing the whole file with embedded data, so the image cannot be altered or removed from the chain without losing integrity. This article describes the basic concept of an imagechain together with building blocks and applications. The two most important issues are embedding methods and block structure.