Reviewerly/exHarmony huggingface dataset:
Training Convolutional ReLU Neural Networks in Polynomial Time: Exact Convex Optimization Formulations

We study training of Convolutional Neural Networks (CNNs) with ReLU activations and introduce exact convex optimization formulations with a polynomial complexity with respect to the number of data samples, the number of neurons and data dimension. Particularly, we develop a convex analytic framework utilizing semi-infinite duality to obtain equivalent convex optimization problems for several CNN architectures. We first prove that two-layer CNNs can be globally optimized via an $\ell_2$ norm regularized convex program. We then show that certain three-layer CNN training problems are equivalent to an $\ell_1$ regularized convex program. We also extend these results to multi-layer CNN architectures. Furthermore, we present extensions of our approach to different pooling methods.
An empirical investigation of neural methods for content scoring of science explanations

With the widespread adoption of the Next Generation Science Standards (NGSS), science teachers and online learning environments face the challenge of evaluating students' integration of different dimensions of science learning.Recent advances in representation learning in natural language processing have proven effective across many natural language processing tasks, but a rigorous evaluation of the relative merits of these methods for scoring complex constructed response formative assessments has not previously been carried out.We present a detailed empirical investigation of feature-based, recurrent neural network, and pre-trained transformer models on scoring content in real-world formative assessment data.We demonstrate that recent neural methods can rival or exceed the performance of feature-based methods.We also provide evidence that different classes of neural models take advantage of different learning cues, and pre-trained transformer models may be more robust to spurious, dataset-specific learning cues, better reflecting scoring rubrics.
Improving Reliability of Quantum True Random Number Generator using Machine Learning

Quantum computer (QC) can be used as a true random number generator (TRNG). However, various noise sources introduce a bias in the generated number which affects the randomness. In this work, we analyze the impact of noise sources e.g., gate error, decoherence, and readout error in QC-based TRNG by running a set of error calibration and quantum tomography experiments. We employ a hybrid quantum-classical gate parameter optimization routine to find an optimal gate parameter. The optimal parameter compensates for error-induced bias and improves the quality of the random number by exploiting even the worst quality qubits. However, searching the optimal parameter in a hybrid setup requires time-consuming iterations between classical and quantum machines. We propose a machine learning model to predict optimal quantum gate parameters based on the qubit error specifications. We validate our approach using experimental results from IBM's publicly accessible quantum computers and the NIST statistical test suite. The proposed method can correct bias in any worst-case qubit by up to 88.57% in real quantum hardware.
An Experimental Investigation of Calibration Techniques for Imbalanced Data

Calibration is a technique used to obtain accurate probability estimation for classification problems in real applications. Class imbalance can create considerable challenges in obtaining accurate probabilities for calibration methods. However, previous research has paid little attention to this issue. In this paper, we present an experimental investigation of some prevailing calibration methods in different imbalance scenarios. Several performance metrics are considered to evaluate different aspects of calibration performance. The experimental results show that the performance of different calibration techniques depends on the metrics and the degree of the imbalance ratio. Isotonic Regression has better overall performance on imbalanced datasets than parametric and other complex non-parametric methods. However, it performs unstably in highly imbalanced scenarios. This study provides some insights into calibration methods on imbalanced datasets, and it can be a reference for the future development of calibration methods in class imbalance scenarios.
Neural Network Robustness Verification on GPUs.

Certifying the robustness of neural networks against adversarial attacks is critical to their reliable adoption in real-world systems including autonomous driving and medical diagnosis. Unfortunately, state-of-the-art verifiers either do not scale to larger networks or are too imprecise to prove robustness, which limits their practical adoption. In this work, we introduce GPUPoly, a scalable verifier that can prove the robustness of significantly larger deep neural networks than possible with prior work. The key insight behind GPUPoly is the design of custom, sound polyhedra algorithms for neural network verification on a GPU. Our algorithms leverage the available GPU parallelism and the inherent sparsity of the underlying neural network verification task. GPUPoly scales to very large networks: for example, it can prove the robustness of a 1M neuron, 34-layer deep residual network in about 1 minute. We believe GPUPoly is a promising step towards the practical verification of large real-world networks.
TZmCFI: RTOS-Aware Control-Flow Integrity Using TrustZone for Armv8-M

Abstract Control-Flow Integrity (CFI) is a class of defensive techniques against control-flow attacks such as Return-Oriented Programming. We propose a light-weight CFI scheme for RTOS-based applications, TZmCFI, which utilizes TrustZone for Armv8-M, a hardware-assisted security feature for embedded systems with tight resource constraints. TZmCFI embodies several existing CFI techniques to provide a comprehensive protection. The traditional shadow stack technique is used to ensure stack integrity and validate function returns. To protect exception handlers, TZmCFI extends shadow exception stacks , which are a variant of the traditional shadow stack technique we proposed in our previous work, for RTOS integration and performance improvement. We conducted an experiment on Arm Versatile Express Cortex-M Prototyping System (V2M-MPS2+) to evaluate the run-time overhead of the proposed system.
Author Correction: Dynamic reconfiguration of functional brain networks during working memory training

An amendment to this paper has been published and can be accessed via a link at the top of the paper.
A Corrective View of Neural Networks: Representation, Memorization and Learning

We develop a \emph{corrective mechanism} for neural network approximation: the total available non-linear units are divided into multiple groups and the first group approximates the function under consideration, the second approximates the error in approximation produced by the first group and corrects it, the third group approximates the error produced by the first and second groups together and so on. This technique yields several new representation and learning results for neural networks:\n\n1. Two-layer neural networks in the random features regime (RF) can memorize arbitrary labels for $n$ arbitrary points in $\mathbb{R}^d$ with $\tilde{O}(\tfrac{n}{\theta^4})$ ReLUs, where $\theta$ is the minimum distance between two different points. This bound can be shown to be optimal in $n$ up to logarithmic factors.\n\n2. Two-layer neural networks with ReLUs and smoothed ReLUs can represent functions with an error of at most $\epsilon$ with $O(C(a,d)\epsilon^{-1/(a+1)})$ units for $a \in \mathbb{N}\cup\{0\}$ when the function has $\Theta(ad)$ bounded derivatives. In certain cases $d$ can be replaced with effective dimension $q \ll d$. Our results indicate that neural networks with only a single nonlinear layer are surprisingly powerful with regards to representation, and show that in contrast to what is suggested in recent work, depth is not needed in order to represent highly smooth functions.\n\n3. Gradient Descent on the recombination weights of a two-layer random features network with ReLUs and smoothed ReLUs can learn low degree polynomials up to squared error $\epsilon$ with $\mathrm{subpoly}(1/\epsilon)$ units. Even though deep networks can approximate these polynomials with $\mathrm{polylog}(1/\epsilon)$ units, existing \emph{learning} bounds for this problem require $\mathrm{poly}(1/\epsilon)$ units. To the best of our knowledge, our results give the first sub-polynomial learning guarantees for this problem.
How teachable agents influence students’ responses to critical constructive feedback

Does a teachable agent influence the uptake or neglect of ‘critical constructive feedback’ and learning within a digital environment? 285 middle-school students engaged with a history learning game in a 2x2 study design. One dimension was inclusion of a teachable agent. Orthogonal was whether critical constructive feedback was presented automatically or only when students chose. Analyses showed that a teachable agent positively affected students’ responses to feedback and mitigated feedback neglect; the results were especially strong for lower-achieving students. Additionally, presence of a teachable agent improved post-test performance for students overall, and this effect was mediated by lower feedback neglect.
Inverse Reinforcement Learning with Natural Language Goals

Humans generally use natural language to communicate task requirements to each other. Ideally, natural language should also be usable for communicating goals to autonomous machines (e.g., robots) to minimize friction in task specification. However, understanding and mapping natural language goals to sequences of states and actions is challenging. Specifically, existing work along these lines has encountered difficulty in generalizing learned policies to new natural language goals and environments. In this paper, we propose a novel adversarial inverse reinforcement learning algorithm to learn a language-conditioned policy and reward function. To improve generalization of the learned policy and reward function, we use a variational goal generator to relabel trajectories and sample diverse goals during training. Our algorithm outperforms multiple baselines by a large margin on a vision-based natural language instruction following dataset (Room-2-Room), demonstrating a promising advance in enabling the use of natural language instructions in specifying agent goals.
High-Capacity Information Hiding Based on Residual Network

To solve the problem of the low capacity of traditional information hiding and steganography, the deep neural network is used to construct a hidden network and a decoding network to hide one image into another. In this paper, the ResNet (Residual Network) has a good effect in feature extraction, and the residual block is used in the hidden network. First, the hidden network encoded the secret image into cover image. During the encoding process, the steganography image mimics the distribution of the cover image as much as possible. Then, the receiver sends the steganography image to the decoding network to obtain the secret image. Experimental results indicate that the scheme will solve the obvious problems of visual cues, also improve the ability of embedding.
Applying AI in Practice: Key Challenges and Lessons Learned

The main challenges along with lessons learned from ongoing research in the application of machine learning systems in practice are discussed, taking into account aspects of theoretical foundations, systems engineering, and human-centered AI postulates. The analysis outlines a fundamental theory-practice gap which superimposes the challenges of AI system engineering at the level of data quality assurance, model building, software engineering and deployment.
Laser Rangefinder and Monocular Camera Data Fusion for Human-Following Algorithm by PMB-2 Mobile Robot in Simulated Gazebo Environment

The paper presents a human-following algorithm for an autonomous mobile robot, which is equipped with a 2D laser rangefinder (LRF) and a monocular camera. As a rule, quality of a human tracking by a LRF is reduced in cluttered environments. We used a monocular camera to increase a human-tracking reliability. In contradiction with popular human-tracking algorithms that apply only a 2D LRF, our algorithm does not impose any restrictions on a type of human’s clothes, and our approach does not require a human head and an upper body to be located within a monocular camera field of view. Several human trackers and variations of our algorithm were compared in the Gazebo virtual experiments within a free corridor and an office room environment. The virtual experiments demonstrated that our method successfully improved a human-tracking quality being employed with the human-following virtual PMB-2 robot.
Bayesian Cycle-Consistent Generative Adversarial Networks via Marginalizing Latent Sampling

Recent techniques built on generative adversarial networks (GANs), such as cycle-consistent GANs, are able to learn mappings among different domains built from unpaired data sets, through min–max optimization games between generators and discriminators. However, it remains challenging to stabilize the training process and thus cyclic models fall into mode collapse accompanied by the success of discriminator. To address this problem, we propose an novel Bayesian cyclic model and an integrated cyclic framework for interdomain mappings. The proposed method motivated by Bayesian GAN explores the full posteriors of cyclic model via sampling latent variables and optimizes the model with maximum <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a posteriori</i> (MAP) estimation. Hence, we name it Bayesian CycleGAN. In addition, original CycleGAN cannot generate diversified results. But it is feasible for Bayesian framework to diversify generated images by replacing restricted latent variables in inference process. We evaluate the proposed Bayesian CycleGAN on multiple benchmark data sets, including Cityscapes, Maps, and Monet2photo. The proposed method improve the per-pixel accuracy by 15% for the Cityscapes semantic segmentation task within origin framework and improve 20% within the proposed integrated framework, showing better resilience to imbalance confrontation. The diversified results of Monet2Photo style transfer also demonstrate its superiority over original cyclic model. We provide codes for all of our experiments in <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/ranery/Bayesian-CycleGAN</uri> .
Convolutional Neural Network Architecture for Recovering Watermark Synchronization

In this paper, we propose a convolutional neural network-based template architecture that compensates for the disadvantages of existing watermarking techniques that are vulnerable to geometric distortion. The proposed template consists of a template generation network, a template extraction network, and a template matching network. The template generation network generates a template in the form of noise and the template is inserted into certain pre-defined spatial locations of the image. The extraction network detects spatial locations where the template is inserted in the image. Finally, the template matching network estimates the parameters of the geometric distortion by comparing the shape of spatial locations where the template was inserted with the locations where the template was detected. It is possible to recover an image in its original geometrical form using the estimated parameters, and as a result, watermarks applied using existing watermarking techniques that are vulnerable to geometric distortion can be decoded normally.
On the Security Margin of TinyJAMBU with Refined Differential and Linear Cryptanalysis

This paper presents the first third-party security analysis of TinyJAMBU, which is one of 32 second-round candidates in NIST’s lightweight cryptography standardization process. TinyJAMBU adopts an NLFSR based keyed-permutation that computes only a single NAND gate as a non-linear component per round. The designers evaluated the minimum number of active AND gates, however such a counting method neglects the dependency between multiple AND gates. There also exist previous works considering such dependencies with stricter models, however those are known to be too slow. In this paper, we present a new model that provides a good balance of efficiency and accuracy by only taking into account the first-order correlation of AND gates that frequently occurs in TinyJAMBU. With the refined model, we show a 338-round differential with probability 2−62.68 that leads to a forgery attack breaking 64-bit security. This implies that the security margin of TinyJAMBU with respect to the number of unattacked rounds is approximately 12%. We also show a differential on full 384 rounds with probability 2−70.64, thus the security margin of full rounds with respect to the data complexity, namely the gap between the claimed security bits and the attack complexity, is less than 8 bits. Our attacks also point out structural weaknesses of the mode that essentially come from the minimal state size to be lightweight.
Real-time communicative perspective taking in younger and older adults.

One core question in studies of language processing is the extent to which interlocutors engage in real-time communicative perspective-taking. Current evidence suggests that both children and young adult listeners are able to draw on common ground (shared knowledge) to guide referential interpretation. However, less is known about older listeners, who are often described as experiencing age-related cognitive declines that could affect their capacity to integrate perspective cues online. In the present study, we examined the extent to which younger and older listeners used common ground to guide the interpretation of temporarily ambiguous descriptions. Participants followed instructions from a Director to click on displayed objects. The target object (e.g., hat with blue feathers) was accompanied by a competitor (e.g., hat with pink feathers) or a control object (e.g., stapler). We manipulated whether the competitor/control was mutually visible (common ground) or not (privileged ground). The results revealed that, although listeners used perspective information to differentiate the target from the competitor in the common ground condition, this pattern was notably weaker in older adults. Whereas measures of executive function showed significant group differences in inhibitory control and working memory, no differences were found in theory of mind. Thus, age-related changes in communicative perspective-taking are not likely due to general declines in mentalizing ability. Furthermore, strict screening criteria for vision and hearing ability allowed us to rule out explanations involving age-related sensory decline. Together, the results advance our understanding of how younger and older adults integrate common ground during real-time referential processing. (PsycInfo Database Record (c) 2020 APA, all rights reserved).
A novel algorithm for searching frequent gradual patterns from an ordered data set

Mining frequent simultaneous attribute co-variations in numerical databases is also called frequent gradual pattern problem. Few efficient algorithms for automatically extracting such patterns have been reported in the literature. Their main difference resides in the variation semantics used. However in applications with temporal order relations, those algorithms fail to generate correct frequent gradual patterns as they do not take this temporal constraint into account in the mining process. In this paper, we propose an approach for extracting frequent gradual patterns for which the ordering of supporting objects matches the temporal order. This approach considerably reduces the number of gradual patterns within an ordered data set. The experimental results show the benefits of our approach.
An intelligent virtual reality technology in the teaching of art creation and design in colleges and universities

The popularization of virtual reality technology has become a brand-new carrier of knowledge and information dissemination and a new tool for art creation and design teaching in Colleges and universities. In this paper, first of all, from the three aspects of the definition and characteristics of virtual reality technology, the integration of virtual reality technology into art design teaching, and the significance of using virtual reality technology in art design teaching, the reform of art creation and design teaching in Colleges and Universities under virtual reality technology is discussed. In order to carry out the application research, this paper then chooses to carry on the simulation modeling teaching of the bamboo forest in the intelligent virtual reality environment, making full use of the intelligent modeling tools, rendering tools and mapping tools to complete the teaching application examples of the art modeling design in Colleges and universities. In the specific teaching of art creation and design in Colleges and universities, the creation and design modeling of individual bamboos, the creation and design modeling of bamboos forest, and the creation and design modeling of different lighting effects and scene rendering are carried out respectively. Finally, from the advantages of intelligent virtual reality technology in art creation and design of colleges and universities, the expansion and promotion of teaching resources, and the synergistic effect of teaching results, we analyze the strategies of art creation and design teaching in Colleges and Universities under the background of intelligent virtual reality technology in detail.
Issues Facing Citizens In Iraq Towards Adoption Of E-Government

Developing countries have established promising e-Government initiatives with the objectives of enhancing the accessibility of government services and information for their citizens. Purpose: This research attempts to explore and investigate the key challenges and Issues that facing e-government adoption and the factors influencing citizen in Iraq. Methodology: The research depended on qualitative approach, the participants comprised undergraduate and postgraduate students at Baghdad university because university students are amongst the adult population for whom the Internet has become part of their daily routine. population of 369, Potential participants for the focus groups were identified from the questionnaire survey. Finding: The findings show that Trust and Awareness significant and direct relationship with adoption of e-government in Iraq. Corruption was found to have significant and negative relationship with the adoption of e- government.
Quantum jump Monte Carlo approach simplified: Abelian symmetries

We consider Markovian dynamics of a finitely dimensional open quantum system featuring a weak unitary symmetry, i.e., when the action of a unitary symmetry on the space of density matrices commutes with the master operator governing the dynamics. We show how to encode the weak symmetry in quantum stochastic dynamics of the system by constructing a weakly symmetric representation of the master operator: a symmetric Hamiltonian, and jump operators connecting only the symmetry eigenspaces with a fixed eigenvalue ratio. In turn, this representation simplifies both the construction of the master operator as well as quantum jump Monte Carlo simulations, where, for a symmetric initial state, stochastic trajectories of the system state are supported within a single symmetry eigenspace at a time, which is changed only by the action of an asymmetric jump operator. Our results generalize directly to the case of multiple Abelian weak symmetries.