Reviewerly/exHarmony huggingface dataset:
Visual Attention Software: A New Tool for Understanding the “Subliminal” Experience of the Built Environment

Eye-tracking technology is a biometric tool that has found many commercial and research applications. The recent advent of affordable wearable sensors has considerably expanded the range of these possibilities to fields such as computer gaming, education, entertainment, health, neuromarketing, psychology, etc. The Visual Attention Software by 3M (3M-VAS) is an artificial intelligence application that was formulated using experimental data from eye-tracking. It can be used to predict viewer reactions to images, generating fixation point probability maps and fixation point sequence estimations, thus revealing pre-attentive processing of visual stimuli with a very high degree of accuracy. We have used 3M-VAS software in an innovative implementation to analyze images of different buildings, either in their original state or photographically manipulated, as well as various geometric patterns. The software not only reveals non-obvious fixation points, but also overall relative design coherence, a key element of Christopher Alexander’s theory of geometrical order. A more evenly distributed field of attention seen in some structures contrasts with other buildings being ignored, those showing instead unconnected points of splintered attention. Our findings are non-intuitive and surprising. We link these results to both Alexander’s theory and Neuroscience, identify potential pitfalls in the software’s use, and also suggest ways to avoid them.
Toward Gender-Inclusive Coreference Resolution: An Analysis of Gender and Bias Throughout the Machine Learning Lifecycle

Abstract Correctly resolving textual mentions of people fundamentally entails making inferences about those people. Such inferences raise the risk of systematic biases in coreference resolution systems, including biases that can harm binary and non-binary trans and cis stakeholders. To better understand such biases, we foreground nuanced conceptualizations of gender from sociology and sociolinguistics, and investigate where in the machine learning pipeline such biases can enter a coreference resolution system. We inspect many existing data sets for trans-exclusionary biases, and develop two new data sets for interrogating bias in both crowd annotations and in existing coreference resolution systems. Through these studies, conducted on English text, we confirm that without acknowledging and building systems that recognize the complexity of gender, we will build systems that fail for: quality of service, stereotyping, and over- or under-representation, especially for binary and non-binary trans users.
Are synthetic clinical notes useful for real natural language processing tasks: A case study on clinical entity recognition

: Developing clinical natural language processing systems often requires access to many clinical documents, which are not widely available to the public due to privacy and security concerns. To address this challenge, we propose to develop methods to generate synthetic clinical notes and evaluate their utility in real clinical natural language processing tasks.: We implemented 4 state-of-the-art text generation models, namely CharRNN, SegGAN, GPT-2, and CTRL, to generate clinical text for the History and Present Illness section. We then manually annotated clinical entities for randomly selected 500 History and Present Illness notes generated from the best-performing algorithm. To compare the utility of natural and synthetic corpora, we trained named entity recognition (NER) models from all 3 corpora and evaluated their performance on 2 independent natural corpora.: Our evaluation shows GPT-2 achieved the best BLEU (bilingual evaluation understudy) score (with a BLEU-2 of 0.92). NER models trained on synthetic corpus generated by GPT-2 showed slightly better performance on 2 independent corpora: strict F1 scores of 0.709 and 0.748, respectively, when compared with the NER models trained on natural corpus (F1 scores of 0.706 and 0.737, respectively), indicating the good utility of synthetic corpora in clinical NER model development. In addition, we also demonstrated that an augmented method that combines both natural and synthetic corpora achieved better performance than that uses the natural corpus only.: Recent advances in text generation have made it possible to generate synthetic clinical notes that could be useful for training NER models for information extraction from natural clinical notes, thus lowering the privacy concern and increasing data availability. Further investigation is needed to apply this technology to practice.
IPGN: Interactiveness Proposal Graph Network for Human-Object Interaction Detection

Human-Object Interaction (HOI) Detection is an important task to understand how humans interact with objects. Most of the existing works treat this task as an exhaustive triplet 〈human, verb, object 〉 classification problem. In this paper, we decompose it and propose a novel two-stage graph model to learn the knowledge of interactiveness and interaction in one network, namely, Interactiveness Proposal Graph Network (IPGN). In the first stage, we design a fully connected graph for learning the interactiveness, which distinguishes whether a pair of human and object is interactive or not. Concretely, it generates the interactiveness features to encode high-level semantic interactiveness knowledge for each pair. The class-agnostic interactiveness is a more general and simpler objective, which can be used to provide reasonable proposals for the graph construction in the second stage. In the second stage, a sparsely connected graph is constructed with all interactive pairs selected by the first stage. Specifically, we use the interactiveness knowledge to guide the message passing. By contrast with the feature similarity, it explicitly represents the connections between the nodes. Benefiting from the valid graph reasoning, the node features are well encoded for interaction learning. Experiments show that the proposed method achieves state-of-the-art performance on both V-COCO and HICO-DET datasets.
Secure Data of Industrial Internet of Things in a Cement Factory Based on a Blockchain Technology

The Industrial Internet of Things (IIoT) has become a pivotal field of development that can increase the efficiency of real-time collection, recording, analysis, and control of the entire activities of various machines, and can actively enhance quality and reduce costs. The traditional IIoT depends on centralized architectures that are vulnerable to several kinds of cyber-attacks, such as bottlenecks and single points of failure. Blockchain technology has emerged to change these architectures to a decentralized form. In modern industrial settings, blockchain technology is utilized for its ability to provide high levels of security, low computational complexity, P2P communication, transparent logs, and decentralization. The present work proposes the use of a private blockchain mechanism for an industrial application in a cement factory, which offers low power consumption, scalability, and a lightweight security scheme; and which can play an efficient role in controlling access to valuable data generated by sensors and actuators. A low-power ARM Cortex-M processor is utilized due to its efficiency in terms of processing cryptographic algorithms, and this plays an important part in improving the computational execution of the proposed architecture. In addition, instead of proof of work (PoW), our blockchain network uses proof of authentication (PoAh) as a consensus mechanism to ensure secure authentication, scalability, speed, and energy efficiency. Our experimental results show that the proposed framework achieves high levels of security, scalability and ideal performance for smart industrial environments. Moreover, we successfully realized the integration of blockchain technology with the industrial internet of things devices, which provides the blockchain technology features and efficient resistance to common cyber-security attacks.
Efficient and Accurate Multi-Scale Topological Network for Single Image Dehazing

Single image dehazing is a challenging ill-posed problem that has drawn significant attention in the last few years. Recently, convolutional neural networks have achieved great success in image dehazing. However, it is still difficult for these increasingly complex models to recover accurate details from the hazy image. In this paper, we pay attention to the feature extraction and utilization of the input image itself. To achieve this, we propose a Multi-scale Topological Network (MSTN) to fully explore the features at different scales. Meanwhile, we design a Multi-scale Feature Fusion Module (MFFM) and an Adaptive Feature Selection Module (AFSM) to achieve the selection and fusion of features at different scales, so as to achieve progressive image dehazing. This topological network provides a large number of search paths that enable the network to extract abundant image features as well as strong fault tolerance and robustness. In addition, ASFM and MFFM can adaptively select important features and ignore interference information when fusing different scale representations. Extensive experiments are conducted to demonstrate the superiority of our method compared with state-of-the-art methods.
Herding on Fundamental/Nonfundamental Information During the COVID-19 Outbreak and Cyber-Attacks: Evidence From the Cryptocurrency Market

We provide an empirical analysis of herding behavior in cryptocurrency markets during COVID-19 and periods of cyber-attacks, differentiating between fundamental and nonfundamental herding. The results show that herding behavior is driven by fundamental information during the full sample period and the cyber-attack days. However, herding is not prevalent during the COVID-19 outbreak, either when reacting to fundamental or nonfundamental information. This finding suggests heterogeneity in the behaviors of participants in the cryptocurrency markets during the COVID-19 period.
Physics model-informed Gaussian process for online optimization of particle accelerators

High-dimensional optimization is a critical challenge for operating large-scale scientific facilities. We apply a physics-informed Gaussian process (GP) optimizer to tune a complex system by conducting efficient global search. Typical GP models learn from past observations to make predictions, but this reduces their applicability to new systems where archive data is not available. Instead, here we use a fast approximate model from physics simulations to design the GP model. The GP is then employed to make inferences from sequential online observations in order to optimize the system. Simulation and experimental studies were carried out to demonstrate the method for online control of a storage ring. We show that the physics-informed GP outperforms current routinely used online optimizers in terms of convergence speed, and robustness on this task. The ability to inform the machine-learning model with physics may have wide applications in science.
Adaptive Methods for Real-World Domain Generalization

Invariant approaches have been remarkably successful in tackling the problem of domain generalization, where the objective is to perform inference on data distributions different from those used in training. In our work, we investigate whether it is possible to leverage domain information from the unseen test samples themselves. We propose a domain-adaptive approach consisting of two steps: a) we first learn a discriminative domain embedding from unsupervised training examples, and b) use this domain embedding as supplementary information to build a domain-adaptive model, that takes both the input as well as its domain into account while making predictions. For unseen domains, our method simply uses few unlabelled test examples to construct the domain embedding. This enables adaptive classification on any unseen domain. Our approach achieves state-of-the-art performance on various domain generalization benchmarks. In addition, we introduce the first real-world, large-scale domain generalization benchmark, Geo-YFCC, containing 1.1M samples over 40 training, 7 validation and 15 test domains, orders of magnitude larger than prior work. We show that the existing approaches either do not scale to this dataset or underperform compared to the simple baseline of training a model on the union of data from all training domains. In contrast, our approach achieves a significant 1% improvement.
A Low Complexity Learning-Based Channel Estimation for OFDM Systems With Online Training

In this paper, we devise a highly efficient machine learning-based channel estimation for orthogonal frequency division multiplexing (OFDM) systems, in which the training of the estimator is performed online. A simple learning module is employed for the proposed learning-based estimator. The training process is thus much faster and the required training data is reduced significantly. Besides, a training data construction approach utilizing least square (LS) estimation results is proposed so that the training data can be collected during the data transmission. The feasibility of this novel construction approach is verified by theoretical analysis and simulations. Based on this construction approach, two alternative training data generation schemes are proposed. One scheme transmits additional block pilot symbols to create training data, while the other scheme adopts a decision-directed method and does not require extra pilot overhead. Simulation results show the robustness of the proposed channel estimation method. Furthermore, the proposed method shows better adaptation to practical imperfections compared with the conventional minimum mean-square error (MMSE) channel estimation. It outperforms the existing machine learning-based channel estimation techniques under varying channel conditions.
Capsule Network is Not More Robust than Convolutional Network

The Capsule Network is widely believed to be more robust than Convolutional Networks. However, there are no comprehensive comparisons between these two networks, and it is also unknown which components in the CapsNet affect its robustness. In this paper, we first carefully examine the special designs in CapsNet that differ from that of a ConvNet commonly used for image classification. The examination reveals five major new/different components in CapsNet: a transformation process, a dynamic routing layer, a squashing function, a marginal loss other than cross-entropy loss, and an additional class-conditional reconstruction loss for regularization. Along with these major differences, we conduct comprehensive ablation studies on three kinds of robustness, including affine transformation, overlapping digits, and semantic representation. The study reveals that some designs, which are thought critical to CapsNet, actually can harm its robustness, i.e., the dynamic routing layer and the transformation process, while others are beneficial for the robustness. Based on these findings, we propose enhanced ConvNets simply by introducing the essential components behind the CapsNet’s success. The proposed simple ConvNets can achieve better robustness than the CapsNet.
Can Offline Testing of Deep Neural Networks Replace Their Online Testing?

We distinguish two general modes of testing for Deep Neural Networks (DNNs): Offline testing where DNNs are tested as individual units based on test datasets obtained without involving the DNNs under test, and online testing where DNNs are embedded into a specific application environment and tested in a closed-loop mode in interaction with the application environment. Typically, DNNs are subjected to both types of testing during their development life cycle where offline testing is applied immediately after DNN training and online testing follows after offline testing and once a DNN is deployed within a specific application environment. In this paper, we study the relationship between offline and online testing. Our goal is to determine how offline testing and online testing differ or complement one another and if offline testing results can be used to help reduce the cost of online testing? Though these questions are generally relevant to all autonomous systems, we study them in the context of automated driving systems where, as study subjects, we use DNNs automating end-to-end controls of steering functions of self-driving vehicles. Our results show that offline testing is less effective than online testing as many safety violations identified by online testing could not be identified by offline testing, while large prediction errors generated by offline testing always led to severe safety violations detectable by online testing. Further, we cannot exploit offline testing results to reduce the cost of online testing in practice since we are not able to identify specific situations where offline testing could be as accurate as online testing in identifying safety requirement violations.
Object detection/tracking toward underwater photographs by remotely operated vehicles (ROVs)

Owing to the shortcoming of image blur, scale diversity, and complex backgrounds, underwater target detection and tracking is a tough challenge in real-world ROVs applications. In this paper, an underwater target detection method based on the class-weighted YOLO network is proposed. The key technique is to construct the class-weighted loss function based on the deep network YOLO to balance the sample difficulty and achieve better results. In addition, an adaptive dimensional clustering algorithm of target box is introduced to further improve the detection performance. Firstly, according to the imaging characteristics of the small underwater target in the side-scan sonar image, the sonar image is filtered and segmented by clustering to significantly reduce noises in each image. Afterward, the speckle detection is deployed to extract the suspected target area in the side-scan sonar image. Finally, the sonar image is segmented according to the automatic threshold segmentation and receive the binary image of the target region. The target scale is estimated by the second moment and the false target is eliminated. Correspondingly, the accurate detection of small underwater target can be achieved. The moving object pixel is detected by background difference and three-frame difference respectively. On this basis, we combine the background difference and the three-frame difference. The background difference characterizes the objects capturing the incomplete message detected by the three-frame difference. Finally, morphological operation is leveraged to eliminate the noise caused by non-static background objects. Experiments on multiple ROVs have validated that, the proposed algorithm can improve the average accuracy and the recall rate remarkably in the dense target detection task that contains 20 targets averagely.
Intelligent Recognition and Teaching of English Fuzzy Texts Based on Fuzzy Computing and Big Data

In this paper, we conduct in-depth research and analysis on the intelligent recognition and teaching of English fuzzy text through parallel projection and region expansion. Multisense Soft Cluster Vector (MSCVec), a multisense word vector model based on nonnegative matrix decomposition and sparse soft clustering, is constructed. The MSCVec model is a monolingual word vector model, which uses nonnegative matrix decomposition of positive point mutual information between words and contexts to extract low-rank expressions of mixed semantics of multisense words and then uses sparse. It uses the nonnegative matrix decomposition of the positive pointwise mutual information between words and contexts to extract the low-rank expressions of the mixed semantics of the polysemous words and then uses the sparse soft clustering algorithm to partition the multiple word senses of the polysemous words and also obtains the global sense of the polysemous word affiliation distribution; the specific polysemous word cluster classes are determined based on the negative mean log-likelihood of the global affiliation between the contextual semantics and the polysemous words, and finally, the polysemous word vectors are learned using the Fast text model under the extended dictionary word set. The advantage of the MSCVec model is that it is an unsupervised learning process without any knowledge base, and the substring representation in the model ensures the generation of unregistered word vectors; in addition, the global affiliation of the MSCVec model can also expect polysemantic word vectors to single word vectors. Compared with the traditional static word vectors, MSCVec shows excellent results in both word similarity and downstream text classification task experiments. The two sets of features are then fused and extended into new semantic features, and similarity classification experiments and stack generalization experiments are designed for comparison. In the cross-lingual sentence-level similarity detection task, SCLVec cross-lingual word vector lexical-level features outperform MSCVec multisense word vector features as the input embedding layer; deep semantic sentence-level features trained by twin recurrent neural networks outperform the semantic features of twin convolutional neural networks; extensions of traditional statistical features can effectively improve cross-lingual similarity detection performance, especially cross-lingual topic model (BL-LDA); the stack generalization integration approach maximizes the error rate of the underlying classifier and improves the detection accuracy.
CokeBERT: Contextual knowledge selection and embedding towards enhanced pre-trained language models

Several recent efforts have been devoted to enhancing pre-trained language models (PLMs) by utilizing extra heterogeneous knowledge in knowledge graphs (KGs) and achieved consistent improvements on various knowledge-driven NLP tasks. However, most of these knowledge-enhanced PLMs embed static sub-graphs of KGs ("knowledge context"), regardless of that the knowledge required by PLMs may change dynamically according to specific text ("textual context"). In this paper, we propose a novel framework named Coke to dynamically select contextual knowledge and embed knowledge context according to textual context for PLMs, which can avoid the effect of redundant and ambiguous knowledge in KGs that cannot match the input text. Our experimental results show that Coke outperforms various baselines on typical knowledge-driven NLP tasks, indicating the effectiveness of utilizing dynamic knowledge context for language understanding. Besides the performance improvements, the dynamically selected knowledge in Coke can describe the semantics of text-related knowledge in a more interpretable form than the conventional PLMs. Our source code and datasets will be available to provide more details for Coke.