Reviewerly/exHarmony huggingface dataset:
Patent claim generation by fine-tuning OpenAI GPT-2

In this work, we focus on fine-tuning an OpenAI GPT-2 pre-trained model for generating patent claims. GPT-2 has demonstrated impressive efficacy of pre-trained language models on various tasks, particularly coherent text generation. Patent claim language itself has rarely been explored in the past and poses a unique challenge. We are motivated to generate coherent patent claims automatically so that augmented inventing might be viable someday. In our implementation, we identified a unique language structure in patent claims and leveraged its implicit human annotations. We investigated the fine-tuning process by probing the first 100 steps and observing the generated text at each step. Based on both conditional and unconditional random sampling, we analyze the overall quality of generated patent claims. Our contributions include: (1) being the first to generate patent claims by machines and being the first to apply GPT-2 to patent claim generation, (2) providing various experiment results for qualitative analysis and future research, (3) proposing a new sampling approach for text generation, and (4) publishing our fine-tuned GPT-2 model and sample code for future researchers to run on Colab.
An Efficient Neighborhood-based Interaction Model for Recommendation on Heterogeneous Graph

There is an influx of heterogeneous information network (HIN) based recommender systems in recent years since HIN is capable of characterizing complex graphs and contains rich semantics. Although the existing approaches have achieved performance improvement, while practical, they still face the following problems. On one hand, most existing HIN-based methods rely on explicit path reachability to leverage path-based semantic relatedness between users and items, e.g., metapath-based similarities. These methods are hard to use and integrate since path connections are sparse or noisy, and are often of different lengths. On the other hand, other graph-based methods aim to learn effective heterogeneous network representations by compressing node together with its neighborhood information into single embedding before prediction. This weakly coupled manner in modeling overlooks the rich interactions among nodes, which introduces an early summarization issue. In this paper, we propose an end-to-end Neighborhood-based Interaction Model for Recommendation (NIRec) to address above problems. Specifically, we first analyze the significance of learning interactions in HINs and then propose a novel formulation to capture the interactive patterns between each pair of nodes through their metapath-guided neighborhoods. Then, to explore complex interactions between metapaths and deal with the learning complexity on large-scale networks, we formulate interaction in a convolutional way and learn efficiently with fast Fourier transform. The extensive experiments on four different types of heterogeneous graphs demonstrate the performance gains of NIRec comparing with state-of-the-arts. To the best of our knowledge, this is the first work providing an efficient neighborhood-based interaction model in the HIN-based recommendations.
Chaotic coyote algorithm applied to truss optimization problems

The optimization of truss structures is a complex computing problem with many local minima, while metaheuristics are naturally suited to deal with multimodal problems without the need of gradient information. The Coyote Optimization Algorithm (COA) is a population-based nature-inspired metaheuristic of the swarm intelligence field for global optimization that considers the social relations of the coyote proposed to single-objective optimization. Unlike most widespread algorithms, its population is subdivided in packs and the internal social influences are designed. The COA requires a few control hyperparameters including the number of packs, the population size, and the number maximum of generations. In this paper, a modified COA (MCOA) approach based on chaotic sequences generated by Tinkerbell map to scatter and association probabilities tuning and an adaptive procedure of updating parameters related to social condition is proposed. It is then validated by four benchmark problems of structures optimization including planar 52-bar truss, spatial 72-bar truss, 120-bar dome truss and planar 200 bar-truss with discrete design variables and focus in minimization of the structure weight under the required constraints. Simulation results collected in the mentioned problems demonstrate that the proposed MCOA presented competitive solutions when compared with other state-of-the-art metaheuristic algorithms in terms of results quality.
Rational Protocols and Attacks in Blockchain System

Blockchain has been an emerging technology, which comprises lots of fields such as distributed systems and Internet of Things (IoT). As is well known, blockchain is the underlying technology of bitcoin, whose initial motivation is derived from economic incentives. Therefore, lots of components of blockchain (e.g., consensus mechanism) can be constructed toward the view of game theory. In this paper, we highlight the combination of game theory and blockchain, including rational smart contracts, game theoretic attacks, and rational mining strategies. When put differently, the rational parties, who manage to maximize their utilities, involved in blockchain chose their strategies according to the economic incentives. Consequently, we focus on the influence of rational parties with respect to building blocks. More specifically, we investigate the research progress from the aspects of smart contract, rational attacks, and consensus mechanism, respectively. Finally, we present some future directions based on the brief survey with respect to game theory and blockchain.
Brief History of Artificial Intelligence

This article reviews the history of artificial intelligence and introduces the reader to major events that prompted interest in the field, as well as pitfalls and challenges that have slowed its development. The purpose of this article is to provide a high-level historical perspective on the development of the field over the past decades, highlighting the potential of the field for transforming health care, but also the importance of setting realistic expectations for artificial intelligence applications to avoid repeating historical cyclical trends and a third "artificial intelligence winter."
Physics-Based Feature Dehazing Networks

We propose a physics-based feature dehazing network for image dehazing. In contrast to most existing end-to-end trainable network-based dehazing methods, we explicitly consider the physics model of the haze process in the network design and remove haze in a deep feature space. We propose an effective feature dehazing unit (FDU), which is applied to the deep feature space to explore useful features for image dehazing based on the physics model. The FDU is embedded into an encoder and decoder architecture with residual learning, so that the proposed network can be trained in an end-to-end fashion and effectively help haze removal. The encoder and decoder modules are adopted for feature extraction and clear image reconstruction, respectively. The residual learning is applied to increase the accuracy and ease the training of deep neural networks. We analyze the effectiveness of the proposed network and demonstrate that it can effectively dehaze images with favorable performance against state-of-the-art methods.
Masked Face Recognition with Latent Part Detection

This paper focuses on a novel task named masked faces recognition (MFR), which aims to match masked faces with common faces and is important especially during the global outbreak of COVID-19. It is challenging to identify masked faces for two main reasons. Firstly, there is no large-scale training data and test data with ground truth for MFR. Collecting and annotating millions of masked faces is labor-consuming. Secondly, since most facial cues are occluded by mask, it is necessary to learn representations which are both discriminative and robust to mask wearing. To handle the first challenge, this paper collects two datasets designed for MFR: MFV with 400 pairs of 200 identities for verification, and MFI which contains 4,916 images of 669 identities for identification. As is known, a robust face recognition model needs images of millions of identities to train, and hundreds of identities is far from enough. Hence, MFV and MFI are only considered as test datasets to evaluate algorithms. Besides, a data augmentation method for training data is introduced to automatically generate synthetic masked face images from existing common face datasets. In addition, a novel latent part detection (LPD) model is proposed to locate the latent facial part which is robust to mask wearing, and the latent part is further used to extract discriminative features. The proposed LPD model is trained in an end-to-end manner and only utilizes the original and synthetic training data. Experimental results on MFV, MFI and synthetic masked LFW demonstrate that LPD model generalizes well on both realistic and synthetic masked data and outperforms other methods by a large margin.
Same benefits, different communication patterns: Comparing Children's reading with a conversational agent vs. a human partner

Storybook reading accompanied by adult-guided conversation provides a stimulating context for children's language development. Conversational agents powered by artificial intelligence, such as smart speakers, are prevalent in children's homes and have the potential to engage children in storybook reading as language partners. However, little research has explored the effectiveness of using conversational agents to support children's language development. This study examined how an automated conversational agent can read stories to children via a smart speaker while asking questions and providing contingent feedback. Using a randomized experiment among 90 children aged three to six years, this study compared these children's story comprehension and verbal engagement in storybook reading with a conversational agent versus an adult. The conversational agent's guided conversation was found to be as supportive in improving children's story comprehension as that provided by an adult language partner. At the same time, this study uncovered a number of differences in children's verbal engagement when interacting with a conversational agent versus with an adult. Specifically, children who read with the conversational agent responded to questions with better intelligibility, whereas those who read with an adult responded to questions with higher productivity, lexical diversity, and topical relevance. And the two groups responded to questions with a similar level of accuracy. In addition, questions requiring high cognitive demand amplified the differences in of verbal engagement between the conversational agent and adult partner. The study offers important implications for developing and researching conversational agent systems to support children's language development.
Connecting Vision and Language with Localized Narratives

We propose Localized Narratives, a new form of multimodal image annotations connecting vision and language. We ask annotators to describe an image with their voice while simultaneously hovering their mouse over the region they are describing. Since the voice and the mouse pointer are synchronized, we can localize every single word in the description. This dense visual grounding takes the form of a mouse trace segment per word and is unique to our data. We annotated 849k images with Localized Narratives: the whole COCO, Flickr30k, and ADE20K datasets, and 671k images of Open Images, all of which we make publicly available. We provide an extensive analysis of these annotations showing they are diverse, accurate, and efficient to produce. We also demonstrate their utility on the application of controlled image captioning.
Contour-Aware Loss: Boundary-Aware Learning for Salient Object Segmentation

We present a learning model that makes full use of boundary information for salient object segmentation. Specifically, we come up with a novel loss function, i.e., Contour Loss, which leverages object contours to guide models to perceive salient object boundaries. Such a boundary-aware network can learn boundary-wise distinctions between salient objects and background, hence effectively facilitating the salient object segmentation. Yet the Contour Loss emphasizes the boundaries to capture the contextual details in the local range. We further propose the hierarchical global attention module (HGAM), which forces the model hierarchically to attend to global contexts, thus captures the global visual saliency. Comprehensive experiments on six benchmark datasets show that our method achieves superior performance over state-of-the-art ones. Moreover, our model has a real-time speed of 26 fps on a TITAN X GPU.
COVID-19 what have we learned? The rise of social machines and connected devices in pandemic management following the concepts of predictive, preventive and personalized medicine

Review, compare and critically assess digital technology responses to the COVID-19 pandemic around the world. The specific point of interest in this research is on predictive, preventive and personalized interoperable digital healthcare solutions. This point is supported by failures from the past, where the separate design of digital health solutions has led to lack of interoperability. Hence, this review paper investigates the integration of predictive, preventive and personalized interoperable digital healthcare systems. The second point of interest is the use of new mass surveillance technologies to feed personal data from health professionals to governments, without any comprehensive studies that determine if such new technologies and data policies would address the pandemic crisis.This is a review paper. Two approaches were used: A comprehensive bibliographic review with R statistical methods of the COVID-19 pandemic in PubMed literature and Web of Science Core Collection, supported with Google Scholar search. In addition, a case study review of emerging new approaches in different regions, using medical literature, academic literature, news articles and other reliable data sources.Most countries' digital responses involve big data analytics, integration of national health insurance databases, tracing travel history from individual's location databases, code scanning and individual's online reporting. Public responses of mistrust about privacy data misuse differ across countries, depending on the chosen public communication strategy. We propose predictive, preventive and personalized solutions for pandemic management, based on social machines and connected devices.The proposed predictive, preventive and personalized solutions are based on the integration of IoT data, wearable device data, mobile apps data and individual data inputs from registered users, operating as a social machine with strong security and privacy protocols. We present solutions that would enable much greater speed in future responses. These solutions are enabled by the social aspect of human-computer interactions (social machines) and the increased connectivity of humans and devices (Internet of Things).Inadequate data for risk assessment on speed and urgency of COVID-19, combined with increased globalization of human society, led to the rapid spread of COVID-19. Despite an abundance of digital methods that could be used in slowing or stopping COVID-19 and future pandemics, the world remains unprepared, and lessons have not been learned from previous cases of pandemics. We present a summary of predictive, preventive and personalized digital methods that could be deployed fast to help with the COVID-19 and future pandemics.
Blockchain based access control systems: State of the art and challenges

Access to the system resources. The current access control systems face many problems, such as the presence of the third-party, inefficiency, and lack of privacy. These problems can be addressed by blockchain, the technology that received major attention in recent years and has many potentials. In this study, we overview the problems of the current access control systems, and then, we explain how blockchain can help to solve them. We also present an overview of access control studies and proposed platforms in different domains. This paper presents the state of the art and the challenges of blockchain-based access control systems.
An Automated and Robust Image Watermarking Scheme Based on Deep Neural Networks

Digital image watermarking is the process of embedding and extracting watermark covertly on a carrier image. Incorporating deep learning networks with image watermarking has attracted increasing attention during recent years. However, existing deep learning-based watermarking systems cannot achieve robustness, blindness, and automated embedding and extraction simultaneously. In this paper, a fully automated image watermarking system based on deep neural networks is proposed to generalize the image watermarking processes. An unsupervised deep learning structure and a novel loss computation are proposed to achieve high capacity and high robustness without any prior knowledge of possible attacks. Furthermore, a challenging application of watermark extraction from camera-captured images is provided to validate the practicality as well as the robustness of the proposed system. Experimental results show the superiority performance of the proposed system as comparing against several currently available techniques.
Does virtual reality attract visitors? The mediating effect of presence on consumer response in virtual reality tourism advertising

An experimental study was conducted to test the mediating effect of presence on the consumer response to tourism promotion using virtual reality (VR) technology. Hong Kong college students (N = 203) were assigned to three experimental groups in which they were exposed to a hotel advertorial on an online blog, a VR 360° video on a mobile phone, or a VR 360° video using Google Cardboard goggles. The results supported the hypothesis that presence plays a mediating role in the relationship between the use of VR in tourism advertising and consumer response. As one of the first studies in the tourism field to investigate the effects of a virtual depiction of a real-world environment on consumption intentions, the findings reveal how VR technology can be used to promote a hotel by generating a sense of presence. This study makes a further contribution to the VR literature by finding that the use of Cardboard goggles did not produce a significantly greater sense of presence in the experience of advertorial viewers. The theoretical contributions and practical implications of the findings for the use of VR technology in branding and promotion are discussed.
Pearson Correlation-Based Feature Selection for Document Classification Using Balanced Training

Documents are stored in a digital form across several organizations. Printing this amount of data and placing it into folders instead of storing digitally is against the practical, economical, and ecological perspective. An efficient way of retrieving data from digitally stored documents is also required. This article presents a real-time supervised learning technique for document classification based on deep convolutional neural network (DCNN), which aims to reduce the impact of adverse document image issues such as signatures, marks, logo, and handwritten notes. The proposed technique’s major steps include data augmentation, feature extraction using pre-trained neural network models, feature fusion, and feature selection. We propose a novel data augmentation technique, which normalizes the imbalanced dataset using the secondary dataset RVL-CDIP. The DCNN features are extracted using the VGG19 and AlexNet networks. The extracted features are fused, and the fused feature vector is optimized by applying a Pearson correlation coefficient-based technique to select the optimized features while removing the redundant features. The proposed technique is tested on the Tobacco3482 dataset, which gives a classification accuracy of 93.1% using a cubic support vector machine classifier, proving the validity of the proposed technique.
Collaborative Fairness in Federated Learning

In current deep learning paradigms, local training or the Standalone framework tends to result in overfitting and thus low utility. This problem can be addressed by Distributed or Federated Learning (FL) that leverages a parameter server to aggregate local model updates. However, all the existing FL frameworks have overlooked an important aspect of participation: collaborative fairness. In particular, all participants can receive the same or similar models, even the ones who contribute relatively less, and in extreme cases, nothing. To address this issue, we propose a novel Collaborative Fair Federated Learning (CFFL) framework which utilizes reputations to enforce participants to converge to different models, thus ensuring fairness and accuracy at the same time. Extensive experiments on benchmark datasets demonstrate that CFFL achieves high fairness and performs comparably to the Distributed framework and better than the Standalone framework.
Missing value imputation in multivariate time series with end-to-end generative adversarial networks

Missing values are inherent in multivariate time series because of multiple reasons, such as collection errors, which deteriorate the performance of follow-up analytic applications on the multivariate time series. Numerous missing value imputation methods have been proposed to mitigate the influence of missing values on multivariate time series analysis. Recently, inspired by the success of generative adversarial networks (GANs) in image generation, the GAN-2-Stage has been used to address the imputation problem with the generative model. Specifically, GAN-2-Stage employs GANs to impute the missing values. However, an extra phase is required to optimize the input random “noise” of the generator. In addition, the imputed values can be very different from real values because of the difficulty in training a GAN and the unstable generation process. Therefore, this paper proposes an end-to-end model to impute the missing values in a multivariate time series. Specifically, we introduce an encoder network into the standard GAN architecture that eliminates the input optimization phase in the GAN-2-Stage. Our generator utilizes real data during training to force the imputed values to be close to the real ones. Experiments on three real-world multivariate time series datasets demonstrate that the proposed model outperforms state-of-the-art methods in imputation tasks and downstream applications, including classification and regression.
DeepThin: A novel lightweight CNN architecture for traffic sign recognition without GPU requirements

For a safe and automated vehicle driving application, it is a prerequisite to have a robust and highly accurate traffic sign detection system. In this paper, we proposed a novel energy-efficient Thin yet Deep convolutional neural network architecture for traffic sign recognition. Within the proposed architecture, each convolutional layer contains less than 50 features enabling our convolutional neural network to be trained quickly even without the aid of a graphics processing unit. The performance of the proposed architecture is measured using two publicly available traffic sign datasets, namely the German Traffic Sign Recognition Benchmark and the Belgian Traffic Sign Classification dataset. First, we train and test the performance of the proposed architecture using the large German Traffic Sign Recognition Benchmark dataset. Then, we retrain the network models using transfer learning on the more challenging Belgian Traffic Sign Classification dataset to evaluate test performance. The proposed architecture outperforms the performance of the state-of-the-art traffic sign methods with at least five times less parameter in the individual end-to-end network for training.