Reviewerly/exHarmony huggingface dataset:
Perceptual hash-based coarse-to-fine grained image tampering forensics method

As an active forensic technology, perceptual image hash has important application in image content authenticity detection and integrity authentication. In this paper, we propose a hybrid-feature-based perceptual image hash method that can be used for image tampering detection and tampering localization. In the proposed method, we use the color features of image as global features, use point-based features and block-based features as local features, and combine with the structural features to generate intermediate hash code. Then we encrypt and randomize to generate the final hash code. Using this hash code, we present a coarse-to-fine grained forensics method for image tampering detection. The proposed method can realize object-level tampering localization. Abundant experimental results show that the proposed method is sensitive to content changes caused by malicious attacks, and the tampering localization precision achieves pixel level, and it is robust to a wide range of geometric distortions and content-preserving manipulations. Compared with the state-of-the-art schemes, the proposed scheme yields superior performance.
Effect of Batch Normalization and Stacked LSTMs on Video Captioning

Integration of visual content with natural language for generating images or video description has been a challenging task for many years. Recent research in image captioning using Long Short term memory (LSTM) recently has motivated its possible application in video captioning where a video is converted into an array of frames, or images, and this array along with the captions for the video are used to train the LSTM network to associate the video with sentences. However very little is known about using fine tuning techniques such as batch normalization or Stacked LSTMs models in video captioning and how it affects the performance of the model.For this project, we want to compare the performance of the base model described in [1] with batch normalization and stacked LSTMs with base model as our reference.
Are Current Voice Interfaces Designed to Support Children’s Language Development?

With the rapid development of artificial intelligence, voice user interfaces (VUIs) capable of speech-based interaction are poised to support children’s language development by serving as their language partners. This paper reports an analytic evaluation of the currently available voice-based apps targeting young children to examine whether and how they incorporate evidence-based dialogue strategies that effectively support children’s learning. We found that, despite the fact that the current apps support a variety of language activities, most fail to carry out open-ended dialogue and provide extended back-and-forth opportunities, thus limiting their ability to encourage children’s language output and increase children’s language exposure. We discuss four design implications for developing VUIs that initiate dialogue and provide feedback in ways that better facilitate children’s language learning.
Markov chain to analyze web usability of a university website using eye tracking data

Abstract Web usability is a crucial feature of a website, allowing users to easily find information in a short time. Eye tracking data registered during the execution of tasks allow to measure web usability in a more objective way compared to questionnaires. In this work, we evaluated the web usability of the website of the University of Cagliari through the analysis of eye tracking data with qualitative and quantitative methods. Performances of two groups of students (i.e., high school and university students) across 10 different tasks were compared in terms of time to completion, number of fixations and difficulty ratio. Transitions between different areas of interest (AOI) were analyzed in the two groups using Markov chain. For the majority of tasks, we did not observe significant differences in the performances of the two groups, suggesting that the information needed to complete the tasks could easily be retrieved by students with little previous experience in using the website. For a specific task, high school students showed a worse performance based on the number of fixations and a different Markov chain stationary distribution compared to university students. These results allowed to highlight elements of the pages that can be modified to improve web usability.
Stylized-Colorization for Line Arts

We address a novel problem of stylized-colorization which colorizes a given line art using a given coloring style in text. This problem can be stated as multi-domain image translation and is more challenging than the current colorization problem because it requires not only capturing the illustration distribution but also satisfying the required coloring styles specific to anime such as lightness, shading, or saturation. We propose a GAN-based end-to-end model for stylized-colorization where the model has one generator and two discriminators. Our generator is based on the U-Net architecture and receives a pair of a line art and a coloring style in text as its input to produce a stylized-colorization image of the line art. Two discriminators, on the other hand, share weights at early layers to judge the stylized-colorization image in two different aspects: one for color and one for style. One generator and two discriminators are jointly trained in an adversarial and end-to-end manner. Extensive experiments demonstrate the effectiveness of our proposed model.
End-to-End Optical Character Recognition for Bengali Handwritten Words

Optical character recognition (OCR) is a process of converting analogue documents into digital using document images. Currently, many commercial and non-commercial OCR systems exist for both handwritten and printed copies for different languages. Despite this, very few works are available in case of recognising Bengali words. Among them, most of the works focused on OCR of printed Bengali characters. This paper introduces an end-to-end OCR system for Bengali language. The proposed architecture implements an end to end strategy that recognises handwritten Bengali words from handwritten word images. We experiment with popular convolutional neural network (CNN) architectures, including DenseNet, Xception, NASNet, and MobileNet to build the OCR architecture. Further, we experiment with two different recurrent neural networks (RNN) methods, LSTM and GRU. We evaluate the proposed architecture using BanglaWritting dataset, which is a peer-reviewed Bengali handwritten image dataset. The proposed method achieves 0.091 character error rate and 0.273 word error rate performed using DenseNet121 model with GRU recurrent layer.
Image Inpainting with Contrastive Relation Network

Image inpainting faces the challenging issue of the requirements on structure reasonableness and texture coherence. In this paper, we propose a two-stage inpainting framework to address this issue. The basic idea is to address the two requirements in two separate stages. Completed segmentation of the corrupted image is firstly predicted through segmentation reconstruction network, while fine-grained image details are restored in the second stage through an image generator. The two stages are connected in series as the image details are generated under the guidance of completed segmentation map that predicted in the first stage. Specifically, in the second stage, we propose a novel graph-based relation network to model the relationship existed in corrupted image. In relation network, both intra-relationship for pixels in the same semantic region and inter-relationship between different semantic parts are considered, improving the consistency and compatibility of image textures. Besides, contrastive loss is designed to facilitate the relation network training. Such a framework not only simplifies the inpainting problem directly, but also exploits the relationship in corrupted image explicitly. Extensive experiments on various public datasets quantitatively and qualitatively demonstrate the superiority of our approach compared with the state-of-the-art.
Attention-Driven Body Pose Encoding for Human Activity Recognition

This article proposes a novel attention-based body pose encoding for human activity recognition. Most of the existing human activity recognition approaches based on 3D pose data often enrich the input data using additional handcrafted representations such as velocity, super-normal vectors, pairwise relations, and so on. The enriched data complements the 3D body joint position data and improves model performance. In this paper, we propose a novel approach that learns enhanced feature representations from a given sequence of 3D body joints. To achieve this encoding, the approach exploits two body pose streams: 1) a spatial stream which encodes the spatial relationship between various body joints at each time point to learn spatial structure involving the spatial distribution of different body joints 2) a temporal stream that learns the temporal variation of individual body joints over the entire sequence duration to present a temporally enhanced representation. Afterwards, these two pose streams are fused with a multi-head attention mechanism. We also capture the contextual information from the RGB video stream using a deep Convolutional Neural Network (CNN) model combined with a multi-head attention and a bidirectional Long Short-Term Memory (LSTM) network. Finally, the RGB video stream is combined with the fused body pose stream to give a novel end-to-end deep model for effective human activity recognition. The proposed model is evaluated on three datasets including the challenging NTU-RGBD dataset and achieves state-of-the-art results.
The Software Challenges of Building Smart Chatbots

Chatbots are becoming complex software artifacts that require a high-level of expertise in a variety of technical domains. This technical briefing will cover the software engineering challenges of developing high-quality chatbots. Attendees will be able to create their own bots leveraging the open source chatbot development platform Xatkit.
How Design Researchers Interpret Probes

Since entering the HCI lexicon in the 1990s, Probes have been interpreted and used in divergent ways as a designerly approach to research. While originally positioned as a critique of dominant user-research methods, literature on Probes rarely reflects on such critical dimensions nor explicitly articulates the intents of using Probes as research artifacts. We conducted interviews with 12 design researchers who have worked with Probes within diverse Research through Design projects, exploring direct accounts of how and why Probes are used in practice. Our interviews brought to the fore the critical concerns behind Probe practices in relation to the language of Probing, relationships with participants, and motivations to challenge normative practices. While the pluralistic interpretations of Probes offered by our participants brings challenges, we discuss how making visible the critical motivations of our research opens up new ways of practicing and disseminating Probes.
Holographic Feature Learning of Egocentric-Exocentric Videos for Multi-Domain Action Recognition

Though existing cross-domain action recognition methods successfully improve the performance on videos of one view ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g.</i> , egocentric videos) by transferring the knowledge from videos of another view ( <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">e.g.</i> , exocentric videos), they have limitations in generality because the source and target domains need to be fixed aforehand. In this paper, we propose to solve a more practical task of multi-domain action recognition on egocentric-exocentric videos, which aims to learn a single model to recognize test videos from either egocentric perspective or exocentric perspective by transferring knowledge between two domains. Though previous cross-domain methods can also transfer knowledge from one domain to another one by learning view-invariant representations of two video domains, they are not suitable for the multi-domain action recognition task because they always suffer from the problem of losing view-specific visual information. As a solution to the multi-domain action recognition task, we propose to map a video from either egocentric perspective or exocentric perspective to a global feature space (we call it holographic feature space) that shares both view-invariant and view-specific visual knowledge of two views. Specially, we decompose the video feature into view-invariant component and view-specific component, where view-specific component is written into memory networks for saving view-specific visual knowledge. The final holographic feature combines view-invariant feature and view-specific features of two views based on the memory networks. We demonstrate the effectiveness of the proposed method with extensive experimental results on two public datasets. Moreover, the good performances under the semi-supervised setting show the generality of our model.
A discrete-event simulation model for the Bitcoin blockchain network with strategic miners and mining pool managers

As the first and most famous cryptocurrency-based blockchain technology, Bitcoin has attracted tremendous attention from both academic and industrial communities in the past decade. A Bitcoin network is comprised of two interactive parties: individual miners and mining pool managers, each of which strives to maximize its own utility. In particular, individual miners choose which mining pool to join and decide on how much mining power to commit under limited constraints on the mining budget and mining power capacity; managers of mining pools determine how to allocate the mining reward and how to adjust the membership fee. In this work we investigate the miners’ and mining pool managers’ decisions in repeated Bitcoin mining competitions by building a Monte-Carlo discrete-event simulation model. Our simulation model (i) captures the behavior of these two parties and how their decisions affect each other, and (ii) characterizes the system-level dynamics of the blockchain in terms of the mining difficulty level and total mining power. In addition, we study the sensitivity of system performance metrics with respect to various control parameters. Our analysis may provide useful guidelines to mining activity participants in the Bitcoin network.
Rate-Distortion Optimized Coding for Efficient CNN Compression

In this paper, we present a coding framework for deep convolutional neural network compression. Our approach utilizes the classical coding theories and formulates the compression of deep convolutional neural networks as a rate-distortion optimization problem. We incorporate three coding ingredients in the coding framework, including bit allocation, dead zone quantization, and Tunstall coding, to improve the rate-distortion frontier without noticeable system-level overhead introduced. Experimental results show that our approach achieves state-of-the-art results on various deep convolutional neural networks and obtains considerable speedup on two deep learning accelerators. Specifically, our approach achieves 20× compression ratio on ResNet-18, ResNet-34, and ResNet-50, and 10× compression ratio on the compact already model MobileNet-v2, without hurting the accuracy. We then examine the system level impact of our approach when deploying the compressed models to hardware platforms. Hardware simulation results show that our approach obtains up to 4.3× and 2.8× inference speedup on state-of-the-art deep learning accelerators TPU and Eyeriss, respectively.
An end-to-end sea fog removal network using multiple scattering model

An end-to-end sea fog removal network using multiple scattering model was proposed. In this network, the atmospheric multiple scattering model was re-formulated and used for sea fog removal. Compared with the atmospheric single scattering model, the atmospheric multiple scattering model could more comprehensively consider the effect of multiple scattering, which was important to the dense fog scenes, such as in ocean scene. Therefore, we used the atmospheric multiple scattering model to avoid image blurring. The model can directly generate the dehazing results, and unify the three parameters of the transmission map, the atmospheric light and the blur kernel into one formula. The latest smooth dilation and sub-pixel techniques were used in the network model. The latest techniques can avoid the gridding artifacts and the halo artifacts, the multi-scale sub-network was used to consider the features of multi-scale. In addition, multiple loss functions were used in end-to-end network. In the experimental results, the model was superior to the state-of-the-art models in terms of quantitatively and qualitatively.
Learning a Dynamic High-Resolution Network for Multi-Scale Pedestrian Detection

Pedestrian detection is a canonical instance of object detection in computer vision. In practice, scale variation is one of the key challenges, resulting in unbalanced performance across different scales. Recently, the High-Resolution Network (HRNet) has become popular because high-resolution feature representations are more friendly to small objects. However, when we apply HRNet to pedestrian detection, we observe that it improves for small pedestrians on one hand, but hurts the performance for larger ones on the other hand. To overcome this problem, we propose a learnable Dynamic HRNet (DHRNet) aiming to generate different network paths adaptive to different scales. Specifically, we construct a parallel multi-branch architecture and add a soft conditional gate module allowing for dynamic feature fusion. Both branches share all the same parameters except the soft gate module. Experimental results on CityPersons and Caltech benchmarks indicate that our proposed dynamic HRNet is more capable of dealing with pedestrians of various scales, and thus improves the performance across different scales consistently.
TransRegex: Multi-modal Regular Expression Synthesis by Generate-and-Repair

Since regular expressions (abbrev. regexes) are difficult to understand and compose, automatically generating regexes has been an important research problem. This paper introduces TransRegex, for automatically constructing regexes from both natural language descriptions and examples. To the best of our knowledge, TransRegex is the first to treat the NLP-and-example-based regex synthesis problem as the problem of NLP-based synthesis with regex repair. For this purpose, we present novel algorithms for both NLP-based synthesis and regex repair. We evaluate TransRegex with ten relevant state-of-the-art tools on three publicly available datasets. The evaluation results demonstrate that the accuracy of our TransRegex is 17.4%, 35.8% and 38.9% higher than that of NLP-based approaches on the three datasets, respectively. Furthermore, TransRegex can achieve higher accuracy than the state-of-the-art multi-modal techniques with 10% to 30% higher accuracy on all three datasets. The evaluation results also indicate TransRegex utilizing natural language and examples in a more effective way.
Set Theory-Based Operator Design in Evolutionary Algorithms for Solving Knapsack Problems

Knapsack problems (KPs) are famous combinatorial optimization problems that can be solved by evolutionary algorithms (EAs). In such methods, a key step is to produce new solutions for each generation. However, traditional EAs cannot guarantee the feasibility of the new solutions, causing ineffectiveness or inefficiency of the methods. Owing to the fact that directly generating a feasible solution is difficult, a practical way is to generate a potential solution and transform it to a feasible one if necessary; more ideally, transform it to the local-optimal solution. Essentially, this transforming process is to map an element of the solution set to an element of the feasible solution set, which can be analyzed and optimized from a new perspective, i.e., the set theory (ST). In this article, we provide new explanations for the transforming process of solutions based on ST and summarize the properties that the transforming process should satisfy. Furthermore, based on the proposed concepts and theories, we put forward the ideas for improving the transforming process. Consequently, some new operators for KPs are proposed. Experimental results demonstrate the superiority of the proposed operators.
Cross-Domain Semi-Supervised Deep Metric Learning for Image Sentiment Analysis

This paper presents a novel method on image sentiment analysis called cross-domain semi-supervised deep metric learning (CDSS-DML). The proposed method has two contributions. Firstly, since previous researches on image sentiment analysis suffer from the limit of a small amount of well-labeled data, which occurs a decrease in accuracy of classification, CDSS-DML breaks through the limit by training with unlabeled data based on a teacher-student model. Secondly, the proposed method overcomes the difficulty of distribution shift between well-labeled and unlabeled data by jointing three losses. Especially, the proposed method constructs an effective latent space with the joint loss considering the inter-class and the intra-class correlations for image sentiments. From experimental results, the performance improvement with CDSS-DML is confirmed.
Arabic Sentiment Analysis of Eateries’ Reviews: Qassim region Case study

Social media plays an essential role in daily life. It allows people to express their thoughts and feelings about available products on e-commerce websites, which is often called an opinion or review. The aim is to see the author's mood or the speaker's attitude, which can be positive or negative about a product. This expression by the people is called sentiment. Sentiment analysis is a machine learning method in which the machine learns and analyzes some textual data's sentiment and emotions (for example, reviews about movies or products). In this research, a real-world data set was collected from customers in the Qassim region using a Microsoft Form survey, containing approximately 1,785 Arabic reviews related to the Qassim region, Saudi Arabia. The overall aim is to find people's opinions regarding different eateries (cafes and restaurants) using sentiment analysis of the customers' reviews written in the Arabic language. The experiments are run using 10-fold cross-validation. The performance of the algorithms was measured using accuracy, recall, and F-measure. Different supervised machine learning classifiers were applied. Support Vector Machine, Logistic regression, and Random Forest achieved the best results. Furthermore, we analyzed the differences between all results in this study. Index Terms-Sentiment analysis, Machine Learning, opinion mining.