Reviewerly/exHarmony huggingface dataset:
Automatic Compilation of Diverse CNNs Onto High-Performance FPGA Accelerators

A broad range of applications are increasingly benefiting from the rapid and flourishing development of convolutional neural networks (CNNs). The FPGA-based CNN inference accelerator is gaining popularity due to its high-performance and low-power as well as FPGA's conventional advantage of reconfigurability and flexibility. Without a general compiler to automate the implementation, however, significant efforts and expertise are still required to customize the design for each CNN model. In this paper, we present an register-transfer level (RTL)-level CNN compiler that automatically generates customized FPGA hardware for the inference tasks of various CNNs, in order to enable high-level fast prototyping of CNNs from software to FPGA and still keep the benefits of low-level hardware optimization. First, a general-purpose library of RTL modules is developed to model different operations at each layer. The integration and dataflow of physical modules are predefined in the top-level system template and reconfigured during compilation for a given CNN algorithm. The runtime control of layer-by-layer sequential computation is managed by the proposed execution schedule so that even highly irregular and complex network topology, e.g., GoogLeNet and ResNet, can be compiled. The proposed methodology is demonstrated with various CNN algorithms, e.g., NiN, VGG, GoogLeNet, and ResNet, on two standalone Intel FPGAs, Arria 10, and Stratix 10, achieving end-to-end inference throughputs of 969 GOPS and 1604 GOPS, respectively, with batch size of one.
Webthetics: Quantifying webpage aesthetics with deep learning

As web has become the most popular media to attract users and customers worldwide, webpage aesthetics plays an increasingly important role for engaging users online and impacting their user experience. We present a novel method using deep learning to automatically compute and quantify webpage aesthetics. Our deep neural network, named as Webthetics, which is trained from the collected user rating data, can extract representative features from raw webpages and quantify their aesthetics. To improve the model performance, we propose to transfer the knowledge from image style recognition task into our network. We have validated that our method significantly outperforms previous method using hand-crafted features such as colorfulness and complexity. These promising results indicate that our method can serve as an effective and efficient means for providing objective aesthetics evaluation during the design process.
Gaze allocation in face-to-face communication is affected primarily by task structure and social context, not stimulus-driven factors

Gaze allocation to human faces has recently been shown to be greatly dependent on the social context. However, what has not been considered explicitly here, is how gaze allocation may be supportive of the specific task that individuals carry out. In the present study, we combined these two insights. We investigated (1) how gaze allocation to facial features in face-to-face communication is dependent on the task-structure and (2) how gaze allocation to facial features is dependent on the gaze behavior of an interacting partner. To this end, participants and a confederate were asked to converse, while their eye movements were monitored using a state-of-the-art dual eye-tracking system. This system is unique in that participants can look each other directly in the eyes. We report that gaze allocation depends on the sub-task being carried out (speaking vs. listening). Moreover, we show that a confederate's gaze shift away from the participants affects their gaze allocation more than a gaze shift towards them. In a second experiment, we show that this gaze-guidance effect is not primarily stimulus-driven. We assert that gaze guidance elicited by the confederate looking away is related to the participants' sub-task of monitoring the confederate for when they can begin speaking. This study exemplifies the importance of both task structure and social context for gaze allocation during face-to-face communication.
Interactive natural language question answering over knowledge graphs

As many real-world data are constructed into knowledge graphs, providing effective and convenient query techniques for end users is an urgent and important task. Although structured query languages, such as SPARQL, offer a powerful expression ability to query RDF datasets, they are difficult to use. Keywords are simple but have a very limited expression ability. Natural language question (NLQ) is promising for querying knowledge graphs. A huge challenge is how to understand the question clearly so as to translate the unstructured question into a structured query. In this paper, we present a data + oracle approach to answer NLQs over knowledge graphs. We let users verify the ambiguities during the query understanding. To reduce the interaction cost, we formalize an interaction problem and design an efficient strategy to solve the problem. We also propose a query prefetching technique by exploiting the latency in the interactions with users. Moreover, we devise a hybrid approach that incorporates NLP-based, data-driven, and interaction techniques together to complete the question understanding. Extensive experiments over real datasets demonstrate that our proposed approach is effective as it outperforms state-of-the-art methods significantly.
Development of a Wall-Climbing Drone Capable of Vertical Soft Landing Using a Tilt-Rotor Mechanism

Wall-climbing drones have many applications, including structural health monitoring of civil structures, such as bridges and high-rise buildings, cleaning of solar panels to improve power generation efficiency, and airplane visual inspections. For these applications, the drone requires a high-payload capacity, and consequently the size and weight of the drone increase. The drone also should not damage the target structures considering the purpose of its mission. Our previous versions of a wall-climbing drone could have high-impact force on the surface where the drone perches and on the platform itself because of the impact caused by a fast pose change and landing speed. In order to overcome this potential risk, a mechanism and a control algorithm for perching on a vertical surface through low-speed pose change are proposed in this paper. The drone platform is based on an X-configuration quadcopter, and a tilt-rotor mechanism is incorporated into the two axes, such that the front thrusters and the rear thrusters are paired. The vertical soft landing mechanism using the tilt-rotors is validated by the experimental tests of the prototype.
Too Many User-Reviews! What Should App Developers Look at First?

Due to the rapid growth in the number of mobile applications (apps) in the past few years, succeeding in mobile app markets has become ruthless. Online app markets, such as Google Play Store, let users rate apps on a five-star scale and leave feedback. Given the importance of high star-ratings to the success of an app, it is crucial to help developers find the key topics of user-reviews that are significantly related to star-ratings of a given category. Having considered the key topics of user-reviews, app developers can narrow down their effort to the user-reviews that matter to be addressed for receiving higher star-ratings. We study 4,193,549 user-reviews of 623 Android apps that were collected from Google Play Store in ten different categories. The results show that few key topics commonly exist across categories, and each category has a specific set of key topics. We also evaluated the identified key topics with respect to the changes that are made to each version of the apps for 19 months. We observed, for 77 percent of the apps, considering the key topics in the next versions shares a significant relationship with increases in star-ratings.
3D Aided Duet GANs for Multi-View Face Image Synthesis

Multi-view face synthesis from a single image is an ill-posed computer vision problem. It often suffers from appearance distortions if it is not well-defined. Producing photo-realistic and identity preserving multi-view results is still a not well-defined synthesis problem. This paper proposes 3D aided duet generative adversarial networks (AD-GAN) to precisely rotate the yaw angle of an input face image to any specified angle. AD-GAN decomposes the challenging synthesis problem into two well-constrained subtasks that correspond to a face normalizer and a face editor. The normalizer first frontalizes an input image, and then the editor rotates the frontalized image to a desired pose guided by a remote code. In the meantime, the face normalizer is designed to estimate a novel dense UV correspondence field, making our model aware of 3D face geometry information. In order to generate photo-realistic local details and accelerate convergence process, the normalizer and the editor are trained in a two-stage manner and regulated by a conditional self-cycle loss and a perceptual loss. Exhaustive experiments on both controlled and uncontrolled environments demonstrate that the proposed method not only improves the visual realism of multi-view synthetic images but also preserves identity information well.
childes-db: A flexible and reproducible interface to the child language data exchange system

The Child Language Data Exchange System (CHILDES) has played a critical role in research on child language development, particularly in characterizing the early language learning environment. Access to these data can be both complex for novices and difficult to automate for advanced users, however. To address these issues, we introduce childes-db, a database-formatted mirror of CHILDES that improves data accessibility and usability by offering novel interfaces, including browsable web applications and an R application programming interface (API). Along with versioned infrastructure that facilitates reproducibility of past analyses, these interfaces lower barriers to analyzing naturalistic parent-child language, allowing for a wider range of researchers in language and cognitive development to easily leverage CHILDES in their work.
Consistency Models of NoSQL Databases

Internet has become so widespread that most popular websites are accessed by hundreds of millions of people on a daily basis. Monolithic architectures, which were frequently used in the past, were mostly composed of traditional relational database management systems, but quickly have become incapable of sustaining high data traffic very common these days. Meanwhile, NoSQL databases have emerged to provide some missing properties in relational databases like the schema-less design, horizontal scaling, and eventual consistency. This paper analyzes and compares the consistency model implementation on five popular NoSQL databases: Redis, Cassandra, MongoDB, Neo4j, and OrientDB. All of which offer at least eventual consistency, and some have the option of supporting strong consistency. However, imposing strong consistency will result in less availability when subject to network partition events.
Measuring presence in video games: An investigation of the potential use of physiological measures as indicators of presence

Presence has become an increasingly central component of Games User Research (GUR) as developments in technology continuously make modern video games more conducive to the sensation of ‘being there’ in virtual environments. The quality of games is now commonly evaluated based on how reliably they elicit presence; however, no standardized objective measure of presence currently exists. This study investigated two physiological measures, Galvanic Skin Response (GSR) and task-irrelevant Event-Related Potentials (ERPs), as potential objective indicators of presence in games. A total of 34 participants were divided into low or high presence groups based on their self-reported presence evoked from experiencing a horror game while task-irrelevant tones were being played. It was hypothesized that presence is associated with attentional resources being fully absorbed by the game, which would lead to less or insufficient perceptual resources available for processing the concurrent game-irrelevant oddball-task. This effect was expected to manifest as a measurable decrease in early ERP component amplitudes. It was also hypothesized that presence would make players react to emotion-eliciting events as if they were real, which would result in more GSR peaks throughout the game while not impacting event response magnitude. ERP components (N1, MMN and SW), GSR peaks/min and response magnitude were compared between the presence groups revealing significant differences in GSR peaks/min and early ERP components of N1 and MMN, but not in GSR response magnitude. The findings support the hypotheses and show that GSR peaks/min, N1 and MMN correlate with presence and have potential as presence indicators.
Dense 3D-Convolutional Neural Network for Person Re-Identification in Videos

Person re-identification aims at identifying a certain pedestrian across non-overlapping multi-camera networks in different time and places. Existing person re-identification approaches mainly focus on matching pedestrians on images; however, little attention has been paid to re-identify pedestrians in videos. Compared to images, video clips contain motion patterns of pedestrians, which is crucial to person re-identification. Moreover, consecutive video frames present pedestrian appearance with different body poses and from different viewpoints, providing valuable information toward addressing the challenge of pose variation, occlusion, and viewpoint change, and so on. In this article, we propose a Dense 3D-Convolutional Network (D3DNet) to jointly learn spatio-temporal and appearance representation for person re-identification in videos. The D3DNet consists of multiple three-dimensional (3D) dense blocks and transition layers. The 3D dense blocks enlarge the receptive fields of visual neurons in both spatial and temporal dimensions, leading to discriminative appearance representation as well as short-term and long-term motion patterns of pedestrians without the requirement of an additional motion estimation module. Moreover, we formulate a loss function consisting of an identification loss and a center loss to minimize intra-class variance and maximize inter-class variance simultaneously, toward addressing the challenge of large intra-class variance and small inter-class variance. Extensive experiments on two real-world video datasets of person identification, i.e., MARS and iLIDS-VID, have shown the effectiveness of the proposed approach.
User-Video Co-Attention Network for Personalized Micro-video Recommendation

With the increasing popularity of micro-video sharing where people shoot short-videos effortlessly and share their daily stories on social media platforms, the micro-video recommendation has attracted extensive research efforts to provide users with micro-videos that interest them. In this paper, a hypothesis we explore is that, not only do users have multi-modal interest, but micro-videos have multi-modal targeted audience segments. As a result, we propose a novel framework User-Video Co-Attention Network (UVCAN), which can learn multi-modal information from both user and microvideo side using attention mechanism. In addition, UVCAN reasons about the attention in a stacked attention network fashion for both user and micro-video. Extensive experiments on two datasets collected from Toffee present superior results of our proposed UVCAN over the state-of-the-art recommendation methods, which demonstrate the effectiveness of the proposed framework.
An Efficient Framework for Sentence Similarity Modeling

Sentence similarity modeling lies at the core of many natural language processing applications, and thus has received much attention. Owing to the success of word embeddings, recently, popular neural network methods achieved sentence embedding. Most of them focused on learning semantic information and modeling it as a continuous vector, yet the syntactic information of sentences has not been fully exploited. On the other hand, prior works have shown the benefits of structured trees that include syntactic information, while few methods in this branch utilized the advantages of word embeddings and another powerful technique-attention weight mechanism. This paper suggests to absorb their advantages by merging these techniques in a unified structure, dubbed as attention constituency vector tree (ACVT). Meanwhile, this paper develops a new tree kernel, known as ACVT kernel, which is tailored for sentence similarity measure based on the proposed structure. The experimental results, based on 19 widely used semantic textual similarity datasets, demonstrate that our model is effective and competitive, when compared against state-of-the-art models. Additionally, the experimental results validate that many attention weight mechanisms and word embedding techniques can be seamlessly integrated into our model, demonstrating the robustness and universality of our model.
A secure image encryption scheme based on cellular automata and chaotic skew tent map

This paper presents a highly secure image encryption scheme for secure image communication and storage. The scheme is based on a chaotic skew tent map and cellular automata (CA). The chaotic skew tent map is used as the initial vector generator for the CA which needs a 128 bit initial bit sequence to generate pseudo random number sequence (PRNS). Using the PRNS, the plain image pixels are first permuted to remove the high correlation among the adjacent pixels in the image. Then the permuted image is encrypted based on a single random number generated by the chaotic skew tent map. The chaotic skew tent map has a very large key space, on the other hand, CA generates PRNS faster than a chaotic pseudo random number generator (PRNG). Therefore, a combination of both chaotic map and CA gives a system with higher key space and faster PRNS generator. The experimental results show a good encryption effects which are capable to resist any kind of known attacks.
Effects of Increasing the Footprints of Uncertainty on Analytical Structure of the Classes of Interval Type-2 Mamdani and TS Fuzzy Controllers

The fundamental difference between an interval type-2 (IT2) fuzzy controller and a type-1 fuzzy controller is the footprint of uncertainty (FOU) of an IT2 fuzzy set. In this paper, we study how FOUs affect the analytical structure (i.e., the input-output mathematical relationship) of a broad class of IT2 Mamdani and takagi-sugeno (TS) controllers. The controllers employ arbitrary fuzzy rules, the Karnik-Mendel (KM) or Enhanced KM type-reducer, the minimum AND operator, and the centroid defuzzifier. The controllers utilize commonly used IT2 fuzzy sets for their input variables and any kind of type-2 fuzzy sets for their output variable. We prove that, with increase of FOUs of the input fuzzy sets, the Mamdani controllers approach constant controllers, and the TS controllers approach piecewise linear controllers. The resemblance to the constant or pricewise linear controllers increases as the FOUs increase. When all the FOUs are at their maximum (to reflect the highest level of uncertainties), the Mamdani and TS controllers become the constant controllers and piecewise linear controllers, respectively. We investigate how change in the resemblance takes place progressively as FOUs increase. We also show that an increase in the resemblance narrows control gain variations for part of the IT2 controllers, which can worsen control performance. These findings implicit controller design-too large FOUs are generally undesirable for the input fuzzy sets because they can make an IT2 controller behave like a constant or piecewise linear controller. Real-time control experiment results are provided to illustrate the theoretical analysis.
Convolutional Neural Network Implementation for Eye-Gaze Estimation on Low-Quality Consumer Imaging Systems

Accurate and efficient eye gaze estimation is important for emerging consumer electronic systems, such as driver monitoring systems and novel user interfaces. Such systems are required to operate reliably in difficult, unconstrained environments with low power consumption and at minimal cost. In this paper, a new hardware friendly, convolutional neural network (CNN) model with minimal computational requirements is introduced and assessed for efficient appearance-based gaze estimation. The model is tested and compared against existing appearance-based CNN approaches, achieving better eye gaze accuracy with significantly fewer computational requirements.
A Two-Tier System for On-Demand Streaming of 360 Degree Video Over Dynamic Networks

360° video on-demand streaming is a key component of the emerging virtual reality and augmented reality applications. In such applications, sending the entire 360° video demands extremely high network bandwidth that may not be affordable by today's networks. On the other hand, sending only the predicted user's field of view (FoV) is not viable as it is hard to achieve perfect FoV prediction in on-demand streaming, where it is better to prefetch the video multiple seconds ahead, to absorb the network bandwidth fluctuation. This paper proposes a twotier solution, where the base tier delivers the entire 360° span at a lower quality with a long prefetching buffer, and the enhancement tier delivers the predicted FoV at a higher quality using a short buffer. The base tier provides robustness to both network bandwidth variations and FoV prediction errors. The enhancement tier improves the video quality if it is delivered in time and FoV prediction is accurate. We study the optimal rate allocation between the two tiers and buffer provisioning for the enhancement tier to achieve the optimal trade-off between video quality and streaming robustness. We also design periodic and adaptive optimization frameworks to adapt to the bandwidth variations and FoV prediction errors in realtime. Through simulations driven by real LTE and WiGig network bandwidth traces and user FoV traces, we demonstrate that the proposed two-tier systems can achieve a high-level of quality-of-experience in the face of network bandwidth and user FoV dynamics.
Fraud detection for E-commerce transactions by employing a prudential Multiple Consensus model

More and more financial transactions through different E-commerce platforms have appeared now-days within the big data era bringing plenty of opportunities but also challenges and risks of stealing information for potential frauds that need to be faced. This is due to the massive use of tools such as credit cards for electronic payments which are targeted by attackers to steal sensitive information and perform fraudulent operations. Although intelligent fraud detection systems have been developed to face the problem, they still suffer from some well-known problems due to the imbalance of the used data. Therefore this paper proposes a novel data intelligence technique based on a Prudential Multiple Consensus model which combines the effectiveness of several state-of-the-art classification algorithms by adopting a twofold criterion, probabilistic and majority based. The goal is to maximize the effectiveness of the model in detecting fraudulent transactions regardless the presence of any data imbalance. Our model has been validated with a set of experiments on a large real-world dataset characterized by a high degree of data imbalance and results show how the proposed model outperforms several state-of-the-art solutions, both in terms of ensemble models and classification approaches.