Reviewerly/exHarmony huggingface dataset:
METIER

Activity recognition (AR) and user recognition (UR) using wearable sensors are two key tasks in ubiquitous and mobile computing. Currently, they still face some challenging problems. For one thing, due to the variations in how users perform activities, the performance of a well-trained AR model typically drops on new users. For another, existing UR models are powerless to activity changes, as there are significant differences between the sensor data in different activity scenarios. To address these problems, we propose METIER (deep multi-task learning based activity and user recognition) model, which solves AR and UR tasks jointly and transfers knowledge across them. User-related knowledge from UR task helps AR task to model user characteristics, and activity-related knowledge from AR task guides UR task to handle activity changes. METIER softly shares parameters between AR and UR networks, and optimizes these two networks jointly. The commonalities and differences across tasks are exploited to promote AR and UR tasks simultaneously. Furthermore, mutual attention mechanism is introduced to enable AR and UR tasks to exploit their knowledge to highlight important features for each other. Experiments are conducted on three public datasets, and the results show that our model can achieve competitive performance on both tasks.
Multimodal Deep Networks for Text and Image-Based Document Classification

Classification of document images is a critical step for accelerating archival of old manuscripts, online subscription and administrative procedures. Computer vision and deep learning have been suggested as a first solution to classify documents based on their visual appearance. However, achieving the fine-grained classification that is required in real-world setting cannot be achieved by visual analysis alone. Often, the relevant information is in the actual text content of the document, although this text is not available in digital form. In this work, we introduce a novel pipeline based on off-the-shelf architectures to deal with document classification by taking into account both text and visual information. We design a multimodal neural network that is able to learn both the image and from word embeddings, computed on noisy text extracted by OCR. We show that this approach allows us to improve single-modality classification accuracy by several points on the small Tobacco3482 and large RVL-CDIP datasets, even without clean text information. We release a post-OCR text classification ( https://github.com/Quicksign/ocrized-text-dataset ) that complements the Tobacco3482 and RVL-CDIP ones to encourage researchers to look into multi-modal text/image classification.
Challenges and Opportunities for Efficient Serverless Computing at the Edge

Serverless computing frameworks allow users to execute a small application (dedicated to a specific task) without handling operational issues such as server provisioning, resource management, and resource scaling for the increased load. Serverless computing originally emerged as a Cloud computing framework, but might be a perfect match for IoT data processing at the Edge. However, the existing serverless solutions, based on VMs and containers, are too heavy-weight (large memory footprint and high function invocation time) for operating efficiency and elastic scaling at the Edge. Moreover, many novel IoT applications require low-latency data processing and near real-time responses, which makes the current cloud-based serverless solutions unsuitable. Recently, WebAssembly (Wasm) has been proposed as an alternative method for running serverless applications at near-native speeds, while having a small memory footprint and optimized invocation time. In this paper, we discuss some existing serverless solutions, their design details, and unresolved performance challenges for an efficient serverless management at the Edge. We outline our serverless framework, called aWsm, based on the WebAssembly approach, and discuss the opportunities enabled by the aWsm design, including function profiling and SLO-driven performance management of users' functions. Finally, we present an initial assessment of aWsm performance featuring average startup time (12μs to 30μs) and an economical memory footprint (ranging from 10s to 100s of kB) for a subset of MiBench microbenchmarks used as functions.
SieveNet: A Unified Framework for Robust Image-Based Virtual Try-On

Image-based virtual try-on for fashion has gained considerable attention recently. The task requires trying on a clothing item on a target model image. An efficient framework for this is composed of two stages: (1) warping (transforming) the try-on cloth to align with the pose and shape of the target model, and (2) a texture transfer module to seamlessly integrate the warped try-on cloth onto the target model image. Existing methods suffer from artifacts and distortions in their try-on output. In this work, we present Sieve Net, a framework for robust image-based virtual try-on. Firstly, we introduce a multi-stage coarse-to-fine warping network to better model fine grained intricacies (while transforming the try-on cloth) and train it with a novel perceptual geometric matching loss. Next, we introduce a try-on cloth conditioned segmentation mask prior to improve the texture transfer network. Finally, we also introduce a duelling triplet loss strategy for training the texture translation network which further improves the quality of generated try-on result. We present extensive qualitative and quantitative evaluations of each component of the proposed pipeline and show significant performance improvements against the current state-of-the-art method.
Three-dimensional entanglement on a silicon chip

Entanglement is a counterintuitive feature of quantum physics that is at the heart of quantum technology. High-dimensional quantum states offer unique advantages in various quantum information tasks. Integrated photonic chips have recently emerged as a leading platform for the generation, manipulation and detection of entangled photons. Here, we report a silicon photonic chip that uses novel interferometric resonance-enhanced photon-pair sources, spectral demultiplexers and high-dimensional reconfigurable circuitries to generate, manipulate and analyse path-entangled three-dimensional qutrit states. By minimizing on-chip electrical and thermal cross-talk, we obtain high-quality quantum interference with visibilities above 96.5% and a maximumly entangled qutrit state with a fidelity of 95.5%. We further explore the fundamental properties of entangled qutrits to test quantum nonlocality and contextuality, and to implement quantum simulations of graphs and high-precision optical phase measurements. Our work paves the path for the development of multiphoton high-dimensional quantum technologies.
LGLMF: Local Geographical Based Logistic Matrix Factorization Model for POI Recommendation

With the rapid growth of Location-Based Social Networks, personalized Points of Interest (POIs) recommendation has become a critical task to help users explore their surroundings. Due to the scarcity of check-in data, the availability of geographical information offers an opportunity to improve the accuracy of POI recommendation. Moreover, matrix factorization methods provide effective models which can be used in POI recommendation. However, there are two main challenges which should be addressed to improve the performance of POI recommendation methods. First, leveraging geographical information to capture both the user's personal, geographic profile and a location's geographic popularity. Second, incorporating the geographical model into the matrix factorization approaches. To address these problems, a POI recommendation method is proposed in this paper based on a Local Geographical Model, which considers both users' and locations' points of view. To this end, an effective geographical model is proposed by considering the user's main region of activity and the relevance of each location within that region. Then, the proposed local geographical model is fused into the Logistic Matrix Factorization to improve the accuracy of POI recommendation. Experimental results on two well-known datasets demonstrate that the proposed approach outperforms other state-of-the-art POI recommendation methods.
An improved opposition based learning firefly algorithm with dragonfly algorithm for solving continuous optimization problems

Nowadays, the existence of continuous optimization problems has led researchers to come up with a variety of methods to solve continues optimization problems. The metaheuristic algorithms are one of the most popular and common ways to solve continuou
Social Boosted Recommendation With Folded Bipartite Network Embedding

With the prevalence of online social platforms, social recommendation has emerged as a promising direction that leverages the social network among users to enhance recommendation performance. However, the available social relations among users are usually extremely sparse and noisy, which may lead to inferior recommendation performance. To alleviate this problem, this paper novelly exploits the implicit higher-order social influence and dependencies among users to enhance social recommendation. In this paper, we propose a novel embedding method for general bipartite graphs, which defines inter-class message passing between explicit relations and intra-class message passing between implicit higher-order relations via a novel sequential modelling paradigm. Inspired by recent advances in self-attention-based sequential modelling, the proposed model features a self-attentive representation learning mechanism for implicit user-user relations. Moreover, this paper also explores the inductive embedding learning for social recommendation problems to improve the recommendation performance in cold-start settings. The proposed inductive learning paradigm for social recommendation enables embedding inference for those cold-start users and items (unseen during training) as long as they are linked to existing nodes in the original network. Extensive experiments on real-world datasets demonstrate the superiority of our method and suggest that higher-order implicit relationship among users is beneficial to improving social recommendation.
Inverse design of an integrated-nanophotonics optical neural network

Artificial neural networks have dramatically improved the performance of many machine-learning applications such as image recognition and natural language processing. However, the electronic hardware implementations of the above-mentioned tasks are facing performance ceiling because Moore's Law is slowing down. In this article, we propose an optical neural network architecture based on optical scattering units to implement deep learning tasks with fast speed, low power consumption and small footprint. The optical scattering units allow light to scatter back and forward within a small region and can be optimized through an inverse design method. The optical scattering units can implement high-precision stochastic matrix multiplication with mean squared error <10-4 and a mere 4 × 4 μm2 footprint. Furthermore, an optical neural network framework based on optical scattering units is constructed by introducing "Kernel Matrix", which can achieve 97.1% accuracy on the classic image classification dataset MNIST.
Generating Synthetic Audio Data for Attention-Based Speech Recognition Systems

Recent advances in text-to-speech (TTS) led to the development of flexible multi-speaker end-to-end TTS systems. We extend state-of-the-art attention-based automatic speech recognition (ASR) systems with synthetic audio generated by a TTS system trained only on the ASR corpora itself. ASR and TTS systems are built separately to show that text-only data can be used to enhance existing end-to-end ASR systems without the necessity of parameter or architecture changes. We compare our method with language model integration of the same text data and with simple data augmentation methods like SpecAugment and show that performance improvements are mostly independent. We achieve improvements of up to 33% relative in word-error-rate (WER) over a strong baseline with data-augmentation in a low-resource environment (LibriSpeech-100h), closing the gap to a comparable oracle experiment by more than 50%. We also show improvements of up to 5% relative WER over our most recent ASR baseline on LibriSpeech-960h.
Gated Mechanism for Attention Based Multi Modal Sentiment Analysis

Multimodal sentiment analysis has recently gained popularity because of its relevance to social media posts, customer service calls and video blogs. In this paper, we address three aspects of multimodal sentiment analysis; 1. Cross modal interaction learning, i.e. how multiple modalities contribute to the sentiment, 2. Learning long-term dependencies in multimodal interactions and 3. Fusion of unimodal and cross modal cues. Out of these three, we find that learning cross modal interactions is beneficial for this problem. We perform experiments on two benchmark datasets, CMU Multimodal Opinion level Sentiment Intensity (CMU-MOSI) and CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) corpus. Our approach on both these tasks yields accuracies of 83.9% and 81.1% respectively, which is 1.6% and 1.34% absolute improvement over current state-of-the-art.
HandVoxNet: Deep Voxel-Based Network for 3D Hand Shape and Pose Estimation From a Single Depth Map

3D hand shape and pose estimation from a single depth map is a new and challenging computer vision problem with many applications. The state-of-the-art methods directly regress 3D hand meshes from 2D depth images via 2D convolutional neural networks, which leads to artefacts in the estimations due to perspective distortions in the images. In contrast, we propose a novel architecture with 3D convolutions trained in a weakly-supervised manner. The input to our method is a 3D voxelized depth map, and we rely on two hand shape representations. The first one is the 3D voxelized grid of the shape which is accurate but does not preserve the mesh topology and the number of mesh vertices. The second representation is the 3D hand surface which is less accurate but does not suffer from the limitations of the first representation. We combine the advantages of these two representations by registering the hand surface to the voxelized hand shape. In the extensive experiments, the proposed approach improves over the state of the art by47.8% on the SynHand5M dataset. Moreover, our augmentation policy for voxelized depth maps further enhances the accuracy of 3D hand pose estimation on real data. Our method produces visually more reasonable and realistic hand shapes on NYU and BigHand2.2M datasets compared to the existing approaches.
Sarcasm Detection with Sentiment Semantics Enhanced Multi-level Memory Network

Sarcasm detection is a challenging natural language processing task for sentiment analysis. Existing deep learning based sarcasm detection models have not fully considered sentiment semantics, even though sentiment semantics is necessary to improve the performance of sarcasm detection. To deal with the problem, we propose a multi-level memory network using sentiment semantics to capture the features of sarcasm expressions. In our model, we use the first-level memory network to capture sentiment semantics, and use the second-level memory network to capture the contrast between sentiment semantics and the situation in each sentence. Moreover, we use an improved convolutional neural network to improve the memory network in the absence of local information. The experimental results on the Internet Argument Corpus (IAC-V1 and IAC-V2) and Twitter dataset demonstrate the effectiveness of our model.
Correction of Automatic Speech Recognition with Transformer Sequence-To-Sequence Model

In this work, we introduce a simple yet efficient post-processing model for automatic speech recognition. Our model has Transformer-based encoder-decoder architecture which "translates" acoustic model output into grammatically and semantically correct text. We investigate different strategies for regularizing and optimizing the model and show that extensive data augmentation and the initialization with pretrained weights are required to achieve good performance. On the LibriSpeech benchmark, our method demonstrates significant improvement in word error rate over the baseline acoustic model with greedy decoding, especially on much noisier dev-other and test-other portions of the evaluation dataset. Our model also outperforms baseline with 6-gram language model re-scoring and approaches the performance of re-scoring with Transformer-XL neural language model.
Full-immersion virtual reality: Adverse effects related to static balance

The use of virtual reality (VR) is associated with several adverse effects including dizziness, headache, and motion sickness. This study investigates how full-immersion VR games cause changes in static balance with associated adverse effects, and whether a fixed or a changing game background is more likely to contribute to such problems. Static balance and adverse effects (eye fatigue and dizziness) were measured in 15 healthy adults under three conditions: baseline; after a full-immersion virtual reality game (PlayStation 4 Pro and PlayStation® VR headset) with a fixed background (15 min); and after a full-immersion virtual reality game with a unfixed background (15 min). Static balance was measured with an AMTI force plate, while eye fatigue and dizziness were measured with the Virtual Reality Symptom Questionnaire (VRSQ) and the Simulator Sickness Questionnaire (SSQ). It was determined that playing a full-immersion VR game had a negative effect on static balance and produced several adverse effects including eye fatigue and dizziness. Moreover, sway velocity and sway length increased significantly in the game with a moving background compared to both the baseline and the game with a fixed background (p < 0.05); VRSQ and SSQ were also significantly higher in this case. It is thus preferable from the perspective of reducing adverse effects that only fixed-background full-immersion VR games be used in rehabilitative interventions.
Learning Formation of Physically-Based Face Attributes

Based on a combined data set of 4000 high resolution facial scans, we introduce a non-linear morphable face model, capable of producing multifarious face geometry of pore-level resolution, coupled with material attributes for use in physically-based rendering. We aim to maximize the variety of the participant’s face identities, while increasing the robustness of correspondence between unique components, including middle-frequency geometry, albedo maps, specular intensity maps and high-frequency displacement details. Our deep learning based generative model learns to correlate albedo and geometry, which ensures the anatomical correctness of the generated assets. We demonstrate potential use of our generative model for novel identity generation, model fitting, interpolation, animation, high fidelity data visualization, and low-to-high resolution data domain transferring. We hope the release of this generative model will encourage further cooperation between all graphics, vision, and data focused professionals, while demonstrating the cumulative value of every individual’s complete biometric profile.
Aligntts: Efficient Feed-Forward Text-to-Speech System Without Explicit Alignment

Targeting at both high efficiency and performance, we propose AlignTTS to predict the mel-spectrum in parallel. AlignTTS is based on a Feed-Forward Transformer which generates mel-spectrum from a sequence of characters, and the duration of each character is determined by a duration predictor. Instead of adopting the attention mechanism in Transformer TTS to align text to mel-spectrum, the alignment loss is presented to consider all possible alignments in training by use of dynamic programming. Experiments on the LJSpeech dataset show that our model achieves not only state-of-the-art performance which outperforms Transformer TTS by 0.03 in mean option score (MOS), but also a high efficiency which is more than 50 times faster than real-time.
Multi-timescale Forecast of Solar Irradiance Based on Multi-task Learning and Echo State Network Approaches

Solar irradiance forecast is closely related with efficiency and reliability of renewable energy systems. Multi-timescale irradiance forecast is a new and efficient way to simultaneously predict solar energy generation on different timescales for hierarchical decision making. This article newly adopts the multi-task learning mechanism to study the multi-timescale forecast for improving accuracy and computational efficiency. A novel multi-timescale (MTS) prediction framework is presented to fulfill the multi-task application, and echo state network (ESN) is studied in the proposed MTS framework. The multi-timescale ESN (MTS-ESN) is proposed to enhance the information sharing among correlated tasks. Simulation results of hourly solar data demonstrate that the proposed MTS-ESN could achieve promising performance at both hourly and daily level in parallel. The MTS-ESN outperforms the single-timescale ESN (STS-ESN), which indicates the information sharing in the multi-task learning is effective in this application.
Programmatic creative: AI can think but it cannot feel

The authors conduct two studies to examine artificial intelligence (AI) in creative programmatic contexts. Findings indicate that consumers have equally favorable attitudes toward human-created and Al-created cognitive-oriented advertising, but form lower evaluations of Al-created emotion-oriented creative content. Programmatic creative ads are effective for rational appeals and utilitarian products, but are ineffective for emotional appeals and hedonic products. The studies indicate that human rather than AI input is needed for creating emotion-oriented advertisements.
Density estimation using deep generative neural networks

Density estimation is one of the fundamental problems in both statistics and machine learning. In this study, we propose Roundtrip, a computational framework for general-purpose density estimation based on deep generative neural networks. Roundtrip retains the generative power of deep generative models, such as generative adversarial networks (GANs) while it also provides estimates of density values, thus supporting both data generation and density estimation. Unlike previous neural density estimators that put stringent conditions on the transformation from the latent space to the data space, Roundtrip enables the use of much more general mappings where target density is modeled by learning a manifold induced from a base density (e.g., Gaussian distribution). Roundtrip provides a statistical framework for GAN models where an explicit evaluation of density values is feasible. In numerical experiments, Roundtrip exceeds state-of-the-art performance in a diverse range of density estimation tasks.
Offline Signature Recognition and Forgery Detection using Deep Learning

Authentication plays a very important role to manage security. In the modern era, it is one, in all the priorities. With the appearance of technology, the interaction with machines is turning automatic. Therefore, the need of authentication increases rapidly for various security purposes. Because of this, the biometric-based authentication has gained a drastic momentum. It is a kind of boon over other techniques. However, this event is not a replacement of drawback but varied ways are adopted to verify folks. Signature is one of the first broadly practiced biometric features for the verification of an individual. This paper proposes a method for the pre-processing of signatures to make verification simple. It also proposed a novel method for signature recognition and signature forgery detection with verification using Convolution Neural Network (CNN), Crest-Trough method and SURF algorithm & Harris corner detection algorithm. The proposed system attains an accuracy of 85-89% for forgery detection and 90-94% for signature recognition.