Reviewerly/exHarmony huggingface dataset:
Long Short-Term Temporal Meta-learning in Online Recommendation

An effective online recommendation system should jointly capture users' long-term and short-term preferences in both users' internal behaviors (from the target recommendation task) and external behaviors (from other tasks). However, it is extremely challenging to conduct fast adaptations to real-time new trends while making full use of all historical behaviors in large-scale systems, due to the real-world limitations in real-time training efficiency and external behavior acquisition. To address these practical challenges, we propose a novel Long Short-Term Temporal Meta-learning framework (LSTTM) for online recommendation. It arranges user multi-source behaviors in a global long-term graph and an internal short-term graph, and conducts different GAT-based aggregators and training strategies to learn user short-term and long-term preferences separately. To timely capture users' real-time interests, we propose a temporal meta-learning method based on MAML under an asynchronous optimization strategy for fast adaptation, which regards recommendations at different time periods as different tasks. In experiments, LSTTM achieves significant improvements on both offline and online evaluations. It has been deployed on a widely-used online recommendation system named WeChat Top Stories, affecting millions of users.
Single-layer vision transformers for more accurate early exits with less overhead

Deploying deep learning models in time-critical applications with limited computational resources, for instance in edge computing systems and IoT networks, is a challenging task that often relies on dynamic inference methods such as early exiting. In this paper, we introduce a novel architecture for early exiting based on the vision transformer architecture, as well as a fine-tuning strategy that significantly increase the accuracy of early exit branches compared to conventional approaches while introducing less overhead. Through extensive experiments on image and audio classification as well as audiovisual crowd counting, we show that our method works for both classification and regression problems, and in both single- and multi-modal settings. Additionally, we introduce a novel method for integrating audio and visual modalities within early exits in audiovisual data analysis, that can lead to a more fine-grained dynamic inference.
Distance Perception with a Video See-Through Head-Mounted Display

In recent years, pass-through cameras have resurfaced as inclusions for virtual reality (VR) hardware. With modern cameras that now have increased resolution and frame rate, Video See-Through (VST) Head-Mounted Displays (HMD) can be used to provide an Augmented Reality (AR) experience. However, because users see their surroundings through video capture and HMD lenses, there is question surrounding how people perceive their environment with these devices. We conducted a user study with 26 participants to help understand if distance perception is altered when viewing surroundings with a VST HMD. Although previous work shows that distance estimation in VR with an HTC Vive is comparable to that in the real world, our results show that the inclusion of a ZED Mini pass-through camera causes a significant difference between normal, unrestricted viewing and that through a VST HMD.
CoreGen: Contextualized Code Representation Learning for Commit Message Generation

Automatic generation of high-quality commit messages for code commits can substantially facilitate software developers’ works and coordination. However, the semantic gap between source code and natural language poses a major challenge for the task. Several studies have been proposed to alleviate the challenge but none explicitly involves code contextual information during commit message generation. Specifically, existing research adopts static embedding for code tokens, which maps a token to the same vector regardless of its context. In this paper, we propose a novel Contextualized code representation learning strategy for commit message Generation (CoreGen). CoreGen first learns contextualized code representations which exploit the contextual information behind code commit sequences. The learned representations of code commits built upon Transformer are then fine-tuned for downstream commit message generation. Experiments on the benchmark dataset demonstrate the superior effectiveness of our model over the baseline models with at least 28.18% improvement in terms of BLEU-4 score. Furthermore, we also highlight the future opportunities in training contextualized code representations on larger code corpus as a solution to low-resource tasks and adapting the contextualized code representation framework to other code-to-text generation tasks.
Robust and Secure Digital Image Watermarking Technique Using Arnold Transform and Memristive Chaotic Oscillators

With the advent of technology and multimedia production, the world has witnessed a tremendous increase in digital media attacks, which duplicates, forges and tamper the data leading to the violation of copyright laws. In this paper, a robust and secure digital image watermarking is proposed, which exploits the chaotic behaviour of the non - linear oscillators realized through Memristive diodes. The proposed scheme relies on a Human Visual System (HVS) model in order to mimic the real-life scenario. To improve the robustness of the proposed approach and to further increase the security of the digital watermarked media whilst still retaining compatibility with the real-time events, Histogram of Oriented Gradients (HOG) and extreme learning machine (ELM) is implemented. Secure key generation by means of scrambling through Arnold Transform and the coefficients of Memristive Chaotic Oscillator ensures extreme security. The watermark embedding followed the pixel transformation based on discrete cosine coefficient modification, and a semi-blind watermarking extraction procedure was carried out through trained ELM models. A detailed analysis has been presented to evaluate the tradeoff between imperceptibility, security and robustness using performance metrics like PSNR, NC, SSIM, and BER. To establish a real-time implementation of the proposed architecture, the simulated results were verified using real-time chaotic signals generated from the chaotic oscillator, which dictates excellent performance against watermarking attacks and image processing tasks.
Teacher-Student Training and Triplet Loss for Facial Expression Recognition under Occlusion

In this paper, we study the task of facial expression recognition under strong occlusion. We are particularly interested in cases where 50 % of the face is occluded, e.g. when the subject wears a Virtual Reality (VR) headset. While previous studies show that pre-training convolutional neural networks (CNNs) on fully-visible (non-occluded) faces improves the accuracy, we propose to employ knowledge distillation to achieve further improvements. First of all, we employ the classic teacher-student training strategy, in which the teacher is a CNN trained on fully-visible faces and the student is a CNN trained on occluded faces. Second of all, we propose a new approach for knowledge distillation based on triplet loss. During training, the goal is to reduce the distance between an anchor embedding, produced by a student CNN that takes occluded faces as input, and a positive embedding (from the same class as the anchor), produced by a teacher CNN trained on fully-visible faces, so that it becomes smaller than the distance between the anchor and a negative embedding (from a different class than the anchor), produced by the student CNN. Third of all, we propose to combine the distilled embeddings obtained through the classic teacher-student strategy and our novel teacher-student strategy based on triplet loss into a single embedding vector. We conduct experiments on two benchmarks, FER+ and AffectNet, with two CNN architectures, VGG-f and VGG-face, showing that knowledge distillation can bring significant improvements over the state-of-the-art methods designed for occluded faces in the VR setting. Furthermore, we obtain accuracy rates that are quite close to the state-of-the-art models that take as input fully-visible faces. For example, on the FER+ data set, our VGG-face based on concatenated distilled embeddings attains an accuracy rate of 82.75% on lower-half-visible faces, which is only 2.24% below the accuracy rate of a state-of-the-art VGG-13 that is evaluated on fully-visible faces. Given that our model sees only the lower-half of the face, we consider this to be a remarkable achievement. In conclusion, we consider that our distilled CNN models can provide useful feedback for the task of recognizing the facial expressions of a person wearing a VR headset.
What Can HCI Learn from Sexual Consent?

Sexual consent has undergone a transformation toward an "enthusiastic" feminist model that emphasizes consent as an ongoing and voluntary process of negotiation and affirmation. This paper considers how such a model can advance understandings of consent in HCI research and design in relation to embodied interactions with emerging technologies that also occur outside of sexual interactions. We apply the popular "FRIES" model of sexual consent (Freely given, Reversible, Informed, Enthusiastic and Specific) to three areas of embodied interaction: 1) bodily-play interactions, 2) persuasive interactions with smart technologies, and 3) intimate interactions with anthropomorphized devices. Based on erotic play practices, we contribute a "TEASE" process guideline (Traffic lights, Establish ongoing dialogue, Aftercare, Safewords, and Explicate soft/hard limits) to advance consensual practice in HCI and develop implementation scenarios.
Knowledge Distillation for Action Anticipation via Label Smoothing

Human capability to anticipate near future from visual observations and non-verbal cues is essential for developing intelligent systems that need to interact with people. Several research areas, such as human-robot interaction (HRI), assisted living or autonomous driving need to foresee future events to avoid crashes or help people. Egocentric scenarios are classic examples where action anticipation is applied due to their numerous applications. Such challenging task demands to capture and model domain's hidden structure to reduce prediction uncertainty. Since multiple actions may equally occur in the future, we treat action anticipation as a multi-label problem with missing labels extending the concept of label smoothing. This idea resembles the knowledge distillation process since useful information is injected into the model during training. We implement a multi-modal framework based on long short-term memory (LSTM) networks to summarize past observations and make predictions at different time steps. We perform extensive experiments on EPIC-Kitchens and EGTEA Gaze+ datasets including more than 2500 and 100 action classes, respectively. The experiments show that label smoothing systematically improves performance of state-of-the-art models for action anticipation.
PHONI: Streamed Matching Statistics with Multi-Genome References

Computing the matching statistics of patterns with respect to a text is a fundamental task in bioinformatics, but a formidable one when the text is a highly compressed genomic database. Bannai et al. gave an efficient solution for this case, which Rossi et al. recently implemented, but it uses two passes over the patterns and buffers a pointer for each character during the first pass. In this paper, we simplify their solution and make it streaming, at the cost of slowing it down slightly. This means that, first, we can compute the matching statistics of several long patterns (such as whole human chromosomes) in parallel while still using a reasonable amount of RAM; second, we can compute matching statistics online with low latency and thus quickly recognize when a pattern becomes incompressible relative to the database. Our code is available at https://github.com/koeppl/phoni.
Fast and Accurate Real-Time Semantic Segmentation with Dilated Asymmetric Convolutions

Recent works have shown promising results applied to real-time semantic segmentation tasks. To maintain fast inference speed, most of the existing networks make use of light decoders, or they simply do not use them at all. This strategy helps to maintain a fast inference speed; however, their accuracy performance is significantly lower in comparison to non-real-time semantic segmentation networks. In this paper, we introduce two key modules aimed to design a high-performance decoder for real-time semantic segmentation for reducing the accuracy gap between real-time and non-real-time segmentation networks. Our first module, Dilated Asymmetric Pyramidal Fusion (DAPF), is designed to substantially increase the receptive field on the top of the last stage of the encoder, obtaining richer contextual features. Our second module, Multi-resolution Dilated Asymmetric (MDA) module, fuses and refines detail and contextual information from multi-scale feature maps coming from early and deeper stages of the network. Both modules exploit contextual information without excessively increasing the computational complexity by using asymmetric convolutions. Our proposed network entitled “FASSD-Net” reaches 78.8 % of mIoU accuracy on the Cityscapes validation dataset at 41.1 FPS on full resolution images (1024 x 2048). Besides, with a light version of our network, we reach 74.1 % of mIoU at 133.1 FPS (full resolution) on a single NVIDIA GTX 1080Ti card with no additional acceleration techniques. The source code and pre-trained models are available at github.com/GibranBenitez/FASSD- Net.
Detecting Alzheimer’s Disease from Speech Using Neural Networks with Bottleneck Features and Data Augmentation

This paper presents a method of detecting Alzheimer’s disease (AD) from the spontaneous speech of subjects in a picture description task using neural networks. This method does not rely on the manual transcriptions and annotations of a subject’s speech, but utilizes the bottleneck features extracted from audio using an ASR model. The neural network contains convolutional neural network (CNN) layers for local context modeling, bidirectional long shortterm memory (BiLSTM) layers for global context modeling and an attention pooling layer for classification. Furthermore, a masking- based data augmentation method is designed to deal with the data scarcity problem. Experiments on the DementiaBank dataset show that the detection accuracy of our proposed method is 82.59%, which is better than the baseline method based on manually-designed acoustic features and support vector machines (SVM), and achieves the state-of-the-art performance of detecting AD using only audio data on this dataset.
Microscopic printing analysis and application for classification of source printer

Identifying a forged printed document with scanned evidence can be a challenge. Microscopic printing is showing random shape which depends on the printing source as well as printing material. This paper presents a statistical analysis of the printing patterns under a microscopic scale, analyses the effect of printing direction, printing substrate (uncoated and coated paper), and printing technology (conventional offset, waterless offset, and electrophotography). The analysis shows a negligible effect of printing direction, yet, using the shape descriptor indexes, the printing materials and technologies are distinguishable under a microscopic scale. As a result, the algorithms based on Support Vectors Machine and Random Forest are developed, with shape descriptor indexes as features, for printing source identification. Both proposed algorithms, equally, achieve a high classification accuracy rate, over 92% accuracy with complex geometric-shape patterns. Thanks to the lightweight and efficiency of the Support Vectors Machine, the study shows promising applications for real-world and potential implementation in the Internet of Things devices.
Pose-guided part matching network via shrinking and reweighting for occluded person re-identification

Occluded person re-identification (ReID) is a challenging task, which aims at retrieving an occluded person across multiple non-overlapping cameras. To address this issue, we propose a novel framework named Shrinking and Reweighting Network (SRNet) that jointly learns global features by shrinking and reweights part features for matching in an end-to-end framework. Specifically, we use a strong backbone that combines some effective designs and training tricks to learn the robust and discriminative global features. Even so, there exist noise-related features due to the occlusion, so we utilize the Deep Residual Shrinkage Module (DRS Module) to eliminate unimportant features by automatically determining the soft thresholds. When aligning two groups of part features from two images, we view it as a graph matching problem and design an effectively Reweight Module for Part Matching (RMPM) to learn self-adaptive weights for part features before the part matching stage, the proposed RMPM can alleviate the influence of meaningless part features in the part matching stage. Eventually, extensive experimental results on occluded, partial, and holistic re-id datasets clearly demonstrate that the proposed method achieves competitive performance to the state-of-the-art methods. Specifically, our framework remarkably outperforms state-of-the-art by 8.9% mAP scores on Occluded-Duke dataset. Code is available at https://github.com/chenxiangzZ/SRNet.
Evaluation of Ethereum End-to-end Transaction Latency

Ethereum is a promising but revolutionary blockchain approach to support rich computational services enabled by Turing-complete instructions, also known as smart contract. Using smart contract as "back-end", any party can publish its Decentralized Applications (DApps) available to all Internet users. In other words, blockchain and smart contract enable evolving existing Internet services and applications to decentralised versions. With the developments of versatile Internet applications, the QoS requirements can be very different. For example, new-age applications such as AR/VR can be quite sensitive to even millisecond delays. End-to-end transaction latency in blockchain is a critical factor that affects the performance of DApps. It is the time duration from sending out a transaction (from the client) to having the transaction recorded on the chain. In this paper, we experimentally evaluate the end-to-end latency in Ethereum and discuss the factors that may affect latency on the Ethereum main-net. It has a particular focus on the gas price because it is a key parameter controlled by users. The experiment results demonstrate some counter intuitive finding, e.g. increasing gas price does not reduce the Ethereum end-to-end latency significantly for a certain range of gas prices.
Toward optimal probabilistic active learning using a Bayesian approach

Abstract Gathering labeled data to train well-performing machine learning models is one of the critical challenges in many applications. Active learning aims at reducing the labeling costs by an efficient and effective allocation of costly labeling resources. In this article, we propose a decision-theoretic selection strategy that (1) directly optimizes the gain in misclassification error, and (2) uses a Bayesian approach by introducing a conjugate prior distribution to determine the class posterior to deal with uncertainties. By reformulating existing selection strategies within our proposed model, we can explain which aspects are not covered in current state-of-the-art and why this leads to the superior performance of our approach. Extensive experiments on a large variety of datasets and different kernels validate our claims.
Enhanced Spinning Parallelogram Operator Combining Color Constraint and Histogram Integration for Robust Light Field Depth Estimation

Accurate depth estimation is an essential part of most light field applications. Previous approach Spinning Parallelogram Operator (SPO) achieves robust depth estimation results in noisy and occluded scenes by fitting lines in Epipolar Plane Images (EPIs). However, only using histogram distances, SPO cannot handle complex occlusions and is sensitive to different parameters. In this letter, we propose enhanced Spinning Parallelogram Operator utilizing color constraint and histogram integration (SPO-CH) to address the above problems. The color consistency between different views is first introduced to exclude occlusions when measuring the histogram distances of occluded areas. We then design a novel Gaussian integration histogram that combines information from other adjacent bins, which improves the performances of noisy and texture-less scenes. Experimental results show that the proposed method effectively improves the depth estimation results. Comparisons with other state-of-the-art approaches show that the proposed method achieves better results for both synthetic and real-world images.
Gibbs sampler and coordinate ascent variational inference: A set-theoretical review

One of the fundamental problems in Bayesian statistics is the approximation of the posterior distribution. Gibbs sampler and coordinate ascent variational inference are renownedly utilized approximation techniques that rely on stochastic and deterministic approximations. In this paper, we define fundamental sets of densities frequently used in Bayesian inference. We shall be concerned with the clarification of the two schemes from the set-theoretical point of view. This new way provides an alternative mechanism for analyzing the two schemes endowed with pedagogical insights.
Older Adults’ Think-Aloud Verbalizations and Speech Features for Identifying User Experience Problems

Subtle patterns in users’ think-aloud (TA) verbalizations and speech features are shown to be telltale signs of User Experience (UX) problems. However, such patterns were uncovered among young adults. Whether such patterns apply for older adults remains unknown. We conducted TA usability testing with older adults using physical and digital products. We analyzed their verbalizations, extracted speech features, identified UX problems, and uncovered the patterns that indicate UX problems. Our results show that when older adults encounter problems, their verbalizations tend to include observations (remarks), negations, question words and words with negative sentiments; and their voices tend to include high loudness, high pitch and high speech rate. We compare these subtle patterns with those of young adults uncovered in recent studies and discuss the implications of these patterns for the design of Human-AI collaborative UX analysis tools to better pinpoint UX problems.
Sensors in blockchain

Blockchain, the technology behind bitcoin, has stimulated global conversation around digital decentralization to connect societies and economies. Connected low-cost sensors within Internet of Things (IoT) networks may soon perform measurements, exchange data, and make decisions automatically over decentralized networks; these systems could improve healthcare, the quality of goods, and supply chain efficiency.