OpenRaiser/Intern-Atlas huggingface dataset:
Render In-between: Motion Guided Video Synthesis for Action Interpolation
Venue: arXiv cs.CV 2021
Year: 2021

Upsampling videos of human activity is an interesting yet challenging task with many potential applications ranging from gaming to entertainment and sports broadcasting. The main difficulty in synthesizing video frames in this setting stems from the highly complex and non-linear nature of human motion and the complex appearance and texture of the body. We propose to address these issues in a motion-guided frame-upsampling framework that is capable of producing realistic human motion and appearance. A novel motion model is trained to inference the non-linear skeletal motion between frames by leveraging a large-scale motion-capture dataset (AMASS). The high-frame-rate pose predictions are then used by a neural rendering pipeline to produce the full-frame output, taking the pose and background consistency into consideration. Our pipeline only requires low-frame-rate videos and unpaired human motion data but does not require high-frame-rate videos for training. Furthermore, we contribute the first evaluation dataset that consists of high-quality and high-frame-rate videos of human activities for this task. Compared with state-of-the-art video interpolation techniques, our method produces in-between frames with better quality and accuracy, which is evident by state-of-the-art results on pixel-level, distributional metrics and comparative user evaluations. Our code and the collected dataset are available at https://git.io/Render-In-Between.
MOST-GAN: 3D Morphable StyleGAN for Disentangled Face Image Manipulation
Venue: arXiv cs.CV 2021
Year: 2021

Recent advances in generative adversarial networks (GANs) have led to remarkable achievements in face image synthesis. While methods that use style-based GANs can generate strikingly photorealistic face images, it is often difficult to control the characteristics of the generated faces in a meaningful and disentangled way. Prior approaches aim to achieve such semantic control and disentanglement within the latent space of a previously trained GAN. In contrast, we propose a framework that a priori models physical attributes of the face such as 3D shape, albedo, pose, and lighting explicitly, thus providing disentanglement by design. Our method, MOST-GAN, integrates the expressive power and photorealism of style-based GANs with the physical disentanglement and flexibility of nonlinear 3D morphable models, which we couple with a state-of-the-art 2D hair manipulation network. MOST-GAN achieves photorealistic manipulation of portrait images with fully disentangled 3D control over their physical attributes, enabling extreme manipulation of lighting, facial expression, and pose variations up to full profile view.
A generalized algorithm and framework for online 3-dimensional bin packing in an automated sorting center
Venue: arXiv cs.RO 2021
Year: 2021

Online 3-dimensional bin packing problem (O3D-BPP) is getting renewed prominence due to the industrial automation brought by Industry 4.0. However, due to limited attention in the past and its challenging nature, a good approximate algorithm is in scarcity as compared to 1D or 2D problems. This paper considers real-time O$3$D-BPP of cuboidal boxes with partial information (look-ahead) in an automated robotic sorting center. We present two rolling-horizon mixed-integer linear programming (MILP) cum-heuristic based algorithms: MPack (for bench-marking) and MPackLite (for real-time deployment). Additionally, we present a framework OPack that adapts and improves the performance of BP heuristics by utilizing information in an online setting with a look-ahead. We then perform a comparative analysis of BP heuristics (with and without OPack), MPack, and MPackLite on synthetic and industry provided data with increasing look-ahead. MPackLite and the baseline heuristics perform within bounds of robot operations and thus, can be used in real-time.
FREGAN : an application of generative adversarial networks in enhancing the frame rate of videos
Venue: arXiv cs.CV 2021
Year: 2021

A digital video is a collection of individual frames, while streaming the video the scene utilized the time slice for each frame. High refresh rate and high frame rate is the demand of all high technology applications. The action tracking in videos becomes easier and motion becomes smoother in gaming applications due to the high refresh rate. It provides a faster response because of less time in between each frame that is displayed on the screen. FREGAN (Frame Rate Enhancement Generative Adversarial Network) model has been proposed, which predicts future frames of a video sequence based on a sequence of past frames. In this paper, we investigated the GAN model and proposed FREGAN for the enhancement of frame rate in videos. We have utilized Huber loss as a loss function in the proposed FREGAN. It provided excellent results in super-resolution and we have tried to reciprocate that performance in the application of frame rate enhancement. We have validated the effectiveness of the proposed model on the standard datasets (UCF101 and RFree500). The experimental outcomes illustrate that the proposed model has a Peak signal-to-noise ratio (PSNR) of 34.94 and a Structural Similarity Index (SSIM) of 0.95.
Understanding the Use of Voice Assistants by Older Adults
Venue: arXiv cs.HC 2021
Year: 2021

Older adults are using voice-based technologies in a variety of different contexts and are uniquely positioned to benefit from smart speakers' handsfree, voice-based interface. In order to better understand the ways in which older adults engage with and learn how to use smart speakers, we conducted qualitative, semi-structured interviews with four older adults who own smart speakers. Emerging findings indicate that older adults benefit from smart speakers as both an assistive and a social technology. Findings also suggest that when older adults learn new technologies in a formal, communal environment there is successful adoption.
Kernel Deformed Exponential Families for Sparse Continuous Attention
Venue: arXiv cs.LG 2021
Year: 2021

Attention mechanisms take an expectation of a data representation with respect to probability weights. This creates summary statistics that focus on important features. Recently, (Martins et al. 2020, 2021) proposed continuous attention mechanisms, focusing on unimodal attention densities from the exponential and deformed exponential families: the latter has sparse support. (Farinhas et al. 2021) extended this to use Gaussian mixture attention densities, which are a flexible class with dense support. In this paper, we extend this to two general flexible classes: kernel exponential families and our new sparse counterpart kernel deformed exponential families. Theoretically, we show new existence results for both kernel exponential and deformed exponential families, and that the deformed case has similar approximation capabilities to kernel exponential families. Experiments show that kernel deformed exponential families can attend to multiple compact regions of the data domain.
Deep neural networks as nested dynamical systems
Venue: arXiv cs.LG 2021
Year: 2021

There is an analogy that is often made between deep neural networks and actual brains, suggested by the nomenclature itself: the "neurons" in deep neural networks should correspond to neurons (or nerve cells, to avoid confusion) in the brain. We claim, however, that this analogy doesn't even type check: it is structurally flawed. In agreement with the slightly glib summary of Hebbian learning as "cells that fire together wire together", this article makes the case that the analogy should be different. Since the "neurons" in deep neural networks are managing the changing weights, they are more akin to the synapses in the brain; instead, it is the wires in deep neural networks that are more like nerve cells, in that they are what cause the information to flow. An intuition that nerve cells seem like more than mere wires is exactly right, and is justified by a precise category-theoretic analogy which we will explore in this article. Throughout, we will continue to highlight the error in equating artificial neurons with nerve cells by leaving "neuron" in quotes or by calling them artificial neurons. We will first explain how to view deep neural networks as nested dynamical systems with a very restricted sort of interaction pattern, and then explain a more general sort of interaction for dynamical systems that is useful throughout engineering, but which fails to adapt to changing circumstances. As mentioned, an analogy is then forced upon us by the mathematical formalism in which they are both embedded. We call the resulting encompassing generalization deeply interacting learning systems: they have complex interaction as in control theory, but adaptation to circumstances as in deep neural networks.
Learning to Explore by Reinforcement over High-Level Options
Venue: arXiv cs.AI 2021
Year: 2021

Autonomous 3D environment exploration is a fundamental task for various applications such as navigation. The goal of exploration is to investigate a new environment and build its occupancy map efficiently. In this paper, we propose a new method which grants an agent two intertwined options of behaviors: "look-around" and "frontier navigation". This is implemented by an option-critic architecture and trained by reinforcement learning algorithms. In each timestep, an agent produces an option and a corresponding action according to the policy. We also take advantage of macro-actions by incorporating classic path-planning techniques to increase training efficiency. We demonstrate the effectiveness of the proposed method on two publicly available 3D environment datasets and the results show our method achieves higher coverage than competing techniques with better efficiency.
Finding the KT partition of a weighted graph in near-linear time
Venue: arXiv cs.DS 2021
Year: 2021

In a breakthrough work, Kawarabayashi and Thorup (J.~ACM'19) gave a near-linear time deterministic algorithm for minimum cut in a simple graph $G = (V,E)$. A key component is finding the $(1+\varepsilon)$-KT partition of $G$, the coarsest partition $\{P_1, \ldots, P_k\}$ of $V$ such that for every non-trivial $(1+\varepsilon)$-near minimum cut with sides $\{S, \bar{S}\}$ it holds that $P_i$ is contained in either $S$ or $\bar{S}$, for $i=1, \ldots, k$. Here we give a near-linear time randomized algorithm to find the $(1+\varepsilon)$-KT partition of a weighted graph. Our algorithm is quite different from that of Kawarabayashi and Thorup and builds on Karger's framework of tree-respecting cuts (J.~ACM'00). We describe applications of the algorithm. (i) The algorithm makes progress towards a more efficient algorithm for constructing the polygon representation of the set of near-minimum cuts in a graph. This is a generalization of the cactus representation initially described by Benczúr (FOCS'95). (ii) We improve the time complexity of a recent quantum algorithm for minimum cut in a simple graph in the adjacency list model from $\widetilde O(n^{3/2})$ to $\widetilde O(\sqrt{mn})$. (iii) We describe a new type of randomized algorithm for minimum cut in simple graphs with complexity $O(m + n \log^6 n)$. For slightly dense graphs this matches the complexity of the current best $O(m + n \log^2 n)$ algorithm which uses a different approach based on random contractions. The key technical contribution of our work is the following. Given a weighted graph $G$ with $m$ edges and a spanning tree $T$, consider the graph $H$ whose nodes are the edges of $T$, and where there is an edge between two nodes of $H$ iff the corresponding 2-respecting cut of $T$ is a non-trivial near-minimum cut of $G$. We give a $O(m \log^4 n)$ time deterministic algorithm to compute a spanning forest of $H$.
Integrating Pretrained Language Model for Dialogue Policy Learning
Venue: arXiv cs.CL 2021
Year: 2021

Reinforcement Learning (RL) has been witnessed its potential for training a dialogue policy agent towards maximizing the accumulated rewards given from users. However, the reward can be very sparse for it is usually only provided at the end of a dialog session, which causes unaffordable interaction requirements for an acceptable dialog agent. Distinguished from many efforts dedicated to optimizing the policy and recovering the reward alternatively which suffers from easily getting stuck in local optima and model collapse, we decompose the adversarial training into two steps: 1) we integrate a pre-trained language model as a discriminator to judge whether the current system action is good enough for the last user action (i.e., \textit{next action prediction}); 2) the discriminator gives and extra local dense reward to guide the agent's exploration. The experimental result demonstrates that our method significantly improves the complete rate (~4.4\%) and success rate (~8.0\%) of the dialogue system.
Cognitive Load and Productivity Implications in Human-Chatbot Interaction
Venue: arXiv cs.HC 2021
Year: 2021

The increasing progress in artificial intelligence and respective machine learning technology has fostered the proliferation of chatbots to the point where today they are being embedded into various human-technology interaction tasks. In enterprise contexts, the use of chatbots seeks to reduce labor costs and consequently increase productivity. For simple, repetitive customer service tasks such already proves beneficial, yet more complex collaborative knowledge work seems to require a better understanding of how the technology may best be integrated. Particularly, the additional mental burden which accompanies the use of these natural language based artificial assistants, often remains overlooked. To this end, cognitive load theory implies that unnecessary use of technology can induce additional extrinsic load and thus may have a contrary effect on users' productivity. The research presented in this paper thus reports on a study assessing cognitive load and productivity implications of human chatbot interaction in a realistic enterprise setting. A/B testing software-only vs. software + chatbot interaction, and the NASA TLX were used to evaluate and compare the cognitive load of two user groups. Results show that chatbot users experienced less cognitive load and were more productive than software-only users. Furthermore, they show lower frustration levels and better overall performance (i.e, task quality) despite their slightly longer average task completion time.
The Secrecy Gain of Formally Unimodular Lattices on the Gaussian Wiretap Channel
Venue: arXiv cs.IT 2021
Year: 2021

We consider lattice coding for the Gaussian wiretap channel, where the challenge is to ensure reliable communication between two authorized parties while preventing an eavesdropper from learning the transmitted messages. Recently, a measure called the secrecy function of a lattice coding scheme was proposed as a design criterion to characterize the eavesdropper's probability of correct decision. In this paper, the family of formally unimodular lattices is presented and shown to possess the same secrecy function behavior as unimodular and isodual lattices. Based on Construction A, we provide a universal approach to determine the secrecy gain, i.e., the maximum value of the secrecy function, for formally unimodular lattices obtained from formally self-dual codes. Furthermore, we show that formally unimodular lattices can achieve higher secrecy gain than the best-known unimodular lattices from the literature.
Detection of Hate Speech using BERT and Hate Speech Word Embedding with Deep Model
Venue: arXiv cs.CL 2021
Year: 2021

The enormous amount of data being generated on the web and social media has increased the demand for detecting online hate speech. Detecting hate speech will reduce their negative impact and influence on others. A lot of effort in the Natural Language Processing (NLP) domain aimed to detect hate speech in general or detect specific hate speech such as religion, race, gender, or sexual orientation. Hate communities tend to use abbreviations, intentional spelling mistakes, and coded words in their communication to evade detection, adding more challenges to hate speech detection tasks. Thus, word representation will play an increasingly pivotal role in detecting hate speech. This paper investigates the feasibility of leveraging domain-specific word embedding in Bidirectional LSTM based deep model to automatically detect/classify hate speech. Furthermore, we investigate the use of the transfer learning language model (BERT) on hate speech problem as a binary classification task. The experiments showed that domainspecific word embedding with the Bidirectional LSTM based deep model achieved a 93% f1-score while BERT achieved up to 96% f1-score on a combined balanced dataset from available hate speech datasets.
Fitness Landscape Footprint: A Framework to Compare Neural Architecture Search Problems
Venue: arXiv cs.LG 2021
Year: 2021

Neural architecture search is a promising area of research dedicated to automating the design of neural network models. This field is rapidly growing, with a surge of methodologies ranging from Bayesian optimization,neuroevoltion, to differentiable search, and applications in various contexts. However, despite all great advances, few studies have presented insights on the difficulty of the problem itself, thus the success (or fail) of these methodologies remains unexplained. In this sense, the field of optimization has developed methods that highlight key aspects to describe optimization problems. The fitness landscape analysis stands out when it comes to characterize reliably and quantitatively search algorithms. In this paper, we propose to use fitness landscape analysis to study a neural architecture search problem. Particularly, we introduce the fitness landscape footprint, an aggregation of eight (8)general-purpose metrics to synthesize the landscape of an architecture search problem. We studied two problems, the classical image classification benchmark CIFAR-10, and the Remote-Sensing problem So2Sat LCZ42. The results present a quantitative appraisal of the problems, allowing to characterize the relative difficulty and other characteristics, such as the ruggedness or the persistence, that helps to tailor a search strategy to the problem. Also, the footprint is a tool that enables the comparison of multiple problems.
Detect-and-Segment: a Deep Learning Approach to Automate Wound Image Segmentation
Venue: arXiv cs.CV 2021
Year: 2021

Chronic wounds significantly impact quality of life. If not properly managed, they can severely deteriorate. Image-based wound analysis could aid in objectively assessing the wound status by quantifying important features that are related to healing. However, the high heterogeneity of the wound types, image background composition, and capturing conditions challenge the robust segmentation of wound images. We present Detect-and-Segment (DS), a deep learning approach to produce wound segmentation maps with high generalization capabilities. In our approach, dedicated deep neural networks detected the wound position, isolated the wound from the uninformative background, and computed the wound segmentation map. We evaluated this approach using one data set with images of diabetic foot ulcers. For further testing, 4 supplemental independent data sets with larger variety of wound types from different body locations were used. The Matthews' correlation coefficient (MCC) improved from 0.29 when computing the segmentation on the full image to 0.85 when combining detection and segmentation in the same approach. When tested on the wound images drawn from the supplemental data sets, the DS approach increased the mean MCC from 0.17 to 0.85. Furthermore, the DS approach enabled the training of segmentation models with up to 90% less training data while maintaining the segmentation performance.
Profit Sharing Contracts between Content and Service Providers for Enhanced Network Quality
Venue: arXiv cs.NI 2021
Year: 2021

It has been a long demand of Internet Service Providers (ISPs) that the Content Providers (CPs) share their profits for investments in network infrastructure. In this paper, we study profit sharing contracts between a CP with multiple ISPs. Each ISP commits to improving the Quality of Service (QoS) for the end-users through higher investments efforts. The CP agrees to share the profits due to the resulting higher demand for its content. We first model non-cooperative interaction between the CP and the ISPs as a two-stage Stackelberg game. CP is the leader that decides what fraction of its profits will be shared with the ISPs. Each ISP then simultaneously decides the amount of effort (investment) to enhance network quality. Here, CP cannot observe individual effort by the ISPs, which poses a challenge for the CP to decide how to share the profits with each ISP. Therefore, we also investigate a cooperative scenario, where the CP only decides the total share it gives to the ISPs, and each ISP then cooperatively shares the profit among themselves. We study the effect of such cooperation between the ISPs by building a Nash Bargaining based model. We show that the collaboration improves total effort by the ISPs and the payoff of the CP.
Characterizing and Understanding the Generalization Error of Transfer Learning with Gibbs Algorithm
Venue: arXiv cs.LG 2021
Year: 2021

We provide an information-theoretic analysis of the generalization ability of Gibbs-based transfer learning algorithms by focusing on two popular transfer learning approaches, $α$-weighted-ERM and two-stage-ERM. Our key result is an exact characterization of the generalization behaviour using the conditional symmetrized KL information between the output hypothesis and the target training samples given the source samples. Our results can also be applied to provide novel distribution-free generalization error upper bounds on these two aforementioned Gibbs algorithms. Our approach is versatile, as it also characterizes the generalization errors and excess risks of these two Gibbs algorithms in the asymptotic regime, where they converge to the $α$-weighted-ERM and two-stage-ERM, respectively. Based on our theoretical results, we show that the benefits of transfer learning can be viewed as a bias-variance trade-off, with the bias induced by the source distribution and the variance induced by the lack of target samples. We believe this viewpoint can guide the choice of transfer learning algorithms in practice.
Classification of Goods Using Text Descriptions With Sentences Retrieval
Venue: arXiv cs.AI 2021
Year: 2021

The task of assigning and validating internationally accepted commodity code (HS code) to traded goods is one of the critical functions at the customs office. This decision is crucial to importers and exporters, as it determines the tariff rate. However, similar to court decisions made by judges, the task can be non-trivial even for experienced customs officers. The current paper proposes a deep learning model to assist this seemingly challenging HS code classification. Together with Korea Customs Service, we built a decision model based on KoELECTRA that suggests the most likely heading and subheadings (i.e., the first four and six digits) of the HS code. Evaluation on 129,084 past cases shows that the top-3 suggestions made by our model have an accuracy of 95.5% in classifying 265 subheadings. This promising result implies algorithms may reduce the time and effort taken by customs officers substantially by assisting the HS code classification task.